 Hello everybody, welcome to our final talk of the session of the day. There's another deaf room in here this afternoon This is Edward Lewis of ICANN Who is having hardware troubles like all of us all the time and I say problems The the microphone fell off the clip And when I look at something that small disappears with my age of eyesight is Mancing and I left my reading glasses my my Other jacket Edward will be talking about the different ways of minimizing any yes luck. Thank you. Thank you. Thank you very much All right, so This I guess this is kind of stars a pet peeve of mine And I talked to the authors about it of the document and they encourage me to bring this and bring it up I'm gonna try to make this pet peeve interesting to software developers. That's kind of weren't going with this So The title here and the subter I have a name of a document. It's an RFC from the ITF came out a little while ago Talking about how to minimize Responding responding to any I'll talk about why why do we have this document a little bit about that an observation I've made a measurement based on dealing with this Then trying to find out why I see the results I saw and then see is this a good thing or a bad thing for the trend of software development and Of course, I have to mend Brett Bert who bears camel at the end of this because that's everything's about the camel So the document why does it exist the Q type Q type any is a feature of the original DNS It's been around since the start And it's had it has basically three uses one's benign that is You can use it to get a lot of information about a name if you go to the author of server and ask it And you can get everything is there you're kind of like snooping on what the author of servers saying about that name Regardless of type it has everything comes back it has a problematic use because some people realize they could do that and Used it to ask recursive servers the same question not really recursive servers don't get all of the records So they may not have all of the stuff and any and all became kind of confused That was the first row down to this being a problem number then finally it became kind of a malicious use of this because if you wanted to generate a large volume of traffic for it for say a traffic flooding DDoS You could ask for all the records any of the records at some name and get a huge mass of stuff Especially with DNS sec of the larger use of names now and flood victims with lots of traffic Those are the three probably most obvious ways of seeing this query go from being something that was useful To something that we really wanted to kind of get rid of so And for while I was very much in getting I want to get rid of this too at one time when I was working in a job That this was in fact impacting us But when this was written I was not part of the effort it had moved on to other folks and I moved out of that job so But in doing this and saying that the this any we had to get kind of stopped using any queries What do you do instead? How do you say no? I don't do that anymore in the DS? It doesn't really happen But there were there were a few ways to look at this one was to say we don't do that But we don't have that way of doing this In the DNS is kind of funny because the DS you say notice someone's risk query In a sense of an error that may make them try again or even more frequently In fact, we see if we stop answering queries for for reasons we see traffic come up instead of go down It goes backwards because people are trying harder to get the information also Yeah, we also realize we want to help those that were doing things the old way backwards capability those that were using the Problematic version of this we want to make them not be pained by this And so the protocol developers tried to come with some way to say no We're not doing it anymore that would appease all of the interests that were involved here And to go to go before I go further the way that says no is that there are three in the doctor There are three different things that could happen instead of sending back everything you have about the name When you're asked for for any any is not all but any is the query You can send back a subset Just pick something and just send send it back minimize it coming back The second thing is to come back with a synthesized H info which in the previous talk Leo mentioned that that was revived by cloudflare's effort to who happened to be the people Participating in this document They brought that back to life saying we'll use the h info record to say This is a minimized any query out there And the third one was try to guess what the query wanted And if you've ever written DNS code you realize a DNS doesn't give you much information beyond here's what I'm asking I don't tell you why so it's really hard to guess when you're writing code I don't know code that can guess is really kind of hard sometimes it might you might have an idea but not always Now this I'm not picking on this document because a document actually mentions the points I want to make and these are verbatim clips from the document The first one says this mechanism does not signal that an incomplete subset has been coming back It doesn't tell you that it's minimized it number two it receives h info in response But it's not possible tell us h info was synthesized for this purpose So then again, it's not really that reliable and in the last one It's not it's possible to guess what if initiator wants but not always so I could tell here that the authors We're not trying to just cut things off, but they really had there's no place to go with that. That's that's great So to clarify my complaint here is I'm not out to stop limiting the query any I think it's really Kind of not a use not necessary to the protocol The problem is that I don't like the way this comes it's becoming non-deterministic You ask a question you get back an answer which might be a mystery or might be the response You can't tell one or the other and this has happened before This is a small issue probably, but I see it the same way We we we over use serve fail to mean errors of DNS sec versus other service failures and There was the the use of the txt record for everything out there And if you remember the spf record at having to do with mail Mail protection, you know that didn't go well So so my little experiment I run something where I do the any I do the any query all the time I use it to look at the tld operator the tops of the tld zones And so I know I have an idea what's going to come back So when I ran this I'd run 13,475 queries for this experiment Oh, I don't normally do this but I asked every name server for every tld at any query. What did I get back? I got about 250 260 nos UDP tcp. I tried both of them And but the thing that got me was I got like nine or ten different ways They said no in that small sample that really bugged me as a small sample nine or ten different ways And then the absolute numbers here are kind of over counts because operators do do a whole lot of work together If one operator changed their decision they might switch 250 responses right away But to give you a graphical view of this and I don't really have a good I got the small crayon set at work. So I don't have a lot of colors So I did this chart this way the first chart shows I asked this many times. I got that many responses. Of course. It's pretty good. There's tlds out there Of in udp. I get a lot of truncation, which is okay. It's understandable. You truncate large response in udp But of the the ones that got through there that were problematic I got a spread don't worry about what they are over there There's a spread a lot of colors in that circle over there And that covers everything including if you look through there, there's the h info down towards the bottom That's the special case for the h-inch info record all the other ones were that was the one type that was seen there except for the empty one Of course, that means nothing came back And I got 10 different ways in udp for tcp a same thing except that I don't have any truncation in tcp And I get back another little color wheel with lots of different variation out there Again the fact that the ns one is the biggest part that only means that somebody does it a lot that way Doesn't necessarily mean there's a lot more use of ns So I started drilling into the numbers because at first this was that was what I thought was interesting to see But then I said well, let me figure out why I got this Why am I getting these numbers? I found that some of the ip addresses out there I ask For different ip for different tlds respond differently So somehow it's being a as a choice based on either per query or per zone or for something going out there So I decided to figure out no, can I find some more patterns here? Well, it turns out that tlds operators out there sometimes don't really Hide and they don't want to hide their versions. So they actually version dot bind actually works for some of them And I'm looking at version dot bind now That's mostly not a great way to get data But I got something out of it And so I looked at the and I and there was like half a dozen implementations that popped up that I could identify Three of them appear on this chart It means three of them were minimizing. There were no there was no evidence from the other three that They were minimizing the mind you have the big four appear So three of the big four and that's bind nsd on NS say bind nsd not and a parody and s or the big four open open source implementations I call Three of them are minimizing some way and you can see what how it comes out here where there is Empty answers from some of them are not from inflation one and three They all do ns only and then there's other types that are spread out there, which I couldn't really explain So I actually did want to explain this So I looked at the configuration information documentation and from what I could tell And nsd there's a refuse any option and it says what it does here on udp. It would truncate and for tcp Works like normal default to snow By 914 6 says that over udp. It'll pick only one of the rsets To go out there. It doesn't say anything about tcp in there. So I couldn't tell what was going to happen I didn't experiment myself default to snow And I looked up not parody and s and I could not find documentation saying how to do it But I found email in but through doing the searches web searches saying that this was just not done So looking at these four big four and how they let you configure this Didn't answer the question of well, how did I have it answered differently per tld? In fact, this doesn't even match up. I have three out of four Doing this even though two say how and how they Um The choices just don't line up with this. It's still kind of interesting to me to figure how this is going on So i'm a bit baffled by this I couldn't see evidence that the operators were able to make choices that looked like it was intentional in the in doing this I didn't see that some operator always answered a certain way In fact, I was baffled by one particular name server which answered the same way for all but one of the zones It was serving up. I don't know So I don't know it's somewhere in the software somewhere in the operations. It's it's not quite lined up yet Um, why does this bother me? I mean, I I don't care much about the any query I'll get into that but in the principles principles of protocol design. I like determinism I want to know query goes out. I should get back our predictable result or a very definitive signal when it comes back to I have this long-term running Ratio with with operators and realizing that We staff the desks with people have less background over time just because of growth and um, so I want the protocol to be simpler One time I heard this from somebody. I wish I could have a better support for this statement But a protocol design should be like a state machine You have state machines on both ends of the channel And as you trans go for you can transition from state to state based on each message going back and forth Or each signal going back and forth timeouts or whatever should be very deterministic If I'm on this side and I want and I think the other side is thinking this and I'm telling them something I want them to go here in the conversation I don't want them to actually kind of like maybe they're over here now I don't know like it makes it very hard to communicate in a protocol when you have this like random number generator essentially popping up there Now granted DNS is not really good at this but it's it's a dream I'd like to have the protocol be very specific so at both sides know where each other's thinking going back and forth Now The How does this apply to my observations here? I'm talking. I want I guess I didn't bring this slide up earlier. I probably should have I was asking From this end about information over here But I know over here These are apex they should have an so a record you should have an s record If they have dns sec They should have a certain set of records depending on what kind of dns sec they have So I have an expectation on this side of why I should come back And I'm not seeing that come back and I can tell it's an error Think about this in a more generic case where I don't know what this name is. Is it an apex? Is it not an apex? I don't know if I get back one one record set. I might think that is all it has If it if it it's better, so it's a little this is this is some uncertainty in my my In my mind here about the way this this RFC has recommended coming back with the don't don't I don't do the full thing Um now should I be able to take to minimize any response is a good fair question Um because I go back and forth. I don't think that this is critically a critical change to the protocol But it bothers me that we have this non deterministic element going on and I'm going to go past this point right now Another experience thing is staff expertise out there I've seen it where I wanted my operator to have much simpler tools to use I want the software they use to be simpler. They don't need to understand the protocol We can't afford to teach them the protocol anymore. We keep bringing more people in to people turning over They want to know how to make things work There are things working out there This puts pressure on the tool makers to make sure the tooling is there. It's in place and it's simple So with the gaps here protocol engineers describes the way software can be written They like functionality like to make things do more things But I find that sometimes they tend to lose sight of what's in operations and what the tool makers have to do excuse me operators Operators are interesting because they they have a job to do that doesn't matter what they're doing They have to do a job. They have to make sure things don't fall apart I work I met with a group of people who are new operators came into run an organization They know how to do operate they can operate anything a bus line a ticket agency now. They're running a dns thing But they're mystified because the protocol really hasn't it wasn't running the way I thought it would go What this does a lot of operators will lead on packaged software to do this defaults and all that Now the software developers and you know, I'm assuming a lot of people in this room are more interested in the software development process Caught in the middle of this and the Brett Brett Brett. I'm sorry. He wears camel talk If you haven't seen that this is a link to He is saying that what straws and I break the camel's back when I keep getting demands for more and more features on my dns software This is what I'm really concerned that this is doing is making this camel like getting closer to the edge So it's a classic dev ops issue maximize functionality versus to minimize downtime So what do you do about this? This is kind of this is what I'm my message here is I want to encourage folks in this room to Get involved with the with the itf process I was told that where were you when this document was being written? I said, well, I was actually there and I kind of got tired of it But people have to go in and say is this implementable is backwards compatibility always desirable Do we have to get rid of some old features and say, you know It also has got a break because we we don't doesn't matter shouldn't be there Can it be implemented? That's what needs people need to have an eye on that when they go to these organizations That they run the protocol development. That's my basic message here for folks Um You know if you're if you're not doing this what I fear is we're going to have to have more dns flag days At some point we get rid of features or features conflicting with each other We need to have more input to the process to to make sure that We don't say in this document. We're going to let any be minimized The response coming back is going to be this you want to have to be like one record You want to have one thing come back saying if you see this stop asking for any more information Assume you're not going to get any and fall back to whatever you need to get your information And that's my basic message for for the This talk here so I have an email my email isn't in the slides if you if you want to ask me about this you can ask me now I guess too So i'm done Thanks Thank you any questions for adverts Shane Hey, this is Shane. It's not really a question. It's more of a comment. Sure. I mean We all know that you can't really trust any queries for anything So it's already kind of performance art, right? It's like you go to a play and the the writer is trying to say something and the actors are trying to say something The directors are trying to say something and you have to interpret what you get back And it's it can be enjoyable, but don't treat it as as science and you're all right Yeah, this was part of the conversation I had with with the the people who wrote the document For for what I've been doing, I have I have a use for it. It's not about my use for it My use for it actually it that works perfect Because I'm asking a specific question of a specific small population I don't deserve to do that. In fact, my code is for years how I had fallback to Ask for everything you expect to see there anyway. It's just more efficient Um, and yeah, but I mean this is I don't think the any query It's not that I'm I want the any query to survive. Don't get me wrong. I don't want to survive I want it to be clearer that you're not doing it That's what I'm after that was when those documents being discussed Um, I first was The h info option was the first one out there and I thought well, you know We don't like to reuse what create a new record type. Oh, we'll take 20 years for that to roll out, right? Um, this is what I'm after we want to make sure that we do changes to the protocol So that's sustainable so that we you know Maybe we you know because and I I understand the discussion that went on too Well, what about in the case where someone was relying on it and we shouldn't take it away from the monosoceros? So maybe we'll do some Is that worth? I mean is it is it worth is that something we can keep doing because that's part of the straws in the camel You know that so that's good. I think if I think if people here felt strong enough that change Don't don't fight this one if a change is coming along in the itf And it's going to change the code in a way that makes the code blow it up big Go to the itf and let them know this is not implementable. And that's what I'm asking So here here Another question So I presume in your analysis you used you used non recursive queries What about the recursive ones and what is the role of a dns cache server in that respect because I was trying to implement or implement the last version of the djb dns curve I also was had to take the decision what to do with the any ones here. So that's Yeah, it's all of it yet. Yeah, I mean You know Well, that's a good question. I mean in my in my work. I always go to the authority of servers That that's because I'm grafting off a code that does that I didn't do recursive. That's a good question What should a recursive server come back with? You know and I if I told you now I'd be thinking on top of my feet, but obviously You know, there are the problems I like my first reaction was a non-imp not implemented our code But if that happens, some folks go into tizzy about that and keep asking other servers now to see if they do it Not input is for non implemented opcodes. Right. They say, right. Yeah So So we maybe that's part of the problem. We don't have the protocol doesn't have enough definitive like This this is a there's there's zone level errors and there's server level errors, right? The server's not going to respond to you go somewhere else But when the zone is out We need to tell them stop asking I've seen meltdown so many ways. I mean, it's not uncommon to a lot of people out there But for a recursive service out there, I mean, I just haven't thought I've never worked in a recursive service field So I haven't thought of it to be honest. Yeah That that should actually that would actually be Part of what is in here because I think The people who wrote this were thinking as a sort of servers to come to think of it because I don't see recursive in there at all And I do know who they were, you know, the the workroom cloud flare is obviously And they were trying to solve some problems also beyond the ones I was just putting out there as a spec But I'll just leave it at that. So Any other questions? Okay. Thank you. Sure You want this now don't steal that This concludes the dns dev room in about 30 minutes the web performance dev room will start here Um, I will stick around for a bit bet some others will so if you have any questions talk to us Thank you for visiting us