 I'm Petr Spacek working for CZethnic on Not Resolver project, but right now speaking generally about DNS and the madness associated with it. So, is a single vendor enough or not? Well, in short, we'll go through how we will select a vendor just to wake you up. I would like to make it more interactive because in the end, we will have some time for discussion how we can make it, we as a vendors, how can we make it easier for you as users of the software to deploy more software from more vendors at the same time and still stay sane with your setups. So, we'll look at the single vendor, what you want to configure then, why it's not a good idea to use a single vendor, then what you get if you have multiple implementations deployed at the same time, what are the problems and what we can do with it. So, selecting a vendor, of course, everyone wants features, features, features, then possibly performance and then reliability is not even on the slide because that's usually, you need to wake up in the middle of night when it crashes and solve it. But yeah, we'll see. So, features. DNS is a madness. This is screenshot from Bert's tool, DNS camel. You can look it up and it has like 200 different RFCs with different features, clarifications, whatever, and you need to pick what you want in different implementations. So, let's try. Who is using dynamic DNS updates? Please raise your hand so we can see. Okay. So, that's less than I expected. So, what we have DNS sec on the slide, who is using DNS sec? Oh, that's even more than dynamic updates. I'm surprised. They go together so well. Well, okay. So, something more interesting. DNS 64, still some hands apparently. We didn't spend time implementing in just for nothing. Okay, electric curves in DNS sec. Anyone? Wow. Really? Oh, wow. Okay. And the name. Does anyone recall what the name is? Recall again. Okay. And is anyone using it? Please use the other hand. Okay, like two people in the room. Okay. Well, not really much. And we could go on and on and on with this game because there is plenty of features to pick on. Different vendors have different subsets and different approach to configure them. So, that's one piece of like vendor selection game. Then you have performance, right? Someone just don't need to care because the throughput is so low that you just don't need to care about it. Some people are crazy and want like thousands and thousands QPS. So, maybe we can try. Who has like at least thousand queries per second on our third at this site? Okay. Varan, you don't need to raise your hand in this game. Okay. So, that was 1,000. So, 10,000. Okay. Still, okay. And more than 100,000 QPS. Okay. Still some hands. So, apparently... All together or one? Huh? All together or one? On one server, of course. Sorry. I should have said that in front. So, again, there is a huge scale from very small deployments to huge ones and some people care about performance, some don't. Okay. Then SLA, of course. If you want to have your DNS up and running all the time because someone is paying you for it, you might want to consider getting a contract with support. So, who has some support contract not without specifying any conditions? Okay. Some people, okay. And who has like 12 hours response time in the support contract? Anyone? And we sell them. Okay. Yeah, that's the other side, yeah. Okay. Well, not really a lot. So, let's see how the presentation goes. Maybe it's 11. Anyway, whatever you do when you are selecting a vendor, one day the phone will start ringing in the middle of night, most probably on Christmas day or something like that because everyone loves that. Well, the reason is that no matter what you do when you are selecting the software, there will be bugs. It's just the matter of fact. So, the question is how we deal with that. The problem is that really there is no single vendor here which is like super reliable or not. I'm not picking on anyone. It's sorted alphabetically. Bind has its own problems. Well, not DNS has its own problems as well. Microsoft DNS is not an exception. Power DNS unfortunately is on my list as well. Unbound belongs to the same club. So, maybe self-hosting software might not be the right solution. So, cloud to the rescue, will it help? No, of course not. Everyone from the big cloud providers had some problem as well. Cloudflare was down because of BGP mess, Dine that's famous, so I will not elaborate. Google had problems from time to time as well. There is no safe place to hide. So, what can we do? Well, you know, the Linux store world couple years ago had famous quote that it was like, oh, if your deployment depends on having all the complex pieces up and running all the time, you have a serious technological problems. You should reconsider. You should have the ability to handle bugs because it will happen. It something will crash no matter what you do. So, yeah, this is the reason why I'm trying to emphasis, put emphasis on diversity, software diversity here. So, ah yeah, there's the quote, I forgot it's on slide. So, if your mission depends on every single piece of complex equipments thing up, you have some serious technological problems. So, you would better have capabilities to handle bugs. That's exactly it. So, sure, of course everyone knows this, but once you are start deploying software from multiple vendors, it has some pros and some cons. So, the pros are kind of obvious, or at least some of them. At least the packet of depth is unlikely, right? If I have two different implementations like no DNS and NSD or something, it's unlikely that both implementations will crash when observing the same packet. So, this makes some types of attacks super hard or much harder. Well, if you think about it for a little more, maybe if you have two different implementations which work reasonably well, you don't need to get up in the middle of night when one of them crashes because if the other is taking over, well, maybe it can wait till morning. And of course, if you have, if you are not depending on single implementation, maybe you don't need SLA with one hour response time or something crazy like that, but you might get two support contracts but which much easier conditions and does price. So, well, of course there are some costs involved as well. You have to pick the vendors first which is super annoying because you have to go through it and test it and so on. But usually if you do your selection right, you will keep the result for a couple of years. So, that's not the biggest problem. Biggest problem is the operational complexity, right? Because if you have, I don't know, NSD and PowerDNS and you want to add a slave zone to both of them, you have to do it basically twice or end times if you have implementations. So, that's super annoying and everyone has like different approach to handling that. Someone has homegrown configuration system, someone tries to buy it and then he has to tweak it anyway because he doesn't do everything. Monitoring is possibly more complex depending on what you want to monitor and how granular statistics you want to gather because of course different implementations are using different terms and monitor different statistics a little bit. So, you might not get exactly what you want from the other implementation. So, of course automation is the key. If you are going to deploy more than one implementation and still want to stay sane, you have to automate, otherwise bad things will happen and paperwork that's like just for the completeness. But besides having pros and cons, there are some serious traps. These are especially associated with OEMs, basically the companies who sell appliances which are typically a black box. So, beware because if you buy an appliance and you have no idea what's inside and then you decide to add a second implementation just as a good measure, for example, as a slave, it might not help you because if the implementation inside the appliance is the same as the slave you just installed for good measure, it might crash using exactly the same packet. So, like for example, Infoblox is shipping modified version of bind. So, it doesn't make any sense to have Infoblox and bind as like second implementation because it's not a second implementation. It's very likely to have the same box. The same goes for other vendors as well. It's of course there are some which have custom implementation but very often it's some open source software. And the important thing is to remember that it's changing over time. So, the fact that you asked your OEM five years ago and they gave you reply, okay, we are shipping NSD inside. It doesn't mean that it's still NSD five years after all. For example, a Secure64 is selling or was selling DNSX sign there which was based on NSD but they in recent years changed to not DNS authoritative. But they are not telling anyone or it's not promoted really. So, you have to ask. So, if you decide to go with OEM, please ask and make sure that you know what's inside. It's important for the diversity. Okay. So, recommendations is very simple. You set list to different implementations at very least in the part of the network which is facing the users or the public site, let's say. And beware of OEMs, double check what's inside. But this is easier set than done because again we are to the configuration nightmare with multiple implementations. So, what can we do? This is the part where I would like to turn to you and ask what you want to configure, what you configure most often because we as the DNS software vendors can do something. We can agree on interfaces for the most common stuff. Of course, not on everything because bind has, if you printed manual with list of options for bind, it would be like this wall and maybe that wall as well. So, it doesn't make to invent common interface for something which is not implemented anywhere else. But it makes sense to agree on common interface for reasonable subset of features. So, for example, I don't know, slaving zones and so on, which is basically the same everywhere and it's different just in the format you specify in the configuration file. But other than that, it's the same feature. So, what do you do most often, right? It's for simple things like deleting zones, there is like no discussion necessary because you just specify basically the operation like delete and then name of the zone and the rest is just sugar, it can be anything. But for more complex stuff like ACLs or T6 or so on, it might be more interesting. Does your ACL depend on T6 or is it just subnet based or is it some ACL which contains more than one subnet or is it combination of subnets and T6 keys? Or there is many options how we can name whatever or put whatever under name of ACL, but different operators have different notion of what's their typical use. So, is there anyone who is using T6 keys for authorizing something in ACL? Please raise your hand, okay. And do you have some more fancy combinations? Can you tell me, yell at me, what do you have like 100 different keys in one ACL or something like that? Or just one key, few keys. Can you clarify few, like 10 or? We can Google few or people few. Yeah, yeah, yeah, yeah, yeah. Oh yeah, okay, so four. Two keys rolling. Two keys rolling, okay, excellent. So we tried to use thousands of T6 keys, but it didn't scale. Okay. So I'm embarrassed to say the evil hacks that we did around that, but we wanted to do that, so. Okay, so it didn't scale because of configuration or the server had a problems processing thousands of keys? Basically every query it would search through all the keys. A linked list of T6 keys and doing the crypto on each one until it found the match, so. Oh, I see, okay, yeah, that's nightmare, obviously. Yeah. Okay. I also think the T6 keys came in nightmare because quite often you configure it per client IP address, sometimes per zone, and so it's one software is different than the other. Yeah, I want to get the feedback. What's most useful? Of course we are not limited to this slide, right? Actually, if you ask me, I would not say it's an API, but we heard before and that is where I was coming from the first name server was bind. And I think the name deep.com syntax is the one which is mostly comfortable, everybody was asked. When I first made my puppet module and converted the name deep.com to an not com, I said, oh my God, it's nightmare. And it was really like going from NSD to bind was quite easy going from to not was really complicated because it had more options also with the ACL stuff. So if you think you want to come in like a common configuration for all just like cope with the name deep.com file, every name server should support it, not necessarily with all the features of bind support, but just master is a slave is, and maybe a GC key that would solve probably 90%. Yeah, do we have anyone from ISC in the room? Ah, okay. Can you share some thoughts about the name deep.com and its future course? Well, looking at this, I'm thinking catalog zones that we have. And I think that's the future of sharing zones between different implementations. With catalog zones, we have for each zone that's well going to be slaved. You can define ACLs with TCIG. And well, you can basically define a very, well, it's really hard to find a subset that would work for everyone, but we have ACLs like IP based ACLs. We have TC keys. So basically if you have a slave that slaves of the master with catalog zones, you can configure all those options using catalog zone and it works. So, you know, now you're moving. Yeah, of course. The thing is that the catalog zones are just the zones, right? In the server, there's a lot of stuff you want to configure like you want to read statistics. And if you have multiple implementations, you want to get statistics from all of them, right? Not just bind. And it's like, yeah, that's the problem. Yeah, of course catalog zones can solve little subset of it. And we would like to solve like the problem in generic way. So you can actually use multiple implementations and not to mess with like, okay, so how do I set catalog zones in Power DNS? And how do I set catalog zones in Node DNS? And how do I set catalog zones in NSD? Because then, you know, okay, excellent. Microphone is passing. So yeah, this is not the first time the vendors have tried to come up with a consistent configuration that can be used for everyone. Not everyone. I'm asking about subsets. I'm asking operators for their wishes for subsets. At least open source vendors set together and it was very hard to come up to a conclusion and the effort was thrown away. And then a couple of years later, people ask again because there is a demand for it because of this story. And then we try again. It's very hard. We give up. So maybe it is a good way not to try to tackle everything. So catalog zones start with that and then do the next one. So take them off bit by bit. I would be my suggestion. Okay, bit by bit certainly. Yeah, we cannot boil the ocean. So I'm gonna open a completely different can of worms. Part of the problem is configuration but even more annoying part is monitoring. Yes, exactly. What would be really nice is if people could just publish like the standard that seems to work easily and nicely is Prometheus style metrics. Bind already does XML style. It would be easy for bind to publish Prometheus style. Unbound has sort of an interface having a conversion where that also did Prometheus style. And if not wanted to do something like it seems like that's an easy low hanging fruit statistic channel to add. Exactly. You know, we need to know what operators want to see because there is no point in exposing everything from inside of the server besides other things it has, it's associated cost. So. Compatible keywords between the vendors, right? Okay. Yeah, statistics might be easier than configuration because that's at least three tonally. Yeah. And monitoring is not only statistics but also simple stuff like what's the status of a zone? Like was it on the slave server? Was it refreshed last time? When was it refreshed? I want to know that it will expire soon before it will just expire. Mm-hmm, mm-hmm. Yeah, that's, yeah. Good point. It's also been supported by all vendors at the moment. Yeah, maybe for DNS like which keys are trusted and so on because in the light of the roll over it would be useful to have interface to look into it and not wait for the midnight or TTL expires. Yeah. Okay. Okay. Any other requirements? I would love to hear them. What you want to see from your server either on statistics site or configuration. Okay, so apparently we know everything. Yeah, excellent. Thank you for that attention. Thank you, Peter. Oh, okay.