 Tom here from Orange Systems and I am with John Todd the executive director over at quad 9 and we're gonna talk about what is quad 9? It's maybe not what you think it is. I've been learning a lot more myself about it And we're gonna dive into like the DNS filtering and some of the Goods and bads and all the details behind it. So welcome John. Hi. Thanks for having me. Oh, it's fun It was great. This all started because I started doing some DNS Phil, you know putting the DNS to the test, which is turns out to be not insurmountable It's certainly a difficult task And you responded as long as as well as a few other people so it's been kind of just a lot of discussion And you joined my forums and we're helping people out Which I thought was really cool and then that led to you coming on here So we can talk, you know throw a video together talking more about the whole DNS filtering. Yeah, DNS is of course from my perspective It's a really interesting topic. It's it's a very active place right now from Both a technical level meaning the changes that are occurring in DNS and also from a policy level Like what is it that we're doing with the DNS and how does it work and how is encryption and filtering and and in some places censorship? Really going to change In my opinion change this kind of a structure of the internet fundamentally So it's an interesting place to be right now Yeah, I mean if the names have to resolve or we don't know where anything is I mean Google points is somewhere But then it you know the whole process after who will points as to whatever we're going or duck to go or whatever your search engine and choices and of course the ISPs have been fighting this whole DNS encryption thing but before we dive into that let's talk a little bit about quad 9 So so quad 9 is a not-for-profit 501c3 we're based in Berkeley, California Although staff is everywhere, of course everything is virtual these days and these days even more virtual than most So we were founded in 2016 Kind of as a distilled the story as quickly as I can An organization another another not-for-profit called Global Cyber Alliance based out of New York Is their job is to make as much of a dent in the cyber security space as they can for as little money as possible As all non-profits have you know there's there's a funding limitation, so they law enforcement agencies in the DA of New York and City of London police their Cyber security, of course, isn't is an constant thorn in the side of these organizations Because of the amount of money that it costs their constituents, right? If you especially New York where the financial organizations are in the City of London again finance those organizations said the GCA You know we're we're losing a lot of money on Cyber security Here's a grant go do something about cyber security. We don't care what You know in fact multiple projects came out of this But one of the things the GCA then said was you know DNS filtering seems like a very inexpensive and easy way to take a chunk out of cyber security issues They weren't however really an operational organization at that time So they came to another not-for-profit called packet clearinghouse based in Berkeley PCH has been around for about 25 years They're the biggest DNS service provider. You've never heard of Because they're not for profit. They have no advertising budget, but PCH runs infrastructure in 160 or so locations. I actually don't know what the count is off the top of my head. I'm ashamed to say Mostly at IX locations in our exchanges around the world. They operate Name servers that serve a lot of the TLDs like more than a hundred country codes use PCH as their Basically their outsourced DNS hosting. Also, there's some very large DNS zones in PCH Two of the 13 root name servers do any cast through PCH So PCH is a very large organization a huge footprint and basically a lot of DNS experience I worked for PCH GCA came to PCH and said, you know, we want to do this DNS Filtering thing. Do you have any experience with it? And it just so happens that yes, PCH had done Some other DNS resolver architectures. So took the project on and then very quickly I was given the project as the project manager essentially and we determined very quickly that the Quad 9 at that point wasn't called quad 9. It was just the DNS resolver Needed a couple of things. The first thing it needed was a very memorable IP address to be used because we saw the success Frankly of Google's 8.8.8.8.8, you know, people can remember that and they can put it in so far The same number is really easy. It is so we firstly we needed a memorable number second is that we needed We needed this to be an independent organization that was as far away from Control by other organizations as we could get and this was because we correctly perceived that Privacy was going to be one of the biggest things that we had to offer So we founded the organization. We created a new not-for-profit that was not part of Global cyber alliance. It was not part of PCH. It's got its own board And its own employees and its own money And so then we were left looking for that IP address It just so happened that timing was really good I ended up reaching out to some folks and actually other people at PCH I ended up reaching out to some folks at IBM and saying, you know, you hey, you've got nine slash eight You've got one two hundred and fifty-sixth of the IPv4 space in the world and you're not really announcing any of it Would you be interested in participating in this project and donating nine dot nine dot nine dot zero slash twenty four And it just so turned out that IBM was considering like how do we get into DNS resolution because we have this interesting IP address But they recognize that that again trust and privacy were key Components of this and that they probably wouldn't be able to offer a service under their own brand because you know It's it's it would be seen as a commercial service. Yeah, and the shareholders would be interested in monetizing it Right exactly and also that there are risks involved in it I mean, you know IBM doesn't necessarily want to have an open resolver attached to them so so there is a there was a push-pull there within their organization, but we found some very Surprisingly, cluful people inside of IBM who really pushed hard for us and after a long process of lots of lawyers They donated meaning that they've they've since given us the address space of nine nine nine zero slash twenty four Which is different than then Swipping it to us meaning that they've actually it's actually ours now. We own okay and so They also then entered into as one of our three foundational partners So packet clearing house global cyber alliance and IBM all put in either funding Real assets such as the address space Or in packet clearing houses case They've put in a huge amount of resources in allowing our servers to be distributed worldwide So that's what quad nine is we were founded in 2016 and since we've we've gone on to be become Quite large from a footprint perspective But we are still fairly small from an organizational perspective. I mean, there's really only there's there's a handful of people working on this But because of the use of open source where we're entirely open source based And also because of PCH is great footprint. We punch well above our weight and So we are on a par with any of the other large global resolvers despite our size and our Our much diminished funding is compared to the billion dollar companies that are running these other systems and so The key primary the key focus with with quad nine is twofold one is privacy We don't have a shareholder To answer to so we're not monetizing users DNS queries We are essentially never recording anything about the IP address of end users. It's immediately discarded We never write it to disk. We never Transmit it out of the location in which it's received So privacy is key and then of course the security component of it Which was the core foundational reason for it Which is that we get a block list and apply that to the DNS queries if you use our 999 service and associated secondaries So if you try to go to a malicious site phishing botnet command control spyware There's a whole host of things that we block if you're if one of our threat intelligence providers has given us that domain We will prevent you from going to it, which is very very appealing for home users Small business and organizations who would typically have a difficult time or could not afford a to implement a Filtering system that's a much deeper platform like doing a firewall with the packet inspection or URL filtering and things like that So we're very effective very cheap free Well, that was the conclusion from my DNS filtering that I really came to was That of all the services and some of them being paid and I was set up some trials of them quad nine was consistently Performing quite a bit better in terms of you know sink holding more and more of the bad sites And that was the narrowest narrower scope of my test. I thought that was really impressive and it kind of made me go How good is this? So impressive that's really due to our threat intelligence providers Quad nine itself. We are you know if you want to use the analogy. We're simply a Looking at the power grid, right? We're just the wires that take the power from the generator to the end user So we don't actually generate this block list We don't we're not in the filtering or analytics business So we have around 19 different threat intelligence providers Of course including IBM, but many others domain tools and the monarch threats dot threat connected. It's a very long list Including some open-source ones, but many of them are closed They provide us these lists of domains and we block on them and because it's this this Widely diverging set of companies and sources that are giving us the list we cover a lot more than most other most other DNS based Block list providers do and so the next question people have for me is typically well Why do these threat providers give you this data that's really valuable data for free? So what we give back to the threat intelligence providers is we give them some volumetric data on their blocks Which it's very difficult for them to get otherwise. So if they give us a domain You know, you know bad domain calm We distributed across across all of our systems and then anytime someone tries to go and connect to one of those hosts in bad Domain calm we don't give back any information about the end user But we do actually send them a note that says hey someone tried to get to this domain And they only get information about domains they give us so they don't get anything about you know random domains They only get us information about the domains they give us and there's a very long set of criteria of like what you're allowed to put in the block list And it can only be malicious. It's only going to be for Blocking purposes. It can't be for marketing or tracking or anything else. So there's a there's a document everyone signs to that extent But that volumetric data is super useful to them because it allows them to figure out which Domains are actually active which ones are ramping up which ones are ramping down And so they find that is useful to their paying customers now how they You know how they filter that and massage it and give it to their paying customers is up to them We're happy if they're making money. We're getting the threat data And then in turn we get an improved set of threat data from them because of this feedback loop So it works out for everybody. There's no data privacy that's violated and We hopefully make the internet a safer place not just for our users But for the users of any of our threat intelligence partners downstream who are using more advanced Components of those solutions and that's fine as far as we're concerned. That's really how it works I really like it. That's a good method and I like that You know when you started out you talk about being a nonprofit and I think we're gonna probably start seeing if you if a Company or a group of individuals really wants to say I want to make a privacy oriented product And I want to give it away in some way immediately your first thought is you're gonna monetize my existence So by establishing a nonprofit, I think it's an important step They kind of allows everything else because you have strict policies not to try to monetize it in that way I've always liked the boxy marlin spikes work on the signal messenger for that You know, he set it up and structured in a nonprofit and they go look we don't have data We rely on donations So I'm not trying to sell your existence or some knowledge of your existence that you know can be quantified so it's a Really clever. It's just a good way and good structure and gives you a lot more confidence in a product Yeah, so we get the bulk of our funding comes still from People like IBM corporate sponsors who are who are find that that threat enrichment model is extremely useful to them And of course, we do get donations from end users. Thanks for plugging us on your site We actually I had some specific, you know Hey, this was because of the the threat analytics that that was done by by you But we're really actually we're we're we're tragically underfunded as most nonprofits are And so what we're really shooting to do is we're trying to find We're trying to challenge the original organizations who we benefit the most, you know financial banks, right? electronic payment companies Email providers all of these people benefit from the blocks that we provide and have real real money associated with it I mean in the millions of dollars per day That we're preventing but it's very difficult to get in front of the decision makers and actually prove out like how's quad 9 benefiting your organization right now and we do But it's it's a difficult process getting, you know, that's a very broad set of people to have to go after so That's one of our biggest challenges is finding out. How do we increase increase our funding pipelines so that we're not We're not always relying on on the trickle of funds that we're And as you go outside the tech space that only like you said the banking and financial institutions It only comes more difficult because their competency isn't technology their competency is running a bank running a financial institution So they don't quad 9. What does that mean? Like that doesn't and be that's before you even got to that What's the DNS and why do I care? Exactly, but but they recognize that they're losing, you know Thousands or hundreds of thousands of dollars due to cyber security issues not just for themselves Not, you know, just not just affecting their businesses where their business might have a breach But their users are being defrauded and that's that's you know looping back in a credit card fraud is a massive one that we prevent So I appeal to your to anybody watching this if you are with one of these organizations put me in touch with your C. So I'm Yeah, how can we actually work together and so the term viral is often overused we kind of do have a viral effect Meaning that if your organization's sponsors quad 9 We are happy to take blocks that are specific to your organization So if your bank is an example and we do this actually right now for several financial institutions If they give us a block of all the domains that they know are trying to defraud them that are phishing domains That are look-alikes right? We will protect any users on our surface from those domains so that it becomes in their Their to their benefit to promote quad 9 to their user base. So like hey if you want to if you want to not be defrauded for your credit cards Use quad 9 we give them data specifically to prevent fraud on the credit card that we know you're a user of so There's kind of a again. There's a roll-up that occurs there and so this is what it's contributed to Was one of the things it's contributed to quad 9's growth You know we up until the COVID restrictions. We were growing at about 3% per week Which was a pretty astonishing and scary number Interestingly in the last three months or so that number that that has stabilized were actually kind of flat although in the last No last week we've actually started growing again probably because people are going back to work but the People when they're making Businesses schools small governments agencies, you know that they Tend to use quad 9 because of the free cybersecurity benefits So we find there's much more traction there where one person can make a decision to change the DNS resolver in the DHCP server You know at the secondary school right so one person's decision can influence a thousand users Whereas if we're talking individual users at home, which is where most people have been for the last three months We have to talk to each and every person we have to convince each and every person to make that conversion So that the rate of growth is slower if we look at end user growth Our rate of growth is huge if you look at you know small small enterprise small government kind of deployments That's where that's where the decision-making happens. They're like well. Why wouldn't we use quad 9? It's free It's private the the DNS queries are going to a server. That's only you know 50 kilometers away Why wouldn't we do that? So that's where we get the most growth Anyway, sorry diverging off of our topic, which was encryption I think laying it down of why you know because It kind of goes hand-in-hand if we don't talk about why you should use quad 9 or who quad 9 is Why should I bother encrypting if I'm handing to someone? I don't trust I think that's a really important piece of the groundwork that has to be laid You can't just throw it at anyone like hey, I encrypt it and I hit it from my ISP that I don't trust But oh, yeah, by the way, I gave it to the other people and they're like they're in cahoots with them and so it doesn't matter Yeah, yep. Yeah, so on to the next topic In where we're kind of heading with this is the DNS encryption. So yeah, I encrypt my DNS Fascinating fascinating work being done in there right now. So DNS is kind of the last frontier as I call it of unencrypted data Almost everything else on the internet has gone encrypted over the last five years or seven years HTTPS, of course, you know, let's encrypt has really done a sea change their kudos to them But you know emails primarily encrypted now, of course, most web traffic is encrypted pretty much everything you look at is got a TLS layer I'm on it's of some sort So DNS was the last frontier for not being encrypted And so obviously that needs to change because metadata is almost as useful as seeing the data itself you can you can infer an enormous amount about An individual or even about clusters of individuals if you see all of their DNS data Especially seeing DNS data in sequence where you see, you know A certain query to a certain domain followed by another domain followed by another domain You can infer that it's either a specific web page being being viewed Because you know, there are signatures if you look at the DNS data for web pages You'll first, you know, the top banner ad is served by this company The first image is served by this part CDN the second image is served by this You know, in other words, you can you can fingerprint Websites just by the DNS queries they do But you can also just fingerprint what people are doing by looking at the general DNS So there's another side that's interesting because it's we always have self-reporting stats from, you know Different websites, but through the DNS data if you have enough of it You would actually have a much better idea exactly how popular that website really is not self-reported But you'd be able to see these websites are this popular And if you combine that with knowing where they're coming from now, you can see what trend right all these people in Detroit, Michigan Where I ma'am seemed to really like this website over here and they go there during this time of the day So you get like you said through the metadata an enormous amount of insight into behaviors very interesting stuff and and Yeah, very interesting stuff So metadata is just as important DNS has been the last unencrypted model So because it's unencrypted there are several parties who are interested in that The first is and possibly the least The least scary although depending on your perspective is our advertisers and so advertisers are interested in what you're doing so they can model Advertisements to you or at least so they can profile you from a demographic perspective That might be the most scary depending on depending on who you ask You seem to know exactly what I want. It's a little it's offsetting. It's off-putting for sure So You know, if you're giving your DNS to an advertising company You might consider that that ultimately is counter productive to your privacy Depends on what the privacy, you know, everyone has different privacy Statements and goals and and wording on their on their formulas, but You It's really kind of a question of principle, you know, who are who is it that you're going to give your DNS to? So and it's not just giving your DNS there have been there definitely been cases where advertisers not simply are the recipient of DNS queries as a Resolver, but where actually isps are tapping DNS and selling that to advertisers as Whether directly or indirectly meaning that they might be selling it to a DNS A passive DNS company which then repackages that and sells it to an advertiser. There's there's all kinds of indirection that can occur in that stack so so You're you're defending really against three different With encryption you're defending against three different participants in the in the stack one is The local of course, you know, the local intercept which were actually local intercept and rewrite So you're sitting in a cafe using wi-fi. You're trying to protect yourself against somebody from seeing that traffic very close to you On an uncontrolled network Or even maybe a malicious device on your trusted network seeing the DNS Replying back before the DNS the remote DNS server is and trying to redirect your traffic to go to somewhere else Or just intercepting and looking at it. So both interception and rewrite are very important topics on the local component meaning in your local network Or even in some cases even on the local client So there's there's malware which will try to do those tricks on your local service Or your local your local machine So the second layer is your isp or your your your trusted transit provider whoever that may be um You know some that chances are low That they will do rewrite Um on just the raw packets But they might you know, there are certain countries where that happens Um, that's for sure like uh, if you're in china Um, you know, you know, there are certain laws restricting what you can do with DNS on the local isp And they'll be rewrites occurring Um Or blocks occurring locally Um, so that's important um, and then often Often that that second layer your transit provider meaning your isp and your dns provider will be the same organization So who you actually send your dns queries to like what what is the ip address to which your dns queries are are being sent Is the third place where you've got to kind of look out for and defend Your privacy and your security. Um, but as I said Looking at the statistics in general worldwide The vast majority of people do not use an over-the-top dns provider Um, you know like like quad niner or google or cisco. They use whatever their internet provider Their transport layer get handed over to them. Uh, one thing I will notice I remember it's been a few years since I've seen this but some of the cable providers had made the decision to block Torrenting sites and things like that and they just sink hold the search engines and things like that So there's sometimes some levels of filtering they may do Because it's you know, they don't want that traffic on their network They don't want you uh, potentially pirated movies Maybe they have their own interests. So there's another layer that does occur is sometimes they do filter at least some pieces of it Yep, you could argue that that's Legitimate because they have terms of service. You've signed the terms of service They, you know, you have a contractual relationship with them and so you have some recourse um, I think that both contractual and legal issues are a difficult one to address because you've somehow you've you've You're in the jurisdiction of those contracts or legal legal arrangements, right? So you kind of have to comply and if you don't like it, you can complain or vote or whatever so There's also things like There were certain providers certain ISPs for quite some time. In fact, I some of them may still do it I want to say that level 3's open resolver did this that if you went to an nx domain Meaning a non-existent domain. They would redirect you to a search engine or you know, you know, advertisements Which is, you know, most people would describe that as unexpected And so that's another kind of problem that people have to surmount. Maybe Maybe there's some person somewhere that thinks that's a great thing. But I know I don't and I think I'd rather have it not resolve and fail versus uh, lambia and some especially because those ad sites into You know the less tech savvy people look like something that might be legit Look like something they might be to click on and they're not necessarily always dangerous, but usually full of junk So they're not necessarily good So I don't think enough people learn from, you know, the great the great network solutions disaster for many many years ago When network solutions decided to, you know, use dot com and start referencing everything that wasn't a domain to their advertising page Like that's still that's still is a thing some places where they'll direct you to advertising But anyway, we're kind of getting off the off the track that the the encryption needs to occur to kind of prevent these to prevent unexpected observation or Redirection of your queries or non contractual or non legal redirection and observation of your queries and and There are three methods recently while actually there's two methods recently and one method that's been around for a while The first real dns encryption method was called dns crypt and dns crypt is actually a pretty awesome protocol That has not received a lot of love. Uh, it never did get ratified or I don't even I'm not even sure if it was submitted as an rfc draft I don't remember. I should know this. I thought I had but I don't but they it never went anywhere Despite the fact that it was pretty nice And and it really hasn't seen much development in quite some time, which means that it's it's getting kind of crafty There are a lot of new extensions to the dns which are quite useful and quite Functional which have not been extended into dns crypt, which is a shame because It it runs over udp. It's very fast. It's fairly lightweight But dns crypt is just not it doesn't have a lot of legs right now We support dns crypt. By the way, it's it's one of the three different protocols. We saw great And there are some Linux based toolkits out there that support it and you can run them on your mac and windows and others So take a look at it The second to come out with With ratification was dns over tls dot And this operates on a different port than dns dns typically operates on port 53 And dot operates on port 853 And so that makes dot Identifiable dot is all tcp. It's not it's not udp We like dot because For actually the for the fact that it's identifiable. It's possible to identify you can't see what's in the traffic But you can see that it is dot traffic And i'll get into why that's a good thing here in a much longer part of this conversation And then the last one to come around is actually doh dns over https And that kind of took the world by storm really because It could be implemented inside of applications, which is a Terrifying concept in itself But it made it so that it was easy to deploy Inside of browsers and also inside of pretty much anything that already has an http stack So applications now can do dns queries instead of relying on the The what's called the stub resolver that's located inside your operating system Usually the application would hand the the request of the stub resolver and then the stub resolver would hand it to the dns Resolver wherever that happened to be So now you have applications making the decision on where they're going to send dns requests potentially different destinations from where the operating system is sending them Which is a problematic troubleshooting issue So those are the three in very rough terms the three different protocols that exist dns over https dns over tls and dns crypt They all have slightly different encryption methodologies, but they are They all provide a relatively similar protection against interception and Rewrite between the client and the dns resolver wherever that happens to be located in the network The you know i'll i'll take a side trip here and say that it's a moot point To to talk about any of these encryption methods at all Depending on where your dns resolver sits We actually encourage people to run their own recursive resolver or actually We encourage people to run a forwarding resolver that sends queries to quad nine But we're also not opposed to people running their own recursive resolvers either we Again because we're not trying to capture all the dns in the world if you want to run your own resolver That's great by us, you know more more Independent resolvers out there the more stable things are anyway Right now encryption covers that last leg between the client and the Resolver the whether it's a forwarding resolver or a true recursive resolver Which is which goes all the way down and does all the dns queries to the root You're really only protecting that last leg so You have to look at where are the risks in your network? And are the risks between the client and the resolver are the risks between the forwarding resolver and the next forwarding resolver? Are the risks between your recursive resolver and the authoritative resolvers? So If you're running your own recursives you really it's good to have Encryption is always good. So don't get me wrong But where's your risk and then figure out where you need to put encryption in your network to defend against that risk um So do t Doh and dns crypt are all between stubs and recursive forwarders or or servers and That doesn't cover the dns between the recursive Resolver and the authoritative's those are unencrypted still so that is a You know, that's the challenge that that right now the dns community is facing is that all right? We've got the client encryption done sort of. I mean at least we have three different protocols So it's one of one or all of those will continue to be used So now we've got to figure out how we encrypt the queries between the recursive resolver and the authoritative's And that's being actively worked on. Um, I expect some answers in that area of coming into the itf And being seriously considered but the next six to eight months maybe sooner. There are already some prototypes out there for doing it Facebook is running a dot server on their authoritative's but it's kind of hard-coded. There's no discovery mechanism It's this, you know, if you know that you're going to this domain, then you should encrypt using dot So it's uh, no, it's a challenge getting them all to agree I think that's the part people don't realize is large companies and they'll have their own opinion They'll say well, we're big we're facebook We'll do it this way and everyone else should do it this way or google may have the same answer But it may not be the same as facebook's So getting that type of information It's so in he's been on channel before is uh, my friend is one of the engineers that lets encrypt and they've talked about the Challenges just in getting people to agree on certificate transparency And how how it doesn't sound like it'd be hard But then the politics of companies trying to get them all to agree if this is the way we do This is the way you do a transparency server so you can see who's getting issued things Sounds easy. Yeah. Yeah, I know that's the the good news is that the in the dns community It's it's actually rather small the dns community is not big and everybody recognizes the problem and There are certain things which I think will get quick adoption. There are only maybe four or five dns software packages out there with with any significant traction and uh, you know, it's it's ISC bind and powered dns Not the not resolver unbound Go dns is starting to pick up some traction, which is good. There are a couple of commercial projects, you know, microsoft has a resolver But it's it's a relatively small community and Everybody knows everybody and so when things like dot when everyone finally agrees on this is the standard I think we'll see rapid adoption between recursive and authoritative again the looking at the dns is an interesting Graph to look at uh, jeff huston at at ap nick has done some interesting research on Like where do what what are the largest world's largest dns recursive resolvers and where do they live? and You know the vast majority of the world's query traffic comes from a very small number of organizations and You can the obvious ones are the big cable companies The big indian and chinese isps You know where a single organization has, you know 300 million people behind it So if you deploy if you if you run the dns shop, you know in for a big isp in india With one decision you're going to upgrade all your users, you know hundreds of millions of users overnight Same thing in the us, you know the big cable companies control a huge portion of the of the dns work. So again We're entering into an interesting phase where decisions by a very small number of people are able to influence Privacy technology decisions at a on a massive scale right browser operators Large isps large centralized dns Resolvers can make decisions which change the way things work on kind of a global basis And again where what the motivations are for people doing those changes I think is really important to keep an eye on that Most of the time those those decisions are made for commercial purposes Right now engineers are still sliding things under the radar, you know even in the big in the big You know faceless organizations where where money is is running the show There are still very good engineering decisions being made To that are in favor of the end user But over time You got to watch out, you know Money tends to corrupt those decisions In very subtle ways. So That's why I'm happy working for a not-for-profit. We we get to be at least a little bit more Honest about what we see happening in the rest of the community Then maybe some others have to be because of their their job security No, I think that's a really important point My friend who works for links foundation is meet comments before because he's had job offers that have Really offered him really high dollars and because of where he used to work He says no I got out of that world of doing things that really violated what I thought was not good engineering decisions In the face of trying to monetize existence. I mean, so Yeah, that's definitely uh, it's important. Um, and it's a slow erosion. It's you don't like you see You don't see it. It's that small chipping away at things that can lead to that. So And ultimately they're always going. How do we get more performance the shareholders always asking? How do we get more performance out of this company? How do we squeeze a few more dollars out there? Like we have all this data over here. We could probably repackage and sell it, you know, okay Yep So anyway, you know, I think there's there's less There's less insidious behavior occurring today than some people might think there is Because honestly, I just don't think that there's been a lot of there hasn't been the focus on dns as a marketing tool Except by a few organizations who are very sharp at that already The the risk is what happens in the future Where is the dns going? What is it being used for will change over time? And so setting things up now with Appropriate safeguards on the data itself meaning encryption as an example But then also structuring it with Where you send your data as it's somebody you trust I think that's just just important But as I said encryption is a big portion of that because now it becomes instead of there being like unknown third parties in the stream Now you're starting to move towards a you know You have a relationship or you have a conscious relationship with the organization to whom you're giving your dns data And then that that narrows the scope of how you you enforce policy And I I think there's an incentive for your large Companies and we'll say like you know the facebook google or even the chinese Based isps and things like that They have a best interest in keeping it encrypted because it helps their security because most of the knowledge that they gain about their users Especially facebook and google is the direct You know knowledge you hand to them the inputs you have on facebook the things you share on social media And they have an interest in keeping that information for themselves because that's their gold That's their oil so and honestly that you're you already are giving you know Company like facebook is an example when you're doing a query on a facebook domain You're already asking facebook for that information Right now you're not there's nothing you can protect against you know face facebook in this particular example Right you already if you're doing a query on a facebook domain You're already giving them that data So you're really concerned about is like who are the other parties in the process that you're giving information about your facebook queries and It's not it's not individual domains. You should be more concerned about it's the it's the it's the entirety of the dns query set And really the only people who have visibility into that right now are the recursive resolver operators Whether it's your isp or whether it's a it's someone like quad nine in the middle That's who you have to build the trust relationship with today. There are some interesting drafts Occurring right now in the itf again that are trying to Split that out where things like oblivious dns or that that Essentially break up your query into parts that neither that actually you'd have to have two recursive resolver operators involved Neither of them understands who the user is and the other one doesn't understand what the query is it's pretty complicated and fraught with fragility And actually somewhat slow But it depends on how how interested people are in this and really the problem everyone's trying to solve is That trust problem like do you trust who you give your dns flow to and and trying to move to a zero trust model? Introduces its own sets of problems It's like the tor model where it handing it off and once you get you know three nodes in removed They don't know where it came from. They just know where to hand it to next But that separation the complexities of it obviously like you said it creates a slowdown It's a fragile system. So it's going to take a lot more deep thinking to get this right dns has worked astonishingly well for a very long time And it's it's becoming more it's becoming more complicated as a protocol because of the issues of trust And and now the the layers of things like encryption and The edns extensions that are being added to dns, which is which are honestly quite useful and well overdue um, but the complexity is increasing and To operate a dns server a recursive resolver Is becoming more of a challenge and that's one of the reasons I think a lot of people actually end up using something like quad 9 instead of running their own because You know just keeping up with the security releases is like that's a it's a big challenge these days and the new features and Various kinds of network Attacks that are occurring against the dns servers themselves, but also against authoritative You know Most organizations don't have time to think about dns, right? It's just supposed to work and The easiest way to make it work like many Services these days is to outsource it to someone who only does that right? quad 9 only does dns We know we that's like 24 hours a day is all we do is dns. We don't do web hosting We don't do email. We don't do any of these other things so And and that's becoming a more reasonable decision for a company to make is to just say all right We we can't deal with the volume of of issues with us. Just get rid of it We still don't really make sense. Yeah, we still encourage people to have a forwarding Cash at the edge of the organization because that improves speed dramatically Even if it doesn't do anything other than just answer your queries for your local customers and then send all of the queries to the An external upstream Recursive resolver It's it's a huge win and we and it also think does things like it's a it's a mixed master for privacy You never see the IP address the clients It translates between rfc 1918 addresses and a public IP address and there's a whole bunch of reasons why affording Cash is a really good idea. And so we encourage that but offloading the heavy lift of the recursive resolution to an outsource provider is Frankly, it's it's the easiest thing to do and probably makes the most sense unless you have a really good tech staff Or or a well written Software package that auto updates and things like that. You know, I know we've talked about PF sense. I'm a I'm a big pf sense user. I think it's great That uses unbound As it's core great package But that but then you have to pay attention to your firewall or whatever it is that's doing those those dns Outbound requests as well and in the most recent one. What was that attack called that was a falloff on unbound It was recently patched in pf sense It's another white paper. It's not a it's not a real world one as much It is a theoretical one that's one found where you could have some improperly Improper dns records to create a reflection attack Yeah, well, it was an ns overloading ns records basically creating bogus ns records Um as a very as a variant of attack that's actually been around for quite some time and we noticed that a while back that it was resurging Um, it actually isn't a theoretical attack. I mean there there are people using it not at the scale that they could have um, and um, but pretty much all the dns vendors patch for that in on a I guess a couple weeks ago. I was at a month ago now. Um, but um, but yeah, pretty much most dns vendors had some of Vulnerability to that that's the kind of thing that it would be really hard for you as a as an end user to keep up with I mean because that's But but frankly in the case of that attack the risks were relatively low because It had to be someone able to recursively query your server most enterprises The only people that can recurse to your server are your are your employees So you would have had to have somebody inside You know they're calling from in the house right you had to find the firewall Doing this kind of attack and you would have been able to track them down But so that that's a good example, but these kind of things happen all the time and there's all these and there's lots of security um features that get added that are not As a result of papers right there's a lot of things that people kind of go, huh? You know, I wonder if you did this and then everyone sits around and talks about it on an IRC channel or on You know sitting around at one of these DNS conferences and goes You know we probably should solve that and create a filter or a rule that fixes that before somebody else figures it out so those kind of extensions are occurring all the time and um It's difficult for for an organization to keep up with that unless you've got somebody who's really dedicated to dns and and Managing that at the edge So yeah, no, it's it's definitely push that update button and trust that really smart people wrote those I wrote those patches for that So that's definitely a very interesting. No Yeah, yep. Um, so getting back to encryption um So as an example again pf sense unbound has the ability now to send queries using do t From unbound out and so actually I I've implemented that in mine quite a while ago Um, very useful. It's not It still needs some patches. It's not a really awesome implementation Because it doesn't send it doesn't pipeline queries particularly effectively So if you have a lot of queries, it's setting up and tearing down a tcp session for each one of those And uh, but but there is a lot of activity occurring now with things like there's a bunch of shim software out there like The dns crypt package is an example. Which does dns shimming for doh um And I expect to see do t or doh included in pretty much every major resolver Very soon now it's included in a few of them and it's it's it's in discussion with everybody So getting that encryption from the client to the to the forwarding proxy or the forwarding cache And the forwarding cache out to another forwarding cache Or another forwarder that's that's happening and then from the recursive resolver to the authoritative That that's closing down very quickly and I I really hope to see most dns traffic encrypted within well I expect to see most dns traffic encrypted from big Resolvers in the next two years. It's going to be a very very long tail like perhaps an infinite tail With getting things like iot devices to encrypt their queries Which is which is because all the libraries out there, you know, they're all 10 years old and they're very stable But getting something like encryption and baked into your refrigerators dns stack is It's a it's not easy. It's it's a challenge. Yeah, that's that's a real challenge And on a topic of like dns over hctps How does that look in on your side in terms of server load? Is that is that a more difficult challenge? I I seem to fire fox roll out and they kind of stalled it back according to the mozilla blogs Uh, I wasn't really clear on why but it sound like they overloaded some of the servers that you were pointing it at I don't I don't know about that Uh, I can't speak to that cloud flare was the one that's that the primary recipient for that traffic I don't expect that cloud flare was overloaded with HTTPS. I mean, that's they're a cdn so it sounded odd to me I was actually just a little bit of reference I was listening to it on uh, Steve Gibson on security now had a whole section of it Because he's trying he was asking the same question He couldn't find a lot of inclusiveness other than some uh, bugzilla reports of we're slowing down the rollout because it seems to not be resolving as fast Which was like well, so there's there's a difference between overloading servers and there's a question of Does adding a layer of of HTTPS onto your dns query slow things down Right and the answer to that still is a little unclear on the second part I I don't think that I don't think that HTTPS Is a significant problem from a protocol perspective a meaning slowing servers down The load introduced is really only on session startup And then once you've got the session started you you keep it open for a while and you do a bunch of queries back and forth on the same socket Um, so I don't think adding HTTPS is an overwhelming issue. We haven't seen it be significant, you know, it's it's Double digit percentages, but you know, is it 10 percent or 15 percent? It's not it's not massive And actually there was someone who did a study on that and I'm sorry. I don't have the paper I don't know my list of papers in front of me, but someone did actually a performance analysis um, and there have also been some Comments made by I want to say Deutsche telecom in there. They they've been doing some presentations. They're rolling out dns server HTTPS in their network as well It's not massive. However on the on the HTTPS side of it meaning the protocol itself Adding additional back and forth Exchanges. Yes, that is the case if you're not sending a lot of queries. You are adding additional packets You know udp dns is incredibly simple one query goes out one response comes back. That's it Yeah, now if you're adding, you know a synac, you know a handshake on top of that And if you're doing very few queries meaning that your sessions are being closed and then reopen for every dns query Yeah, it's going to be slower There are ways around that Or there are ways I shouldn't say there ways around that there are ways to mitigate that delay Such as keeping your sessions open for longer And so you don't have that handshake There are ways of pipelining queries so you get multiple queries in a single request so There was again. There was some papers written on looking at the effectiveness of HTTPS. Is it actually better Because retransmits are are handled in a different way In other words udp dns when it skips There's different delays that occur with how often you send a lost packet for being replied And so under different packet loss circumstances under different, you know latency circumstances HTTPS might actually be better I would say that's going to be in fewer, but that's that's my gut feeling Does it add an appreciable delay? I I don't think it does right now And I think as again as these stacks get better both on the recursive The recipient side and on the transmission side We'll see that improve further still, you know dns server quick is coming Which reduces the transaction load even further so I don't I don't think it would be an appreciable delta. There there have been some papers which have said it's huge I haven't seen that in practice. I mean I've I've done testing with it and didn't he granted nothing actual, you know Benchmark wise, but just usage turn it on use it see if I have any issues I couldn't notice between going to the browsers that were using it versus the browsers that weren't I didn't notice anything about it But I I mentioned from an engineering standpoint you look at on paper you see there are more packets going back and forth, but as you said Proper, you know speed servers that are close to you It happens so fast. It kind of doesn't it's not where the delay probably is opening up a web page Yeah, it's it's it is it is a minor portion But of course, you know you talk to the to people who do web page performance design and you know 10 milliseconds is a big deal and and rightfully so but remember also a lot of these dns requests are done All at the same time. So it's 10 milliseconds, but then that's divided out across however many dns requests you have to do in the page So it turns into fractions of a millisecond slower Yeah, I don't I guess my comment is there are quite a few papers some with contradictory information Practically speaking from my perceptions. I don't notice any difference Um Your mileage may vary Um, so we're waiting on we're waiting on a lot more research to come out as far as how it implements or how it affects rather the The implementations of dough in each individual use case. So as an example your recursive resolver might have a much lower perceived latency on a dough stack than your browser might so I have found from an administrative standpoint doing it administration It creates another challenging layer of well, your computer can't resolve it. This application doesn't work But your browser works great. Oh, you have a local dns problem, but the doh that you turned on in your rouser With firefox. Yeah, so it's one more layer of troubleshooting So there's there's been a lot of fun thrown around about why isps are worried about dough and People think of it as in the following order They think oh the isp is sniffing my dns and they're marketing it And that's the primary reason that they're opposed to me using An over-the-top provider like quad nine That probably isn't the primary reason that they're scared That is that is on the list for some of them. Some of them might harvest your dns data A lot of them most of them probably don't But again that question of well, are they interested? Now that's been raised to their attention level. Would they be interested in the future? So But I think most of them are more terrified of just what you described the support questions You know a user calls in and says I can't get to you know food.com Okay, well, you know pull it up in your browser and it says failed But when you type ping on a in a command window, it works fine Why is that? Well, your browser uses a different resolver and what happens when each browser uses a different resolver or your And your iot devices use a different resolver than what your dhcp server is handing out and You know from a support perspective. They had to get that gets kind of worrying Which is Getting back to one of my comments so much earlier in this in the conversation is this is why we prefer d.o.t Although that preference is becoming less obvious over time because d.o.t is really The first thing to roll out d.o.t was android Um, interestingly enough again, google's big organization. They have competing engineering issues sometimes but android pi rolled out with auto upgrade for d.o.t Meaning that if you are running on android pi and your dhcp server handed back to you an ip address That would respond on port 853 Then android would automatically upgrade and say oh great encryption is available Let's use it and it would but it doesn't change the ip address to which you're connecting meaning it doesn't change the policy Arrangement with whoever your provider is So that's great. We think that's a fantastic way of doing it. In fact, that's the way that chrome Is rolling out with d.o.h It's slightly different in that the chrome model says if we see that your system resolver is using an ip address That is in this list that we have of approved d.o.h servers then we will automatically convert to d.o.h Meaning that they're not changing Where your browser goes As far as dns requests, they're simply changing the protocol So they're looking at the operating system saying what what are you using? Oh, you're using we we recognize those guys. We'll change over to d.o.h But the same destination Nice keeps at least a consistent policy. We think that's the great. We think that's the model to go with Um, I didn't realize it was doing that and I think that's important That completely makes sense if you're already sending it to this particular server They just changed the connection, but it definitely, uh, I like that It's it seems to make the most sense and it's the least surprising Um for that's a good point The principle of least surprise And so, uh, that's the way chrome was doing it and that's the way android did it and we think that's the way to go Now windows has announced That they will be in the future Making all future versions of windows support d.o.h at the system level We think that's a good idea We actually really like that because that means that we're going to try to shift away again from each application doing its own d.o.h Communication and move back to asking the operating system You know, are you doing something encrypted and you know, here's my dns query you go figure it out for me um, that's Still, I believe the right way to do things because it centralizes your dns queries in a place that has well understood parameters and hopefully then is open to integration with vendors like antivirus vendors and other people who who have a desire a willing desire to put in filters and policy On the system level. So again, we think that that when microsoft's way of doing it is also the correct way where They're using d.o.h at the moment they have And they're I'm not even sure what they call it. I don't remember the name of it But they're early early beta prototype versions of this support. They have a fixed list Just like chrome does where they have a list of approved d.o.h vendors And if they see that the operating system if they see the dhcp servers handing back An ip address which maps to one of these approved d.o.h servers Then they will automatically upgrade the encrypt to an encrypted d.o.h session Both chrome and microsoft have said that that's an interim solution that they hope to be able to do automatic upgrade based on policy dns based policy instead of having this hard-coded list With quad 9 and cloud flare and google in it They hope to eventually migrate to a model that simply says well if the if there is a trusted d.o.h service offered by that ip address then we'll switch over so That's the way it's going to progress over time. We're still kind of in the introductory Stumbling steps of how to do automatic discovery and upgrade of dns d.o.h is a little bit more challenging than d.o.t And d.o.t because d.o.t uses a specific port number It's very easy to figure out if you should upgrade because you should just say all right This is the ip address dhcp gave me. Let's try connecting a port 853 into a query. Oh, it succeeds Upgrade the connection d.o.h is much tougher because d.o.h Of course operates on port 443 so All kinds of things can be served on port 443. What's the url? Is there really a d.o.h server running there? Also d.o.h operates on names It doesn't it doesn't use ip addresses. You typically give a name to a d.o.h server So you're going to say my d.o.h server is dns.quad9.net Well, okay. Now you have this bootstrap problem I have to look up dns.quad9.net in order to connect to dns.quad9.net. So how does that work? so How the certs work how you do how do you trust? answers In a pre-trust environment. That's that's where everyone is right now and trying to figure that out Yeah, that bootstrapping is a little bit of a problem on there because I think someone had said one of the Methodologies that some of the filtering companies like that make enterprise it filtering to stop people from doing that Is breaking the bootstrap process so they can sink hold it before it starts therefore people still have to go through whatever Filtering policies exist on that business or corporate network. Yeah. Well d.o.h just kind of blows that out of the water um Because a doh server can be any system that supports d.o.h right or that supports tls. So yeah, you can't tell Excuse me. You can't tell that a server is a d.o.h server Unless you know something about it and so it is possible to disguise Your d.o.h queries in the same stream of traffic as your normal web queries. So um, that's I mean d.o.h is a great protocol, but from a policy perspective, it's somewhat worrying Um, and we support it by the way, so i'm saying this out of two sides of my mouth, right? We support d.o.h but at the same time it's a little bit scary because When when it is possible for every web browser to automatically and by default be a vpn That is going to cause some backlash from organizations and enterprise in particular That are trying to manage The destinations to which their employees connect for for very legitimate reasons for both malware distribution and and security issues, but also just from filtering perspective. There are there are Reasonable conditions under which filtering can be applied There are unreasonable conditions under which filtering can be applied, but it is impossible to tell the difference between reasonable and unreasonable um, and so d.o.h makes it so that um even people who Even organizations who want to be cooperative are going to be forced to be uncooperative And they're going to be forced to put in what we would probably think of draconian measures to prevent um d.o.h from from undermining their security model Um, so I expect unfortunately I expect d.o.h to drive um a rather sharp increase in disconnection from the internet Meaning end-to-end disconnection from the internet and forcing users in enterprises and in in some cases in entire countries um to be put through Proxying servers which unwrap every single query and examine it in detail because That's the only way to enforce any policy is to unwrap all packets so preventing Preventing end-to-end because it's now it's now the case that vpns are so omnipresent that You can't trust any traffic If you're if you're a network operator, you can't trust any traffic to be non malicious because you can't see it So that's my concern with d.o.h. Um quad nine um has We've we've committed to not running Any other services or any other content on our IP addresses which contain d.o.h servers This is kind of a a bone. We're throwing to enterprise And and in some cases national governments who say well Um d.o.h. Is dangerous. We can't we can't block port 443 um Or we or maybe we can block port 443, but we don't want to so we've said all right We're operating our d.o.h servers on a very small set of IP addresses and the only thing that's on those IP addresses is dns So if you want to block quad nine You know, that's your right to do so we can't prevent you from doing that If we try to be clever and you know distribute our our d.o.h servers across a lot of different random IP addresses We're only putting fuel on the fire of of people Ultimately disconnecting their users from the internet and we're not interested in that So we're saying all right. You want to block quad nine? Here are the set of IP addresses that we operate on Block us at an IP level You know It's it's then up to your end users to complain to the administrators as to why they can't get you know Open dns resolution, but that's not Uh, you cannot solve a policy problem with technology Very true does not work No, it doesn't work and the more you try to force a policy solution with technology the more you'll discover that other policies are Layered on top of the technology which ultimately, you know, somebody with a gun solves that problem, right? You know, ultimately laws or not necessarily a gun But like if you're an enterprise your enterprise operators just gonna say, you know what? We can't trust any of you We'd like to but we can't trust any of you so we're going to disconnect everyone from the internet Um, and you're all going to go through this filtering proxy and that's the way it's that's the way it's going to be There's no more end-to-end and we think that that's that's not a good result D o t allows per port blocking. So if you wanted to block quad nine D o t you simply block port 853, right? easy Right, you don't have to block end-to-end for the whole internet You can just block port 853 and you solve the problem mostly You know the question of is this is this Is the death of the internet, you know details at 11 is is this Is this is this a little bit premature? Yeah, it is we're not going to see that, you know in the next year or two years five years Maybe and is dns really going to be the reason for this No, probably not the only reason But we would like it not to be a contributing factor um At least here at quad nine. We would like it not to be a contributing factor. Yeah, which is why we're We're bigger fans of d o t, but you know the cat's out of the bag right now. Doh is happening Um, it's happening at scale. It's not just going to be quad nine. Obviously. It's not already And it's not going to even just be the centralized dns providers. It's going to be malware operators It's going to be small isps. It's going to be all these open resolvers Which are already kind of littered the internet which are unmanaged and and out there for various good or bad reasons so Pandora's box is open um We can only try to encourage good behavior, but we cannot we can't compel it We'll see with how this plays out in the next in the next five years Yeah, an interesting thing that we haven't seen yet, but this like you said opening up new opportunities Is we've seen malware that hijacks routers that hijacks computers and changes where their dns queries go Wow, how how much more fun can it be when a browser? But the operating system itself has just decided to use something else. How do we As an it provider, how do we defend against it because we're going? I don't know why you keep going to this website because I ping it from the command line and it goes here Oh, look something hijacked where your doh goes. So it's going to be interesting Yep Yeah, and so filtering used to be easy just by blocking certain ports You could block You know most of the threats like if you block port 53 you could restrict your dns Just to a local resolver which had some policy applied to it. Great I can't really do that anymore. It's becoming Uh, it's becoming very very difficult if not impossible Um Side note actually this this might interest you again because I know a lot of your users and are using pfSense one of the things that I I suggested a while back was wouldn't it be great if your firewall? Uh, and I'm sure somebody is already doing this. So I'm gonna like the first comment You can be as oh so-and-so has this implementation already, but wouldn't it be great if your firewall did not allow connections to IP addresses for which there was not a recent dns query which resulted in an a or quad a record That seems like a reasonable thing, right? If you're if you're If your laptop tries to connect to a raw ip address that has never had a dns query associated with it Why is that right? What what's actually doing that? Um because quad 9 is a filtering dns service Um We protect you against domains that are known to be bad All right, so that gets that gets a pretty good percentage of of malware and phishing sites and all kinds of other stuff But it doesn't get all of it And in fact, it might not even get the majority, but it gets a good portion Because still a lot of malware uses IP addresses. It just opens up a connection to some random IP address out there Wouldn't it be useful to be able to Stop your system from connecting to IP addresses unless it had done in dns query in other words Unless it had validated a domain against one of a blocking dns service like quad 9 um, so I actually had somebody write a a module for pf sense that looked at the unbound logs on pf sense and and dug through them in real time essentially within a few milliseconds and said all right If a query goes to unbound meaning the local resolver in pf sense And then pf sense resolves it whether through actual recursive resolution or through a query out to a forwarding resolver like quad 9 A response comes back and it's valid right a real IP address Then a firewall rule would be inserted to allow that That client to connect to that IP address And if there wasn't a dns lookup, you wouldn't be able to connect outbound um, so I had someone write this module and Honestly, it took a long time for the module to get written and it took like five months and uh, Kind of lost momentum on it, but I think I put it up on Did I put up on github? I think I did great. I can leave a link to uh, well if you can If it's not on github, I'll publish it. Um, I don't remember. I don't remember what we did with it But um, it sort of kind of worked. It was still a little bit buggy and uh, we just lost I honestly lost traction with it because of the delays and we got focused on other stuff Um, but that seems like a really useful thing To be able to say dns is a prerequisite for connectivity out of my firewall um I think it would be a really interesting, uh, when you're trying to find indicators of compromise It should be something that sets off flags and encourages investigation might be a good way to From a debugging standpoint too Look at what gets flagged because there's probably especially as you if we were to deploy this in a larger scale network I would probably find Not necessarily always nefarious for probably a lot of stupid things And iot devices that's decided to hard code some ip address But those are still interesting Dives into what might be going on and of course like you said in terms of indicators of compromise or potential malware That is common that they just go right out because they don't want to trigger any of these sync holding filters They're going to go call out right to an ip address. So yeah, that's Anybody that's interested in working on that, um, we're interested in implying it. It's really it's not a it's not a quad nine It's not it's not specific to quad nine. It was specific to pf sense. Um, there's kind of a proof of concept But it would it would certainly improve the effectiveness of a dns filtering service of any type whether you run one locally or whether you use quad nine. Yeah anyway, so there's there's Uh, lots of interesting things happening right now in the dns field Encryption is is certainly a challenge But as I said the long tail is going to be very very long Um Again with different things, you know dns is an unencrypted protocol is going to take 20 years to go away Um or more. I mean I still I have I probably have 12 or 13 year old equipment actually not. I'm thinking about I've got Got 13 year old equipment just sitting right here on the table. Um, but I'm configuring as new Um for for stuff. Um, gotta One of these uh rack bots. Oh environmental sensors. Um, I use them for temperature and humidity But I was looking at the firmware as you know 2007 and like That's not going to support it. Yeah, that's like this for the new protocol Yeah, that's definitely going on a vlan all by itself, but it can't talk to the outside world Um, but those kind of things, you know, these devices are going to be around and in fact We still see people using ancient versions of windows and and mac os That's it's going to take forever to get up to encryption Standard so here in the Detroit area, um, you know due to the manufacturing. There's a ton of manufacturing equipment Um, matter of fact, many of the large the very large supplier here just outside of Detroit that does Cabinet manufacturing still runs os 2 warp and their answer is named broke It good news is it's so old. It's offline. So it's safe. Yeah, that's You know the safest machines are the ones encased in lusite. So yeah os 2 is essentially encased in lusite at this point, but yeah, um, yeah, I mean that's DNS encryption is going to do great things for um, I think the vast majority of people because again these updates and um You know a browser vendor or an operating system vendor can push things out to hundreds of millions of end users in a week Um, and that solves the vast majority of the problems and that's I think really what we're trying to deal with You know, how how much of the internet can we give better privacy and security and and a higher level of trust Um, and that's what I think everybody in this industry is working towards that. There's always going to be that long tail Um Interesting studies about how long it's going to take for rollouts for isps, right isps who have uh dns mask is another Program that's used quite frequently in embedded systems. So a lot of european ISPs have dns mask embedded in the home router so that it acts as a as a forwarding cache Great and speeds up queries but You know interval to upgrade You know, they're talking like well minimum minimum was five years maximum is never We don't even know what's out there. We don't know how to upgrade it. We don't you know, those companies have gone out of business So we don't even have the firmware I mean Yeah, take a very long time for this to happen Yeah, there's no one replaces the cable on them until the consumer complains about it and then they go. Oh, wow that thing still worked Yeah, exactly Exactly. So so hope is not lost but it's gonna be on you know, it's gonna be a lot of this is going to be on the end user to To update their settings More frequently though, it's going to be the Small it, you know, the if you're a big company, it's the it administrator in your office Or if you're a small company, it's the it administrator for the whole company They're the ones who are going to be able to make the most change the fastest By changing some just a few components of how they do their networking, you know, install a forwarding cache Locally in the network and then send all your queries out encrypted to quad 9 right or Install a local recursive resolver that's doing doh to end users and then send your your queries out to the roots However, you want to do it, but it's it's it's going to be difficult to get this rolled out to all the edges Completely but we can make a big difference With some of these new protocols and and methodologies pretty quickly one server at a time That's uh one one little piece at a time. We'll get there. Well, this is great All right, what are the questions? Do you have I'm I'm full of I think we did answers. I know I think we've covered the bases We got you know who quad 9 is we why dns is Important to be encrypted how hard it's going to be encrypt and uh, and what can just individuals do besides switching to it? I mean I mentioned before in a tweet that there's a donate button on top So I mean if you every every couple dollars helps please do Yes, as you said, it's a uh underfunded operation much like all nonprofits Keep this private because they offer a free dns service But not free as and you're the product free as in because people take the time to throw money at them So our goals are the end-users goals, which is very different than pretty much everybody else Yeah, I mean they eat a lot of ramen noodles That's that is how it works, but it's uh, it's a labor of love for sure I see you have a lot of passion for it. Well, thank you for joining in and uh, you can find out everything just uh hit quad 9 bus ways to find everything Quad 9.net. So all right. Thank you. Thanks for your time. Appreciate it. Bye And thank you for making it to the end of the video If you like this video, please give it a thumbs up If you'd like to see more content from the channel hit the subscribe button and hit the bell icon if you like youtube To notify you when new videos come out If you'd like to hire us head over to laurancesystems.com fill out our contact page And let us know what we can help you with and what projects you'd like us to work together on If you want to carry on the discussion head over to forums.laurancesystems.com Or we can carry on the discussion about this video other videos or other tech topics in general Even suggestions for new videos. They're accepted right there on our forums, which are free Also, if you like to help the channel in other ways head over to our affiliate page We have a lot of great tech offers for you. And once again, thanks for watching and see you next time