 Thank you for coming to this talk. My name is Yang Tang. I'm a maintainer of Quoding S project. I'm from OVN and this is my college job from Google. Hi there, I'm John Bellamerick. I'm also a maintainer of Quoding S and also deeply involved in communities. And as Yang said, I'm from Google. In today's talk, we are going to discuss several things. First, we're going to make a brief introduction about Quoding S in case another one is familiar with Quoding S. We're also going to provide an update for the past year or so. And then we're going to discuss about community bridge, sponsored by the foundation and the Google summer code programs. The reason we want to emphasize those programs that for the past year or so, we have made tremendous progress and we have lots of contribution from different from different school students and they counter a lot of to Quoding S. So we think it's worth to mention to to recognize their contribution and give people some insight on what how things going on in those two programs. And finally, after that, we are going to discuss about the demo plugin in Golang, which is a source-based service discovery, which is the focus of this session as well. And at the end, it's going to be a Q&A session. Thank you. So what is Quoding S? If you're not familiar with it, Quoding S is one of the earliest CNCF projects and it's a DNS server written in Go. So you might ask why do we need a new DNS server? Well, what we found was that traditional DNS servers were really geared towards traditional infrastructure and in cloud environments, cloud native environments, the requirements change very quickly. We have a lot of underlying infrastructure services coming going more quickly than you'd want to reload your buying server. And so we sought a need for a DNS server that could handle those use cases where the use cases themselves are evolving quickly and also the underlying services and things that we want to look up are changing quickly. So this project was started by Meek Gibbon, who I believe was at Google at the time. He's now elsewhere, but he's still our project lead and our sort of benevolent dictator and he does a great job of leading our community. But over the last few years, Quoding S has grown really quite dramatically on becoming the default DNS server within Kubernetes, which has increased its usage quite a bit of course, but also by integrating natively with different cloud provider API services. So you can attach to Route 53, you can attach to Azure DNS, you can attach to Cloud DNS, Google Cloud DNS, and you can do all of those in the same process and have it actually serve up records related to all of those things. And there's also a lot of other innovative features involved like or that we've developed like I'm able to do DNS over Jrpc, which folks are using to put their DNS save on the pushbox. There's all kinds of sort of interesting experiments, but I should say. What are some of those experiments in the last year? So it's been because of COVID, it's been about a year since we've done an actual update to the community. And so it was back in July 2019, we released 1.6.0 and more recently, 1.7.0 and coming presumably already released by the time you see this will be the 1.8. In any case, through that 1.6 cycle, we added a bunch of more integrations like to Cloud Providers, Azure and Cloud DNS, we already had Route 53, we added some additional ACL functionality and improvements in the cache to allow you to search tail options. In 1.7, we pulled out the communities federation plugin, some kind of plugin from the original federation control in Kubernetes, which is effectively dead. And so we removed that to reduce the service area for problems. And we also added DNS64, which is a plugin for helping with IPv64 transition. The big thing in 1.7, that those of you who are familiar with core DNS and have been using core DNS need to know about is that we changed the metric names. So in previous versions, the metrics names didn't follow the conventions that the Prometheus project publishes. And so we revised the metrics names to be consistent with those conventions, which helps fit in with the rest of your metrics. But of course, if you built dashboards of old metrics, this is a transition. In the upcoming 1.8 release, probably the biggest most interesting thing is a little bit under the hood, we have a transfer plugin now. So previously, the zone transfer configuration, transferring zones out, was configured on a per back-end plugin basis. And now we've built a plugin that handles it all for those back-end plugins to make it easier for authors of those plugins to support the transfer. So those are the major things we've done. Of course, there's been hundreds of bug fixes and minor improvements throughout that last year. But those are the big things that are user-facing and invisible. Okay, now I'll talk about the core DNS community update. For the past year, we have lots of updates from the community. First of all, our number of contributors has increased from 150 to outweigh to 238 contributors. That's a big sign to all the developers that come into core DNS. We also have additional maintenance. Now we have 24 maintenance in the community. We have 30 public adopters. By the way, the public adopter only means if they're the company, like Enterprise, they use core DNS and they're willing to reveal their name. We add their name to the adopter's list. But I'm pretty sure that a lot of companies, a lot of Enterprise, they're using core DNS silently. By the way, in case if you're a company or Enterprise, your institution use core DNS and you would like to share your experience with the community, I would like you to add a poll request to public adopters to increase this number again. Finally, core DNS reached a milestone of 6,600 stars. This is a gigantic milestone compared with a year ago. And we certainly want more stars down the road. So if you haven't done so, please just click start to core DNS repo in GitHub. Many people would ask, if I have a question, how do I get help from core DNS community? There are two sources of, two places that's going to help you a lot. It's a GitHub handle. In GitHub, you should be able to find most of the maintenance and most of the maintenance are very active and you can find help every day. There's also a Slack channel, CNCF Slack channel, that's a core DNS Slack channel that's going to find a lot of help as well. There are some additional places you can find, like a website, like blogs, like Twitter, those are all places you can find if you need any help. Next, I'm going to discuss about the community bridge program. For anyone that's not familiar with community bridge program, this is a program sponsored by Link Foundation. Every year, Link Foundation sponsors at least twice for community bridge programs for different open source projects. The core DNS has been fortunate enough to be selected to participate for the past two years. In 2019, one student finished a so-called Google Cloud DNS plugin, which is a very important plugin and is part of the default plugin in coding as repo. In 2020, another student finished a coding self-check on Kubernetes. Essentially, it allows the coding to automatically repair itself if something goes wrong with coding itself. Those are the very exciting contributions from the students. At the end of the day, the community bridge program will sponsor the student. Once the student finishes a project, the Link Foundation will pay a student to the student. This is both financially motivating and it's a very good way to help the open source project, especially for core DNS. As many of you already know, core DNS also participates in Google Summer Code and the people are familiar with Google Summer Code as well. For the past four years, core DNS has been participating in Google Summer Code and many students participated and contributed to core DNS project. There are very interesting contributions, things like a DNS tab, like ACO, which is access control list plugins, all part of the default plugin. Finally, the CEO, Chinaka, he is a student from India. He contributed a project, so-called machine learning by DNS detection. This project is now the part of the core DNS repo, but it is an interesting external plugin that's actually do very exciting things. Essentially, the project the student is doing for machine learning plus core DNS is that many people, when they talk about DNS, they say, okay, DNS is just infrastructure thing. It's not a lot of interesting things. It's not a lot of excitement, but for this student, he basically built a plugin that allows the core DNS to forward the query to an outside server and this outside server is essentially using machine learning combined with TensorFlow to decide if a query is Malaysia's or not. If the query is Malaysia's, the outside machine learning server will return back to core DNS, say, hey, this DNS query needs to be blocked. And then that's that complete whole story, which is very nice. If you look into his website, this consists of many components. It has a very good plugin that's written in Golan. It has a nice UI written in Python plus and it also has TensorFlow integration building a course model. So those are the excitement that has been introduced by this student. So I think it's the worst to mention in our discussion here because as many people are aware that actually there are a lot of things you can do with DNS if you are a little innovative. So that's the thing we want to show. The next slide is we see that the student even provide a nice UI on the left side. On the right side, that's convoluted neural network model this student used to do the detection. It's a very nice start to see people getting this field to combine the machine learning, combine the DNS infrastructure and combine the security in place. And also the student even have a very nice logo as you can see. He named this project as Emil Bridge and he has a very nice website which is a good thing if you have time to know Brazil. Now we start about a discussion for server discovery and the server discovery discussion is the main focus of this session. When people talk about DNS, a lot of people read the question about say discovery say okay in this day and age you'll have sdns out of the place you can define a network and whatever you want. So why is DNS still needed or why do you still need DNS or why do you need to use DNS for service discovery? I think there are several factors. Now first of all DNS is a nice induction. It may not it actually allows you to give a lot of flexibility. For example if you want to move from one place to another let's say you have your infrastructure out on google cloud and in one day maybe you say okay yeah because of some user requests I want to move to AWS. So what are you going to do? Of course it's going to be very hard to do the migration if you're using sdns solutions to make sure that the IP is as the same. With DNS a lot of times it's just a simple DNS record change and you can redirect the user to another cloud or to another place and we want. The DNS is also a very nice distributed system. It may not be the most sophisticated distributed system in nature but it is really scales to internet and it scales really well. Another reason many people use DNS is that DNS is very pervasive in IT infrastructure and you have DNS in cloud vendors. You have DNS in your Kubernetes cluster. You also have DNS with your existing IT infrastructure that's managed by IT admins and because the DNS is so pervasive you can easily wire everything up into one place and use DNS as the central source of truth for all the services and that's why the service discovery has been a nice area for DNS or server. Thanks y'all. So before we jump into how we build our source-based service discovery plugin it helps to have a little bit of a discussion about how queries are within our DNS. So on the left hand side here you see a core file. A core file is the basic configuration file for core DNS. Essentially it's a bunch of stanzas. Each stanza has a zone and a port more or less. So what you're seeing there is for example organization.com is handled on port 53 by the first stanza. On-prem.organization.com is handled by the second stanza and everything else is handled by the third stanza. So when a query comes in off of TCP or UDP port 53 the tree on the right kind of describes how that query, the flow that query is going to go through. So query coming in the top of the flow it's going to be routed based upon the configuration of those zones and the stanzas. It's routed to be handled by a plugin chain which is defined by a particular stanza. So if we look up food.organization.com it's going to go to that routing point and it's going to say okay food.organization.com let's make the longest possible match in DNS labels and of course on-prem doesn't match. organization.com matches dot matches but organization.com is longer. So it's going to get routed into organization.com plugin chain it's going to use so it's going to just get the cash plugin if it's cash to be sent out and otherwise it'll be forwarded on to 123.3.401 which will actually have work best. Now anything on-prem meaning this is a sort of anything falling under the on-prem.organization.com so food.organization.com we get routed to that stanza which would get forwarded and everything else then gets sent through to the other stanza. So this is how you configure different zones to be handled in different ways in a sort of stock vanilla coryns. So then the question is how do we change that behavior? Well we change that behavior by creating our own plugin so it's not going to change the routing but it can add another layer of routing in this particular case. So the demo plugin which is available on our coryns GitHub organization in the demo repository is a plugin that will return a different result based upon the source IP address of the plugin and in this particular case we look at you know 172 zero slash eight and or localhost network and we return 1111 otherwise 888. And that's the wrong deck. So any plugin in coryns is actually pretty simple. It consists of three functions well you can have more but there's three that you need to have. You need to have an init which is what all it does is actually register the setup function with caddy. So what am I talking about caddy? Well coryns was originally was originally a ford web server called caddy but we we're no longer a ford. We haven't been for many years but what we do use is we use caddy's infrastructure. So caddy has it's a going web server and it's it's got a sort of set of primitives for managing servers for handing graceful restart for handling the configuration for handling this whole setup process of plugins and and so we use that for some of our infrastructure on the this the internal server functionality and as it happens one other thing that's that happened in 1.7 I talked about different versions earlier I believe it was 1.7 well maybe it was 1.8 but about to happen is that caddy has gone on to a version that we don't want to to go with so we've actually pulled the original caddy code internally so that's kind of more under our control now but nonetheless the first thing you need to do is you need to tell the the core file parsing functionality what to do when it encounters a particular directive that describes your demo describes your your plugin so that's what the init does it says here's this function you should call setup and the setup function actually parses then that that configuration so like when you set up forward you say forward dot to some particular server name that is that that's what gets passed into this setup function is that dot and then you know essentially all those parameters you passed with that gets called once for each time the plugin appears the directive to register within it appears within the core file finally you need a serve dns method so the serve dns method does exactly what like what it sounds like it basically handles a dns request and processes it and basically either processes it returns the response or decides that it doesn't want to process it and passes it down the function so what do those look like they're actually like maybe less than 20 lines of code um the init function like I said super simple registers the plugin just says okay in our case our plugin's called demo um so what we're going to do is uh register demo and the action there is server type dns is because there's also a http server type because it's caddy we mentioned but so it's just always dns for us but action is basically the name of the function that should be called or a pointer to the function to be called when when you encounter when the caddy parsing routines encounter that directive demo so the setup function on the other hand is passed this caddy controller that's essentially a pointer into the configuration a tokenized version of the configuration file so the c dot next consumes the word demo which is the directive so we just kind of typically consume that and throw it away and then we go through and start to look at all the different options that might have been passed those could be passed directly after it or they could be passed within the within the curly braces it within the core file so in this case we don't have any arguments so we just uh essentially uh puke if we come across something that um uh if you added arguments there then we add this plugin in the setup we add this plugin to the chain so essentially we're saying here's the the handler uh here's here's the the handler to call um when uh when query comes in and it's going down and trying to process it um here's the one that the previous plugin should call in the event that it doesn't want to handle this particular request so finally then serve DNS this is the meat of it this is this is what gets called when um by that that chain when um when a request needs to be handled by this particular plugin um just does a few things one it creates this sort of convenience um struct called a state that's a request object um and pulls the uh the query name off of that that's what name does um since we are sort of uh brain dead we don't allow you to configure it we just hard coded 888 here um and then we just look at the the state.ip that's the client's source ip and if it starts with with one of these things um we we reply with something different and we log in so um obviously this is not a very useful plugin because um that's not an answer we want typically but um normally a plugin would examine the type of the query uh we potentially look it up or it might manipulate it in some way finally we the final piece to making all this work is that we need an actual core file um when we create uh we need to call it super simple um the plugin doesn't get inserted into the chain until it sees that directive we talked about earlier the directive we put in was demo so we have to put that into the core file finally building it so one interesting thing interesting meant in a somewhat pejorative way because i i don't particularly like this but it's just how it is right now is that uh that plugins cannot be loaded at runtime plugins are only loaded uh only built uh done at build time so we have to actually go in there's a file called plugin dot cfg and that file has an ordered list of plugins and in fact not only can they they not be loaded because right it's go it's statically compiled but they actually uh can't be reordered um that's actually my bigger complaint not so much that they that we need to recompile uh because i like statically compile things but um but that we can't reorder so that's my my uh my my little side comment there maybe we'll fix that one day we haven't done it yet um but nonetheless you need to go in and decide where do you want this to fall so that means that um for example the forward plugin comes very late in the plugin dot cfg file because typically we don't want to forward something until we've given an opportunity for uh say the kubernetes plugin or the file plugin to handle the query first so um similarly things that um like um rewrite rewrite comes very early in the plugin chain because rewrite takes the request modifies it hands it down the chain for other things and then it's given an opportunity to um to to do something more with it on the with the response on the way out so you have to make a decision about where your plugin should fit in this case well we are doing a source-based service discovery we're actually um we're actually essentially a backend we're deciding what to return uh based on the query and so um and given that we want this to sort of take priority over what might be in a file or might be on some DNS server out there we want to put this early in the plugin chain so we would go into plug-in cfg and we would drop it in um uh somewhere early in there um and then um finally we just build there's a nice docker command you can use to build so you don't need that go go installed locally and it creates the file for you and and you can go so we don't have a live demo for you today but what we do have is a link to the source code so you can just click on that or visit that in github and um take a look at what what what it is and and give it a try and um after that uh feel free to reach out to us on some of the channels that yang mentioned earlier on slack or wherever it may be if you if you run into trouble thank you finally uh once you've written that demo plug-in and worked it and it's working great and you realize you've got all these crazy use cases that you want to do then please join us and become a contributor um you know we um there's there's lightweight ways like like uh help to help out the project like yang mentioned star us or uh if you if you use um cordy and s which if you use kubernetes you probably use cordy and s even if you don't know it um feel free to add yourself to the adopters or jump in on our github or slack channels um we uh you know if you if you show good judgment write good code and you give us a good solid pr um you know you you'd be a contributor and if you do one that uh that requires um potentially ongoing maintenance then we'll watch you to be a maintainer especially if you had a plug-in um uh because you know many hands make light work um when when there are changes that come in uh or problems with a particular plug-in we like to have the author of that be a maintainer of it uh in the vast majority of cases so um we'd love to have you have you take your new skills and contribute there thank you very much and we'll look forward to the q&a