 Hi guys, welcome to the designate operators deep dive talk and I think we've got to get started now. Yeah So first off my name is Graham Hayes. I'm a software engineer in the HP DNS as a service team we develop designate we also run the designate install that's in our public cloud at the moment and And we're developing the DNS as a service for the HP helium private cloud as well Hear me. Okay. Hi. My name is Ron Ricard. I'm a senior cloud engineer at eBay I'm responsible for DNS as a service within eBay Okay, today. We're gonna go over Several things with you. We'll give you an overview of designate. We'll talk about the rest API We'll talk about designate neutron We'll talk about designate central and we'll talk about designate sync Hopefully you guys are all here because you want to hear about DNS as a service. That's that's what designate does We're hoping hoping it'll be part of open stack soon designate consists of a rest API Central and sync those are the three main pieces to designate designates communication Shouldn't be a surprise to anybody if you've done any other open stack components It's all through message queue, you know rabbit rabbit MQ for example This is both for its external communication for example between designate sync and neutron or designate sync and Nova Internal communication is also through rabbit. That's between designate API and designate central or designate central and designate sync Um Designate client is available for you which will allow you to manage the servers the domains and the records And it's very important to understand when I say manage the servers What we're really talking about is name service records. Okay, and and if you come to our Workshop a little bit later today. You'll actually get hands-on experience with that and that'll become a little bit clearer to you Hopefully More functionality is available though for you in the rest API Okay, so the client is is a subset of the of the API, but there will be more functionality available in there for you It's important to understand a couple things as we go through these slides Designate is the source of record for DNS records for the domains that it manages Okay, so if you're an admin, maybe you already have some heartache about that You just heard me say that and you're like that's fine But designate is the source of records for for the domains it manages Domains are owned by tenants. It's also another important concept in designate If you want additional information about designate, we do have read the docs out there you can go out there and Get more information about it So as we're talking a lot of this stuff is covered in the read the docs if you need more information Maybe maybe maybe I wasn't clear Graham wasn't clear go ahead and read about it Okay, I do want to point out before we get on to the next slides that I'm going to use the word domain And I'll probably out you'll probably also hear me here use the word zone There is a distinction between a domain and the zone I understand that distinction most of you guys probably understand that distinction as well When it comes to designate though, there is a one-to-one mapping at least now there is between a domain and the zone so it's okay if I Use the word zone where I should be using domain and vice versa. Plus if you're from Keystone I believe they have a concept of a domain as well and probably confuse you so Okay Cool. So this is the basic architecture for designate I'm probably the easiest way to run through how it works is to run through a user making API call So in the bottom left there, you'll see that the user makes an API call into designate API it authenticates the token against Keystone and then puts the request on to the message queue this gets picked up by one of the central nodes and Depending on what's happening it'll either go to the database and get the information it needs or it'll write to the database and Then it sends the information to the back end now But a back end is what manages your DNS servers and that bits entirely pluggable. So at the moment We support a couple of DNS servers, but it's easy very easy to add new DNS servers in the future Just by writing a pluggable back end piece and that's really part of central But because it's pluggable we make it a separate piece on an architecture diagram So once the user has say created a domain or Created a record they can then query it directly to the DNS server Through the side channel. So we have two entry points really for people to One for inputting information, which is the API and one for getting the information back, which is traditional DNS okay, so the designate REST API has Matured over time version one Provides the basic functionality for you allows you to manage the servers again when I say servers and talk about name server name service records It allows you to manage domains the zones and allows you to manage records We have an experimental version to which adds additional functionality and again this this functionality is not captured yet in the designate client But it'll allow you to do things like import and export zones Manage your top-level domains It'll allow you to manage your zone black lists and it also has Information in there related to neutron which allows you to work with managed floating IP pointer records And there's another link for you all again on the read the docs. I've mentioned the designate client before Again, if you come to the presentation a little bit later Workshop you'll get a little bit hands-on experience with the client if you don't come think keystone keystone has a client You're able to type keystone give it some command does some things Designate client works the same way Nova if you've worked with Nova same kind of deal And again the functionality is a subset just to reiterate subset of the entire designate API right now It works with servers domains and records So I'm going to be talking a little bit about the new features in the v2 API for the next little bit So the first thing we've one of the first things we did now at the moment is the ability to import and export buying nine zone files from and to designate So we've implemented this with just content types So if you make a get request to a zone with its ID You will get back buying nine compatible zone file which you can put into a server somewhere else if you like For the same the same way you can import a zone file if you have a huge amount of zone files You want to import it to designate for the first time you just push them up with a text DNS content type and it'll load the zone and all the records into Designate for you. We also have a zone extractor tool which Was written to allow people to generate Zone files if say if you have a very complex setup for buying nine zone files This will collect all of the zones and put them into easy to manage zone file form Individual zone files for loading into designate In the v2 API as well we allow you to choose What top-level domains you want users to be able to use as part of designate by default? It's wide open if you have no tlds specified. It's you can use whatever you want But this is for some people who are running in a public cloud situation This means you can assure people only register legitimate top-level domains And because it's an API this can be done on the fly. There's no need to Right out of zone right out of config file and reload the API and centrals You just your support guys can keep adding in new top-level domains as they become available We have some sample data in there, which was up to date as of a couple of months ago, but the zone file at the top level domains keep getting created so That's why part of the reason it became a rest API endpoint was to allow easy management of it We also allow blacklisting zones so a zone blacklist is effectively a rejects that runs against every single zone creation request and This allows you to set it to block people using particular keywords or patterns for registering zones For example at HP.com at HP cloud. We try to limit the use of HP comm domains to internal users so as part of that The Whenever a user makes a zone request a creation request. We run these rejects against it and make sure that it's valid It's important to note actually for the blacklists on the top-level domains If you had a customer that really wanted something and had a legitimate business case For overriding the two checks if you have we have it with all it's all a policy based Set up so you can allow your support staff or your admins to create the zones On their behalf if they if they need it and then it then they'll be able to be able to use it like any other zone If it had passed the creation checks so this What we have in Neutron it's new functionality in the last three or four months is we allow people to manage Pointer records for their floating IPs in Neutron This means that operators can just delegate the entire reverse zone for their floating out people Into designate and not have to worry about dealing with user requests for managing for creating pointer records So currently the way we have it the way it's designed is When a user makes a request for a floating for a pointer record We go to Neutron and get all of the floating IPs that they have so this means that users can only set pointer records for For IPs that are in their tenant on Neutron So I'm going to run through example calls for that You probably can't read that but this for example you do when you first do a get request to the root of the floating IP resource you'll get a list of all of the floating IPs that you have available and If any of them have pointer records set already it'll show up here as well So this is if you get this is a single floating IP record So we have the address We have the PTRD name, which is what will be the pointer record that gets put out onto the DNS server and we have then the ID which is built up of your region and The floating IP ID that Neutron provides so that's just a simple get to the floating IP address you can then post our patch make a patch request with demo 001.example.io and this will create the pointer record in Designate and on the DNS server so that so after this call now that pointer is live and It can be queried So let's talk a little bit about more more about What what does designate central do? It's important to understand that? When designate was architected the API was actually Written in such a way that it doesn't do any work per se. It's just it's just it's just a front-end a facade For for for the work most of the work and well actually all the work happens in designate central It's the it's the core location for for for the code It works in a plug-in fashion There's a plug-in for the storage Sequel alchemy there's plug-in for the back-end drivers We have power DNS and sd4 free IPA right that's free IPA Dinect and bind 9 It is important to note though that the bind 9 needs some work Okay, so if you were interested in implementing Back-end driver. It's actually fairly trivial to do. I've done it for ebay You just need to be able to create update and delete t-sig keys Domains record sets and records now with that said While it was easy, it's actually going to get easier in in Juneau We are going to be introducing Concept of mini DNS in the designate Mini-diagnosed mini DNS will be used to push data using zone transfers I believe in Juneau there's be a ex fur but afterwards Incremental zone transfers will will will come along. It's going to simplify the back-end drivers even more than they are now We won't have to deal with The record creation anymore in the back-end driver Or the record sets. You'll just have to deal with the you know the DNS servers specific way of creating domains and probably t-sig keys so this is the a Simplified architecture that will be in place once mini DNS is being used so now So when this is in place this will allow The back-end driver will only have to deal with creating his own and deleting his own Every time that a user tries to update a record or create a record or delete a record central Designate central will tell many deal will put the information into the storage layer And then it'll tell the back-end manager Which all it does is once it gets a update It'll send a DNS notified to your DNS servers the ones that you have serving customer requests So what they do then so they can just then do an AFX or so which is a zone transfer from mini DNS and Get up to date one of the major reasons we did this we were looking at doing asynchronous transfer and Async asynchronous zone creation and record creation and we were coming into all sorts of problems with Being consistent on ordering and then we realized that DNS already has this solved This is a FXR is solved solves that issue So that's one of the major reasons we have mini DNS It'll also allow a lot of future potential development As we go forward designate sync so we talked about designate API We talked about the second piece designate central designate see sync is the third piece of the designate architecture Designate sync was written For you the customer or the user of designate to meet your business needs Okay, so what it's going to do is it's going to consume events from Nova Neutron and potentially other services And it's going to turn those events into DNS operations What events and what operations are performed are determined by custom handlers which are plug-ins in the DNS sync? designate sync is going to ship or ships with to plug-in handlers which are They're good examples, but whether or not you're going to be able to use them. It's kind of dubious But there's a Nova handler and there's a Neutron handler in both cases What they they're tied very much to a single domain And they allow you to add an A record into that into that domain. So like I said, but they're great examples They're there to show you the simplicity of actually writing a custom notification handler But you probably will end up writing your own okay These are the example of the kinds of events that You may get from your from from from other Open-stack components in the environment. So you've got your events that come from Nova looks like we got some events from Trove some events from Neutron Okay, I Don't know if you guys can read that or not. It looks even small to me from up here This is this is an example of the payload that comes in an event now where the why the payloads important is designate sync actually Passes that payload off to the custom Notification handler and then the custom notification handler the piece that you you will write You can actually pull out data that you need to create whatever it is you need to create. So for example You you fire up a VM you start up a VM you need to create and pointer records The data needed to create the end pointer records is right here inside this payload you have your address which is one of the one of the bolded items there and you and maybe potentially you're going to use the display name as as the Name of the as the label for that record and so that's there in the payload When you when you extend the class for the notification handler, there's only three Methods that you have to implement and and and there and again. It's very simple to do There's there's a method which allows you to It's to Tell the handler or to tell sync which exchange topic that you're listening on or you you want to consume What event types do you want to consume remember the screen with the event types? What event types is this notification handler gonna gonna handle and then finally? Probably the the main code is the is the process notification where where you're actually taking the payload Maybe even the context And the event type and you're turning that into into something some DNS operation So for example, you're taking the payload and you're you're creating an a record you're taking the payload and you're creating a record and a pointer record Okay With that said I was going to walk you through How eBay is using detonate sync because because I feel it's important for you guys to understand that this is Probably the piece of designate that you will Need to do some work with okay Hopefully the community will have provided a back-end for you for whatever DNS server you have sequel alchemy is going to meet your needs and You're going to need to you're going to need to write a custom notification handler for your business. Okay, so I'm going to go through this it's going to be a little bit complex But what what I'm kind of hoping that you'll get out of this as I talk about how eBay is using Designate sync and what our custom notification handler does when I'm hoping you'll get out of this is the flexibility Okay, the power behind this this particular Component in designate is it provides a lot of flexibility to you as a customer to get designate to do what you need it to do Okay, so our initial design at eBay. We took a very simplistic approach It was very granular We had we had tenants and we had we had a single forward domain and a single reverse Domain associated with those tenants and again member tenants own domains and designate we took the context from the notification message and Then we created the a and pointer records for for our VMs as the VMs were created We delete it those as the as the VMs were deleted It turns out turns out that that was a little bit too simplistic for us. We need to make things more difficult And I'll talk to you a little bit about that We have a concept of virtual private clouds VPCs and and we have and basically we have a production VPC. We have a dev VPC. We have an external VPC Those are represented in designate as tenants. Okay, there's special tenants So users are not members of these tenants all the other tenants where users actually are creating VMs for our Members of these tenants. Okay, I say members of these tenants. They're associated with these tenants These tenants own the domains Okay, and they may and they can they can own many domains So we may have one tenant that has eBay comm PayPal comm Etc etc etc and another tenant that owns other domains both forward and reverse domains Okay, in order for us to determine which Zone We're going to add the record for when a VM is created. We're using Nova's instance metadata Okay, so when when when you fire up a VM we pass the zone name We pass the host name and we've asked the VPC name In the as metadata inside of the message Inside the message and that's in the payload now It's available at that point to the code that I'm writing the notification process code of the process of notification code and And I can take that data out and then I can determine what my a record is going to be I could determine what my pointer Pointer records are going to be now the only the only thing you probably need to understand there is Because we have VPC names and then we have tenants there may not there There's a one-to-one mapping between the name that we get in the metadata and the tenant That's actually in the configuration file for us the designate configuration file that mapping So we make that mapping so we know what tenant we are then you just need a user because you know a user needs to be in that tenant in order for it to add the a or pointer record and We have a service account for that and that service account is a member of All of those tenants so we have a service account. It's a member of the dev dev tenants a member of the prod tenants member of the The external tenant, okay, and then that service account is used to create the end pointer records Okay, so that was complex Maybe maybe doesn't meet your business needs or anything, but again what I wanted to point out here is it's very flexible the the model that designate sync provides to you you're gonna have your own business needs, okay, so you'll be able to sit down and design something that's that's That works for you, so Graham's gonna talk a little bit about another use case so this is a Sample handler for it. It's using a particular API. That's about to be merged. It should be merged in a week or so but this is a solution that would allow you to have per tenant zones for For your for your users, so we're exposing a Endpoint to the v2 API called options which allow tenants to set key value pairs of options in designate and This would allow the notification handler to get in the event and from the context You can tell what user it is and get the default domain Value from our from our database and then use that value to create the a record and the pointer record so effectively And this you'll just see this here. I haven't done any other setup This is just the prototype of the process notification Function, so we get it we get the domain ID from central which goes to the database and looks up the option for that tenant We create the record set For this for this in this domain with the right name, which could be the name of the host or whatever You decide you want the pattern to be as Ron said because it's You write the code for this you can make that you can make it as complex and Or as locked down as you want so you can give the users no options or you can allow them More and more flexibility so from this we're just taking the name from the name from the payload and Setting then the the floating IP address as the as a record in that record set and then setting it off the central Which will publish it live on the DNS server and It's a very this is a very simplified example, but it shows How easy it is for you to? write a powerful Function that suits you and your business case Something like this would probably more appropriate for a very large private cloud or a public cloud, but This could be set up to you for any use case really So just reminder we have a workshop today. It's in B 3 1 4 in I Which is upstairs at half half one If you're coming can you make sure you have vagrant on virtual box? And if you're on Windows an SSH client I'm we found out that if you don't have SSH installed beforehand It gets a bit scary But if you're if you if you've heard anything you're interested in or you want to see this in action We're doing a step-by-step live install at half one. So please do come down Yeah, so you'll get a you'll get a live install You'll you'll actually get to see the client will use the client will will touch briefly on the rest API And how to use the rest API, but then we'll we'll dig right in and the client You'll get some examples of the client and you'll also see sink in sink in action specifically We're gonna we're gonna exercise the One of the default Notification handler that's that's there the the nova one so you'll get to see a VM get created And you'll see the a record Get created in the DNS back end. So So are there any questions Please if you could go up to the mic give us your name Maybe who you work for so we can determine or we can direct our higher admiration properly Hey, my name is Richard. I work for Richard Stram. I got a couple questions. First Is there any support or anybody working horizon support yet for for the API? Yes, we have horizon support and HP public cloud currently and As soon as we get incubated we'll be open sourcing that hopefully and it should be open the community then What about DNS that got in here? I wasn't sure what? Anybody working on DNS that get for the API or It's on the road map. It's on the road map. We've talked about it It won't be coming in Juneau, but it is on the roadmap. All right last so the interactions between Designated neutron like when you specify a network and you can specify the DNS and or the search parameter for your solve.com Like right now every tenant has to have a you know DNS mask is everybody the same resort resolve calm search parameter because it's specified at the global level any way of being able to have any kind of interaction between neutron and designate so that designate can take over some of the the Creating the proper DNS cop on a pertinent basis or anything like that We we've been talking with neutron a bit, but We were trying to do a designate is mainly aimed at being a third of DNS not real not recursive I know there is we are looking at integrating with neutron, but not quite in that way yet. I Think there's probably there's probably scope for that to be fixed a neutron I'm don't know how much integration that designate could do there which if we stay as a a third of DNS and last the when you talk about It is Designate hooked into like the salameter events. Is that how it's like knowing when to create exactly? All the events there we were showing our salameter and you expose you expose the events the same way you do in salameter So for example in Nova you have to turn on the state change for VMs You do the same thing for designate and we'll actually show you that at the workshop today Hi, my name is Kashi Fally from risk management solutions. So my questions Actually, I think you answer one of them was that because you're an authority DNS server That means all your VMs or servers will still use your recursive So now okay simple blocks all that doesn't it will do is pass the records to okay That's straightforward You mentioned the CLI interface, which is great like for Linux admin want not blah blah We have a lot of Windows admin as well And so what happens is there any sort of web interface that they can use if they need to because obviously that's gonna be the authority You need to make sure everyone's going to the same place and some people aren't comfortable with the command line Yeah, we have a horizon plugin, which is the web interface for open stock. It's coming It's coming so yeah, and there's a rest API So I mean if you don't want to use the horizon plugin you can write your own gooey The other question I've got is does this require open stack in any sort of form can you run it standalone? So if I wanted to bring it in today, and we don't have a perspective day We're looking to bring it in so it's like a phased approach can I bring this in today? And you know what the plug-in table blocks myself and carry on or is the Yes, you could install it today. You would currently we use horizon our keystone for authentication Okay, so I'm but you can run it unauthenticated if it's in a private environment You can run it unauthenticated, and you know, or it's it's just simple middleware that runs on the API That processes HTTP headers and does something with it in our case It goes off to keystone and gets a response But if you want to the middleware is particularly difficult, okay So you could you could influence on the customer and so so if I understood DNS That's the next question so if I understood DNS sync properly. That's like a notification piece or there's in a so one of the challenges we face today is with cleanup where you tell And I'm not saying it's info blocks is full But you tell him for blocks of the leader record and it deletes a record. It doesn't delete the C names You didn't have to go back and see names or any other subsequent records And you have to make sure you check for every single possible, you know scenario So is that basically what DNS thinks about so if I get a notification from Nova or any other sort of event I can just trigger that to do the cleanup process. That's that's correct because it's it's custom model Well, you write you write the code on what it does So if you have a records and C name records to clean up pointer records it you you go ahead and write the code for that It'll do it you get the notification. It does it so last question Maybe this might be slightly off topic But I try to see your view on it So one of the most painful things I'm going through right now and unfortunately I have to do it next week when I go back Which is why I'm asking this question is about house name conventions now You're you're doing a Guinness record thing here, right and having a cloud and seeing what AWS do with house name conventions and Working from a traditional enterprise and trying to make them go into a cloud and telling them that you can't have you know Roll DC cluster number Whatever you flavor you want in the host name because you've only got 15 characters for whatever reason where Windows analytics workshop So that's why the 15 character limit What does does designate imposing limits in terms of what you can so does it add does it take only give you x number of characters? Because you're in a cloud like environment so that you can provide some sort of unique form Is there any recommendation you can have or you know? Any sort of ammunition you can give me would be great We're kind of agnostic we support the RFC For host name length anyway With the notification handlers, there's a lot of there's a huge amount of information in there But if you're 15 characters, it's gonna be difficult to get that all that information into a host name No, but I don't mean I don't mean the meeting that standard Right, so my point is that to be in a cloud environment is very difficult to if you if you're thinking traditionally in an enterprise Yeah, and you're thinking oh, yeah For example, I've got a skill server. It's the first node in the cluster and it's in you know In Iceland and it's in rack 15 and but to have all that and manage that and create some sort of database for me That's crazy, right? Whereas this is why one of the reasons Amazon and I think I believe I'm not 100% sure rack space have these unique IDs And then you use tags. So how does designate from a user perspective? How does designate enforce some sort of if you're gonna if it's for cloud right doesn't it sort of cloud? How do you enforce that people don't Do that a lot you just expect them to manage that themselves. We leave it entirely open to themselves. Yeah, you can It's up to you But if you have requirements around names and you want to limit Maybe not even allow the users to do that the custom notification handlers there You can you can come up with your own scheme for how you want the names to be or you can or you can take the data out of the Like set out of the notification event that the user enters display name or whatnot and use that so it's really it's really up to you Everybody has their own has their own way of handling this. So Designate isn't okay imposing anything on you Right I can go into more detail about people waiting now if I explained to you what we do today, then you understand Yeah, let's go ahead and take it offline. Yeah, you talk about it drops at the end. Yeah Yeah, come on So my name is Anthony Vega. I'm from Comcast. I have a couple questions about you know We talk a lot about pointers and a records and see names Do you guys handle qualities and to an extension to that? Do you handle v6 transport for resolving as well? Yes If the host it's running on can respond to v6 We're agnostic. I think your DNS server can deal with v6 requests It'll work quite a yeah, that's fully baked. That's fully in there as our v6 Pointers as well. Excellent. Thank you to extend that running your DNS server in Perhaps a VM inside of OpenStack as an advanced services VM or something. Would you be able to support? Actually pulling from neutron and and binding to a dual stack setup I'd like to a customers So the VM that you actually run the authority resolver on Would you be able to support actually binding directly to that and pulling a data out of neutron for setting all that up? We don't currently It's an interesting idea though. Yeah, I'm We're open to ideas especially around integration with other open-sock projects. Yeah, I'm okay I'd like to talk to you a little bit more about that. I'll stop by afterward. Thank you I Steve Pearson HP cloud Does central manage the SRA record for you or do you have to increment that yourself with the serial number? I mean, it's all managed It's really yeah, it's on for you. Yeah, so is it possible to create zones which are not RSC compliant? We don't think so I hope hope not If somebody has found a way, please tell us we'll fix it I found all sorts of little ways of breaking it for instance you can have a CName which coexist with another record for instance one problem. I've had it's very easy to miss those little things That's one of you thought about Yeah, that's definitely blocked CName with other records. Yeah, we did block that. Okay. Any other questions? You mentioned some limitations in Support for by 9 what is it is it advanced features or is it just basic functionality? Wow It's basic functionality it'll be fixed in Juno with the mini DNS To make a long story short Creating record seems perfectly fine and and actually it's probably this is just a bug that we can fix now And then when many didn't ask because along by 9 will just get better, but when you delete a record It doesn't get deleted immediately. It waits for another change to occur and then that record goes away I noticed that but it's a bug that we can fix I think all of us were kind of hoping that we'd get the mini DNS stuff going and then all all of that management stuff goes away So any other questions? Okay, well, I will I do encourage everybody to come to the next session It'll be a hands-on session. You'll get to see What we talked about in action. So, okay, good