 Check one sibilants sibilants check two Hello So we have a lot of things to cover. So I'm gonna try to be very quick quick introduction about myself I'm a software developer with Rackspace. I've been doing open stack for about 10 months. It's a fairly new I'm working in the private cloud team Our product now is on a stack forage or a sensible deployment. So if we check it out member of the API working group we're talking about that today and I'm also a contributor mostly a heat at a little bit of horizon Also on the side I do a lot of things with flask. I wrote the book From O'Reilly on the framework and a bunch of other things and I spoke and the last two pycons About flask and rest API's and I also have a blog you're welcome to check it out and also blog on the rack space developer blog And hi there, my name is Everett Taves. I'm a developer advocate also at rack space in our developer experience group I helped found the API working group, which we'll get into a bit later with Jay pipes and Chris. Yo also a Python open stack SDK contributor and I've been working on Open stack related technologies for the past three or four years pretty pretty close to its inception I write and speak as well and I write more on blogs and such so if you want to check out more detail You can go right ahead and we're going to be sharing a link to this presentation at the end of the presentation So you'll be able to follow up afterwards With whatever you're like, so I'll turn this first two-thirds of the presentation over to Miguel Okay So I'm assuming Some of you many of you will be writing applications that need to talk to an open stack cloud So you have a lot of options. I'm gonna give you an overview of those Options, so this is pretty much a diagram where each of the core boxes are an entry point Into the open stack clouds. So all these are options that you can you can take to to make the cloud do things So as you can see, it's a fairly complicated. So I'm gonna walk you through it one step at a time So there you have your cloud, which is a bunch of services, right? You have Keystone, Nova, etc Some of them are optional. Some of them are core modules So you have them there and as you probably know each of these modules has a rest API So with one small exception that I mentioned in a little bit It's very important to know that this is I'm not sure people realize this that everything that you do with the cloud They it goes through these APIs So this is it So if it's not in the API is it gonna be done everything else on on the full diagram That's on the left it all goes through this So this is the little exception that I mentioned. We also have AWS compatible APIs We're not gonna talk about those here, but I mean you should know that there if you need to be migrating from AWS So For each of the modules, there's also a Python client library that you typically install with pip and basically wraps the corresponding rest API and You can use functions or object-related programming to talk to the cloud In recent times There was this new project. It's called the OpenStack SDK It tries to consolidate all these many Python libraries that you need to use to talk to individual individual modules and If you use the OpenStack SDK that this is a single library that talks to many most of the Of the core modules in the OpenStack cloud so The the OpenStack SDK goes through their same rest APIs, but it's a unified client. So If you're using Python you're golden, this is a really good option But you know Python is not the whole world if you're not using Python This is a no-go. I mean it's nice as they these options are you know, they're not for you So There's also third-party clients. So these are unofficial clients created by the community This exposes the rest APIs wrappers to the rest APIs for for different languages and platforms You may find something that works for you Because they're unofficial. They usually lag behind the support. That's obviously in the rest API But also in the Python clients which are official. So, you know, your mileage may vary It's sort of a hit-or-miss situation with this So going even higher level now, we have the command line clients so each module also offers a command line client and note that in the diagram I drew boxes around the Python library and the Python command line client and the reason is that there's a very heavy Coupling between these two they're actually the same package. So you you install both together. They're the same Python package So these give you access from the command line. You can use this Exercise the rest API through the Python client from the command line Now like the same effort that was done with the OpenStack SDK on the library side It was also thought that it was going to cumbersome to have all these many command line clients Different command line clients. So there's also an effort here to consolidate all the clients into a single command line application and that that's called the OpenStack command line client and For some reason I'm yet to understand. It doesn't go through the OpenStack SDK. At least not yet it goes through the individual Python clients the client libraries and And Okay, so so one of the problems with the Python clients command line clients are very useful To do scripting so you can stick them in bash scripts and it's super easy to use Unfortunately, if you need to do something a little bit more involved where for example, you need the output of one command and Use some of that on the next command then you have to do crazy things like parsing or scraping the output Which is made for human consumption not for scripts So, you know for very simple things good option something a little more involved not not so much The the final client that we have is horizon the dashboard and the dashboard also goes through the individual Python libraries that are the Python clients and Of course, this is great if you need to do something quick manually But you know automation is not there you can't use it for anything that that's really building a workflow so My job here is trying to convince you that the rest API's are the best option I'm gonna try I'm gonna tell you good things and bad things about them But let me start with a quick introduction a lot of people You know that that not that they don't work with APIs, you know every day They think rest API is one word and really there are many types of API's and rest is one type and The main thing with rest is that it's an architecture that enables services to be highly scalable so that that's the main thing with rest API's and The interaction with the rest API works pretty much in the same way you interact with a web server through Through a web browser like browsing Twitter or Facebook that kind of thing basically the client sends a request and the server responds and With that response the client gets some information makes a new request and so on and it's Conceptually extremely simple. That's why I like it so much. I think it's underrated Rest API's are well implemented. There's a lot of common knowledge that applies to all of them even outside of OpenStack So you don't have to go to the manual so much when you know that the framework is a rest API So there's a lot of things that are known in advance Unfortunately, I don't have a lot of time to talk about rest. I get carried away every time I start talking about rest But I gave a talk last month at PyCon in Montreal. So it's called it's your rest API restful It's on YouTube. So search for that if you want to know in that talk I described the six principles of rest in detail and I gave a demonstration with the Python API So feel free to look at that. So I'm a good news first kind of guy And also the good part is shorter. Unfortunately Well, so let's get this out of the way The main most important thing about the rest API's is that they're universal You can access them from anywhere as long as you have an HTTP client You don't need a library. You don't need a command line client. You don't need anything HTTP client. It's all you need. You can access the rest API fully. All the options are available to you With no restrictions So that means your phone can talk to OpenStack A microcontroller with an Ethernet stack can talk to To an OpenStack cloud. So it's available everywhere So if you need to remember one thing about this talk, I think it's this It's very important So now Specifically about the way the OpenStack APIs are organized One thing that I think it's very nice is that we have all these modules and you can combine them to build clouds Have different features so that they're specifically designed for a purpose so Probably the core modules are going to be always there, but then you can add and remove all of these other Components to to make the cloud that that's specifically designed for for your purposes. So Since you have clouds that may have different different shapes and forms One thing that I think it's also very good is that there's this concept of the catalog So when you start talking to an OpenStack cloud you you have access to the catalog and the catalog is a list of all the services that are available and For each service you get the the main URL the endpoint to start talking to that service so basically You only need to know how to initiate the connection to the cloud and then you get back the catalog and You don't have to hard code anything in your application if you want to talk to Nova In fact, I don't even know what what port Nova is I don't remember because I never need to know that because I go to the catalog or my application goes to the Catalog and gets the URL from there and the port is there. So I don't need to remember that and that's very good This is one of the pillars of the rest architecture. You can see it in my by contact. It's called hypermedia So I think this is a very nice implementation of hypermedia So the authentication flow is also very good. It's very straightforward. It works the same for all the modules So once you learn this Basically, you get to go we can talk to any OpenStack services Basically, your application needs to know the authentication URL so in the same way when you when you go to Twitter to check your your stream The only thing you need to remember is that you need to type Twitter calm You don't remember all the other URLs are inside Twitter For OpenStack, you need to know the authentication URL. So that's your entry point into the cloud so you send your credentials and OpenStack will send you back the the catalog and an access token then you find whatever you want to do in the catalog whatever service you need to talk to and You start sending requests directly to that service using the endpoint that came in the catalog nothing that you need to hard code or know in advance and For every request you have to pass your token and and that module will know who you are So it's very simple to explain and it's actually very simple to do It's it's it's a great thing Okay, I'm done. I'm done with the good. So there are a lot of bad things So I'll get started I don't know you probably know about this we seem to be version happy kind of version happy in OpenStack Basically releasing a new version of an API fixes a lot of problems. So that we have too many in my opinion And it's it's really confusing What I would like to see in the future and Ever deny and and the API working group are working on this is that we Basically changed the design of these APIs so they are Evolvable so that you as a client know that not all the things that you get from the API You will understand because the API is constantly evolving But that doesn't mean that you're gonna be broken It means that anything that you don't understand you ignore and the things that you understand are there and they're always the same So I would like that that it will work quite a ways from from this right now. So This is I put an example I said I work on heat and this this gives me heat a lot Many times and I hope if you if you write a client you don't do this This is a bad example many times you find that The catalog is incomplete the catalog doesn't have really all there is in in this in the cloud and this is a good example keys Heat needs to use keystone v3 so This is actually today in and heat Heat goes and reads the authentication uri from the config file steals it from keystone and and then it does a replace of v2 v3 Hoping that v3 will be there so this is wrong in so many levels that I don't even have time to explain and and I I'm hoping that for Liberty this is gone and I quit if it's not probably and and we in in the OSID the the start for start for a chance of deployment we have to To keep this basically we make an exception to make this work because right now if we wanted to do a full Keystone v3 deployment this will be broken Another really big thing that goes on is that the client and the servers by client I mean the Python clients are heavy coupled and the main reason for that is that There's one team for each module and that team works, you know end-to-end on the on the actual server API the the Python client and the command line client it that they're all fixed, you know at the same time So it's really inevitable that there are some Bad things going on the server that are compensated for in the client So when you come through the API and you're bypassing the client then you find these weird things So that that's very common Part of the problem is that there's an emphasis in testing all these modules through the Python client The testing through directly through the rest API is Less there's not so much coverage. So we need to improve that Remember I said that the catalog was a great implementation of hypermedia Unfortunately, when you start talking to individual servers There's no hypermedia or the hypermedia implementation is not that good so You as a client may be forced to hard-code URLs for different things You're gonna have to go look in the docs what the URLs are for different resources And then hard-code them and I have two examples here from a glance on the left and Nova on the right You can see that they hard-code URLs They don't hard-code the the catalog portion, but they hard-code the right the right part The glance one is specifically interesting even a hard-code subversion number So that code will break when when there's a version change. So not good. I mean if it's in your power, then do this These are the official clients by the way, so ideally I like to see this fixed there too, but we'll see Another thing that's fairly common and It gives me sleepless nights is the these actions concept that's so common in in so many of the API's and The rest architecture says that everything is resource-based So you talk to a resource and the actions are Predetermined the actions are part of the rest Structure you you can do a crowd-type action. So create read update and delete So either URLs represent should represent resources and here you see that some of the API's Abuse that and start putting actions in the URLs. So these are more things that you need to hard-code if you're writing an application Some are really bad that the the second one in the heat portion It's actually I mean if you can tell me what it does. I mean what was it validating? I have no idea, right? I mean you have to Go read the code or or the documentation to find out exactly what what's so what's doing? So it's also something that I hope will improve over time Remember I said that The API's have universal access sort of lied a little bit. There's a tiny lie there Browsers are special probably in a different universe not in this one. So that universal Qualification doesn't apply to browsers They have strict security requirements. So When you are running a web application that's based on say website a The JavaScript that comes from that application cannot talk to any services outside of that website a Because of security, right? They they basically want a secure environment That's called a course. It's a cross-origin request sharing So course is a method that you can use to enable the browsers to do that to talk to different services, but the The trick there is that that that third-party service in this case It would be the open stack services should need to green light that exchange So if you connect to website a for an application and then that application loads in your browser I want to talk to open stack open stack needs to say okay website a is allowed to talk to me And that that hasn't been implemented in any modules except Swift, which by the way has a very nice implementation But but none of the others have this so you can't do it So if you Google this you're gonna find some people that hacked the services to an enabled course In manually you just need to just mark with config files and stuff The method that I prefer that I always use is to use an API proxy So I put an API proxy in my website a And I send everything to that proxy and if you're interested hit me You know on Twitter or whatever and I have a proxy that I'm very happy to share. It's written in flask as you can guess so and Then I have a bunch of slides here for inconsistencies. This is a big theme In the API working group. We want things to be consistent and they're not so Pagination is one of the worst offenders in my view. I think I counted four different methods to the pagination Going from not able to so not implemented to three different implementations which I show here They're slightly different. So what when you're working with these with services you have to be aware of, you know What what implementation they have they're all different That and they also don't have hypermedia So ideally you want pagination to give you links to next and previous pages. None of that is implemented So hopefully this would improve Filtering and sorting are two other areas where Implementations vary or they're non-existent. So the experience it's it's different depending on which service you talk to Same as with pagination Metadata is a topic that we talk a lot in the API working group and here we have Most of the implementations are sort of consistent or close enough, but Swift is it's something that's abysmally different So we have to be aware of that and Another one that this is the result of the API is not getting enough testing Since this is all going through HTTP a big thing with the the interactions are the the responses that you get which have these Status codes that are from the HTTP Specification and depending on the service the same type of operation will give you a different response code So your client cannot generically You know a right code that acts depending on those responses and examples are Asynchronous requests which many many services respond with 200 which is a code for okay But the correct one for for that type of thing will be a two or two and likewise A couple other things. So, you know, it's something that you need to be aware. You're gonna have to be flexible for the time being and Support different times of responses and know how to react to them. So I'm gonna end there with the the bad stuff and now Everett is gonna tell us how We're gonna solve the problem Thanks, Miguel Right. So what are we gonna do about it? We've got some good lots of bad We need to do something about it, right? So our solution is the API working group and this is a bunch of words on a slide I realized that but go ahead and take a few seconds to to read the whole thing because it took us like a Month to put together these two sentences Because that's how we do So the gist of it is, you know, we want to improve the developer experience for those people using the API the We're really concerned with the API and up So all of those people who are actually using the API to build their applications and to operate their applications on open stack The implementation stuff and down underneath the API. That's out of scope for us We're concerned about the API and up and all of the people who are using it in one way or another and We would like to look at this holistically So that when people come to an open-stack cloud wherever it may be They're presented with a platform that actually works together as a coordinated set of services a Coherent set of services that looks like they were all developed by a group of people who are actually working together In fact, they are But the communication and the consistency between them could be much better And that's what we're here to help with and we'd also like to improve the design a lot of these APIs while we're at it Some of the design choices were made Maybe in in simple ignorance They weren't aware of the particular rest principle behind some Decision that they could have made better. So we're here to inform them of that aspect, too There's a very big educational aspect to what the API working group does and So we go ahead and we create these guidelines that live in this repo and get published as documentation on specs dot open stack dot org and We go out into the world into open stack land and you know promote convergence of all the APIs of all the different services around this set of guidelines Whether people are creating new features for an existing API or whether they're creating the next version of their API So I'm really focused on the deliverables in the API working group Actually want us to do something and not just talk about Guidelines and create them and hopefully one day somebody follows them. I want there to be some hard deliverables for us the first being Actually taking and doing some analysis of the existing APIs Going through, you know, say Nova, Cinder, Glance, Neutron And what do all of their errors look like across all of those different services? Are they the same? Where are they different? Is there any similarity at all? The answer is no So we need to improve that but we need to know where we are right now before we can do so Then we go ahead and we create the guidelines This can vary from a simple paragraph on, you know Return this status code when you're doing an asynchronous create Very simple very short to much more complex guidelines around tagging of various resources or Consistent errors across all the different servers our services Now reviews are kind of where we put the rubber on the road. This is this is the idea behind reviews So not only do we review our own guidelines within the API working group because we want to make sure we have good quality guidelines That are themselves consistent But when people actually are writing API impacting code for those various services So they're off in their own repos or own projects. They're doing their own thing all over the place But the one thing that brings it together that lets us know that they're actually writing code that will impact the API is This API impact flag that they can add to their code commits So they write in there their code commit comment API impact And then we're able to discover that review and realize that there's actually some code out there That's going to impact the API and it could be just a Specification some a plan to write some code that's going to impact the API or it's actually Code being written to impact the API So we'll get a list essentially a report in Garrett of all of these things All these reviews that are going on and we'll go out and see Basically how well That code or that spec is actually following the guidelines or if there is even a guideline to follow for that API impacting code and Then if it follows it plus one There's the carrot, right? You say plus one good job. You followed the guideline, you know have a plus one Effort have a plus one, right? If it doesn't follow the guideline Then you can say you know what there's actually a rule written about this and it's over in this repo here I can link to it very specifically and you know you should probably follow it because then you'll be consistent with all of the other services across open stack and You know you put a minus one on there because you'd like them to follow the guideline. So there's your stick So this is kind of where the the rubber ultimately meets the road and we can actually get some traction within these various projects and Because we're dealing with all of the projects across open stack ideally we do a lot of collaboration There's a lot of collaboration with the cross project liaisons These are people within those projects who are dedicated to you know integrating with the API working group and talking to us and And keeping tabs on what we're doing and then there's IRC meetings and and the mailing list etc. etc. etc many ways to communicate and Bugs question mark So what does that mean? So one one other deliverable we may have is when a Service you know an API of a service isn't following a particular guideline. We go out to that project Go to their bug page and say oh, you're not following a particular guideline That's a bug. Please fix it in the next version of your of your API Now whether that's a good idea or not. I'm not actually sure I'd love to hear feedback on that because to me that feels a little bit like declaring war and I don't know that that's a good way to get people on board and actually get them to follow your guidelines on the other hand it could be a way to inform them that their API is out of whack with the guidelines and they would actually appreciate Receiving that bug so that they can do something about it in the next version We'll see how it goes We got a lot of traction in kilo This is really just a handful of quotes from people who have been interacting with us in the API working group I know there there's a lot many more instances of this sort of thing John Garbut the Nova Ptl Discovered Miguel's guideline for tagging just as they were implementing tagging in Nova and he was Happy grateful to see it. Excellent You've created this thing I can reuse a ton of this work. You've saved me a lot of time We're gonna be more consistent across all these different services Salvatore from a neutron core wanted our feedback on some micro version stuff in neutron Rocky in the logging working group is working on creating better logging error messages Across all the different services I'm working on a guideline for improving the errors across all the different API's The key piece between those two is like sub error codes So if you've got error codes in the logs Sometimes oftentimes you're gonna want to see that same error code in your API. So there's a connection point there So we're working to actually make that consistent too so when it comes to getting an error in the API you can actually correlate that with an error in the log and Fix your problems much faster for your users improve that developer experience shorten that time to fixing problems in your cloud and This last one here is a particular instance Interest sorry for me Alex from Nova came to us and joined our group and he and Matthew Gillard have been working with J pipes on a new model for interacting with the Working group and so as they're working on the Nova API and They're doing something new or doing something different They're gonna come to the working group guidelines and see if there's something there that covers it if it's not there it's on them to actually create something to engage with us and Create a guideline for that new thing that they want to do in their API or that simply isn't covered yet in a Guideline that's already there and so we'll go ahead and review it and create a guideline for it And then use that to inform the original development that they were doing in the first place So I think this is a great model for working with all of the different Services all the different open stack projects out there in terms of how we can actually get more traction Within the API working group with the different projects, and I'd love to see more of it So this wouldn't be a open stack session without saying polar requests are accepted If you're interested in helping out if you care about the API's which is to say if you use open stack You care about the API's whether you know it or not because all of those different surfaces that Miguel talked about to the left of the API All of that stuff depends on the foundation of the API and if the API is crap It's gonna bubble up as bugs and inconsistencies and and wonky errors in the rest of those layers So believe me you you care about the API You can meet with us on IRC chat with us on the open stack development mailing list Or in the hallway here or on Twitter. However, you want to give us feedback. Give us feedback, please Actually already learned a bunch of great things just by chatting with people at the summit these first two days We have a session at 340 just upstairs if you've got access to the design summit You can follow this link to take you to the design summit sked page for it So with that, I'd like to thank you very much for your time and attention. This is Miguel. I'm Everett That's the one link you need to write down or remember if you care to follow up the rack.to good bad API's and I'm contractually Obligated to ask you or mention that we are hiring People of all natures in rack space developers You know ops people Sales marketing business ha ha like bring it on come visit us in the booth or or you can head up this link at the bottom here Thank you very much