 Okay. Hello, everyone. My name is Everett Taves. I'm a developer advocate in the Developer Relations Group at Rackspace, and I'm here today to talk to you a bit about the different programming languages that people are using to run applications on OpenStack. So first, just a little bit about myself. I still do get the question quite often, what exactly does a developer advocate do? It's kind of a nebulous title to a lot of people. So first and foremost, I am a developer. I write code for the majority of the time, meaning, you know, like 50, 60% of the time. I am a PMC, a project management committee member and committer on a project known as Apache J Clouds. We actually just recently became an Apache top-level project, and this is Java SDK for multiple clouds, actually, and it also works seamlessly with OpenStack clouds as well. So the other aspect of this job is the advocate bit, and that's the guy you see doing the song and dance before you today. I, you know, I build things and then run around the world and tell people about them and how to use them and why they're important and what the benefit is. This is me on the various Githubs and Twitters and Google Pluses and what have you. I've also done a fair amount of operations in my time. In a previous job, I was building OpenStack clouds. We actually built the first OpenStack cloud in Canada. It was a multi-region cloud with a region in Alberta and a region in Quebec attached by a high-speed research network. So to that end, I'm a co-author on the OpenStack operations guide, and this book has actually just recently been adopted by O'Reilly, and they're going to be publishing it after we do a bit more editing of it. And it's free to download at this URL here. I'll be tweeting out the link to this presentation at the end a little bit later today, so you can pick it up and find the link from there. Okay, so when I talk about using different programming languages to talk to OpenStack, what am I really talking about? I'm talking about what's usually known as SDKs, what we typically refer to as an SDK, but what really is an SDK? It's a software development kit or software development tool kit. That's what it stands for, and it's a pretty common term in the industry. If you're a developer, you've probably heard it. How many developers do actually have in the audience today? So maybe about a quarter or a third. That's great. Also, can be known as a language binding because you're talking to some API in a particular language, a language that's specifically written for that API. The language is bound to the API, hence language binding. Or sometimes it's just known as a wrapper, and these are all synonyms for the same thing, the kinds of terms you'll hear thrown about when you're talking about these sorts of things or SDKs in general. So where do SDKs actually live? Where would you actually put them in relation to your system? Where do they live in relation to the cloud? How are they used? What are the communication channels? So as we know, OpenStack is just infrastructure with an API. It's a way to access cloud resources programmatically, and in this case, the API is an HTTP restful API that you talk to over the web or over a network. So in the past, people might have just had their system and talked directly to the API, coding to the API, those restful calls, get this resource, post, a.k.a. create some other resource, and doing that directly from their programming language. That's really not an ideal situation for developers. That's not the most productive they can be with their system in relation to using resources in the cloud. What they really should be doing is using an SDK to talk to that OpenStack API because there's a lot of benefits that you get when you adopt one of these SDKs that sometimes developers aren't quite aware of until they're really far down the road in the development of their application, and they start doing that load testing and start doing those more complicated operations, and they start getting stung by certain little edge cases and use cases they might not have anticipated when they started out building their systems. SDKs alleviate a lot of those problems, and if your system is actually running within the cloud, nothing really changes. You still have your SDK as part of your system, and it's still talking to the same cloud API. No endpoints change, nothing really actually changes. Whether your system is running in the cloud or outside of the cloud, it can still talk to a cloud API, an OpenStack cloud API, the exact same way. Okay, so why would you even bother using an SDK or a language binding or wrapper or what have you? Okay, so you've gone and you've deployed OpenStack, you've installed it yourself, you've had one of the many vendors install it for you, perhaps you're using a public cloud that's powered by OpenStack. Now what? What are you going to do with this thing? You've got this great thing that everyone's talking about and everyone's really excited about, but what are you actually going to do with it? Well, of course, everybody wants to develop applications on top of it. That's really the primary use case for an OpenStack cloud is application development, and we want developers to be as productive and as efficient as possible when they're developing those applications on OpenStack. So how do SDKs help? I would say one of the primary things that they really help with is creating those requests and responses and consuming the responses. So if you're talking to an HTTP RESTful API directly, you have to build up a URL, like get some certain resource, you want to get the server details or you want to post to the server's API to create a server, then you need to create the correct headers so that your request is well-formatted. And then, finally, if you're creating a server or something, you want to put in other parameters that are in the body. So you send the request off to the cloud, and something's going to happen with the API, something's going to happen under the layers of OpenStack, and a body and headers are going to come back in a response to your system. Now, if you're just working with the direct HTTP RESTful API, that's all just text going back and forth, and that's not the kind of thing you're really used to working with in your programming language. You want to be using the objects and data structures that you're used to working with in your language. You want to be able to make a method call. You don't want to be worried about fiddling with all these different little bits and pieces to send off a request. When you get a response, you want some sort of dictionary or map structure that your algorithms are already optimized to use. So you don't have to worry about fiddling with all these different little things and getting the formats exactly right because it's actually really complicated and really finicky to get right. So the SDKs take care of all of this stuff for you. They handle the whole request and response cycle. When it comes to working with OpenStack Clouds in particular, you need to authenticate against Keystone. First thing, it's really the first thing you do every time, you authenticate against Keystone and you get a token back along with the service catalog telling you where all the different OpenStack services live that you need to work with. Now, the funny thing about that token is it's really only valid for 24 hours. It's configurable but typically the token is only valid for 24 hours and you use that token to access the different services, Nova, Glance, Swift, what have you. But if you have a long lived running process like a web server that's going to be running, should be running for days, weeks, months, years, right? Of course, that's much longer than 24 hours. So that token is going to expire. And when that token expires, the request that happens next when you're using that token will get a 401 back and that means unauthenticated. So what do you do? If you were writing the code yourself, you have to worry about, oh, I constantly have to catch this case where you know, has it been 24 hours? Do I need to re-authenticate? If you're using an SDK, that just happens under the hood of the SDK. It will transparently re-authenticate for you. So you don't even have to worry about it in your application code. And this is exactly the kind of scenario that can happen when a developer is just working along if he was working without an SDK, just doing testing. Of course, he's probably not going to be testing over 24 hours. He might not even consider that case because of just doing short lived tests. But then when you go into test or production, all of a sudden, these processes are running for longer than 24 hours and you start getting these errors about being unauthenticated. Pagination is a problem that we've had since the early days of computing science, since the early days of databases. Anytime you have more data than you can bring back in a response, you need to paginate it somehow, just as we paginate or as Google paginates the Google search results because millions are coming back. Same thing. In Swift, you can have tens of thousands, hundreds of thousands, millions of objects in your containers. Just bringing a list of those objects back is a lot of data. So you need to paginate it. And the SDKs help you consume those pages of data really easily as you need them, depending on what's necessary for your use case. If you want your users paging through the data. When it comes to starting up certain resources in open stack clouds, they're not always ready instantly. Take for instance, pun intended, a server in Nova. It's going to take anywhere from one to maybe even five minutes to get started. So you need to wait for it to be in that active state before you can actually start using it and doing something useful with it. And to find out if it's in that active state, you need to pull the API and ask, are you ready? Are you ready? Are you ready? Are you ready? Are you ready? Are you ready? Are you ready? Sorry, that was my impression of an exponential back off algorithm. So the SDKs make it really easy for you to do that. There's usually a utility method that just says, oh, wait for this resource to be in this sort of state. And it's just simple so that when your code will block until that resource is ready and you can start doing something useful with it. Now something that goes hand in hand with state polling, because you're asking, are you ready so often is rate limiting. This is something that a lot of clouds, cloud providers, whether they're public or private implement, just to prevent either abuse of their APIs or possible denial of service attacks of their APIs. So you can only send requests, so many requests per minute per hour per day. And it prevents people from overloading your API servers. So it's just like driving in the car with the kids. Are we there yet? Are we there yet? Are we there yet? Are we there yet? Are we there yet? I've had enough time out. Go to your room for 24 hours. The exact same thing happens in the cloud. If you ask, if you request, too many times you'll literally get sent to your room for 24 hours and you won't be able to actually use your account for those 24 hours, depending on, you know, what your provider's rules are. And you might have to pick up the phone and call them and ask them to re-enable your account. Errors are a fact of life. There's going to probably be network errors at some point when you're working with a cloud API. There could be a faulty switch between you and your cloud API. There could be wayward packets. Anything could cause any kind of error when you're making a request to the cloud. The trouble is you don't actually know if these errors are transient. You don't know if they're intermittent or not. It could be just the kind of thing where you need to try again because that switch comes back online or the throughput gets cleared out. So the SDKs can be configured to just retry a number of times when they get an error. If they're getting a 500 back, some sort of problem on the service side of things for the cloud provider. Well, just try again X number of times, three number of times. But if it's actually more than that, well, then there's probably a real problem and you need to kick the error up to your user so they can deal with it. The OpenStack APIs are not perfect. They're pretty good. They're very restful and they're relatively consistent. But it's those little inconsistencies between the different APIs. They've all been developed by the OpenStack developers at different times, at different periods, on different teams. So there's these little inconsistencies between the different APIs. And the SDKs can smooth over those inconsistencies. And you just wind up calling similar methods on similar APIs and getting similar results, even if there's little minor differences in the background. So it just makes it a better developer experience using an SDK. The SDKs are always packaged in whatever is the most appropriate package management system for that language. For example, in Java, it would be in Maven, so you can just add it as a dependency in your Palmfiler or whatever. Or in Python, it would be in PyPI, so you can get it via pip. Ruby, it would be a gem. So it just makes these things really easy to consume and put them in as dependencies in your system. Of course, they're open source. You always want to keep this sort of stuff open. And even the other public cloud providers open source their SDKs, because there's no actual, you know, real monetary value within the SDK itself. It's the service that's providing the value, and it's the service that you're paying for, not the SDK itself. So it always pays to have an SDK as open source. So if there is a problem, the developers can dive right into the code and see exactly what's going on under the hood. And typically, when you get, you know, you have an open source project, whether it's an SDK or open stack itself, you get this great community along with it. In particular, I know that with my work in the J Clouds community has been really positive from the very start. It was founded a long time ago, about four years ago, and it's just been great working with everybody there. And it's really interesting to see the people, the new people coming into the community, trying new things and getting help from other people who have tried those same new things. Or they're just doing something totally different. And, you know, they want to go in a completely different direction. And it just helps feed into the community these different use cases that people come along with and these new approaches that they're trying. And it just winds up being a really strong and interesting and vibrant communities around these open source SDKs. Typically, the SDKs are all in one. So, by that, I mean, you know, you can use any of the services available in your open stack clouds. So, you know, if you want to start using, you're using Nova now, but oh, now your provider has deployed Swift. While you don't need to download a new SDK or anything like that, it's already part of the one that you're already using. And you just start calling the methods for those particular APIs. Because we're working with open stack, you're not getting locked into any particular proprietary cloud provider. And in some of the cases, these SDKs are multi-cloud SDKs, meaning they actually work across multiple different either public or private cloud providers. So, you can avoid lock-in. It's not a silver bullet coding to these, you know, abstractions, but it certainly eases your transition when you're trying to achieve application portability and not getting locked into a particular vendor. So, you know, what do you get when you actually adopt one of these SDKs? You get production ready code. Like I said, a lot of these SDKs have been around for a long time and they've been, you know, kicked and beaten and stomped on and just used in every conceivable way and, you know, lots of bugs have been filed, lots of bugs have been fixed. So, they're really ready to go. You get example code. I find that an example can be worth a million lines of code. You know, if you're trying to do something either, you know, just kind of basic in the cloud and you're just getting started, that example that shows you the proper path can be totally invaluable and get you going very quickly. Or if you're trying to, you know, do something a little more complicated, there might actually be an idiom within that SDK, within that language that, you know, makes it easy, but discovering that idiom on your own can be quite difficult. So, the example code can show you the right way and get you moving along a lot quicker. And we do believe in documentation. You know, it's not always the first thing on everyone's mind, but we always make sure that part of what it means to have a feature complete in the SDKs is documentation. So, people can either get started quickly or they can get that detailed reference documentation for the SDK so they know what kind of values they need to feed to the different parameters when you're, you know, trying to create a server or put an object into Swift. Okay. Thank you for sticking with me for so long. This is going well. Let's take a little bad slide break here. This is just plain awful. You know, video in a presentation is never good. Blinking text is an absolute no-no. Old memes are a joke. Text too small and obscured. It's awful. Yeah. That's me on a particularly snowy day in Alberta. It was a lot of fun. Unfortunately, you can't hear the laughter of my children at my stupidity here. Okay. Anyway. So, let's start getting a little more, a little more detailed and focused about what we want to do with the OpenStack SDKs. Let's actually define some criteria about, you know, what makes, what really makes up an SDK and let's set a bar for quality for OpenStack SDKs. Some criteria that will help us choose what's an appropriate OpenStack SDK. So, you know, going back to what I was saying before, we obviously need that language binding. It's got to be written in Java or Ruby or Python or whatever to talk to the OpenStack RESTful HTTP API. So, of course, you need the language binding. You need a getting started guide. I mean, this is something that's absolutely crucial to developers. I know that when I go to start consuming any sort of web service, whether it's a cloud API or, you know, an SMS API or what have you, I always look for those keywords, either getting started or quick start. And that's the first thing I click on, because I want to get going in five minutes. I want to kick the tires on this thing right away and see if it can do what I have in my head that I'm expecting it to do. So, if I can try it quickly, that's absolutely crucial. And so, we need to provide that experience for developers who are building applications on OpenStack. It needs that detailed reference documentation. So, when you're doing the more complicated operations in the cloud, you have a guide. You have a reference that you can, you know, get into and find out exactly what it is you need to do. Example code. I explained the value of it earlier. And, you know, OpenStack itself is Apache License V2. So, it's kind of a bit of a question whether or not the OpenStack SEKs that talk to OpenStack should be Apache License V2 or just V2 compatible. So, that was a bit of an open question still. So, this criteria is actually detailed here on the OpenStack Wiki. And this is work that myself and some others did in previous design summits coming up with this criteria. You know, we're working towards having a defined set of OpenStack SDKs that developers can use to get started with. So, what are the OpenStack SDKs? What am I actually talking about? Well, on that Wiki, there's also a list of the known SDKs. So, using the criteria as kind of a filter for the known SDKs, what are we considering adopting as the OpenStack SDKs? For Java, J Clouds meets the criteria pretty well. I think all of the SDKs that I'm about to describe aren't 100% meeting the criteria, including kind of the default Python SDKs for OpenStack. But we need to start somewhere. So, for Java, J Clouds does a great job of meeting most of the criteria. For Node.js, packaged cloud is a good front runner. For PHP, there's PHP OpenCloud. This is SDK that was developed by Rackspace because there wasn't really a strong OpenStack SDK in the PHP community. So, this is one that we started ourselves. In the Python world, there's the default Python-star clients. Now, these are actually the command line interfaces also for OpenStack, and they can be used as SDKs. There's also another Python SDK that's going to be known as PyStack coming out, for Ruby. The whole Ruby community seems to have coalesced around Fog. This is another one of those multi-cloud libraries that can work across multiple public and private cloud providers. For .NET, again, this was an area that Rackspace saw where there was a bit of a gap, in fact, a gaping hole. There was no real .NET SDK for OpenStack, so we started OpenStack .NET. Another very strong up-and-coming language is Google's Go. More and more systems are starting to be built in it. I think it still has a way to go before it has full mainstream acceptance, but it'll probably eventually get there. So, again, there wasn't really one for OpenStack yet, so Rackspace has initiated another Go SDK that's going to be known as Go for Cloud, and it's going to be planned to be a multi-cloud SDK as well. Ultimately, where these things are going to not actually live themselves, because they're all separate open source projects around in the OpenStack ecosystem, but we want to bring it together in one place where developers can find everything that they need to get started with OpenStack in one place, and so we want to bring developer.openstack.org into existence, and this was a design session I led earlier in the week with Tom Feifield, one of the OpenStack Foundation members. So hopefully we'll be able to get this off the ground and just give developers a starting point for when they're working with an OpenStack Cloud. So once you've got these OpenStack SDKs, developers are going to start building really cool stuff, and a lot of that cool stuff is for other developers or other operations people. So you start getting this ecosystem of tooling around the SDKs for the OpenStack ecosystem, and for example, there's Chef, which uses Ruby Fog, Puppet, which also uses Fog. Ansible uses PyStack. Vagrant, I believe, uses Fog. Jinkins can use J Clouds to start up build slaves on different clouds. Word is kind of similar to Chef or Puppet in the Java world. It can use J Clouds. There's this PHP plugin for WordPress that will back up your WordPress site to a Swift provider, and many, many, many, many, many more tools built on top of these SDKs. Creating a whole ecosystem. Okay, so now I'm going to do the standard patches accepted pitch. I'm certainly encouraging everyone to use an SDK. I think that's pretty clear. That's pretty obvious. But I'd also like to encourage the providers and the vendors in the OpenStack ecosystem to contribute to SDKs as well. Because really, application developers are expecting SDKs when it comes to talking to an API. This is SDKs are an absolute base level of functionality when it comes to talking to web service APIs like an OpenStack Cloud API. For example, I mean AWS already provides SDKs in all of these languages. They obviously didn't start all of them at the same time and have them all available, but now that they're pretty mature, they're all there. Likewise, Microsoft provides SDKs for all of these languages. Google, their Compute Engine, provides SDKs for all of these languages. So very much, these are expected by developers. And developers who are building applications on top of OpenStack should expect them to. So it's all of our responsibility to contribute to these SDKs. Sometimes the OpenStack contributing companies can get a little obsessed with commits to github.com slash OpenStack. And it's perfectly understandable. I mean, it's their bread and butter. And they also get those lovely stats about the commits. And that's good. And I understand it. But I think OpenStack has matured to the point where the OpenStack contributing companies need to start thinking about commits to other repos around the world about the wider OpenStack ecosystem. And I get that developer time is certainly finite. I'm painfully aware of that. But really, we're in service of the application developers. I mean, that's ultimately what OpenStack is about, is increasing developer velocity so we can build stuff. And so it also enables DevOps and automating application deployment. So Rackspace has been working really hard at this stuff. We started our developer relations group a little over a year ago. And we've been working on these SDKs. We've put a lot of code behind it. And we're reaching out to developers in all different ways and manners. Conferences, meetups, forums, Stack Overflow, Twitter, IRC, just anywhere developers are, we want to reach out to them and help them work with OpenStack. And of course, Rackspace. We've seen HP do a lot of great contributions to both J Clouds and RubyFog. And we're looking forward to further contributions from them on those SDKs. But after that, things kind of trail off. And I haven't seen, at least for the SDKs that are on the Wiki there, if you look at the GitHub commits, I haven't seen a lot of participation from any of the other companies. And like I said, I understand this. Developer time is finite. And it's tough to commit resources to every open source project. But again, coming back to developer expectations, the OpenStack contributing companies need to understand that SDKs are an absolute base level of functionality for OpenStack Clouds. So not to really call out the different companies, but just to give some examples of where it would be great to see some contributions. Or even just saying that these companies are adopting an SDK of any kind to help their developers access their Clouds or the Clouds that they've built for their clients. You know, it'd be great to see Dreamhost adopt any of these SDKs for accessing their OpenStack compute and storage Clouds. It'd be great to see IBM throw their weight behind some of these various SDKs. Canonical loves getting into the mix and most any OpenSource project, likewise with Red Hat or Suze. So all of the distributions would also benefit from contributing to SDKs because, like I was saying before, once you've deployed OpenStack, so what? Now what do you do with it? Well, you want to develop applications on it and you need SDKs to do that. And again, it all comes back to the application developers. And we're in service of those developers. And OpenStack has matured to the point where we need to start thinking about the application developers first and giving them the tools they need to be productive and efficient in the Cloud. So the whole point of the presentation here is simply to please use or contribute to one of the SDKs. Thank you very much. My name is Everett with Rackspace. And again, thank you for your time and attention. There's a few minutes for questions here, if anyone has any. Or afterwards, it's fine too. Yep. Is there a difference between using SDKs for an external project or use Python SDKs for contribute to OpenStack? I mean... Yeah. And what do you mean? Give me an example of an external project. I mean, do OpenStack contributors use SDKs to communicate with other OpenStack projects instead of directly call an API? Instead of directly calling the API. So I mean, we need to make the distinction between application developers. So are we talking about application developers who are building solutions on top of OpenStack and only consuming the API? Are we talking about OpenStack developers who are actually building OpenStack itself? Yeah. Which one are we talking about? It is the same kind of SDK use. Oh, sure. Okay. Yeah. Okay. That's fair. So the short answer is no. Application developers, using the standard default Python client SDKs, those would be used by application developers or by operations people who are writing scripts because it's also the command line interface to OpenStack. So it would be application developers or operations people or DevOps people who are using those SDK slash command line interfaces. The OpenStack developers who are building OpenStack itself, if they're actually in the server side code, then they're typically going to be making calls using the RPC mechanism within OpenStack. And that would hopefully be via Oslo the OpenStack common library. So there is that difference between the two. Although the Python client libraries are a repo under github.com slash OpenStack, they're client libraries as opposed to the OpenStack server code. Does that answer your question? Yeah. No. Okay. Great. Thank you. Okay. I guess my question is what are the most widely used? So I'm on the OpenStack wiki looking at the different SDKs. What are the most widely used ones? My product has a pearl bundled with it. So I'm looking at the pearl one. But if I was to start with something, what would you suggest? The cross-cloud ones, maybe? Yeah. So without getting into like real specific details, one of the things that as a public cloud provider, Rackspace can do is actually analyze what's using our cloud, essentially. The clients that are using our cloud, we examine the user agent. It's one of the headers included along with the request. And so we examine that. And we found that the absolute number one most popular SDK is PHP. PHP is huge. It is still used all over the web by a ton of developers. I mean, Facebook is written in PHP. Second is probably RubyFog. And third is JavaJ Clouds. So those would be the top three that we've seen in our experience as the top SDKs. So a quick follow-up on that. I see two for PHP. PHP dash OpenCloud and Zen service OpenStack. Yeah. So that Zen service OpenStack was one that we stumbled across earlier. And so the list on the wiki there is kind of a full list. It hasn't really been filtered yet. I don't really know what kind of shape that the Zen OpenStack one is. So you're talking about PHP OpenCloud. I'm talking about PHP OpenCloud. Okay. Cool. Thank you. You're welcome. My question is how J Cloud can be applied in OpenStack? So is there any example for it? How can Cloud... Yeah. If I want to use how I can do... Oh, how you can use J Clouds? Yeah. For OpenStack. Okay. Yeah. So I mean, do you have an OpenStack Cloud yourself on premise? Okay. So I would encourage you to go to jclouds.apache.org. And at the top, there'll be a link that says providers. And by providers, we mean what are the different cloud providers that work with J Clouds? If you click on that, you'll get a list of all the providers that J Clouds works with. One of them is OpenStack. So if you click on OpenStack there, it'll show you a couple of quick examples on how you can get started using J Clouds with OpenStack. And if you have any problems, come visit us on IRC. We're hash J Clouds on FreeNode. If you just search for J Clouds, you'll see how you can... Again, you'll hit the jclouds.apache.org. There's a community link. You can join our mailing list. And we're happy to help people getting started with it. You're welcome. I want to know if there is a guide to how to organize our source code. I think specifically to the automatic test, when I tried to implement an application and I didn't find help... Sorry, it's in... English is not easy for me. I think about... I see that there is automatic test with stocks, with tempest, with a lot of things. I don't know if there is a recommendation to be a two-auto test for the product. So automated tests for the product in relation to a particular SDK. For Python. Oh, for Python. Yeah, so I believe that... I'm not actually familiar with tempest. That's what you were referring to, right? Or whatever, just something for testing clouds. Currently, I have an application. I need to do auto test and with Jenkins. And currently, I don't know what is easier way for me to start. Okay. So you want to automate your testing. You want to start doing continuous integration. Okay, great. So Jenkins is a great place to start. You need your unit test to eat. That's job number one in your code, is have it all unit tested. And then adopting a tool like Jenkins is a great place to start. You can use, and I'm just about out of time here, you can use... There's a J Clouds plugin, actually, for Jenkins that can start up Jenkins slaves on any clouds, including OpenStack clouds. But really, that might be something you need later. Yeah. Just use Jenkins or Jenkins as a service, like CloudBees. If you don't want to run your own Jenkins stuff, just as a way to start learning about it and kind of figuring it out. Does that help? Maybe we can talk after, sure. Okay. So I guess we're out of time here. I can, if you want to ask a question afterwards, that'd be great. Again, thank you very much.