 Yeah, I'm not sure if that's the right home for this, but I guess mostly what I wanted to talk about was, you know, where OpenStack can fit into development tool chains, operations tool chains, where do all of the projects that Rax, sorry, OpenStack has now, where do they fall, how can you use them, things along those lines. So the first thing I really wanted to start out with was just the sort of picture of like where OpenStack was and where it's heading. Initially, we started out with these, the five main projects, right? Everyone knows about these, and these honestly still get talked about the most. These are, you know, computer as a service, object storage, image. But where, what's been dominating a lot of the conversations the last year or so has been really all of these services, right? So this is getting away from the core OpenStack into more of the cloud service model, right? They mentioned some of these up here before us, this database is a service, this is networking is a service, this is bare metal, cloud queues, I mean, there's all kinds of services now coming out of OpenStack. And this really isn't where it ends either. There are now a number of, you know, there's more than this, I couldn't even get all of them on here. I just don't even know all of them, to be honest, but there's a number of incubated projects or projects that people are trying to kick off right now, like the Solom one, which is around CI CD and application delivery, or the Taskflow one for doing workflows against OpenStack. And then you've got all of the config management systems on the other side, which not only are being used to build OpenStack, right, to actually build and run OpenStack. They're also being consumed on top of OpenStack, and they're providing the provisioning layer for OpenStack, for instances on OpenStack. And in case of some of them, like Ansible, they've also got orchestration in there as well. And to be honest, there's probably another ring on the outside of this, like random tools that people have built, right? So like StackTac, RackSpace uses pretty heavily, and the guys from PayPal converted Asgard from Netflix, OSS, into an OpenStack version, right? So there's all these tools that companies are building, and they're open sourced, and they're out there, kind of hard to find, but they're there. So kind of where I wanted to go today was taking a look at some of those projects and figuring out where they fit into your tool chains as a developer, as an operations person. So let's start out with defining DevOps and the tool chains, at least for the scope of this talk. Review of OpenStack practices, so perhaps OpenStack won't directly help you in one way, but maybe you can follow a practice of OpenStack. How OpenStack could help you if it provides a specific service, and what's missing, which I'd really be interested if anyone has input on that part. So to get started, I'm going to loosely define the term DevOps. This is not a DevOps talk. I'm not going to try and define that for you. But in the scope of this talk, it is the tool chains of dev development text, the tool chains of operations text, they're changing, right? They're changing along these lines of where DevOps is taking them. So I try to find some good, like what is DevOps? And I try to throw together some of these things. I found these on Google, Google Image Search. But trying to take this idea of a developer and assist admin and turning them into Superman, right? Everyone seems to think that DevOps is the answer. In reality, it's not, but what it is, is evolving their tool chains, right? So for developers, maybe this is taking on some traditional operational tasks, like being on call for application deployments, right? Maybe that's a new task that developers aren't used to. Or maybe developers are getting involved in application deployment through something like Fabric, Capistrano, or any of the other ones you could possibly use. Or quite possibly they're getting involved in the infrastructure's code piece. So operations, maybe you're using a big management system. And developers are actually getting involved in the infrastructure. For operations, maybe the evolution for them is actually starting to adhere to development principles. So along the same lines with something like Chef or Puppet is treating infrastructure as code and committing that code to source control. Testing that code, just like you would expect that your developer would do against your live production application. Because essentially, your infrastructure is like an application. Again, that's committing the source control for like your big management. This might be new tooling, right? I think Netflix is probably the poster child of this. And even though they're not OpenStack, they're the best at sort of, they've home written a ton of tools for working with Amazon, AWS, and EC2. And they've open sourced them because they needed these tools. So maybe part of what operations is evolving to is developing internal tools, and maybe operations are getting into application performance. Before, maybe they only cared about infrastructure performance. What does my CPU load look like? Which is kind of starting to become less important in application response time. Application performance is becoming a little more of a goal. And of course, there is the business evolution as well, which is breaking down the barriers between the two teams. All right, so let's start defining the dev tool chain a little bit. And let's see where OpenStack fits in. You don't have to forgive my simplistic keystone boxes, but they work. So to start with, there's obviously the develop phase. You're going to be working on application. You're going to be developing against it, which will generally lead to some sort of testing, unit testing, integration testing, smoke testing. That's sort of a cycle back and forth until you have something you're happy with, right? This may move to the commit phase, commit your code, get SPN, whatever it might be you're storing it, which will generally actually kick off another testing run, right? Like you're going to test your production code. At some point, you'll reach a build stage, right? You'll either package your code up, maybe you'll tar it up, maybe you'll build an RPM or a Debian out of it. Or if you're like OpenStack, you'll just, well, actually they use tar. But they'll ship it to GitHub and let people download it. You may artifact it. You may place it in some central place where all of your app servers can reach it. And finally, you'll deploy that code, right? And then once that code is up and running, it will generally have some sort of application monitoring after that fact and followed by issue tracking. This is obviously a pretty simplistic view of a development tool chain. It's going to change for every company. People are going to do deployment completely different. Some people are going to follow the formation model or excuse me, the template model was something like CloudFormation, Cloud Foundry, Heat. I mean, you can kind of name those off. Some people aren't going to do artifact or even building. Hopefully everyone is committing their code. That's probably something everyone's doing. So how can we follow OpenStack, right? So how can we think about how OpenStack does things and what OpenStack is providing us? So I'm marking these boxes with a star if we can follow an OpenStack methodology, right? And I'll mark the boxes with the little o if OpenStack's providing something directly for you. So a star, OpenStack's not going to help you develop, right? Necessarily, OpenStack's not going to write code for you. OpenStack's not going to do any of that. But you could follow OpenStack's methodology. They have a very, very tight Python standard, right? Every project is Python. Every project must go through. Every commit must go through PEP 8, test, right? Lint test to make sure it fits. They have very, very strong Python standards. Maybe you can follow that model. But what OpenStack will provide you are language bindings, right? Actually, we talked about this before and there was a question. OpenStack may not directly provide that, but things like Fog, things like J Clouds, those are going to help you integrate with OpenStack. And that directly affects your development. That directly affects your application. So for testing, again, OpenStack's not going to necessarily write your test for you. It's probably, I'm not sure if testing as a service is ever going to become a thing. Based on the last six months, it probably will. Someone will come up with testing as a service for OpenStack, I'm sure. But you could follow OpenStack's methodology, which is all their tests are in talks, at least currently they are. But there's also a new project that's been spun up recently called OpenCafe. And that does fall under the StackForge. It is on GitHub under StackForge. It is not an actual OpenStack project, but it has been built around teams using it for OpenStack. I think it was Rackspace that did it, don't quote me on that. So there is a project under StackForge to help you with testing, right? It is not testing as a service, it is not TravisCI for OpenStack. But you may be able to use it for your company. As far as the commit workflow goes, again, OpenStack's not directly going to help you there, but they do follow a pretty strong Garrett gate before you can get in, before you can commit code into it, right? So it's more than just code, commit, and deploy. They're code, try and commit, right? Get code reviewed, make sure people are happy with it before it actually gets pushed in. So OpenStack's not going to help you there, but they have a pretty good methodology that you could follow as well. As far as building your code, OpenStack follows the TAR model. Essentially, they TAR up their package and they throw it on Launchpad. If you want it to go that way, you can. I presume most people don't, but you certainly could. As far as building and artifacting go, it's really been up to the OS distros, right? It's been up to Ubuntu and Red Hat to kind of do that for OpenStack. But TAR is certainly a build methodology. As far as artifacting goes, OpenStack essentially follows the Launchpad. It places that TAR file on Launchpad. That is your central artifact repository. So lastly, at least with the direct tool chain would be like deployment. OpenStack will, however, help you with application deployment. This is, there's a couple of things going on right here, but right now around deployment, Heat's probably the most well known. It is essentially the orchestration layer for OpenStack. It's probably coming of age about now. I think people are going to start actually using it and deploying it. But there's some new efforts underway, one of which would be Solum, which is more focused around container-based application deployment. As well as integration with CI and CD tools. That's been one of my biggest complaints so far, which we'll talk about later. Is not having good integration with the CI servers like Jenkins and the other ones out there. So there are, I would say two, there are two OpenStack projects to help with application deployment right now. Solum being very, very, very young and I think was pitched like a month ago. So I don't know if I'd use that in production yet. As far as application monitor goes, OpenStack's not going to help you there yet. But Solometer exists. You can look at Solombers code, look at how they're monitoring essentially OpenStack itself, and you might be able to use that methodology in your application. And issue tracking, they follow the launch pad. They use launch pad pretty heavily. So that's what I would say how OpenStack may, may not help your development tool chain. There's only four direct places where I feel like it's going to help you. But you could follow the methodology everywhere if you really wanted to. So what about Ops? Ops is a little less of a perfect flow. It's more of kind of like discreet, discreet elements of a tool chain. So these would be like inventory, what machines do I have? What images am I using, right? Am I using some sort of config management to build my images? Obviously application deployment is, traditionally it was an operations job. Remote execution, once I have machines up and running and I want them to do something, something different. Remote execution orchestration, perhaps I am following the template orchestrated model, monitoring both application and infrastructure monitoring. Issue tracking and identity. So controlling who has access to what, right? So first up with inventory, I would start with saying OpenStack does provide you that. OpenStack provides a nova API. OpenStack provides you an inventory of your machines that are running. That you can get via the API, you can get via CLI, you can get via the dashboard if you wanted to. OpenStack does provide that for you. It also does that for images. It does provide the glance API to tell me what images I have, how I'm gonna boot them, it doesn't necessarily provide you building images. But there are a lot of tools out there to do so and there's a lot of instruction. For config management, OpenStack isn't gonna do that for you. That's being left up to some of the other providers like Chef and Puppet, but Chef and Puppet are both heavily used for deploying OpenStack. So if you wanted to figure out how to use Chef or Puppet in some really advanced ways, maybe have a look at the Puppet manifest for OpenStack or the Chef cookbooks for OpenStack. They're on Stackforge, they're all there. Stackforge, by the way, is github.com slash Stackforge. It seems to be the place where people are just putting OpenStack related projects or code that could be useful for multiple people in the community. As far as deployments go, we mentioned heat before. As far as if you want to get into template-based orchestration, OpenStack will provide that for you as a service and Solom, again, is getting into this container-based application deployment. It skipped over remote execution because OpenStack doesn't, it's not going to provide that for you. Orchestration, heat does provide that. As far as monitoring goes, it can provide some infrastructure monitoring from the OpenStack side, not necessarily within the guest level, but within OpenStack from the hypervisor's point of view. Issue tracking, again, they use Launchpad and identity. They do provide the identity API and integration with LDAP systems. So what's missing? I would start off by saying OpenStack is missing AWS level integration. So I don't necessarily mean AWS compatibility, although there's certainly people in the community who think it should be. But I mean AWS level integration. When I go look at Chef Cookbook on just some generic Chef Cookbook, there's usually an AWS recipe for dealing with block storage. I feel like we're missing that with OpenStack right now. Or if I go to website XYZ, whatever it might be, generally speaking, I can hook it to my AWS account and use that as an inventory system. And I feel like we're missing that with OpenStack right now. AWS integration with a bunch of third party tools and sites. OpenStack inventory integration, I feel like it's missing. There's a lot of, this kind of goes in with AWS level one. But there's a lot of internal tools who can't talk to OpenStack to determine how many machines I have, anything along those lines. OpenStack is really trying to beat everything. But a lot of people have IPAM internally that they want to continue to use that cannot talk to OpenStack. So CICD tools integration was a big one. Jenkins, for instance, doesn't know how to talk to OpenStack, right? There's not a Jenkins plugin to talk to OpenStack. I can't say build, test this, build an instance on OpenStack and run this test code and then destroy that instance when you're done. I can't do that with some chef stuff, but I don't really know of anyone else that's doing that right now. That's what Solom, part of what Solom is trying to fix is integrating with some of the CICD tools out there right now. As well as provide the application delivery. And then workflow, I would say is honestly probably one of the biggest things missing as well. Think of Amazon's simple workflow service or something along those lines. If I want to be able to say, spin up an instance, download this code from Swift, or download these files, process it, and then destroy everything when I'm done. I can't really do that with OpenStack right now. That said, there is a project underway called Taskflow to get this done, right? So I think that the CICD tools and the workflow were recognized as problems and people are trying to figure it out. And that's pretty much it. That's what I got. If you want to contact me, you can rack Ninja on Twitter. My email address is there and these will go up on SlideShare after this. If anyone's interested. And then if you do look at the slides online, there's links for some of these resources, some of the things we're talking about. So there's plenty of time left, anyone has any questions, go for it. Or comments. I'd love to know what anyone thinks OpenStack is missing. If we start an open discussion on that. Yep, I don't know, I think he's going to try and bring a mic. Okay, thank you. My name is Hasher from Microsoft. In regards to the last slide that you presented, the missing CICD topic. I want to understand if there is any initiative or any venue that I can go to to understand, for example. Some of the standard practices that might be in existence around building such the CI infrastructure for OpenStack. That's something that our team at Microsoft is working on right now. And I've been attending some of these test tracks that are offered at this summit. I wanted to understand if there was a group of individuals or if there's an internal team that I can refer to with such questions. Like within OpenStack? Within OpenStack. I think that there is, unfortunately I don't know who they are. But I believe that there's a talk at some point this week talking about their testing infrastructure. I have seen some mention of it that talks about the entire workflow. I get to Garrett, the Jenkins infrastructure. I'd be very interested as well. Yeah. Okay, thank you. Hi, sorry. This is Lakshmi from IBM. I'm wondering where would you place in this list tools that would help say rolling upgrades or upgrades of an application or some canary testing, things like that. As far as, so can you say again, I heard upgrade of applications. What else was there? Also some kind of, yeah, let's say rolling up, rolling up data for an application. That's a typical like DevOps kind of scenario where I want to continuously keep the application updated. Yeah, so should OpenStack help with that? Or? No, so my question is here, do you think it's covered in one of those tools or? I would personally probably consider that a workflow. Workflow, okay. Right, but that's a workflow at a slightly higher level than what OpenStack may be involved in. So maybe that's a really combination of how you're deploying it plus a workflow engine, right? Application upgrades to me or is some sort of workflow, right? Do something, spin up new instances maybe, grab my new code, okay? Everything looks good, out of floating IP, whatever that workflow, it's multiple steps. So I would consider workflow being the primary thing that's missing. But if you're using something like heat to deploy your infrastructure and application, all of a sudden workflow is not really missing, right? If you're using a template engine to deploy all of it, redeploy, point the IPs to the new one, and you're up and going, right? It's that easy. Okay, thank you. Any other thoughts, questions? I just wanted to back. My name is Adarsh. Does OpenStack has anything recording closed loop DevOps? What's that? Closed loop, feedback from your mechanism from the production back to the developer. Does OpenStack, I would say OpenStack at its core does. There's a number of people developing against it. When something breaks, essentially new bugs get opened and people work on it. I think because it's an open source project and a massive one at that, it's probably not as tight of integration as you would see inside of your own company. But I don't know if you can follow their methodologies, right? Open source projects are a completely different animal. You might be able to follow the way OpenStack does it, but it's probably a little too slow. And possibly a little too loose to really follow that. I wouldn't say that there is a loop that you could follow from OpenStack. Unless you follow the standard sort of like commit, deploy, bug, fix bug, release in six months type of thing. Anything else? Questions, comments? Cool, well I'll let you guys go. I think they're probably a beer or something at the other place by now. So I'm not going to keep you guys too long.