 Good. Yeah Hello, everyone Sorry for being late a little mishap in my schedule So I'm here today to talk to you about Using open-stack as a software factory I'm here with Medi Abakouk. He's one of our software engineer. He's the guy really doing the work I'll be the one presenting today of because he's telling me that my English is better However, if you have any questions, I'm sure you'll be the guy answering them Okay so very briefly We both work for in advance in advance is a Small company that has been in the open-stack area for the past two years that has existed for five years Recently we grew when I arrived at in advance. We were about 50 people Today less sorry 20 people and a year later. We are now 100 people We have an office in Paris. That's our headquarter We've got an office in Montreal and we are opening offices in the US In Asia a couple once in Bangalore and Singapore soon. So the reason why I'm on stage today To state the obvious Open-stack make developers envious and dubious Why that? Why is open-stack? such a Projects that when I talked to in house developers To tell me how do you guys do it? How can open-stack? handle 400 commuters per month and More than that. I think like little statistics 405 Developers are coming from 250 companies have More than 10 integrated projects and yet Be able to release a new version every six months that For the last version add close to 400 new features or blueprints without any major inconsistencies That's the big question that people are asking How do you guys do it because most of the time I talk to these in house developers? they have a big team of about 50 people and They don't know how to Consistently merge their work They spend month in their development cycle just merging their work and all of this in open-stack Is done with a single product manager release manager is right here Tiri carries and apparently this guy still has time to play ping-pong. So how does this happen and? Tiri would be in a very position than I have to explain to you the detail of this But the answer really lays in the process In the process, but not only that also in the fact that Without having a formal TDD Environment it is absolutely required for any patches that enter open-stack to come together With its unit test and it's becoming more and more recommended for new functionality to come with its functional tests and together this process with the ability to Continuously test what is being added as not breaking what has been done before Which can be summarized in this little diagram here? is really the key to Being able to handle so many developers so many companies contributing to so many projects yet Make sure that at every step of the process nothing breaks So if we detail the the process we see that Well anything that you will do in open-stack start with either a blueprint. That's for a new feature or a bug report that's to Fix something that is not behaving the way you want and generally you're going to be Getting the code locally on your machine Coding away your fix or your new functionality Do a quick test on your local machine and as soon You're satisfied with it. You're going to push it to Garrett So what does Garrett do the first thing Garrett does? Even before anybody else reviews your code. It's going to be talking to Jenkins They is going to perform the full test suite Not only against your patch But against the wall project So that if your patch Introduces a regression anywhere in the project you'll know immediately Immediately there is a little guy named Jenkins that is going to come and put a minus one on your contribution if that passes then Your peers are going to be able to review your code and eventually core reviewers are going to be able to approve your code and that's when Jenkins enter again Reduce a test. Why do we reduce a test after the approval because there can be Some period of time that occurred between your first patch submission and the approval and During that time what could have happened some other people in some area other areas of the project could have contributed other features That's only you're breaking with your patch so we don't want to merge something that is Breaking that and if that's worked then it is merged and when it is merged What's happening the status of your bug report of your your the initial blueprint is going to get updated and Let's say that we've got now multiple sets of patches at some point The ptl together with a release manager decides to do a release to just have to do a tag in get and that Automagically builds a release. So this process together With the test-driven implementation Is making open sack possible one of the great thing about this process is that everything Regarding the process itself is in get so that mean that the description the code The infrastructure the deployment of the infrastructure didn't needed to test your patch The everything that you that is possibly needed is treated as go as code in get meaning that technically anyone can improve that infrastructure when they think there is a failure and that happens all the time on the open stack project and Overall, it's really 100 time better than having to write your documentation your how to compile your project build your project to be able to test it locally because in every software houses where we had such a documentation And we were forced to have such a documentation By the time I was using the documentation the process are already changed and the how to we followed step by step Would never be up to date This doesn't happen on open stack. We do have some failures Sometimes there is there are race conditions But we've got a few infrastructure guys that are there to help us figure that out and fix them quite fast So in the end we at in advance found this process so good that we felt that we had to share that process outside of the open stack project and In order to share it. Well, we're a business we built a solution We we are offering a solution to customer to reproduce this process internally So what what is it that we offer? Well, if you look at it from far away enough We offer a way to deploy an open stack cloud Which is going to be quite key because this open stack loud is going to be what is running going to be running all your tests and We deploy on top of it as well The various tools that compose as a software factory So we are talking here about get Garrett Jenkins mainly there are quite a few others Then you connect it to wherever You maintain your bugs wherever you maintain your blueprint that varies Then you create a new project and you can immediately start coding so Why is it so great? Well, first of all, it's a great way to improve the efficiency of your development team there is a great value in Having everything that is being done in your development team being stored in a central location if you share stuff you're making sure That everybody can access it You're making sure that you can set Access rights and you can control these access rights to limit visibility if you are working on some sensitive matters You're making sure that you're through this tool instigating a contributing model inside of your company and One of the things that I've learned is We are all a lot better when we collaborate when we co-contribute to something There is no way a single man today can know everything About any environment and it's only through the exchange that we have one with the other that we are able to build better things and This kind of process Really contributes to it because it provides the infrastructure needed to have easy and fast collaboration Another game from it is that you won't need a full release team I've seen large projects use up to 10 people in the release team the build management team generally it's called and These guys generally have no hair after doing this job for about three months and Another benefit is the fact that developers are Well like most computer users like a gas They always want to expand more and They will always find a good reason why they need more hardware to test what they're developing The great thing is because we're using open stack as a basis for this We have a mean to factorize that hardware across the development team So instead of having to buy Let's be excessive 10 machine per developer You can have I don't know 20 machine for the world team that are going to be sharing But the main thing is Let's put the team to work as soon as possible as soon as you start a project You don't have to wait for ages to it for everybody to understand how the project works You can get to it very fast Another great thing about this environment is we can extend it a little bit to manage reproducible environment and When you're not in the open source business Very often you have to custom tailor your environments for a given customer So let's say that you've made a special release of your great software that I don't know Prince Bills For your big customer in India This customer is going to be working with his version for quite a long time a year maybe two maybe three and And three years later He hasn't upgraded but yet he calls you With a bug do you think you will have maintained an environment that he exactly the same as your customer That you can where you can reproduce the bug? Not really the great thing is that By using reproducible environments Because everything is stored as code we can rebuild this in Environment the specific environment in the exact binary state it was three years ago Through a simple click you just ask to restore at a given point in your industry of your project You also have a mean for developers to Generate a test and a debug environment using whatever release they want at any point in time and Of course, you're sharing resources so that with Jenkins that is also going to be launching new environments to perform the test on demand another Great area is the ability we have to parallelize the work very often You have multiple option in a project and that's a case in open stack You've got cases where you want to test with KVM other cases where you want to test with said and if you had to Rebuild the full hardware environment Manually to do the test this would take ages The great way is the great thing is that with open stack you Can create virtual environment that allows to test exactly the same thing with very few differences from the real world You can also accommodate more developers on a single environment than you would have normally and You also drastically limit wait times If you're not using such an environment, you would have to wait for your colleague to have completed a Modification before you're able to do yours Here you can really apply distributed methods So finally what you're doing is that you're really industrializing your development process and You can release more often Releasing more often. Well, this is something that In open source we strongly advertise release early release often, I guess everybody has heard of that and This is no joke when you can release you have Points in time to which you can go back you have points Which can be used to check what you've done with a product manager or product owner or with a customer and The more often you do so The more able you are to course correct How you're developing your product and this is really really a key for More agile software development so How is it? delivered by in advance to our customer So to deploy the base open stack installation We use a tool that we developed called ed ploy Together we is puppet modules, which are the community puppet modules that we co-maintain with quite a few others Including red at including so many others. I won't be able to list them ed ploy is a tool that we Are developing its open source to valuable on github slash in advance. You will find ed ploy the reason why we Decided to develop yet another deployment tool is Very simple. We wanted something that's used images as the basis of deployment on machine Something that understood that you needed some time to do rollbacks on your deployments Something that could distinguish what is Code from what is data on your machine so that when you do a rollback you're not losing prior work And this is why we started ed ploy in order to deploy what's on top of open stack meaning the components That transform this open stack deployment into a software factory Well again, we use puppet But in order to orchestrate the deployment of these tool Well, we use heat There's nothing better to reproduce orchestra to produce orchestrated environment today on open stack that then heat and Whenever heat doesn't meet our requirements. We have the possibility to modify it Another thing that we did because we Said hey In order to use gits right now We have to mount a file system and mounting a file system Means a lot more complexity when I outgrow this sort of this file system capacity a lot more complexity in order to provide high availability so Why don't we modify? Get and we use dull which in this case To be able to use Swift as a back-end So that's only you have a scalable highly available redundant storage for get I Told you about having the ability to Store environment So that we could get back to them at the same binary stage Later on in the history of the project and storing binaries Binary blobs again. That's a job for Swift Every time you tag a release If all the components of the release are stored there so that mean that you can Do ACL to give public or not public access to the binary you've produced you can Have an history of all the binaries and get back to them whenever you want another thing that we did is We found the Initialization process of a new project a little bit tedious So We built a web UI That allows to generate everything you need in order for this project to be kickstarted And finally because there are no two use cases are exactly the same when we deal with real customers We have to do some customization To implement a specific workflow some people like to have to review some people like to have one some people like to have known Well the one that I've known we try to convince of the white Some people you don't use a red mine launchpad or Jera. So we need to work with them Some people are used to Mercurio. Oh great thing gets has a very nice way to become the Underneath repo for about any dvcs that you can think of and Some people come with Exotic language saying oh will that works and we have to show them that it works nicely in their specific environment So I would have love to tell you hey this project is available today right now all the Modification that we've made in upstream projects that I described have are already in the upstream projects That's our policy at in advance. We have this Delivery low at in advance is that we'd never ever Deliver a patch to a customer that has not been merged upstream that doesn't mean released But if upstream hasn't merged our patch our customer won't get it So this is what applied to To here so the the dull witch modification all the other modification that women that we made are already upstream but the wall package is Slated to be ready Sometime in the next remount of course We are dogfooding this and our very first customer for this product is ourselves because that's very helpful for us to be able to Handle our deliveries for customer and we've got Currently one alpha customer With whom we are we are working and validating what we are doing as we progress okay, so that's it for this presentation, but We'll be happy to answer your question I'm happy that maybe we'll be here to eventually help me if you guys get too technical So for the part that you use a swift as a gate back end Did you open source that part? Yeah, that's in the dole which project which is a get implementation in python You'll find our what is the project name again dull witch. Thank you very much