 Thank you very much for attending my lightning talk on CICD and open infrastructure. It's not specifically not only for OpenStack, it is for any platform that you would like to try to accommodate this. Also in the future it will be for bare metal, which is very much looking forward to at the moment. We are initially focused on OpenStack because it's some known quantity for us. We've been doing OpenStack deployments for a long time. I've been with Mirantis for almost six years now and I've seen OpenStack grow from something that people had in the data center with a 10-15 node installation to just try it out, to dip their toes into the water, to something that a lot of major companies have guests here from Volkswagen for instance and I see other familiar faces that they are using for their infrastructure. OpenStack of course is a very complex environment, it takes a long time. If you would ever try to manually install an OpenStack cluster you would find out that this is something that you do not want to do. I have at the very beginning at Mirantis that we actually did installations manually and we found very quickly that this was not the way to go. So we came up with a tool named Fuel and Mirantis OpenStack that became the sort of envelope of fuel. Fuel was a deployment tool, it was based on puppet, it was not CI-CD and this is not what this talk is about but the limitations became obvious eventually. One of the reasons, one of the things was that we only focused on deploying OpenStack and this is not enough anymore in today's world. If you build an environment and it will change over time, it will evolve, it will grow, it will be upgraded to the next version and if you try to do this again, try to do this with an environment that has just been deployed with a deployment tool you have the choice between scripts, script horizon, like the event horizon with the black hole where it starts pulling in all the scripts that you possibly have written and it will demand newer and better scripts all the time and you know how well that eventually ends. Nobody knows how it works anymore, nobody is willing to touch the code base because if you touch it in the wrong way then you will inadvertently create something that will simply not work anymore and nobody will know how to fix it. You have a lot of tribal knowledge which has been a problem with that quite a few of our customers encountered. Not only if people leave the company which is obviously a rather big thing if a key person leaves the company but also if you are running a 24 hour environment in the morning somebody makes a change in the evening the night shift comes in they have no idea that the change was there something all of a sudden doesn't work anymore what do you do you cannot go with a scripted environment you cannot go back and see what was done you do not have a universal source of truth of what was done and if you ask a software developer they are not going to let you have a major project with just the environment yeah we are editing a file here and just putting it back into the code base that simply does not work we have human error we have diverging configurations which is also a major problem that we have encountered in multiple customers that changes were made but not to all the nodes eventually you have incompatibilities you have you get failures and the failures are very very difficult to diagnose so if we want to apply CICD to an open infrastructure of course the very first thing that we need is a code base and this code base has to be it can be it can be something arbitrary we are at merantis we are using salt stack for this because salt stack has a very rich feature set for controlling open stack environments with salt but obviously that alone isn't enough salt stack itself is just again a way to script things to make things happen in a scripted form so on top of that we have to have a methodology for automated code deployment you have to be able to take the changes that you made in your code base and your salt stack code base and transfer it into your actual environment ideally even into your staging environment tested in the staging environment and put it back and put it into production environment you should have automated testing of course and the most important piece is no manual interaction with the cloud components if you have to log into KVM into a controller node to make a change manually then you're already getting to the back to the point where you have the diverging conflicts where you have the risk of things not working right because somebody made the change only to controller 1 but didn't do it on controller 2 and 3 and it works for a while because they restarted only controller 1 and then later on if there's a failover between service on controller 1 and controller 2 then all of sudden it stops working and then of course we have to have continuity across individuals which means we will need to be able to see what the people that were ahead of us in time did to the environment so I already showed this this morning at the different talk this is the environment that we eventually came up with it's the whole environment is called MCP but it is fully based on open source components we have the well-known Garrett Jenkins environment for CI CD we make changes to the code base by checking changes in and Garrett and having them approved by the appropriate approvers obviously you can also set it up so you can approve your changes yourself but that's kind of not the point if you're if you're making changes yourself and approving them then it you are violating that far for ice principle where you have to have some out someone else make sure that did that what you did was not accidentally mistype look at that look at the piece of code and try to find a missing semicolon this is that that's exactly it so reclass in between takes the human readable code changes and makes creates a set of intermediary files that actually used to implement the changes in the environment and you have the configurations that are coming out of that this in the code base you will find generated directories where the entities are that you are not supposed to touch and then you have the Jenkins which has a set of pipelines if you want to know more about that come by your booth we have demo set up that we can show where we can see what the pipeline like this looks like and what it's used for and then the deployment with the salt formulas into staging and then into production so from the field this is something that I wanted to talk about a little bit we have quite a few customers who have both who have more space clouds and have mcp based clouds and the universal feedback has been pretty much from everyone who has been working on both environments is that most was great for what it was supposed to be to be doing but it was not completed was not doing what it it would not help you operating the cloud after day one mcp has been different we have obviously had a maturing product we had in the very beginning the very first version of mcp was let's say a little bit finicky but by now we are at a point where we universally find that dealing with mcp in any given situation is much easier than dealing with the old way of doing things that was not CRCD based and one thing that I would like to highlight especially is supportability we have time and again we have failures at customer sites we used to spend a lot of time just trying to figure out where the problem might possibly lie mcp helps a lot with that because you can actually go back into Garrett and see what changes have been made over time especially changes that have been made by a previous shift and that find at least a set of potential issues that you can easily address sometimes it will still be something that you will have to search there the old-fashioned way but in most cases or in very many cases you will find that there's as much easier to be able to look at what has changed and also be able to tell us what has changed not nothing nothing has changed so it will so you will be able to find a fault much faster I'm out of time unfortunately if you would like to know more please stop by the booth and I hope that you found the talk useful and thank you for thank you very much for attending