 So, my name is Thomas and I'm an engineering manager at Red Hat, and I work on Red Hats. So excuse me, this is my screen-sharing. I work on an engineering Red Hats understack, specifically a part of the understack director. But the thing I want to talk to you all about today is the Triple O Project, which is an upstream open source project, our open stack, and it's the foundation that I used to build with the director. But it's the Triple O itself that I want to focus on this afternoon. Triple O represented by this one of our stands for open stack, our open stack. And essentially, what the project is about is reusing open stack components and technologies as the underlying infrastructure that we use to manage the deployment and life cycle of your actual real open stack. I'll get into that more in detail, but open stack on open stack is a multilayered thing. Quick summary of what I'm going to talk about. High level vision. Quick overview of the open stack components and how we use them. The Triple O concept, how Triple O goes through the process of defining and deploying a scalable open stack that is actually suited to customers and uses real world use cases. Some of the features that we've got right now are some of the things that you're going to go through and then questions if we have time. I'll try not to hand it around, but it doesn't make that possible. So, the mission of Triple O is full non-stable management for an open stack deployment, which really breaks down into three different areas. The planning that needs to happen before the deployment actually takes place, setting up network technology, setting up service parameters, and it's also classifying at hand, estimating what the number of machines of different capacity you need, the roles you want to put them into, all that stuff. Then you find yourself orchestratively actually putting the bits onto the machines, according to the roles that you might actually have for configuring the services, and then doing some checks at the end to make sure that it's actually working very well and that it's a functional cloud. And then the final part is ongoing operations. There's some conversation this morning about the difference between deployment and day two, which is a very important distinction between getting the thing to resist in the first place and then being able to scale it out, deal with the heart of the things and the services customer demand increases. Triple O is to support high-alcoholic operation mode, and whether that will get into a fairly big summary of the bits of open stack that we are using for our use case. So what we have on the left here is a subset of the cross-storage systems that we have coming out of use for open stack. And this is the content of the respective units, which is all about deployment, workloads and virtual machines. So, in the open use case, no one provides command and choice services for computer nodes, or orchestrates a network of virtual machines. It's the element of the notice that receives the request for the virtual machine on a certain size with a certain set of characteristics so well understood. Neutral network management sets up a virtual network through VAs. It creates photo IPs with maps of external traffic to enter the process itself. Trans is an image repository. Whenever I open a stack, typically, if we have a large number of services that are trans, it has all the features I support in that container. And this is going to specify usually what I have. That's how I've realized all the features. Skipped by our electronics, what is it called? That is what it did. HIT is an illustration. It was initially designed by a server that allows to houses, cloud formations. It's extended beyond that now. And it allows you to define complex or simple applications that you want to have put out onto your cloud. So when you've got, say, a multi-tier application with a database back then, and there's an application server that went from there into a multi-storey job, HIT allows you to define that collection machine as an entity and deploy it. And HIT has to have features to do with scaling out and fixing multi-application developments when they go along. All of these, in the way that I've just described them, are core OpenStack components which are used in the awful lot of our heavy case where people are putting their jobs in the cloud. Treblavidance, which is the OpenStack, OpenStack is used as same components, but used them as the infrastructure and the tool for deploying and managing OpenStack itself. So to be more specific about that, we've got the same sets of services, but in this case, they're deployed onto mathematical machines, actual physical ones, and structure of the machines. They're going to be your practical OpenStack instance. In this case, NOVA is providing the control service for deployment onto physical hardware. The new change is still the matching of flavors, basically a recipe to say, I want the machine to have this much RAM and this many corals and all that. It's still matches those flavors against and makes a shape for you to see in that way or to actually deploy it, but what's actually being deployed is what exists in physical machines to expect. What is said networking again, but in this case, it's great ports or pretty ports or routes to allow physical machines to connect to each other. In some cases, neutral actual physical switch configuration works as SDM plug-ins for particular supported hardware. Glass is playing the same role of containing OS images and when the user defines their cloud deployment, which matches against a particular entity with clouds, and that's the software that gets put in the machine. Now, I'm not going to skip ironic on this, this will run through because in this actual open-stack developer case, ironic is actually fun. And ironic is another open-stack project and it is all about managing actual physical servers and making them bend out of way as a service. So, ironic explains the API, it lets you empower certain machines and sort of do the lifecycle around your stuff is typically done with VMs all the time in the cloud, but there are actual physical machines behind what ironic does. Heat is doing the orchestration. I think people are using heat in some typical, typical case on that open-stack to define a multi-node application and something about the relationships between the nodes which ports into the accessible. So, in our case, we do exactly the same thing, but what heat has within its entire definition is all the attributes of the production open-stack infrastructure that you're going to deploy so that the heat application payload is your real-world startup. So, you're going to have that complex and the rich set of heat amplifiers to define an awful lot about service placement and so on. So, in case I haven't laid at this point heavily enough yet, we have two clouds in this scenario. What we call the undercloud is the simple command machine which is the installer, effectively, for the real-world cloud, the actual production cloud. The undercloud is a persistent instance. It's not just an installer, like I said, but also covers up the day-to-day operations so the undercloud continues to be available and continues to be consistent. After it's all finished, the undercloud is the same in the inside which will add the hardware scale all the other top. So, the undercloud is where the services that I just ran through accessible and available is not something that any other person can cover up so I think I'll have a cloud result. It is your management interface for the overcloud and the overcloud is your actual payment. This is why you're doing this. This is the thing that users will consume which is typically going to be a discount on a large number of chains and a lot of tenets and so on. It didn't take long for the MGVL to have worked through this idea of putting over a stack and putting over a stack to save over. Could you put over a stack? I bet for a small one, it's called ROOGLOW which is how you're using virtual machines to simulate that they're metal and you then do the deployment. It's actually quite a useful project because you don't have to try this because I've already done this in terms of interactions with open stacks. You don't even have to remain to design your devices. So, that's ROOGLOW in a nutshell. It works in particular cases, deploying two machines. Wrapped up by is to identify which machine the machine is going to be the manager. This is the undercount. The install number, it's also the operation interface for dirty stuff. This is where all the stuff happens. That machine is installed with the triple assault on itself which is a cut down on the special purpose open stack business. Like I said earlier on, it's got metal frets, it's got heat, it's got neutral. One machine ready to treat the other machines as clever as it was. How they got that first machine, the next step is to get all the other machines and register them with the first one. The registration is about telling Oroic the service I mentioned earlier on which makes actual physical machines appear as a service not like for years. Letting Oroic know that if you found a metal manager it's really about the IPMI interface and the credentials. So different machines have various different implementations of BMC interfaces whereby machines can be power cycle attributes which can be modified. Triple O needs to know about that because that's the channel that's going to use to reach the machines that Oroic needs to know about. Once the hardware has been registered Triple O is able to which is really what happens with the inspection is the machines are power cycle and I serve it the reboot and reboot onto any rounders which boots on each machine runs all over benchmarking tests and gathers come from the mental information the number of calls and the number of disk sizes all of that information is hard to manage and I'm posted into a service called Oroic Inspector which is a very long triple O node GIFs and what I created to have having inspected the hardware networking needs to be configured there are multiple ways that networks can be set up from very simple to very complex where we have a there's a book for a thing called network isolation where individual types of traffic with the use of management traffic and the tenant traffic and the storage all be sent across different networks either discrete freelance or share a NIC or across different NICs however this one to set up a network and we have quite a lot of flexibility that needs to be applied what's that is done one of the machines which is going to be the first controller in the old cloud needs to be selected and we have support for five different roles at the moment only a controller where it runs most of the old traffic control services keystone, neutral clouds that you have and so or it can be a computer in which case it's going to run post-vm it's going to run kvm we also have the ability to specify the machine should it be storage which needs for either split to more clouds or set for so in a certain case here, a set of roles one controller, two computer nodes there is some amount of process switching from controller to controller things that we have found in going out config this process for real customer or otherwise is that it pays to integrate as much validation as we can the entity work for you the definition of everything about the old cloud where the services they've had in a sort of 2-2 months is complex and misconfigurations so A411 which isn't actually tagged on a physical point on this switch which ends up breaking the whole thing it helps to find those things as early as possible so we have a set of validations which will look at all sorts of things that have been defined and make sure that they're actually ready to print in this case the machines are actually given their roles which is basically the machines are given a less image or all of the services are started parameters which are defined associated with the services are pushed to the node which is the puppet on the nodes to continue the parameters configure the services that thing gets deployed we are in a simple case the the next thing is just to do a more scalable and highly valuable approach in this case we've got multiple racks of machines an increasing number of computer nodes and three control nodes the we can support from the single control which is quite useful if you've got some more hardware to play around with in actual real use cases we follow the active users who do our highly valuable configuration we put all the control systems in our three nodes and we just make an agent proxy to do an agent management across the services so we have a single point of frame this red one here is your tremolo on the cloud node which isn't actually customer facing it's kind of essential because it's aimed to your manager but it's not customer facing but everything else is your action to deploy production with generating infrastructure so highly available services and some value resiliency because it's very important but in this example we're just scaling up further by having more more computer nodes which I think is that larger numbers of fields can be run and can carry on more users can work at the same time turning to the features that exist in tremolo some of those things which were completed at the end of QLO or something like that APOs one of the big things about tremolo one of the reasons why I like thinking about service is that we are reusing open services services as a installer on our management tool which has a couple of advantages for people who want to be able to use it it means that the services that they're using as a management tool are the same open stack services that they want to get which means that you don't have to learn some new vendor specific set of tools just to do the install knowledge that you have from using tremolo is directly applicable to you which will take your management production because it's the same AP guys it's the same services and also for the development of tremolo it inherits all of the improvements that are happening in the vital community that is open stack but in a more ironic neutral way in the future and the capabilities of open stack they will have to be sure we are a management product because it's the same thing so it has to say that we have commanding tools which we mentioned we have a web UI we have a lot of integration from an Excel management project called manage at a queue those REST APIs as you guys have in the chat the open stack REST APIs are pretty repulsive so those are the APIs for tremolo commanding operations that I've mentioned ready state is important operations is one of the things that we've been working on in the context of ironic you can't just find a machine in your data center and start using it for a particular purpose there's a whole lot of setup that has to be done before you put the REST on it depending on the role the machine is going to have there are a lot of settings that need to be worn when the other virtualization extensions want to be enabled obviously you don't discourage it you need to be built and set up set a certain way and ironic in tremolo has the ability for someone to really get out of that so far as we've done much of the work to actually manage the REST as well so you find this machine you decide it's going to be a complete node you go into the BIOS and you set all the other things and they're probably away that's what happens hardware matching, people go out machines adding inventory based on the capabilities we have support for post-deployment validation HAA configurations, pacemaker that's a day-to-day operation that is using the undercloud to redeploy our data to to make the cloud bigger the same thing can be done to shrink it obviously if you want to take out all of these other features that we have now set integration and put it to either to deploy a set server or to hook into the existing set deployment for external storage the ability to integrate the external hardware load balance and the part of defining how you want your overcloud to be set up and specifying that there is this external load balance between services all year hardware integrated by a worker already talked about network integration is about the ability to work with the existing networks in an ideal case as engineers create a deployment tool we like to think well everything will be brand new we'll just tell the customers this is how things need to work in reality customers have an existing infrastructure that they want to reuse network operators we're doing things and what's needed is to integrate open stack or everything else into an existing framework and triple lower house network isolation to work with a customer's approach to networks a huge area which has been on a lot of work recently this is one of the areas where the difference between simple proof of concept and actual real customer of course is updates in particular upgrades upgrading between open stack versions as you know there's an important major open stack release every six months and a lot of new features it's very important that people have a network part whenever they've achieved actually execute a lot and allowing for actual customer deployments with complex network technologies highly available services and clusters and so on allowing for those things to be upgraded from one open stack version to another with minimal disruption and maximal automation has been a real engineering challenge but it's something that exists in triple lower house the other thing that makes that a real challenge is integrations of third party components if people have their own particular hardware stored triple lower house to bring into deployment then that makes the upgrades a good deal a lot of the work recently has been around making upgrades actually work the kind of upgrading the kind of deployments that real customers have we also added support for IPv6 not a lot of open stack components necessarily have robust IPv6 support integrated today Drillmo does IPv6 basic deployments at the moment we can either have networking to be IPv6 or IPv4 the ability to have a dual stack where you can mix and match and move on to the other big feature that we added based on demand from customers particularly larger telecommunications customers is TLS the ability to have all of the API endpoints for both the overdrive and the cloud to be encrypted and there's some notes on the slide which I want to repeat about options about statistical management generating the cells or using the third party but that was another large recent addition to TLS container as a computer every one of those containers containers is getting a huge amount of attention and we are the exception in the most recent release of TLS we have the option for the computer specifically rather than just putting an OS image which contains all of the services and over and over and KVM and so on also users can deploy or send us a topic as their base on the iOS and then put all of the services out in a sort of containers which are run on top of Docker we're going to extend that one of the big advantages of containerization for this use case is it makes one run back I don't know how many of you have experienced systems administrators but if you do updates to our PN based distributions it's never been similar to the third complex containerization it makes that far easier to test because you're moving from one instance to another that's probably something to be referred it also lets you mix what would otherwise be incompatible versions of libraries and so on so containerization is something that is interesting about this there's going to be more last couple of things that I'd like to finish in a moment that's coming up. Composable roles is the ability to make the roles of the machine multiple threads form a control or computer swiftly turns storage roles and that's about it we want to make it so that the roles can really be anything that needs to be required a dedicated neutral controller is a vital case for that split stack is about separating out the management of the hardware and getting all of the physical machines ready to use a set of operations once we have that done you could do the first stage keeping the hardware ready and then potentially use somebody else entirely for the second stage of orchestrating the services maybe Kubernetes in a bunch of containers so that's going to affect the ability to approach we've got a UI coming and then further down the road a single open cloud managing an array of open clouds H&M in the cloud a bunch of possible torsions more containerization and then that with the tenant support which is a whole lot of the kind of roles I have to finish there I'm afraid there's a bunch of URLs here which tell you where to find out more about trim load and audio which is open stack distribution which trim load tends to work most immediately with as I said at the beginning upstream load systems project is part of the open stack community and absolutely welcomes contributions of all colors that is about my time I think