 Okay, everyone looks like we're gonna start. Thank you for joining us. My name is Maria Veraccio. I first want to say that this is a sponsor session at the Summit, so Red Hat is part of the Opening for Foundation, and as such we get one sponsor session. So if you see a lot of the content is very Red Hat heavy, this is why, and this happens in this kind of sponsor sessions. And with me is Chris Jones. We're gonna do a demo today. And before the demo, I'm gonna try as fast as possible to tell you what is it gonna be about and why we're doing this. I'm gonna try to leave most of the part for the demo. So where, how do we get here? So as you know, Red Hat has a wide portfolio for hybrid cloud infrastructure, OpenStack being one of them and a key component of the on-prem platform. But OpenShift heavily based on Kubernetes is there as well. And our customers have been asking for a way of working with both of them together. You probably have heard the foundation talk about Loki, the Linux, OpenStack, Kubernetes infrastructure. Here you're gonna see that challenged a little bit. And I don't even want to come up with a new acronym, but you're gonna see Kubernetes both at the bottom and at the top of that stack. And I'm gonna tell you a little bit why. A while ago, years ago, we had customer requests to run the OpenStack control plane in a virtualized way. And so a lot of our customers use Red Hat virtualization. And we announced that this product was kind of going away. And when we did that, and it still hasn't, but when we did that customer said, okay, we need another way, because we really want to leverage the way that we manage the control plane of OpenStack in an easy deployable way. So we said, fantastic, we're gonna run that same control plane on OpenShift virtualization. And we released that in our release 16.2, it came as GA, it's gonna be available on 17.1, which is the release that we are announcing at this summit. But we said this is not enough. Just running the virtualized control plane of OpenStack, which is OpenStack services, which have been containerized for a long time on top of a platform used to run containers just seems like a little bit much. We can definitely continue to leverage and modernize the operation of this if we just run it straight and natively in Kubernetes. Obviously, this is not something new. This community knows how to run OpenStack services in Kubernetes for a long time. We haven't sort of jumped on that bangwagon because for us is hundreds of customers that translate into a lot of mission-critical workloads, billions of subscribers that we just can't rip and replace. So we needed a way for this to happen in place and to modernize those operations in sort of a kind, less disruptive manner. So this is what the director operator looks like today. This has already been in general availability. You can see several common unknown faces, right? So triple low is there, heat is there, and ansible. And then for the hardware provisioning side, we're leveraging multiple cube birds to run the VMs and metal cube to handle the infrastructure. So if you already have that, why go the extra mile, right? Like why continue to modernize this if that's serving a purpose for your customers? Well, this, indeed, this model has served us well for a long time. We had resiliency provided for a pacemaker. They two operations were managed in a certain way with triple low. And then we had a set of observability and monitoring features that were also available. So why did we want to modernize on Kubernetes? Again, we wanted that fine green OpenStack control plain services. We wanted to leverage the operator framework available in OpenShift. We wanted that replication of failover available at the control plane for those services. And then we also want to provide additional observability for the full stack. So a lot of that is coming up highly for other Red Hat sessions in this summit where we're also going to talk about observability and we're going to talk about how we do a lot of this. And then what about the data plane? So in there, we wanted to eliminate certain pain points, but still understand that our customers and their workloads require OpenStack APIs and they're still virtualized and there's still a ton of virtualization available and will be available for years to come. So the data plane is not going to change in terms of the way that we handle virtualized workloads or in the way that those run on rel and still have a five year life cycle of support. But we're going to go do away with a lot of other complexity that has been built in time with menstrual heat and puppet and sort of standardized and Ansible while still maintaining that long life. And obviously improve upgrades, which is a part of that I've dedicated a lot of my time here at Red Hat. So is this only for new greenfield deployments? Absolutely not. So this is sort of the key piece of this and this is sort of why we haven't done that in the past. We're not interested in bringing a brand new product or a new way of doing things only for new deployments. We have to bring existing deployments that are in production forward. So it's a low friction path of adopting the existing data planes. So we have our customers and that's our first priority. So we need to create a new control plane and then we'll repurpose the existing one. And then what happens with director going forward earlier this year, we announced that Red Hat will no longer continue to contribute to triple O as a management of the platform because we have indeed been working on this. If some of these slides are a little bit similar to what we talked about last year in Berlin is because we have continued to build on that story. Last year we told you we were going to work on it and this year we're going to show you what we did. So yeah, again, the starting with our next release 18, the management of OpenStack will exclusively in this form factor. So the control plane will run exclusively on OpenShift and then 17 will be the last release that we support director. And this is a little bit of high level of how things look like. And with that, again, supporting this meta operator, you're going to see a lot of this as part of the demo. So here's the big picture. And now Chris help us out and tell us how this looks like. Yeah, Thanks, Maria. All right. So what are we starting with here? We've got OpenShift is already installed at this point. It's running three masters, three workers, and we've got two pre deployed rail machines that have been using this compute nodes for this deployment. The operators that we need have been pre loaded into OpenShift and some container images have been pre cashed into my computer nodes, but nothing is running. So the stuff that we've got here, there's two little terminals down at the bottom. One on the left is monitoring for the existence of the control plane. Right now it can't find. And one on the right is monitoring the data plane, what we call the collection of compute nodes that you have. So those will change state as this is running on. And then in the middle there's a terminal that I'm going to be doing stuff in. I'm going to be using a tool called canine s. It's not part of OpenShift specifically or Kubernetes itself. It's a third party tool, but it's very useful for being able to quickly walk around through OpenShift or Kubernetes APIs, show lists of jobs, pods, pull up logs for things, edit some of the CID YAML files. And so that's a nice thing. So we'll get going here. And the first thing I'll do is show that we are indeed running OpenShift 4.12. And if I grab a list of operators that are currently available to us, we can see there that we've got a couple of OpenStack related operators. Those are the ones that we've been working on. And then there's a couple of networking related operators, which are standard things that we're pulling out of the normal kind of OpenShift registry to help us connect up networking in containers. OpenShift with networks on real hardware in rail machines. So the first thing we need to do is give OpenShift the CID YAML file that describes the control plane and tell it to deploy that, which it is now doing. And when I launch canine s, we start to see that there are a bunch of pods. Some of these, the ones with manager in the words, those relate to the operators, those are already there. But things in these lists that show up as blue or it's supposed to be red, they might look more sort of salmony color. Those are things that have been created or instantiated for the first time. And when we pull up the logs for each of these things, we see the first things that happen basically as database things. Same as it would be for deploying OpenStack any other way, the first thing you need to do is have a database and start putting things in it. A list of pods that we can see here, if you can see, there we go, so we can see rabbit is being started, like the control plane services are being brought up in a dependency tree kind of order. And it's going to go through and configure each of those appropriately as it builds its way up the stack. So one of the key things that we want to do here is have fewer layers between the operators, the human operators of this and what's actually happening under the hood. So you're able to use the normal Kubernetes OpenShift things to interact with this very directly. And this is the resource for the control plane. So this is the YAML file that I actually fed in to deploy it. And I want to show you what it's like to make changes in this. So in this case, I'm going to skip down to the section for Cinder. And you can see that really all we're showing, all we need to configure for each one is like here's a container image to use. Here are some networking things that you need to know. So like there, it's going to be on an internal API network. You can give it my IP address and a number of breadth because it's trying to eliminate the amount of stuff that you have to know about that we have to have pre-coded into things like AAA heat templates for those of you who are familiar with with AAA or directed from the past. So I'll pull up a list of pods again. And we're starting to see now here the actual services are booting. And as a general rule of thumb, like in this sort of deployment, when you see NOVA being mentioned, it means you're usually getting close to the end because NOVA depends on the most things. So that tends to be one of the last things to be started. So those are actually NOVA API cell conductor and metadata services being started, which means that where we are now about two minutes into this, the control plane is done. So that's kind of cool and nice and quick. So we can see the list of pods now is much longer. We can see the various control plane services already up and running familiar names like Cinder and Glantz that you would always expect, which means the next thing we need to do is kick off the data plane. So I need to pull out a small amount of metadata from that deployed control plane to be able to feed into the YAML file that I'll give it for the data plane, which is the next step here. So we give it a second CID YAML file that describes the data plane that we want and now it's off to the races deploying that. And when we go back in we'll see a very similar kind of interface. So even though this is all now happening on the compute nodes, we still get to view it through the control plane through the open shift underneath it. But this time, instead of pulling up logs and seeing things like alembic migrations, the logs are all just going to be ansible output. And this, again, it's fewer layers. You can get straight to the information you need. If there's a problem going on here, you can see right in the ansible log exactly where it's happened, which step, which role it's part of, which machine it affects. So while the data plane's running, I'm going to make a change to the control plane. For some operational reason I've decided I want to enable debug logging for Keystone. So previously this would have been a triple O heat template parameter you would have needed to specifically toggle individually. But because we want fewer layers here, we have a flag you can set here, a custom service config, and then any arbitrary text you put in there will be written into the configuration file, the iini file for that service, so in this case Keystone. So I've just added a default section, set it to debug equals true, and if I now pull up a list of the pods and then filter just for Keystone, we can see that there's a new one already running, that's the red one or some any kind of pink one, and as soon as that one is up and running, which it already is, the previous one will be terminated. So in a very short span of time we've been able to make a change to a service without having to do a full stack update. You can also do things at runtime, so this is not modifying any of the yaml files, but we're just telling OpenShift at runtime, hey I want to increase the number of replicas for that service, so in this case we're going from one to two, and again filtering for Keystone, we can see that the second one is already spinning up, and in basically the same amount of time it took for the previous operation, we'll have a second instance of Keystone running, and we do. So I'll just show the logs for that instance that it's picked up, that same debug logging configuration, and indeed it has, and so that we've been able to make several changes to a service, actually while the data plane was deploying, and OpenShift is managing all of the life cycle of that forest and allowing us to operate very precisely on just the things that we want to. Right, so this is going to be bouncing around more of the jobs and pods, they're all just doing Ansible at this point. There's a lot of Ansible that we had previously in TripleO director, for the last couple years the TripleO project has been working on consolidating that into a number of what they've called consolidated roles, so all of the different bits of Ansible from all over the place have been brought together in Unified, and we've been able to take advantage of a lot of that while doing this, so we're able to set things up very similarly, how we would have done before. And if I pull up the data plane resource, I can show the CID YAML that we use for that, so where for the control plane we were describing the services that we want, for the data plane, the compute nodes, what's really more important is describing the hosts themselves. So this is an entry for one of the two compute nodes that we're deploying here, we have to give it its IP address, an SSH key to be able to talk to it, some Ansible variables which in this case is dominated by the networking IP addresses we want it to have, and then an Ansible role at the bottom there, EDPM compute that we want it to be configured with, and so you repeat that for each of your different compute nodes, and then that will go through and do all of the bits that you've asked for. So let's come out of that, and if we go back into KNNS, I'll show up the list of jobs, we can see there's a series of data plane deployment jobs, the ones that have gone white, those have already completed, if we were to watch that list more and more would be filling in as different parts of the Ansible run, and really what this needs to do is take these bare rel nodes and turn them into things that actually have open stack running on them. So this is containers that are running on one of those control plane, sorry, compute nodes, we can see that some networking services are already starting up there, and that's really repeating the same process for each of the services that we have to have on that machine, like does it need packages installed on the host, does it need configuration files written or directories created to host things, we need system de-unit files to describe the services that need to run in those containers on the compute nodes, and then actually starting them. And so it's going to work its way through the kind of dependency stack in the same way as on the control plane until we get to the point where everything is running. You might have noticed in the configuration for the compute nodes that it had multiple IP addresses, this is something that we're bringing forwards from the director-based open stack that Red Hat has shipped for a long time, it's very important to us that all the different classes of network traffic are separated, and so you now have a situation where there's containers running inside OpenShift and bare metal rel nodes, how are you going to join up all of the different networks there, that's a very complicated virtual networking thing, but we've been able to figure stuff out there using things like Metal LB so that we can say this is a network only for internal API traffic, that's the open stack services talking to each other, and those are the only bits of traffic that will be on that network. Tenant networking can be isolated, storage is very often isolated there for people who run sort of CEP or their own external partner storage services. So we can see here that there's just more Ansible running, that's really all that's happening at this stage, a bunch of configuration for the compute nodes, and again it's working its way up through various things, if there's a problem here you can see it, it's creating, I'm not sure if you can all see that, but it's creating system D unit files for iSCSI-D at the moment if you wanted to use that, and really this just keeps going until we start seeing words like NOVA and Libvert. Now this is not quite as fast as the control plane was, but we're getting pretty close to the end at this point, which is really nice, because this is a much simpler kind of operation, we haven't had to create huge template files and wait for stack updates to run, and every time you want to change something you don't need to do another stack update. So we're getting pretty close to the end, it's hard to know what else to say about when you're doing something like this, because it's all just the same as before on the compute nodes, that's one of the key goals here is, because we're reusing a lot of the Ansible and we're keeping standalone rel nodes instead of making those OpenShift workers as well. The configuration that's being done, the services that are being, you know, started up here, the configuration files, the package dependencies, they're basically all identical to what they were before, and like Maria was saying, we want to be able to adopt existing compute nodes from a director-based deployment into one of these OpenShift-based deployments, and a key requirement for that is, you know, the underlying state of that machine needs to be very close between the two, you don't want to adopt a machine and immediately have to apply a huge Ansible playbook to it that will change absolutely everything on it and disrupt all of the things that are running. So the two should actually come out being essentially identical, like IP addresses will change and SSH keys will change, but, you know, the services that you have been configuring for a long time and been setting up in a way that is reliable and that you trust and that works for your customers continues to be the case, it's the same rel nodes, the same services, the same configuration, we're just managing it through a different interface and the control plane is the bit that's changed the most. So we can interact with a lot of these services from the OpenShift end of things using, like the logging I've shown you, Ansible logs, those are Ansible logs that are coming from the compute nodes. You won't be able to interact with the logs for the services themselves that easily at the moment, where we're looking at more ways to be able to improve the management experience as we go from here. And all right, we can see that we're starting some Libre containers. So we've got to be getting pretty close to the end at this point. This is now again on the compute node, we can see that Nova Libvert is up and running. I'll also mention the way we're deploying Nova is changing a little bit in the sense that previously we would always deploy just without cells. So by default you'd be all in cell 0 and you would have to make a conscious choice to start using cells. That's going to change in this world, we'll be deploying things into cell 1 by default if you don't ask for anything else which makes your your path from there to adopting additional cells a lot easier. They're all available to you straight away. All right, so Data Plane resource has been zoomed up and it's done. So we're now at 11 minutes in and we've deployed both a control plane and the compute nodes. So I mean we could stop there but this wouldn't be an open stack demo if we didn't actually boot a tiny VM on here. So I'll connect through into a container on OpenShift that's already got the OpenStack client installed and ready and then I've written a little script for myself here to to show off some some of the things that you would expect to see there. So like an endpoint list, all of the standard services are up and running. The compute services list shows that we can see the compute nodes, they're known to know where it's happy that they're up. Same thing for the the networking there, the controller gateway and manage metadata agent for OVN are up and running on the compute nodes and talking to the control plane. And now we have to do all of the normal kind of open stack commands. So we create flavors, we'll grab a Cirrus image, like the point is from this point on this should look exactly the same as any other open stack demo. The fact that the control planings running on OpenShift from here and out should be completely irrelevant and completely transparent. This is a tenant doing tenant things and they don't need to care where your control plane is. The same way they wouldn't have done whether your control plane was running on bare metal or in VMs, now those containers just happen to be hosted in OpenShift from from their perspective. Hey Chris, sorry to disrupt this, but while this VM is booting, so one of the things that we know is that our customers or the majority of our customers are running OpenStack and also OpenShift, whether that is side-by-side in their data centers or whether that is shift on stack or whether that is running sort of the director operator. So a lot of our customers are familiar with both, but there's going to be a ton as well that have not and that have been familiar with OpenStack and OpenStack operations and then this is new, kind of similar to our engineering team at Red Hat, that was sort of solely focused on OpenStack. So obviously for this, OpenShift or Kubernetes has been a new skill throughout the teams that are, have been running, you know, Nova, Neutron, whatever. So can you tell us a little bit about the ramp up? Like how has that been using this new tool and why have you learned? Yeah for sure, I mean I think it's the case for for most of our engineers who started working on this, that this was their first exposure to Kubernetes to OpenShift, even to using Go underneath for rating operators. The feedback that we've had from them has been very positive. They've really enjoyed getting to know a newer system which gives them more access, more control, more flexibility. It's easier for them to interact with and to test. And from my perspective as an engineering manager, I think it's very fair to say that there are more people in the world who have skills like Kubernetes or Go than have triple O skills. Triple O has served Red Hat extremely well for the last decade or so, but it is a very niche skill set that applies really only to one product. And now we're able to hire people who have more generally applicable skills. And I think that's probably going to translate for customers as well that you won't need to spend time training people on triple O. It'll be we need someone who knows about Kubernetes, which is a much easier to hire for skill. All right, so we're at a point now where the VM, I don't know if you noticed, actually booted and was running a little while ago, this is just going a step further than that rather than just leave the VM booted running. We've created a floating IP address network. We're setting up the security groups at the moment that will allow that group to be able to access outside world. We'll be able to ping it an SSH into it. In fact, that's the firewall role that's going in at the moment, allowing us to ping those machines. And then I think with that done, we should be at a stage where we can attach the floating IP address to the thing and we can ping it. So that's the quote unquote public IP address of that VM. I can SSH into it as many of you probably know Cirrus's password is gocubsgo which you don't get to see here but gocubsgo and then once we're inside the VM, we should be able to ping the outside world. And we can. So there it is. We've deployed the control plane services on OpenShift. We've deployed compute node services on some rail nodes. We've joined them all up with with the proper virtual networking between them and we can boot instances and connect to the outside world. Thank you. I think should we do some questions? Yeah, I think we have a few minutes for questions. So with OpenShift virtualization on being a thing based on Kuber, where do you see OpenStack versus the OpenShift virtualization? Where do you see the use cases for those things differentiated? Well, the first thing is that OpenShift virtualization gives you the ability of using Kuber to run the VM on top of OpenShift itself. So from it's all in the Kube world, it has the life cycle of Kubernetes or the life cycle of OpenShift and it's sort of tied within there. This is OpenStack. So this is infrastructure as a service. So it has sort of a wider set of use cases, the scale and the sort of tried and tested abilities of OpenStack. So as sort of Chris mentioned, there's a ton of OpenShift at the beginning when we were sort of putting out the control plane. But as the data plane comes in to show, it's still very much OpenStack. Whereas with OpenShift virtualization is still VM inside OpenShift. Hopefully that makes a difference. Yes, over there. How is it? Well, we're not using Helm. So right now in this demo, we're only showing basically the deployment piece of it. If anything is another deployment mode. It's a new installer type thing for OpenStack. What you haven't seen here is that with OpenShift, we also have the stack observability stack. That's going to be the same for both. It's going to be bringing up a ton of other services in terms of CI CD that we can then leverage for the rest of the group. And we just picked operators to wrap around the OpenStack services so that we can serve them that way. And we're using the operator OLM, the operator lifecycle management to also keep track of all those possible combinations. We also want to make sure that we're shipping something that the entire stack is supported. Say that. Yeah. Yes? Well, so, yes. We are going into this new world, right? Like the first thing that we're doing is changing the way that we deploy. And we have seen, I don't know if you have ever seen a deployment of OpenStack in an open infra or OpenStack summit for that matter. It's a first for us at Red Hat to do the full deployment because we could never fit it in a session that's 30 minutes, which we actually haven't used the entire 30 minutes for. So that's a first. Now the possibilities are endless. We're not using a new installer here. We're using a whole entire product that has a whole entire community on top. So we could start using a ton of things. So we have looked into data foundations. There's some limitations of what we can do in the control plane. The data plane is still very much open stack on rail. So that's still debatable. So how we implement things there's going to be multiple use cases, I think from a product management perspective is going to be wrangling customer needs and then engineering excitement into trying to do a shiny new thing and try to fit that into a roadmap that we can actually deliver things. But yeah, I think we're very excited of all the possibilities that are opening up. Yes. The the what's the dates playing for open shifts and yes, they're separate. Yes, correct. OpenSack services. Yes, it's sitting on OpenShift. Putting the data plane of OpenSack on top of on top of OpenShift that is a possibility. It wasn't the first use case that we were trying to solve here. Like we said, like I said before on the previous question we already have OpenShift virtualization to run VMs on top of OpenShift. Can we run Nova on top of the thing on top? Yeah, we yeah, we probably can. But it's just not it hasn't been our first priority because like I said our existing customers and bringing them data playing forward was sort of number one. And I think if you did want to have a single pool you would have to be able to scale down OpenShift workers to scale up OpenSack compute nodes. So it would be possible to move machines between the two but right now that they're managed completely separately and neither side knows about the other ones working on it. Yes. Were there any plans for Nova Scheduler against Cooper? On Nova Scheduler against Cooper? I think it worked trying to talk in sort of about the same thing hasn't been the extreme question whether Nova wants to support that. Yeah, it hasn't been our priority that's all I think that. Yes. So Chris can talk about it but I can tell you that just even from a developer experience just the amount of time it takes to deploy alone and make a change and that you don't have to do a stack update and wait the debuggability is different. As you can see there's a ton more answerable there so it's somewhat human readable if you want to talk about Yeah. So I mean from from the performance and scale side of things we're initially targeting basically the same as we have right now with Bare Metal Director. It's not like the OpenStack services you know their containers running on machines right now in director their containers running on machines. So there's the performance of those services is basically the same as it was before on like the like hardware. Leading back to one of the other questions we're hoping that we'll be able to leverage some of the things in OpenShift to allow us to scale better. So for example like I mentioned we're deploying Nova with cells by default. So that means that each pool of cells that you deploy gets its own like rabbit instance and OpenShifts primitives give us a lot more ability to open up that world. So for example you could have a different instance of the database per compute node service. So if you know that I don't know say your glance is going to be really really heavily used and that's going to need its own database or its own rabbit or something that's a lot easier to express in OpenShift's world than it was in director. And that's something that we're going to look at definitely from the performance and scale side of things but our initial target is kind of parity with where we were before. Which is very aggressive even to comma to parity because we're talking about essentially and no way of installing and just to be able to in this short amount of time get to get to a place where we can see things the same way that speaks to a lot of the reuse that we have done on a ton on a ton of the things and there's been some refactoring of it but it just speaks about how well these two products work together. Other questions? Maybe time for one more. Yeah I think we're at time but all right well thank you very much and I hope you'll enjoy the rest of the summit.