 Let's get started folks. Good morning. Let me introduce myself. My name is Rupak Parekh. I'm co-founder and VP of engineering at Platform 9. I have Amirish Kapoor here. He's architect at the vSphere engineering team at Platform 9. And we have Pushkar, who is a part of the engineering team for the vSphere. And we have Ken Hui who is sitting over there. He's director of technical marketing at Platform 9. So today, we are going to talk about OpenStack and how Platform 9 was able to make it work with existing brownfield environments. Before we go into the technical depth of the topic, let me talk about Platform 9, its architecture a little bit, just for a couple of slides. And then I will have Amirish talk in detail about the architecture. So Platform 9 was founded two years ago in 2013 by early VMware engineers. We wanted to use OpenStack and make it really easy. So think about Platform 9 as an easy button for OpenStack. We also provide OpenStack as a service. We host the OpenStack controller in our data center. You bring your own servers. We connect to that. And boom, OpenStack is up and running. And we are hiring. We are based in San Francisco Bay Area, San Ivel. So if you're interested, I would be here after the session. Quick overview of the architecture. As I spoke about, we have the blue box on the top, which is the Platform 9 cloud, where we run NOAA API, scheduler, conductor, glance API, registry, keystone, and various other OpenStack services, along with some Platform 9 services which really make it happen. The green boxes down on the bottom are the data centers of our customers and their hypervisors. So on the left-hand side, you have Linux, KBM. This is what we support, which runs a Platform 9 agent connecting back to Platform 9 cloud. And on the right-hand side, we have the vSphere environment where you have the ESX hypervisor with VMware vCenter server, which connects to a Platform 9 appliance, which is essentially the same agent, but in an appliance form, connecting back to Platform 9 cloud. And this is how we connect it and make it work. So coming to this talk, why are we doing that? Why do we want to make OpenStack work with the Brownfield environment? Well, what we have seen is most of the project starts really small. You have one or two hypervisors with some virtual machines running. You add more servers. You have some more virtual machines running. Before you know it, you need a cloud. And you haven't really realized it yet. Virtualization has been there for ages. And people have large virtualization set-ups without a true cloud. So how do you go from there to true cloud? And believe it or not, when you talk to our customers, infrastructure as a service is still new. People have virtualization, but cloud is not there. So we want to remove the silos. And for most of all, we have seen strong customer demand for that, especially on the VMware side, where they have a lot of virtual machines running. They want to use OpenStack, but they have existing virtual machines. How do you take those virtual machines into the cloud? And the last thing that I want to mention, Platform 9 has been working with OpenStack for the last two years. So we want to contribute this particular thing back to the community. And Amrish is going to talk a little bit more about the challenges and the solution that we have came up with. Over to you, Amrish. Thanks, Rupa. So now that we spoke about the motivation for why all of this is useful, let's get into the how. Once we started looking at this problem, the first thing we wanted to do was classify the different asset types that people cared about having discovered from their environment. So for each host or cluster, which is a currency on the vSphere world, we were interested in discovering not just VM instances, but the networks that connect to their IP addresses on these networks, as well as any images or templates that these instances respond from. This is applicable on KVM just as it is on vSphere. So I'll be switching between both hypervisor types during my talk. So starting with VM instances, the way we approached this problem was to create a periodic synchronization task that would check for instances on the hypervisor. Again, the same applies to both vCenter and KVM. Any new instance that was found on the hypervisor would then be reported as a new open stack instance with the appropriate metadata, whether that includes a name, UID, block device mapping, et cetera. Now, to complete this loop, instances that don't exist on the hypervisor but had previously been reported to the database also need to be cleaned up so that the synchronization is maintained on both fronts. So all of the sync task components basically query the hypervisor and tell NOVA conductor, here's a VM instance I found, add it to the database. Now, that's fairly straightforward. But VM instances can't really exist in a vacuum. So since this discovery occurs at startup, our deployment has to take care of creating a special service tenant and an admin user within that tenant who owns all of these instances that are discovered. Now, all these instances are then owned by this user under the special tenant. But this doesn't mean that you can't go in and create further tenants at appropriate times once you've deployed your open stack cloud. So that was one of the problems that we had to address once we started discovering these instances. In addition to this, we also had to worry about flavors and quotas. Like I said, instance discovery happens at startup. So at this point, you don't have any images. You don't know what images these instances were spawned from. You don't know what flavors they were created from. So the simple solution here is to create an unknown image and an unknown flavor and let instances be built on top of that concept. Now, it turns out that you can still continue using the VM instances as you would normally use them, even with this information missing. So things like power operations, instance ownership operations, snapshotting, et cetera, still works. But the one key missing piece is that if you don't use quotas, then the instances are pretty invisible to open stack from an accounting perspective. So even though we used the concept of an unknown flavor, we still took into account all of the quota-related information for each instance. So for example, this included cumulatively establishing the block device mapping so that it would count towards the quota. So when the instances are discovered, they're also discovered as having consumed a certain amount of quota within that tenant. Moving on from instances, we took a similar approach to discovering networks as well. We had a periodic synchronization task that would go into the hypervisor and look for what networks it was reporting. For vCenter, this is basically the switches, pod groups, et cetera, that get reported as part of the cluster. On the KVM side, we're reporting Linux bridges that are attached to that hypervisor. So the impact of this is that if a network is added to the hypervisor post the fact, it automatically gets discovered and reported to OpenStack as an OpenStack network as well. Because this synchronization task is not just a one-time thing. It continuously keeps monitoring for changes in both directions. So if a network goes away, it makes sure that the database is updated to take that into account. Similarly, if a network is created, it makes sure that OpenStack is plumbed with a new network as appropriate. So once we have the instances and the networks, the next step is to glue them together and see if we can discover the IP addresses of the instances that we've got in our cloud so far. So the synchronization task here, again goes to the hypervisor, asks it, okay, these are the instances I know. Tell me what IP addresses are occupied by these instances. And we've already discovered the networks in the previous step. So I know how to map those IP addresses to the networks that are now being plumbed into my cloud. This information is reported to NOVA. So all of the interfaces, IP addresses, all of the metadata for the instances now updated into the database tables. The key impact here is that now, if you're using any other external to OpenStack technology, let's say you have your own DHCP server setup, which is serving IP addresses to your instances, it doesn't need to know about OpenStack because this discovery goes in and periodically keeps pulling the IP addresses that each instance reports. So you can continue using whatever other infrastructure pieces you have while still letting OpenStack know that this is an instance, this is the IP address it consumes, these are the quotas that it contributes towards. The one point to note here is that we're still talking about the context of NOVA networks. We're not talking about the neutron yet. This is future work, we'll get to that in a later slide, but all of this is strictly within the confines of NOVA network. So once we have all of the networking sorted out as it were, we also wanted to look at seeing if you could use template images. Now, this is vSphere templates, so this is strictly in the vSender world, and vSphere templates are very similar to VM instances there. So the discovery process here is very similar to how you would discover an instance. You'd go in and say, what are the templates that are housed on this data store that I care about, and I'll report them as new glance images. So glance thinks that these were new images that were created within OpenStack, but we're basically the agent that is creating those images. The key point to note here is that with vSender, it's very easy to convert between virtual machine instances and templates. So your discovery service has to make sure that if this conversion happens, let's say you convert a virtual machine instance to a template, you have to make sure that NOVA drops that instance and glance gains an image. Now, this is great from a vSender perspective, but what about other image types? So we wanted to give a path for users to also be able to use VMDKS QCOW 2s, any other image types that are not vSphere templates. And the way we approached that task was to set up a predefined folder under each server that could house template images, or sorry, other image types that users cared about plumbing into glance. So it essentially became a special watch folder, and we had a file watcher process that would go in and keep monitoring that folder for changes. If there were any existing files in there, those would get reported as glance images. If you added more files in there, those would get reported as new glance images as well. So all images that were copied to this folder were essentially discovered automatically. You didn't have to deal with glance. You didn't have to try and convert your images into a particular cloud tenant, et cetera. All of this was taken care of as part of the initial deployment. Now with all of these pieces, we can then put together a complete picture of how all of them work together in concert to create an open stack based private cloud that is actually fully plumbed with all of your existing assets without you having to do anything special. So you can see that the hypervisor at the center of it all is providing all of the VM information, the network information, the IP address information, as well as the image information that is needed to set up essentially a private cloud with all of your existing assets. So all of these go into setting up a fully functional private cloud and you haven't even done anything in terms of operational stuff yet. And this is essentially what we did in our environment as well. So we put all of these tasks together. We made an initial deployment which was based off of our platform nine vSphere infrastructure that we're using internally. We authorized the data stores and clusters that we cared about. Once all these tasks had kicked in and settled in, we were able to create a completely seamless transition to this new open stack cloud which was now serving our underlying infrastructure. This whole process was completely seamless. The teams that were using vCenter actually continued using vCenter while this conversion was in progress. And the reason for that is the synchronization tasks that we spoke about actually keep both the views of open stack as well as the hypervisor in sync. There isn't a one way communication here. So usually what happens with open stack is that open stack is a source of truth, right? Once you put open stack in place, everything has to go through open stack down to the hypervisor and any information that comes back is kind of not really understood by open stack. So because we put this in place, we could enable teams to continue working with vCenter the way they were while all of this was being plumbed. Once it was in place, teams could switch over to creating new quotas using all of the open stack API private cloud goodness and continue working with that API without skipping a beat. And that's how we set up our internal infrastructure. To dig deeper into this a little bit, I'm going to hand over to Pushkar to go through a live demonstration of how all of this works. Pushkar. Thank you, Amrish. So I'll try to have a live demo if the network allows me to do that. So this is the general platform nine UI that you will get. This is the platform nine open stack controller which is hosted in the cloud. I have one ready for the vCenter environment and another one ready for the KVM environment. What you generally have to do is you'll have to download an appliance for the vCenter environment and an installer for the KVM one. I have already done that and that's why it's waiting for authorization. So basically when we deploy the appliance it will go ahead and discover a few things about the environment so that the authorization step basically adding Nova compute becomes slightly simpler. For example, when I click on authorize it asks me what clusters I want to use with this deployment. I'll just choose these two clusters which are our internal dev test and clusters. And I'll authorize all the data stores currently available in that under that cluster. What this did was it will now install Nova compute on that appliance and along with the platform nine components that Amri just talked about and it will start the discovery process as and when it starts running. While it's doing that, let's switch over and see the vCenter environment itself in a bit. So this is the actual cluster that we are looking into. So I just authorized the VMW dog food and the VMW test clusters. So we should start seeing them in some time. It takes about usually around three to five minutes but then the time varies depending on the number of instances that are there in that cluster and also depending on the number of clusters that you have authorized. While it's doing that we can actually go ahead and so because it's still under the discovery I'll just go ahead with the KVM workflow as well. It's similar to that on the vCenter. Here again it's asking for asking me to authorize it. I'll just assign it the hypervisor role for now and not just click next, next as usual. This will also, this will operation also at the same thing the Nova compute is now being installed on the KVM environment and it will start the discovery process as and when the Nova compute has been installed and configured. So the KVM environment is a demo environment. It's not a live environment, it is a live environment but it's not one of the actual clusters. It's a demo environment. So it has just three virtual machines and just one network for now. So once the discovery completes we will see that one network and the three virtual machines pop up here. So at least now it has understood that there are two clusters but it's still waiting to discover all the virtual machines. It depends on the connectivity also between the appliance and the vCenter environment. Okay, so now the discovery has been completed on the VMware environment. We can see that it has discovered 35 instances that are running and if you see the networks all the networks on those two clusters have been discovered. Let's go to the instances. Even the IP addresses of the instances have been discovered and it's not just the platform nine UI. If I shift to the horizon UI, I have set up DevStack Horizon to point and pointed it to that particular open platform nine controller, till it refreshes, let's go over the KVM environment. Okay, so even here the discovery has been completed. You should see, yeah, it has discovered that one network that was there on that hypervisor and on the instances, yeah, all the three instances are now visible here. Like Amrish said, for vCenter view, especially we discover templates. So what happens if you convert an instance into a template or back? So let's try to do that right now and we should be able to do that. Let's go to this cluster and I hope Kevin is not going to use this VM for some time. I'm just going to convert it to a template. Now this can sometimes takes time depending again on connectivity that I have right now. Okay, so the VM has been marked as a template in the vCenter view. So we should be able to discover that changes instead of waiting for the 10 minutes or whatever the time of the periodic task might be. I'll just manually go ahead and refresh it directly. Again, now because I'm refreshing, it can take time depending on the size of the vCenter cluster and the number of VMs that are there because all the information there will be refreshed. Let's go back to the instances view and we should not see the Kevin's VM now. Let's see if it is seen here, not yet. But at least it has been partially completed. It has disappeared from the instances view. It now shows up in the images view. So basically you can perform operations on the vCenter and they will be discovered, they will be discovered in the platform, open stack. Let me try to find the credentials and I will be able to show you that view as well. Thankfully I had saved those credentials somewhere. Good, yeah, so the instances pop up. Thank you. Let's look at the images. This is taking surprisingly long time today. Yeah, and all the images show up in the horizon view as well. So it's actually an open stack controller, so basically you can use horizon or platform and it's up to you. Now I'll hand it over back to Amrish. So what Pushkar demonstrated was essentially how long this whole process takes in real time. So it really does take the couple of minutes that it took to discover all of your existing assets. And the key point that we want you to take from this is that once that discovery is complete, that is actually a fully functional open stack private cloud. So we could have gone ahead, created new instances, we could have gone ahead and created new coders, new tenants, new users. We could have used all of the existing open stack API and all of that would have worked just as if you had set up this private cloud from scratch, except that it's already plumbed with everything that you care about in your infrastructure. Before I summarize, I wanted to raise a couple of points that we took as part of the implementation details when we went down this path. Now the discovery that we spoke of, we essentially thought of two different ways we could implement this. And the first was to integrate it with each of the parent components that it belongs to. So you'd create a discovery portion in Nova, you'd create a discovery portion in Glance, with networks, et cetera. The advantage of this is that you can implement this at the interface layer, you can define proper APIs for how to implement discovery and every hypervisor or appropriate driver can choose to implement this discovery as appropriate. The disadvantage is that every component that needs discovery will then have to build this at the implementation layer. And that can be a bit of a disruptive change for a whole bunch of open stack components. The other option is to implement this as a separate component altogether. So all of the discovery related work can be housed in its own service, in its own component. The advantage here is that it alleviates the problem from the previous point, which is making sure that you can add discovery independently of other components, so there's no disruption to the existing components. The disadvantage is that this service then becomes pretty complex because it has to understand a whole bunch of different open stack components, it has to understand a whole bunch of different drivers, other under-reach open stack components, and then be able to report discovered artifacts back. So we chose the first approach, but I just wanted to lay out the different options we considered while thinking about this. And the reason I wanted to lay that out is because when it comes to contributing it back, these are the sort of discussions that we hope to have within the community. In terms of future work, as I alluded to earlier, we're working with Nova Networks today. We're still trying to think through the process of how this would work and fit within the neutron context. We're also trying to see how we could build this with volume discovery and what makes sense in Cinder as well. But lastly and most importantly, we wanna start talking about how we can work within the community and understand the contribution process and figure out how to give this back. Before I take questions, I'll quickly summarize what we just spoke about. We found open stack as do our customers, a great solution to cloudify your internal infrastructure. But if that's the course you're taking, if you're trying to cloudify your existing internal infrastructure, you need to be able to also leverage all of the assets you've built up over time. This is why we created a lot of these pieces to be able to go in and discover these assets and report them to a fully functional open stack cloud. The goal was to be able to be completely seamless, which as you saw it was. The goal also was to be able to get up and running within minutes, which again, as you saw it was. And we can reliably claim that you can have a open stack cloud up and running in minutes instead of months, which was our whole goal. That's all we had. I'll open up the floor for questions to all of us. I was beginning to think it was a flawless presentation and there were no questions. Thanks for the presentation. So this one question I had is, so the whole workflow that you demonstrated, so at what level of scale have you actually tested this out? I mean, for a couple of VMs you have demonstrated this, but actually in your probably testing environments, like at what level of scale have you gone and then what is the kind of recommended a number of VMs or instances that you suggest here? So we actually not just tested this, but this is an actual live production use and we have folks that are using this in two levels of scale. One is the number of hosts that they have, the number of hypervisors and that we've tested in the order of hundreds and we've worked with VM scale in the order of thousands. So that's where we're at today, which is not to say that the process doesn't scale beyond that, it's just what we know of. And in terms of recommendation, if the hypervisor can handle it, the discovery can handle it, right? It's not like the discovery adds more load to the hypervisor. Because you have a periodic task running behind this and then that might take its own time to do the discovery, so I just wanted to basically understand into that perspective. The semantics of that periodic task is actually not very different from all of the existing open stack. NOVA has a bunch of tasks that run periodically to check power state, et cetera, right? It's actually very similar to that. So anything that applies to those tasks applies to these as well. So after you've completed the discovery and you have this new environment, is the idea that you would continue to use it with both views? Like people are still using it as if it's a VMware environment and others are using it as an open stack environment at the same time, are there any problems or conflicts that come out of that? Ideally you'd want them to switch to the open stack API because that's why you set it up in the first place. But the reality is that people do still continue using vCenter specifically because it has a nice interface, it has a well-established set of expertise around it, which is not necessarily the case on KVM. So that's why we built this whole synchronization piece, not only to be able to discover it, but there's a whole bunch of stuff that vCenter supports that open stack just doesn't, right? Like Pushkar demonstrated the template to instance conversion. You just can't do that in open stack. So you have to go to vCenter and do it. And we had to make sure that the pieces that we were building were resilient to that. So we do allow that dual view, but if the whole point is consuming the resources, you should be doing that through open stack because that's the best way to do it. And that's the reason people want to open stack because vCenter is mostly administrator only. They want to open it up to their internal users. Right. So we also have some t-shirts available in the front rows. So please help yourself. Okay. So there are no more questions. Thank you. Thank you.