 Okay. Can everyone hear me fine? Sounds good. Thank you very much for showing up. When we saw initially about 100 people signing up for this session, we figured out, okay, probably we'll have to take the keynotes room or something like that. But right now it seems quite optimal. So, today we'll do 45 minutes overview overview of what one may do with OpenStack and VMware being in the same room, being in the same environment. So, we'll do a little bit of a slide where, no this by PowerPoint, I promise. We'll do a demo at the very end and we'll discuss about use cases. So, the main question is why, given the OpenStack and the environment, given different hypervisors available, why one may want to integrate with VMware and which use cases are better enabled in a such way? So, before we begin, a quick round of introductions and I will actually start with my co-presenters. I'm Santos, I'm a product manager at VMware. I'm a product manager at OpenStack. It works, it works, it's good. Hi guys, my name is Evgenia. I'm from Mirantis and I run partner integration team. So, my team was the one who made sure that Mirantis OpenStack district actually supports VMware as another technology. As one of the beautiful options. So, I'm Dmitry Nolakovsky, I'm a sales engineer. So, I work in the fields in different places, originally based in Ukraine, but haven't been there for quite a long time. I work with customers and that's why I like to talk about use cases because I mean it's great to integrate things with each other, it's much more interesting to see how these things work together. So, the first question to go through is actually why, as I was starting. So, OpenStack as you all know is a technology which allows one to combine a number of technologies in the data center on the same API. So, it may be a number of hypervisors, a number of storage technologies, a number of network technologies. And it all comes down to specific features which your environment needs from a VMware hypervisor, maybe from an Oracle VM to run some Oracle stuff and so on. So, before we go into use cases, I actually want to do a quick quiz. So, please raise your hand if you have some of VMware's products in your infrastructure. You guys are doing good business. Okay, please raise your hand if you have an OpenStack installation in your infrastructure and it's beyond POC, so it's running some real stuff. We're getting there. And please raise your hand if you tried, if you already played with integrating the two. So, OpenStack and VMware. As I said, we're getting there. So, let's take a look at a couple use cases. So, these are the ones which we see in the field and which actually generates, which we've seen generating a lot of customer interest. That's why we in Mirantis and VMware and a bunch of other folks are investing into getting VMware's hypervisor properly supported in our deployment toolkits. So, number one is from where the whole thing started with OpenStack actually is a developer enablement. So, in such use case, what we usually see is that the company is happily running some infrastructure using VMware. So, for example, vCenter with vSphere obviously, some maybe vSend, maybe some other storage back in it and some workloads already working without cloud enablement on top. And the organization becomes interested in enabling developers with all this agile, DevOps kind of self-service style to be able to provision stuff on this infrastructure without needing to interact with admins all the time. So, that's actually the business driver which comes to customers and they come in and say, okay, so we want all this DevOps style, developer enablement style, development agility and that's one of the scenarios we're dropping in OpenStack on top of existing infrastructure with VMware which is already running. Becomes like a good way to do a first step and to enable developers with OpenStack APIs. The second use case which is actually my favorite one is what we call heterogeneous infrastructure. So, as I was saying earlier with OpenStack, you can combine different technologies under same management umbrella. OpenStack API, OpenStack front-end and so on and so forth. And given that, it becomes possible to have number of hypervisors in the environment. For example, KVM to run some type A workloads. VMware to run some type B workloads which run better on VMware. Oracle VM to run Oracle stuff and get the certified configuration and all still managed under the same roof of OpenStack. So that's the heterogeneous infrastructure. And well, I've been talking about use cases and I always like to show this guy. So it's like in our domain, people often talk about integrating this and that without clearly defining why. So that's kind of, I keep reminding myself with these guys that it's not like we just integrate these things, we just need to actually support some certain use cases and understand why we do it, not just put things together. And the third and the main one which we will expand on today is catering to application needs. Since different applications, different workloads have different requirements from what they expect from infrastructure, from what they want infrastructure to handle for them or what they can handle by themselves. And that's also use case in real enterprises when some company has a VMware infrastructure or some stuff on it, wants to have some other types of hypervisors and technologies sitting nearby, but still unite them under the same management umbrella. So as you can see, these are like three sides of the same coin, although it's not the way how it works. So the use cases are quite closely related and coupled, but the business drivers are usually different. So at this point we come to actually the problem and I will turn over to Santosh in a second. So the problem with the use case number three which we are going to dig deeper now. And the problem sounds like, okay, so I have in my organization, I have a number of development tracks, developing the applications which have been around, for example, for quite some years, even before the cloud arrived. And I also have some developers which ran on AWS and which do all these DevOps and continuous integration style. How do I make all of them happy with a single private cloud? And that's what we are actually going to elaborate on. So with this I'm turning over to Santosh. Thanks, Dimitri. Can everyone hear me? So before I jump into the different types of apps and what the actual problem is, I wanna talk about a user story to kind of set the context. So I was talking to this customer who's a telecom company, not allowed to bring up the name here, so let's just say telecom company. And they're running two data centers. One data center they use where the developers, they go have fun developing mobile backends or scale out web applications and stuff like that. And they have another data center dedicated for running IT workloads which the IT has been building up over a period of time. And now they're trying to kind of manage both these data centers using a same management platform, using the same infrastructure provider and they chose OpenStack. So the point here is, enterprises over time they have different applications that have different needs and they're looking for tools that help them manage the different applications which have different needs using the same tools, same management tools. So looking at applications, you can classify applications broadly into two different types. One is the cloud-ready applications, the cloud-y applications. So what do I mean by cloud-ready applications? These are applications that are aware of the fact that they are running on a cloud infrastructure and they exploit this fact to be massively distributed which means based on demand, they can automatically scale out. If there is more demand, they can spin up more instances, more virtual machines to scale up. And when there's less demand, they can scale down. They are stateless, which means they don't have any local data, they don't have any local cash or data which gives them kind of implicitly HA which means if an instance goes down, nothing wrong, you can automatically, the application spins up another instance to take its place and there is no downtime. So these are cloud-ready applications and these applications are happy running on cloud like your internal cloud on OpenStack or AWS. And the other type of application is the classic application. So enterprises, as I mentioned earlier, like over time they've built up applications that need some kind of guarantees in terms of reliability and availability from the infrastructure. And these applications, if you look at them, they're typically not very distributed. They run out of a single virtual machine or maybe two virtual machines and they don't, because they're not distributed, they cannot scale out on demand. Whenever there is need for increased capacity, you have to manually add more CPUs or add more storage or add more memory. So you have to scale them up manually. And another aspect of classic applications are, they process data locally, so they maintain kind of local caches. So they're stateful, they maintain state. And because they're stateful, you cannot afford to lose that state. And that means that to be able to maintain that state and to be able to provide high availability, these applications depend on the underlying infrastructure to do that. Which means you cannot just take that stateful, the classic applications and not throw them on a cloud that does not guarantee or that does not provide reliability or availability guarantees. So what do we do? We have, what do we do with classic applications? Obviously, we need a single management tool to be able to manage both cloud-ready and classic applications. One option is we could throw all the classic applications out and just live with cloud-ready applications. And if I suggest that, most of you are going to think that I'm a stupid person giving a stupid idea and you'll be right. And so we cannot do that. So you cannot just throw your existing classic applications out. And the other option is you can rewrite all your existing classic applications to be cloud-ready. There are a lot of guidelines and stuff online to rewrite that. But the point is you don't want to re-design, redesign, tinker with your application to be able to fit what the infrastructure gives you. You just have to be able to write your applications to do what they do best and leave stuff like reliability, guarantee, reliability, and availability to the infrastructure to provide to you. Say for example, one really good example is NFV, network function visualization. So in simple terms, what you do is you take all your network functions, like routers, firewalls, and you virtualize them. And these are good at doing network-related functions, like firewall does, firewalling, router does routing, so on and so forth. You don't want to worry about having to build HA into it and fault tolerance into it. And you want to let these applications do what they do best and leave the rest to the underlying infrastructure. So some applications, you just don't want to rewrite them. They're not meant to be rewritten to be cloud-style. They expect the guarantees from the infrastructure. And the other problem with re-writing is it's expensive, both in terms of time and money. If you're like Mr. Scrooge, if you have all the money in the world, all the time in the world, and most importantly, if you can ask your customers to hold on to what they're doing until you rewrite your applications and then come back to you, then maybe you could rewrite. But there's a lot of time and money involved in re-writing these classic applications to be cloud-ready. So back to square one. We have this nice tool, OpenStack, which gives really nice APIs to manage your infrastructure. And how do we use this OpenStack to manage both your classic applications and your cloud-ready applications? And to tell you how, I'll hand it over to Virginia. OK, yeah, I guess you can hear me. So there are three presenters, like it's mess, yeah. OK, so I'm going to talk a bit about the solution, about how actually can we do and marry VMware and traditional hypervisors like KVM and OneCloud. And you know, guys, we have a huge presence of VMware in OpenStack community. And the whole OpenStack team, like VMware team, they contribute a lot. And actually OpenStack community embraces it so that a lot of work was done in Nova, Cinder, Neutron, which is amazing. So yeah, and we have different types of drivers and plugins there. So actually, and this is how the whole OpenStack ecosystem that enables and integrates VMware solution looks like. OK, we have Nova. And for Nova, we have this center driver. So as you know, guys, especially those who actually have this VMware plus OpenStack clouds in place, yeah, you know that actually Nova Compute driver works through the center API and manages ESXi clusters, not just ESXi itself, but the group of ESXIs and Nova considers this ESXi cluster as actually like one compute node. This is how it works. And actually it provides you a lot of nice features with that. There is a Cinder driver, which is like VMDK. So it allows you to use vSphere data store as a backend for Cinder. And as a real backend, you can have vSan or EMC storage or whatever. So it depends on how you actually vSphere cluster is configured. It's in below vSphere. Yeah. OK, Glance. So for Glance, the same VMDK driver, but for storing images and getting them to the cloud. For Neutron, there is an NSX driver. You know, guys, the SDN is a buzzword in OpenStack. So VMware has NSX solution. Which is kind of really nice fit if you're enabling VMware for OpenStack. Because this is the same family of technologies, like it's all from VMware. So looking at this picture, you can see the best of the breed if you are talking about how to bring VMware to your OpenStack cluster. But it doesn't mean that this is the only option you see. OK, so having all these drivers and plugins you will be able to install OpenStack on top of it and manage through OpenStack APIs VMware virtual resources. And moving forward and answering the question how actually to build heterogeneous cloud and how it usually looks like. So here's the very typical picture of deployments that our customers are usually looking for. So what they really need, they really want, they wanna see both, for example KVM and vCenter as hypervisors in their cloud, just one single cloud. Sometimes they want to have two clouds, but it's not a heterogeneous cloud anymore. So you see, we have some compute nodes responsible for managing KVM and a couple of compute nodes managing actually yes, excite clusters via vCenter. You don't see the networking part here, but let's assume it's NSX and NSX driver, NSX plugin. Also, if you wanna play you, we always can use Nova Network, but it's not for production, so everybody knows that. And how we usually solve this problem on how to distinguish these compute nodes and compute resources. Usually you can do it using host aggregates, availability zones, or even different types of images. So also we have regions and cells, but usually what we offer is different availability zones for different types of compute resources. And I guess that's it for my part. I'm going to pass the ball to Santosh and he's going to talk a bit more about VMware and application that can run on that part of OpenStack cloud. So please go ahead. Thanks, Evgeni. Thanks, guys. Before I jump into that, how many of you were at the keynote yesterday when they rolled in the BMW I8? Cool car, huh? So towards the end of the speech, Stefan Laird, he made a very good point. He mentioned that OpenStack is just one tool in your toolkit to be able to build your cloud, but you need more than OpenStack to have a reliable cloud running. And I kind of like it when he said that because I was not planning on leveraging that, but now that I have a good point to lead into my talk. So what are the other pieces that you need to build your cloud apart from OpenStack? So OpenStack gives you the nice consumption layer with APIs and it abstract services into compute network and storage, but you need something underlying. You need a hypervisor for your compute. And vSphere gives you HA for your application when they need it. It gives you fault tolerance for your application when they have to keep running without any downtime when the applications in your instances dies or when a host goes away. You need some kind of reliability guarantees and vSphere gives that to you with HA, DRS, and Vimotion. And going back to his risky note, when Matt Haynes from Time Warner Cable, he was talking about what kind of features they were looking for when they built his cloud. Like one of the important points he had upon his slide was high availability and DR. So these are features that are important to build your clouds to be able to provide the kind of guarantees that your applications need. And vSphere provides all of that to you. And for applications that need networking functionality, like for applications that want to deploy networks automatically, like network functions, like routers, firewalls, NSX provides a highly scalable network virtualization software. Applications can use that to leverage the underlying infrastructure to dynamically create networks, routers, and use them for their applications. And on the storage side, VirtualSan provides a cost-effective distributed virtual sand-based storage, which based on solid state devices are combined with your spinning disks to provide a cost-effective storage solution for your applications. You can do things with vSan, like you can create a tiered architecture, like tiered storages, where you can have different types of storage and that instances them. And this is just the infrastructure, right? Now you have the infrastructure and you've deployed OpenStack and you have a cloud and up and running, but that's not the end of it. You need tools to be able to manage and operate your cloud. And that's provided with vCenter operations and login site where you can use these tools to manage your OpenStack cloud. You can use the same tools that you use to manage your vSphere cloud, your vSphere tools, the same vCenter operations and logins that you can now use to troubleshoot and operate your OpenStack cloud. With that, we jump into the demo and give it back to Dimitri. We've actually checked a bunch of demos which were done on the last summit and we decided that ours will be pre-recorded, just to many surprises. Okay, not okay. Scopes and stuff. Okay, so as we were saying and we'll say later today, there is a number of tools and ready to use toolkits which you can deploy OpenStack with VMware already. So there is an offering from VMware, there is an offering from Mirantis. Canonical folks also say that they have something around this. So what's important to understand is that all these solutions are using the same drivers from upstream. So that's just the question of who kind of does the nicer installation. And here in this demo, I'm actually running two ESXi nodes on this laptop using nested virtualization. I also have a vCenter appliance and I have a shared storage backend with NFS to power the live migration for me. So here you see the horizon on the admin page on hypervisor's outlook and you see the only one hypervisor. And this hypervisor is the instance of Nova Compute's vCenter driver which exposes behind itself two ESXi hypervisors. That's what's important to understand. OpenStack right now consumes VMware's hypervisor as a black box. So it knows how to talk to it but doesn't know what's happened. What happens inside? Well, knows but to some extent. So what I'm doing here is I'm showing the beautiful hypervisor and then I'm probably showing the vSphere client. So this is the admin interface of vCenter. And you can see here is that I have two ESXi nodes, two ESXi nodes under this vCenter cluster. So I have a high availability enabled for this cluster. What does HA for a virtual machine means? If virtual machine goes down or if the entire hypervisor goes down, then vCenter and vSphere will work on restarting this virtual machine from a shared storage as soon as possible. Here I actually did a bit of learning myself and I found that there are two options for doing HA. One is host monitoring. Second one is VM monitoring. So for VM monitoring you need a virtual machine with VMware tools installed inside and in such case, vSphere will be able to monitor a very specific VM. In case it goes down, it will be possible to restart it. Host monitoring is for restarting all the VMs which were on a somehow died ESXi node. Also here I have for this cluster, I have enabled DRS. So DRS is a distributed resource scheduling. This is the mechanism which allows a vSphere cluster to balance and rebalance the virtual machine among ESXi nodes. Given that there is a shared storage and it's possible to migrate them between the nodes. That's actually something which people often ask from OpenStack and it's not there. And also for this cluster, I have enabled the vMotion. So this is the inverse native life migration for virtual machines, which given the presence of shared storage in my setup allows me to move one virtual machine from host A to host B, hopefully without downtime. So what I'm doing now is I'm going back to OpenStack Horizon and I'm creating a new virtual machine. So since it's a virtualized environment, I'm not very heavy on resources and I'm using just a small image with seros. Enough to demonstrate that it can stay alive while running on VMware. So I launch it and yeah, full disclosure. So since the setup is quite small, I used, I move it to stitch some points. So it's not always that fast as it shows here. So it's take some time. And the virtual machine lands to vSphere. So right now in vSphere web client, I can see that it's actually running here. And right now, well, at least in my setup of OpenStack, I couldn't get an OVNC console from Horizon. So in order to get to console, I'm actually using vSphere's native console. Now what I'm doing is I'm assigning a floating IP to this machine to be able to ping it from my host. So from that point, I will be using this ping as like an indication whether machine is alive or not yet or already not alive. So right now it's alive. Probably next I will do live migration, yes. So now I go back to vSphere and I say, okay, probably I need to move this virtual machine to another host on whatever reason. Maybe I want to do some maintenance. And I go through the native vMverse, native vSphere's live migration flow. So here I actually have two hosts in this cluster. If I would have multiple clusters or multiple hosts, I would be able to point to a specific cluster where I want this machine to migrate or otherwise leave it to vSphere to decide where this virtual machine will go. What's important is that OpenStack has no idea that this is happening. So vSphere is playing internally with placement of virtual machines. I know on which host it runs at the moment, so I will just select a different host to do the migration. And I say, yes, let's do it. I was hesitant, don't know why. I come back to ping and on the top right of the screen you see the progress. So it's being moved to another host at the moment. And luckily I'm not losing a single packet. So in this quite slow setup, actually if I do 10 live migrations, two of them will lose one packet. In real world, it's not supposed to happen. And now I'm doing the most evil part of the demo. I'm killing one of the hypervisors. So I just migrated, I run nested virtualization with VMware Fusion. So I just migrated a virtual machine to hypervisor two and I'm killing it. So I'm just shutting it down like it went out of power. So it's dead. I will show that ping is lost. Yes, it's lost. So from this point, vSphere kind of quickly catches up with the fact, oh, it's a stitch. vSphere quickly catches up with the fact that it lost one of the hypervisors and it migrates the virtual machines to a different one, to surviving one. It is, this is not fault tolerance. So there is some time when machine is actually down, which takes for vSphere to do the actual migration. In my virtualized setup, it was taken actually up to the couple of minutes. In real world, it's working faster on hardware, on real hardware. Another option is to do fault tolerance here. So it's not on the demo, but I want to say a couple of words that for a virtual machine, which is provisioned from OpenStack and runs on vSphere, it is possible here to do like the right click and enable fault tolerance protection for it. This will mean that basically vSphere will run a sort of a clone of this virtual machine with less resort consumption on another hypervisor. And in case if primary hypervisor dies, it will quickly switch over to another one with hopefully no downtime at all. vSphere didn't like the storage which I used in this demo, so I couldn't unfortunately record the fault tolerance maybe next year. So here, the virtual machine came to another host, to another host. You can see that there was some packets which were actually lost during this process because again, HA, it's not no downtime. It's as few downtime as possible for a virtual machine. And it's back alive. Yep. So as I was saying, the whole point and the whole interest from my side of integrating OpenStack and VMware was to give the answer to the question of many customers and prospects who are saying, okay, I have some applications which have been developed in a classic manner, like Santosh explained. How do I make sure that I don't get them killed if I start using OpenStack? So that's basically one of the answers. You can use VMware in parallel, you can buy some new licenses or reuse existing ones and use them, utilize the VMware part of the infrastructure to help the applications which need more support from the infrastructure. Okay, so that's it with the demo. You're welcome to ask the questions in a couple of minutes. The last section here is, the last section from me here is, not the demo, I'm done with the demo. How to get? So as I was saying at the very beginning right now, multiple companies and we are proud to be one of the first at Mirantis work on enabling their deployment toolkits to work with VMware. So in Mirantis, there is a number of options how you can do it already today. So number one and kind of the one which you obviously will see being a VMware user is you can consume VMware integrated OpenStack, VO. That's the product which was announced early this year, right? Is it still in beta or already in GA? It's still in beta, yeah. Still in beta, okay. It's until Q1 of next year. Q1, okay, good. Number two is, Mirantis is partnering with VMware since Hong Kong summit and we are doing the similar kind of integration in our distribution, Mirantis OpenStack. So we enable deployment of OpenStack clusters combined with vCenter as a hypervisor. We enable connection of Cinder as a block storage to vCenter and we enable a certain use cases with NSX as well. The long-term goal and the stretch goal is to have the automated scenario of deployment of heterogeneous clouds. Right now we do it yet through some configuration adjustments as well. And if you feel adventurous, you can always go with do it yourself. The drivers are in upstream. The documentation is out there. So if you like building things from bottom to top, I liked it once. I was using Gen2 Linux. So you can go with do it yourself as well. Okay, so that's all with the slideware and videoware. You're welcome to ask questions. We have questions. Thanks for that. I think the biggest problem in terms of this kind of context when we have tried to have a mix of different type of private cloud system in one framework, the networking part is really headache. And you start using networking. Yeah, networking. So you assume that actually NSX already utilized that what if we don't use NSX, we use just vShield or some traditional, you know, other type of solution then how we can deal with the networking part? I told you that will be the first question. You want, do you want to start? Yeah. That's all. So yes, in this presentation, we're using NSX as a reference on one simple reason, it's battle tested. So VMware has been doing it, we've been doing it. But we at Mirantis, we kind of strive to enable as much choice as possible. And we track other folks such as Plum Grid, some just Nuage and other virtualization technologies to see and to elaborate on working stories of multi hypervisor environment with different options for networking, not only in NSX. But NSX is battle tested, it works now. So that's why it gets promoted. In next six months in killer cycle, I would assume that there may be also an open source solution available, given some developments in SDN right now. And well, obviously, I mean, VMware looks at the market, Mirantis looks at the market. So this is today the safe bet is NSX in six months, I guess we will have some more diversity and it's good for the market, it's good for VMware as well. Any other questions? Well, I had two, the first was about NSX. So that's done. The second was about legacy because in many use cases, customers, they want to have open stack up top of VMware, but they want also open stack to be able to manage all the VMs that are already existing. And that's sometimes quite a nightmare, especially for the networking part, but even just to have the discovery features to be able just to see the VMs at first and just to be able to kill them when they don't want them anymore. So is there, I mean, some solutions right now or do we have to wait more? That's actually a very good question and I did not anticipate it. The even more interesting aspect of it is that VM in VMware, in pure VMware environment and the VM in open stack are also slightly different concepts, right? So in open stack, you run them from images as a templates or from block devices as kind of already pre-set up and some more configured and with some persistent data templates. And in VMware by default, you create a virtual machine and you just install stuff on it. So that's actually a very interesting question and I'm looking on Santosh to figure out whether you guys have already thought about this problem. It can obviously be done manually like the white cloud, very close engagement but we don't have a tool and just to finish on that. So that's for VMs and what about networking? For instance, if someone tells me, I want to use my V-switch configuration that I'm used to in my ESX, what's your advice there? Do you buy an ESX or do you go with something else? I mean, without an ESX. Without an ESX. Yeah. Without an ESX, if I understand correctly, you just want to be able to use your V-switch with open stack. So with the current product and beta, we have a driver for the V-switch as well. It'll be upstream so it's not just specific to the product. It'll be upstream then when it's accepted, reviewed and accepted, you'll be able to get that upstream but it will be available before it goes upstream in the product because it's our product and we can push it in. So the driver for DVS, the standard switch which is running on ESXI nodes, that's kind of elaborating further on the answer of if I don't have NSX. So obviously with a setup like this, you'll have less capabilities and less flexibility on what any SDN gives you like with logical networks on demand and services injected inside. But if you just need to run VMs, probably the option of running with the driver which folks will upstream for DVS is even better than running with Nova Network. Thank you. Any other questions? I've worked for a large company doing a lot of enterprise application hosting on VMware and all these legacy things and we are just starting with OpenStack projects. I wonder why would we do it? Why, what would be the motivation to merge OpenStack and VMware world? What's the benefit for the customer? Are you seeing customer cases that are operating SAP Clouds, Oracle applications and all these legacy enterprise applications being managed through OpenStack? And if so, then why? Imagine a use case when the use case number one, which as I was actually talking about, so developer enablement. You have VMware infrastructure which you happily run and it works great. You have the licenses. You don't, I mean, it's good for you in terms of like costs and so on so forth. But all of a sudden you get this new team of the developers who want to deploy applications in a cloud way. And maybe they also use some kind of a public cloud like Rackspace or, well, it's not a public cloud anymore, like AWS for example. And they want to run a private cloud as well. And you don't want to buy a bunch of hardware for them to run their isolated cluster on KVM. So you connect to OpenStack to VMware. You give them some quota on VMware hypervisors, on VMware storage, on VMware networking. And you still get them managed through the tools which you are familiar with. For example, VC Ops operation manager. But they will not come to you and fill the tickets. Okay, create a virtual machine for me. Create something else for me. Increase me the quota and so on. They will go through OpenStack API and through OpenStack user interface in a completely self-service manner. But your existing and happy infrastructure will serve them. Okay, thanks. But when thinking of running the existing installation and supposing that there is no import process for importing what's already running, then it means that I would have to, I would need to have a big motivation to really do it. Supposing that I will have to reinstall and already deploy everything through OpenStack to the VMware background. It's not a big, all the problems. There are some applications that are very suitable for OpenStack. There are some applications that are not suitable for OpenStack and trying to kind of force them into OpenStack may not be the right thing to do. Yeah, that's why I'm asking for the experience and the motivation why people might be doing this. Well, I mean, being completely blunt, one of our customers didn't want to buy the cloud. Sorry. And they have been using and they are still using a very extensive setup of vCenters and vSpheres. And they were quite nice, but they wanted to do this developer enablement thing which I was talking about. So they got a few new development tracks which needed private cloud capabilities. But there was no kind of, because the company evaluated and decided that they don't want to go one more step up the stack with enabling the cloud through vCloud. So they decided that for API and cell service parts they wanted to do OpenStack. And we're doing it for them. So, and they actually didn't care that much about importing existing running stuff. So there is an intention, for example, later to investigate some virtual machines which are running there, which have been put there through the usual admin process like create a ticket and so on and so forth. There is an intention to investigate, okay, let's do a block device snapshot, select import this stuff to OpenStack. Let's instantiate it as an OpenStack managed virtual machine, but it's not the top priority one. The top priority one is to make developers work faster. Yeah, okay, thank you. You're welcome. There are quite a few things which are being sublimated into the hypervisor of vSphere 6 from vCloud director. What's the plan there for the OpenStack enablement of those kinds of features? Well, vApps, for example. Yeah, absolutely, yeah, yeah. So for those of us that are on the vSphere 6 beta, then there's nothing we can currently do with you. I mean, all the OpenStack code and the drivers that are available. But if you're already cussing Cahood, then, yeah, it's a... Currently, we are testing it. We are testing our product on vSphere 6 as well. And then we release it, there'll be something. Okay, thank you. I'm not going even to try on this question. Any other questions? We're running five minutes late, but if you folks have questions, we're glad to answer now or after the talk. Two, three, four, five. Okay, that's it. Thank you. Thank you so much. Thanks for being here. Thank you.