 I want to thank you all for coming out. I appreciate it. I hope this is the cheese at the end of the maze. If you had a hard enough time trying to find this place the first time, down, going through not that set of stairs, but this set of stairs to find the right room. But welcome, and I'm glad to see you all today. My name is Russ Pavlichek, and some people may not know who or what I am. So I've been around since the open source world since 1995. And I converted to a Linux desktop in 1997. So I've been hanging in there for some time now. And I've got a reasonable past in the open source world. I did a lot of talking and speaking inside the early days, including a few years on the old Linux show webcast. By the way, these slides will be up on zenproject.org. In fact, there's actually a copy that's almost identical that's up there right now. So if there are things on the slides you want to capture, just go to zenproject.org and you can get hold of it. But I've done a number of things. I've got a really big mouth, and I know how to use it. So that's why I'm up here. And I actually was out of the open source world, as a lot of people who started, you join a company that's using open source and not necessarily producing open source despite your best efforts. And then you end up with that company being sold to a company that's allergic to open source. And so you're in purgatory for a while. And so when this job became available, I jumped on it. Because I technically work for Citrix. However, that being said, I do not have a single goal that relates to Citrix. All of my goals are about zenproject. So I stand up here as one that wants to do the best for zenproject. And whether or not there's a Citrix product that uses zen that makes some money, well, that's great, because maybe they'll keep sending me a paycheck. But outside of that, I have no particular interest. Just a word about the zen stack, because this is kind of an interesting point of confusion, even among people who've been working with zen for a while. Zen has a whole bunch of pieces. The hypervisor, which we'll talk about. The zen API, better known as Zappy, which is sort of the cloud layer. Or cloud enabling layer, which we'll talk about some. The zenproject itself is a Linux Foundation collaborative project. Has been now for roughly six months. And that's the stuff that I'm going to be talking about. There is something that was called zenserver, which was a commercial product, which now also is open source as a open source project. It's kind of a distribution of zen. It's a disk you put in, and it comes up, and it's much more user friendly in that regard. But that's not the subject of what I'm talking about today. Zenserver uses the zen hypervisor, which is what we will be talking about. But there was a lot of confusion, because when Citrix bought the old zen source company, which initially dealt with zenserver, a lot of people thought, oh, zen is now closed source. Well, nothing could be further from the truth. Zen is open source, period, end of discussion. And I just want to make that clear, in case there are some people that hear old things. So let's start first with just the notion of what's the cloud problem. How many people were in the IT world one way or another prior to the arrival of cloud? Show of hands? OK, so some of you folks will find this familiar. It's kind of interesting when I do this, if there are a lot of younger people, sometimes this is educational, so I have to kind of go through this. In the old world of IT, stability was everything. It was constant service availability. When you left at night and you came back the next day, the report you wanted to hear is nothing changed, nothing, absolutely nothing. That was perfection. That was wondrous. Because when things changed during the night, someone got a phone call, most notably about 3 AM, saying, the HR server is down. We have to get it up right away or you don't have a job at 9 AM. So somebody is running around trying to solve problems, trying to deal with things. When things changed, that was bad. And that was the mindset of IT for decades. And so status quo was everything. Change was bad. And in fact, when you had to add something to your IT situation, you had new servers coming in, what did you do? You beat it into submission until it became part of status quo. You unpacked it, you uncrated it, you tested it, you loaded it up, you made sure that it was identical to every other one in the row, and you made sure that it became part of the status quo. In fact, it was standard inside most of the organizations that I worked with as a consultant and so forth, that IT guys could just walk down the rows of machines and without looking and say, OK, this is the HR rack, this is the database machine, this is this machine, this is that machine. You knew where everything was all the time because that was the nature of IT, stability. Now in the cloud, there were issues. How many people were cloud advocates inside their organization? We had to go up to someone and say, you know, we should be doing this. How many people really enjoyed that? The hands just went down. There's a reason, you know, because in the cloud, it's a different thought. It's not about stability. Everything's staying exactly the same. It's about availability. When the service is there, when you need the service, you want it there, period. Do you know where it is? Do you know what hardware it's running? No. You don't care. In the cloud, as long as the service is there, when you call it, success. So you think, well, that's not a big change. But then you start to realize that in the cloud, change is good. It's not bad. And when you start telling people that, especially if you were a cloud advocate, people could start acting, reacting to you as if you were going to your saintly grandmother who goes to church every other day and saying, I'm becoming a saint mist. Or as if you were saying, I enjoy stomping puppies to death. Why? Because everyone in IT knew that change was bad. Now you're coming along and saying, in the cloud, change is good. You want things to move and to breathe and to react to you. So suddenly, change is everything. If things aren't changing, something's not working. Because the value of the cloud is that it meets the need as the need arises. So as you need things, they appear. As you don't need them, they disappear. That's the cloud ideal. So you've got this difference of opinion that comes up pre-cloud and post-cloud. And this is documentation that I'm not a graphic artist. This is just sort of looking at the basic layers. You've got a cloud orchestration layer. You've got a virtualization layer. And then you've got operating system and apps. And so you've got all these various pieces that have to play together. And the layer we're going to talk about today primarily is in that virtualization layer. That's where Zen tends to play. We do sort of touch on the orchestration layer, but only touch. Why? Because the cloud is young. I mean, I've been working with the cloud since before it was called cloud. I was with a startup that did what was then called agile infrastructure, because we didn't know what the heck to call it. It just was stuff moving around autonomously, according to design. And then the cloud word got thrown around and said, OK, I've got to say, well, yeah, that's basically what we're doing. But if you look at the history of cloud, every couple of years, concepts change and migrate, because it's very young. We're not quite sure what cloud is going to look like in five years, because the cloud today doesn't look like anything at all like it did five years ago. So it's maturing. And you want to make sure that you have this separation between these two layers here. Because if you're getting tied into a cloud orchestration layer by a vendor or what have you, that has this is what the cloud will be. Well, guess what? In five years, it's not going to be. So part of the Zen concept is we want to be that hypervisor layer, the virtualization layer, that enables your cloud layer. So as cloud matures, we're there for you. We're not there to push you into our mold. That's kind of one of the basic concepts of Zen. So what about virtualization in the cloud? We know it must be stable. If you're going to be doing production level work, you want it to be there. When you say go up, you want it to stay up. When it says go down, stay down. It must be secure. Big issue these days. I wrote a blog post that actually kind of made a lot more traffic than I thought it was about cloud security being the 800 pound red herring in the room. It's the one thing that everyone talks about, but it's the one thing that few people really understand the basis for. And we'll talk about that in a little bit. Cloud must be configurable on a large scale. How many people have been in a situation where you find the old user sitting at the machine paradigm being used, and you think, no, no, no? I mean, I've got a gas station, like about a half mile from my house, and on top of each set of gas pumps, there's supposed to be a video screen showing me products and services and advertisements and stuff like this. And most of the time, what I see is a blue screen with a box in the middle saying, C colon is running out of room. Click here to continue. That paradigm doesn't work inside this sort of thing. And likewise, in the cloud, if you're running 1,000 different VMs or 1,000 hardware machines, you don't want 1,000 consoles to be going to. The cloud paradigm has to be one of one control for many, and it has to be efficient for you. It must take orchestration once again. If you have something that is static, it's not going to play in the cloud. Some brain has to be able to tell it what to do when you want stuff done. It must be multi-tenant. Not such a big deal maybe if you're just doing an internal cloud for your particular organization. But if you're a service provider at all, one of the biggest bugaboos is you can't have company A's cloud sort of overlapping with companies B's cloud. So company A can look at the HR system on company B. You want them to be totally unaware that they may actually be sharing the same hardware. There should be no visibility at all. So multi-tenancy is really important. And it should not lock you into one concept of cloud or one provider for cloud. There is a commercial one out there that is doing a very good job of trying to provide that lock-in for its customers right now with a recent announcement I heard. It was actually quite impressive in that regard. But the Zen culture, being an open-source culture, is we want to be there for whatever it is that you want to do. So we're going to let you pick, and we'll be there to provide the service. Now, how does Zen answer some of these needs in the cloud? Well, first off, it is stable. It's got a rock-solid record. How many people have ever used the Amazon cloud? Congratulations, you are Zen users. The rack-space public cloud uses Zen. Verizon just came out with a new Zen-based cloud. There are lots of these very large clouds that pick Zen. There's a reason for that, because we are stable. Software is very good and stable. We just celebrated our 10th anniversary, by the way. There was, we've had a couple of parties throughout the year. An email, I think, a posting just went out just on Monday about this actually being, Monday I think was the exact 10th anniversary of its announcement as being open-source. So we're very proud of that. It's served us well. And you have a number of partners as part of the Linux Foundation that are supporting Zen. And that's just a few of the bigger names that people recognize, but there are a bunch more out on the website, zenproject.org. You can find them all out there. Also, Zen is very secure. Now, most people know what SELinux is. There is something very, very similar to SELinux called Flask, which is actually created by some of the very same people who did SELinux. That is to say, everyone's favorite United States government agency, the NSA. And whereas there may be a lot of criticism about the NSA these days, no one has ever accused them about not knowing about security. And Flask provides that, and we'll talk a little bit about that later. It gives that sort of level of security to the VMs themselves. There's another principle called disaggregation, which we'll talk about a little bit. Disaggregation is a very powerful concept of being able to take out certain drivers and load them into their own dedicated VMs for the purposes of both security and performance. It's a really powerful concept. And then there's also sorts of architectural advantages dealing with a type one hypervisor, and we'll talk about that in just a couple of minutes. I do have a security talk, actually, how many people saw George Dunlop's talk in this room yesterday? Anyone at that one? Basically, it's my version of his talk, but if his talk, I think, is recorded, listen to his talk. It's quite good on some of the finer points of Zen security, really worth the time. Zen is also configurable at scale. This is, of course, a really important side of any sort of cloud situation. We've got multiple tool stacks, and we'll talk about what they are and why, to give you all sorts of command line capabilities. It is not a GUI-centric thing out of the box. Well, why? Because GUIs don't necessarily scale well, and also, if you notice, when you start dealing with a GUI, frequently it imposes a concept on you. And when we're talking about cloud, we don't want our concept getting in the way of your cloud concept. We want to enable your cloud concept. So, the GUI is not the center of the universe. In our world, the tool stacks are. So, we can easily integrate with things like Puppet and Chef and all these various orchestration tools that you may want to use. Now, about the tool stacks themselves, we can see that we actually have three entirely different tool stacks. We've got the default one known as XL, used to be called XMXL, sort of XM on steroids. That looks at the VMs and the host as one unit. So, I mean, when you're looking at one machine doing virtualization, one hardware machine with VMs on it, XL gives you that perspective. There's another one, of course, if you're working with KVM, you're used to working with Libvert and Bersh, and we can play in that area as well. We do that for compatibility. So, it gives you a slightly larger focus if you have a mixed cloud. And then we've got Zappy. And Zappy is sort of the cloudish layer. So, if you want to not look at the individual machine, but you want to look across machines to coordinate, et cetera, you might want to be using Zappy as a toolkit. Very powerful stuff in there. So, the three different toolkits have three different perspectives for three different reasons. You can choose what works best for you. Now, different solutions, there are commercial solutions that use Zen all over the place, and different ones pick it for different reasons. Oracle is using the XL toolkit if you use Oracle's virtualization. All the way up to Zen server, which, of course, uses the Zappy toolkit. It's a question of what fits best. So, if you're going to roll your own cloud or you're working with other clouds, it's up to you which toolkit's going to fit best. And once again, we want to facilitate what you have. And then the online cloud services that are out there, there's a number of clouds that use Zen under the covers. Once again, they're using different toolkits. Why? Because it fits their perspective on how to control a cloud. We're here to enable, we're not here to force you into doing things our way. So, let's take a look at Zappy for a second. What's involved in the Zappy layer? The Zappy layer includes things like basic VM life cycle. So, you know, snapshots, checkpoints, all that sort of good stuff, migration. Storage migration is in there as well. Resource pools are in there for flexibility. There's event tracking and upgrades. All these sorts of cloudy things that you're expecting to be enabled on a cloud layer. The Zappy layer will give you if you want to work with that and then put your orchestration on top of that. So, if that fits, that saves you a lot of time. But once again, we're not here to force anyone into anything. And if you take a look down below, there's an entire page that talks about all sorts of issues. Once again, for latecomers, these slides actually are online on zenproject.org now and they will be posted as well, I think, within the conference website. But there's plenty of URLs that you'll see throughout these slides. So, if you're looking for more information, because this is an overview talk, there's all sorts of stuff out there. Also, Zen multi-tenancy, incredibly important. I worked for a little while with a company that was trying to do a cloud with a hypervisor layer that really wasn't very good when it came to multi-tenancy. And that's a disaster if you're a service provider. I mean, it's critical that you block off for each individual organization a cloud that looks like it's your own and you don't see the other guys that's sitting next to you that may be on the exact same machine as you physically, but you can't see it. So, that multi-tenancy is absolutely important when it comes to anything above an internal cloud. Even in an internal cloud, you may want your human resources department to see only its machines and not the payroll machines and not the development machines. So, multi-tenancy, even an internal cloud can be incredibly important. And Zappy is a good layer to be working with if that's a problem for your organization. Zen doesn't lock you in. I've mentioned this before. We want you to pick what makes sense for you and we want to be there for you. I see too many providers that are falling back to an old commercial mindset, which is, okay, we gave you the tool, now we'll give you the layer on top and you're gonna work with that. And our concept of the world is going to be the one you play in. That's not the Zen way. So, pick what works for you and we'll try to meet you there. And if it doesn't work with you, guess what? We're open source. Join in and help us make it work for you. Because we want to be that level of a provider. There are a number of orchestration choices out there right now that deal with cloud. There's Zen Orchestra down the bottom. It's actually, that's a project that's sort of got a reboot about a year or so ago. It's trying to do just sort of a native Zen API and are not, excuse me, not API, but native Zen GUI. And they're actually doing some really neat things. And if that's something of interest to you, I'm sure they'd be thrilled to death if you join in with them. But then there are other things, of course, like Apache Cloud Stack. You see these fellows around at this organization. Open Stack, of course too. Open Nebula. So all of these things, we're trying to play well with others. That really is it. So we don't wanna restrict you from picking something that's reasonable for you when you're way of working. Now let's just do a minute on the health overall of Zen Project. Because if you've been around, how many people have been in the open source world for more than five years, just show your hands. How many people at one point or another in your experience heard Zen is dead? Yeah. And I won't bother to say who was saying it, but absolutely untrue. So let me just touch on this briefly. We are part of the Zen Foundation Collaborative Projects, which was a great move for us. That includes the things you see here, the hypervisor, Zappi, the ARM hypervisor, which there's some really splendid things going on in that area. If you're looking at ARM servers or even ARM mobile, there's a lot of work going on for Zen on ARM. It's really a very sweet story and I think I've got a slide on that near the end. If I don't have a slide there, just someone's screaming at me at the end and I'll talk about it for a second, but it's great. The Mirage OS, we'll definitely get to that. And then as far as the actual project self, the governance is somewhere between Linux Kernel and Apache in the way it goes. So I mean, it's very much an open source style. There's nothing strange about this thing. So I mean, it's very, very viable in the open source community. This slide's a little old, but you get the sense that the newer picture's even better when you look at the 2013 numbers. You take a look in 2010, you saw relatively few organizations contributing. And you notice that the big one on the bottom, they're Citrix at over 60%. By the time we got to 2012, you have many organizations, key organizations now working and suddenly Citrix is now a minority share. We're the largest, but we're doing less than half. That's by design and that's the way we want it because a couple of years ago, and this is a subject of a different talk. In fact, Lars Kurth this afternoon is doing a talk about lessons learned from Zen. If you wanna hear the story, go to his talk. But in a nutshell, when we sort of pulled back from the open source community and weren't playing as well as we should, participation dropped off. And a couple of years ago, a decision was made consciously to fix this. And this is some of the results you're seeing here. And I think he actually has a slide that includes 2013, which is even nicer looking. So we are getting more and more interested parties, both on the individual level and on the corporate level that want to see Zen survive because it's valuable to them. So participation is definitely growing. A few more Zen features that I just wanna touch on. One that doesn't, you know, we mentioned before was Zen Motion via Zappy. That's got all sorts of interesting things to it. There's a high availability piece as well called Remus that doesn't get as much public acclaim maybe as it should. Sometimes you have to kind of, depending on how it's packaged for your individual distribution, you may have to kind of build some pieces yourself. But it's actually quite interesting. We do support a wide variety of control domains. And control domain in Zen terms is the thing that can talk to the hypervisor. And it also does some driver work which we'll talk about in a minute. But it is, you don't have to be running Linux to be running Zen. NetBSD is a perfectly viable control domain for Zen. And there's work going on inside some of the other BSDs too, I believe at this point. So I mean, we're, once again, we're not just trying to force you into our mold. We've got all sorts of guest domains. BSDs, Windows, Linux, you know, also I think on the control domain, I think Open Solaris, when Open Solaris was a little bit more viable than it is at the moment was also a reasonable control domain. And so we also have multiple virtualization modes for performance reasons and we'll talk about that in a second. So let me spend a moment on the hypervisor architecture because this is actually interesting to know because when you look at other hypervisor solutions, they do it quite differently than we do. And there are benefits to each one. And unless you know how they are built, you won't know whether it fits you or not. So here is a textbook type one bare metal hypervisor. What do we see there? We see that there's the hardware layer on the bottom. The hypervisor itself runs directly on hardware. It's not part of an operating system. It is itself running on hardware. And then above that, you've got the VMs. You've got the virtual machines. So that's kind of your textbook type one. Here's a textbook type two, little different. You've got the hardware layer, but then you've got this host operating system that kind of sits on the hardware layer. And it provides the hypervisor capabilities through that host kernel. And the device driver sit in the host kernel. And then the VMs sit on top of it. So it's a different concept. Zen, in all of this, is type one with a twist. Okay, so here's the picture of type one again. Hypervisor running on bare metal VMs on top. This is what Zen looks like. Hardware layer, hypervisor sitting on top, VMs on top of that, but you notice the difference, the device drivers and models. Not in the hypervisor layer, why? How many people have been played with Linux for at least 15 years? Anyone? Okay, good. How many people remember what it is to get a nice new piece of hardware, slam it in your box, you say, oh crap, no driver. Gotta find another distribution, gotta update something. The problem of drivers is real. We're kind of, we see it less these days because the driver coverage in Linux is really, really good. So what the Zen architecture is, is to have a control domain, because one of the things that isn't normally inside of your type one picture is something's gotta talk to this hypervisor. Something's gotta tell it what to do. So that'll be the control domain and it will, at least by default, give you device models and drivers. That means that your drivers are as up to date as your control domain is. So if you've got a, you know, if you're running Ubuntu on your server and running it happily, that'll make a dandy control domain, because it knows everything it needs to know about that hardware. So whether it's Linux or BSD, you can use whatever in the control domain there to get the job done and that's where your driver support is. So just to be clear, because some people mistake this, the hypervisor itself is not in the Linux kernel. It's separate, but there are certain things in the kernel that have to be there to enable it to get started and so forth. So all of those things are in the default kernel build. How many people remember the day, you know, whether either you yourself or you heard someone else say, I had to load a Zen kernel? Yeah, that used to be a problem in the day, that if you wanted to run Zen three, four years ago, you had to go to your distribution, you had to look in the special packages area, whatever it was called, you had to pull out the Zen kernel, the kernel that was built for, especially for Zen, throw that in, substitute it for what you normally would install, and then you could run Zen. If you listen to the last talk later on, you'll hear the details why, but basically we weren't playing very well with distributions and we were making them do extra work and part of it was we weren't upstreaming our Zen patches into the kernel well. That's all done. Right now the kernel tree has all the Zen you need. You don't have to worry about that anymore. That's an old problem. Right now, just about every distribution, notable exception is REL6 has everything that you want to run Zen. You just install the package, reboot, do some configuration and you're going. It's fairly simple. REL6, Red Hat, back in the days when Zen wasn't playing very well with others, invested heavily in KVM, business decision. So now they have decided in REL6 they don't want to deal with Zen. So you can't easily get there. You have to do some work to do it. However, the folks with the CentOS group decided, you know what, there's a lot of REL5 people running Zen who'd really like to run Zen on a REL6-like system. So they introduced, I think it was in the 6.4, a kit so that you can actually run Zen on CentOS 6. That was something that they decided they wanted to do. Our team, of course, helped out. But, you know, it's really exciting because if you were running REL5-ZEN and you want an upgrade path and you didn't want to have to go through changing your hypervisor, you don't have to. You can go through CentOS 6, that's an option for you. And of course, there's more information down below. Let's take a look at a few basic Zen concepts. One is the whole notion of control domain, which I mentioned before. And that is the thing that talks to the hypervisor and by default gives you driver support and device models for your hypervisor. And then you've got guest domains, which are just the VMs that you want to run, the things you're really interested in doing. So we see that the console talks to the control domain and then you can use the management tool stacks to talk to the other VMs or control the other VMs. So, you know, that's a little bit more complete picture of what you have. Now the concept of disaggregation, and this is actually incredibly powerful. By default, disaggregation isn't used. You have to make the mental decision to do it. But it's incredibly powerful in that you can take some of the elements of the control domain, certain drivers and so forth, and actually give them their own dedicated VMs. Now, as far as the guest VMs are concerned, this is all invisible. So the guest VM doesn't know, doesn't care. But if you're trying to host, say, you know, hundreds of machines, hundreds of VMs on one host, you don't want it going through one network driver. Because that could bottleneck. So you could actually split it out into multiple network, individual network driver VMs for performance reasons. So now you don't have the bottleneck. You just have to decide to do it. Another piece of this and referred to George's security talk to get more information is the security part of that. Those things can be wrapped with Flask and other tools so that it's a driver domain. And if someone actually violates the driver, someone hacks your network driver, say, they have just inherited a very, very small universe that won't allow them to do much. Because they won't be in privileged code. They won't have access to disk. They won't have access to disk. They won't have access to much anything. So you've actually prevented, you've made security a lot sounder. You've prevented an attack and made the attack algorithm that the attacker's gonna need to do much more complicated because now he has to do multiple attacks to get in to find anything useful. So it's actually incredibly powerful concept. And the lifetime on a driver domain to start one up very quick. So if it fails for some reason, let's say you have a driver that you know is kind of buggy, restart can be as little as a fraction of a second on these ones, yes. You did say by default all drivers will be in Dom Zero. Yes. So what's the step that you're going to go through? When you do a standard install of Flask, what's the complete way? Okay, the question is, how do you get to the disaggregation? If it all starts off as being part of domain zero, how do you get through disaggregation? Actually, I would refer you to George's slides from yesterday, the security talk. He actually has a nice slide that gives you the ABC plus a nice reference for it. So for getting the details, I don't wanna go and bog everyone down with details right now. I don't know if it's automated or the... It takes a couple of configuration steps that you can do very, very quickly. There's no distro support for that. Well, out of the box, no. We may start to see some of that inside the next couple of years, but that kind of remains to be seen. But it's just a configuration issue, primarily. Yes. When you talk about the freight domain. Yes. And then... Okay, this is getting a little deep in the weeds, but the question is, if you've got the network driver in its own domain, do you have things like open V-switch? As, and being the evangelist, I'm not nearly as deeply technical as some of the other guys, but my understanding is that the V-switch is in a, I believe in its own different universe. So it simply knows how to talk to it. So it's, I don't believe it changes anything, but I'm not sure. If you see me afterwards, I'll see if I can get a direct answer for you from one of the engineering fellows, because we do have a number of engineers here at the show, but I don't wanna lead you astray there, but I believe it's distinct. Let's talk a little bit about virtualization types. Now, I have to admit, the first time I went through this, because there's a few virtualization modes in Zen, it can cause brain bleed if you haven't been here before. I'm going to keep it really, really light. And if you just track with me for a couple of minutes, I think you'll see the point that I'm trying to get at by covering this. Power virtualization, if you know the term, means that the guest kind of already knows that it's running in a virtualization mode. And instead of trying to talk through a hardware specified driver, which has a lot of difficult little bits to make the hardware work, instead it's more or less sort of talking to a little socket that just says, here, do this. And it comes back and says, yep, done. Well, that's much more efficient than doing all the stuff that's needed to support the hardware layer. So power virtualization is very good for that performance aspect. However, you have certain operating systems that don't power virtualize very well, particularly if it's closed source. You may have issues trying to get the support there. So we also have hardware virtualization, which just, it acts like a giant piece of hardware in software. And full virtualization is another thing, another name for hardware virtualization. Now that's the, so you have the two extremes. One is the VM knows it's being virtualized. The other is the VM is totally unaware that it's being virtualized. Now there's a couple modes in the middle. And this is where the brain bleed can get in, so I'm just gonna do this really quickly. But power virtualization on HVM drivers and the one coming hopefully in 4.4 PV and HVM, which try to narrow down the scope some. So it's actually sort of mixing the two in two different ways. So in one, it's using disk and IO drivers, but it's not really modifying the guest OS at all. And it works better than HVM. The other is PV, but it's using the MMU. So it's a slightly different mix. Let's take a look at the next two slides, we'll tell the tale. And this looks like an eye chart, so like don't freak out on it. But here you see the stack going from fully virtualized down to fully power virtualized. And so we've got this range. And if you look down here, you'll notice that the combination of what's virtualized and what's not changes with each one. Now what does all this mean? Next slide, we color things in, gets a whole lot clearer. Green is good for performance. The red, maroon, poor, gold, it could be better. So now you see that where we started at the full extremes, we can draw, as we draw near to the two middle ones, we can get optimal performance, one being the hardware virtualization mode and the other being the power virtualization mode. So what we've tried to do is to bring the extremes together and get you to the optimal modes, depending on which way you wanna go. So it's there and we're working to make the performance better and better, because in the old days, sometimes you get people saying, well, Zendon perform well from our workload. Well, it may have been because one of these combinations was off. And so what we're trying to do is bring it to the optimum. By the same token, don't freak out when you look at this chart because Zendon will try to give you the best one it can see, but you have the right to go into configuration and say, no, I'm gonna change the mode. I wanna see what it looks like in the other one. So if you're doing bench testing and you wanna see how well it performs, you've got the option to flip on another mode, run your test and say, was that better or worse for me? Once again, it's up to you. We want you to have what you need, not tell you the way it's going to happen. So we're giving you that opportunity. And when 4.4 hits, sometime after that, when PBH matures, we think that the future is going to be in PBH and PBH VM because those seem to be the absolute two best performant ones, but the other ones are gonna be around. So you'll have that option to pick one of the others if you wish. Yes. That's an interesting question. I honestly don't know. The question was, is this configured by hardware box or by guest? It's by guest mode. Yeah. And that's gonna be more than the same box. Yeah, I think it's by guest. On it, forgive me. Once again too, I can verify that for you later. Here's a little bit more on disaggregation. Disaggregation, there's a couple of papers out there. Once again, you can find the links online, but there's one called breaking up is hard to do. And that talks about some of the details of what this disaggregation means and how it's handled. If you've ever played with cubes OS, they give you these wonderful, interesting desktops that look like they're all part of the same machine, but they're actually multiple VMs. Each one of the different windows you see there is a different VM. And it's playing with the disaggregation concept. It's a really powerful concept and I think there's gonna be uses for it that we're not really seeing quite yet. Is that widely deployed? Well, the question is, is it widely deployed? The capability is there. I don't know honestly how many people are taking advantage of it and to what extent. That is one thing too, by the way. One of the things that happened in ZenProject this year is we've always had a developer community. We're trying to reach out and make a better user community because questions like that as to what users are using it for, sometimes a lot of time we don't know because they're using it and they're just using it. This year we had the ZenProject user summit at LinuxCon North America, which is the first time we've ever done a strictly user event. We want to do more of that. So if you find yourself in a potential user camp, we'd like you to participate because we want to hear what you're doing or what you need to do because that'll help us. So I would say just beyond the watch for user events having to do with Zen. Here are some of the benefits of disaggregation. I'm just gonna try to clean up pretty quick here because we've got just a few minutes. I've mentioned most of these before. It's better security, flexibility, robustness, performance, scalability. I mean, there is a paper out there on these nproject.org site. Wei Lu did a presentation at an event about six months ago where he tried to do tiny, tiny VMs and could manage to scale to something like 4,000 with a couple of patches. And there's wild stuff there but when you have the disaggregation, stuff like that becomes conceptually possible and practical. A little bit more on that. Let me do the, this is just pictures, but here's kind of the big beefy, thank you, here's the big beefy VMs that you expect to see. They got everything stacked in them, et cetera. That's before disaggregation. After disaggregation, you'll see something that looks more like this. Where suddenly you have individual VMs potentially for each individual driver. You can have one separate for your disk driver, for your network driver, storage, et cetera. They can all be placed in their own little boxes. Now, there's security advantages and we'll revisit the disaggregation with this in just a second. As I mentioned, Flask which is essentially SE Linux for VMs can manage to tie each individual VM down if that's what you really want for security reasons. And this was NSA based, there's a whole section of the wiki on that. And the sorts of things it provides, can this guest talk to other guests? What devices can it talk with, et cetera. So it gets that granular sort of thing you expect with SE Linux. And so you get to this situation, you see the red box here. If someone violates the network driver, I mean that's a frequent point of attack in security. What have they received? Instead of taking out the entire control domain and now having accessed all sorts of privileged stuff, they have exactly one driver domain which doesn't have the privilege to do anything else if you've tied it down with Flask. So and particularly if you're in the cloud, what happens if your network driver goes down? Well in most cloud systems it says, oops, something's wrong here, I need to restart this. So they may violate the driver and then seconds later, boom, the driver gets restarted. You know, it's a nice security feature because they violate, they get in but they don't win, they don't really get anything. And in fact, you can set it up so that it'll just destroy what they just didn't get you back to known state. Let me talk because I've got about three minutes now. ARM hypervisor, this is very, very cool. How many people are looking at ARM in the future? Is there discussions? Very cool area. We've got support for V7 and V8 of ARM. It's come along really well. And here's just a, if you're a geek like me, if you're a geek and you ever get excited about something or you've ever talked with a geek who's excited about something, that's what it was when I was talking to one of our guys who was doing a lot of the ARM work. Because he said if you look at the ARM situation, and this is sort of your basic ARM diagram, hypervisor mode, kernel mode, user mode, and then you've got your device tree stuff over here. Then you start looking at how Zen fits on it. Well, the Zen hypervisor fits directly into hypervisor mode. He said to his knowledge there is no other hypervisor that will work alone in ARM hypervisor mode. It never has to cross into the kernel mode. It never needs the privilege because it just fits. Likewise, the kernel space maps into the kernel for a control domain. User space becomes the user space for domain zero. Man, your clock's running fast. And the device tree, what happens to that? Well, that becomes the device driver area inside domain zero. So it really looks like the ARM people and the Zen people were almost using the same specification when they put together their architecture. And I've been told they got a minute left. One mode to rule them all. Whereas we had four modes in x86 and we had two that were optimal, there's only need for one in ARM because it just works. It is the optimal mode. And take a look at lines of code difference. X86, these are a little old numbers, but they give you the idea. X86 hypervisor, 100 to 120,000 lines of code. By the time you get down to the new ARM code, 17,000. Fast, tight, secure, much less to go wrong. It's really a sweet story with ARM. Just a moment on MirageOS, wonderful sort of concept of a sort of library-based VM, a small VM nimble that works on one thing and does it really, really well and fast. So that's where Weilu was dealing with potential of thousands of these things on one host. As we get into an area where the cloud has services that just do individual services, Mirage plays really interesting game. And you can find more information about Mirage out on the website, et cetera. So I'm out of time here. There is URL that you can pick up from here. Really interesting stuff. What's next? We just finished 4.3 a couple of months back. You can see some of the things that are there. We've got better security, more ARM stuff, even spice support, better NUMA support with this past one, which is really cool if you're playing in the NUMA area. Information about the roadmap on the bottom for what's being worked on for 4.4. And as I said, the CentOS stuff is tremendous. We're working more with the orchestrators, like Cloud Stack and Open Stack and Zen Orchestra and so forth. More disaggregation stuff to come because we want that to be easier. And we want to try to make it easier to be working with Libvert and stuff if you're doing a mixed Zen KVM situation. A few things that if you want to join into all, we have document days monthly. In fact, next Monday is a document day. If you're going through documentation, you say, well, this doesn't explain it very well. You can pitch in and help us upgrade the documentation. That's what documentation days are. We just try to get together and clean things and add things that are missing. We have, prior to each release, we do have test days, really useful. We've got the mailing list, IRC, and then the new zenproject.org website. If you don't know where to go, go to zenproject.org. That's kind of the bottom line. We want that to be the one touchstone spot to find anything information-wise that you need about Zen. As far as events, we've got a developer summit that starts tomorrow here. We had a user summit last month in New Orleans, and there's a hackathon that appears periodically as well expected to be sometime in the spring. So if you're a zen hacker, you want to join in on that. References, thank you. I will take questions in the hall because otherwise people will carry me out by my arms. But thank you very much for showing up. I appreciate it.