 Hello and welcome to scale so as you know scale is a community event and we couldn't have it at the prices We do without our sponsors. This particular room is sponsored by Q connect Q connect is a small Booty that specializes in in networking and building relationships within Southern California to connect top talent to our clients in the tech industry They actually host things like our Linux LA in their offices So without further ado, here's Russell Pavlichak to give you a great exciting talk about Zen. Thanks Good afternoon How many people Saw my talk yesterday in the unicolonial track just show of hands. I will warn you that there's gonna be a few minutes of overlap so, you know, please no throwing up tomatoes or anything but Because we are gonna cover a little bit of that territory here But there's a lot of other stuff as well. So hopefully it'll be worth your while the short slide obligatory on who I am I'm not going to sit here and read it to you, but basically I've been around for a very long time and Until recently actually up until today. I was a Zen project evangelist, but as of tomorrow, I'm laid off So anyone with job opportunities see me afterwards. That would be really really nice but I've been I've been an open-source advocate for a very long time and It's one of the things I actually enjoy immensely But today I want to talk a little bit About the idea of innovation on a hypervisor, you know We're used to thinking in terms of containers being the new stuff and and it is But there's actually some very interesting things that that's going on innovation wise on a bare metal hypervisor and You know innovation is one of those magic words, right? You know, it's the thing that every time someone puts new puts lipstick on a pig. They call it innovation you know So many things are just you know hackneyed placid tripe really when it comes to innovation It's not supposed to be that way, but that's what's happening in the way things are being marketing. So You know innovation really is is the ability to see something in a different way than you've seen it before And it may be reapplying things that you've That you've learned in a different way, but you should come up with something new and different So it should be about borrowing from the known To make something create something that is currently unknown You know many innovations are just reassembling things that we've already understood, you know The cloud cloud was hailed as a great innovation But those of us who have been around for a few years kind of remember that a lot of these concepts Existed on a thing called a mainframe But they've been reapplied to a different set of hardware with a different set of rules and because of that it's become innovative So that's that's a you know a reasonable use of innovation That's that's actually reusing something we knew but applying it in a very different way So The result does have to be significant Now in the Zen project with which is a bare metal hypervisor. We are Using its nature as a bare metal hypervisor to innovate in various ways and we'll cover some of these things in the next few minutes One effort is Zen automotive Which is rather interesting the notion of having a hypervisor in your car Things like real-time virtualization that's something that we've never really Thought about in most ways, but now there's actually a need for this and what do you do if you have real-time Jobs that have to be virtualized Arm-based hypervisors that's that's another issue and of course things like unicernals which are Really quite interesting So let me just back up and try to give you a little foundation. What's a bare metal hypervisor? Here's a little you know textbook pictures here. This is a picture of Your type one or bare metal hypervisor, and it's relatively straightforward. You have hardware on the bottom You have the hypervisor layer sitting on top of that and then you have VMs which sit on top of that It's a relatively straightforward model There is a second textbook Type of hypervisor, which is called a hosted or type to hypervisor and here We see that rather than having a straight hypervisor layer There is a host operating system which provides the opera which provides the hypervisor function It's embedded into the host itself So it looks a bit different and the flow is a bit bit different now which is superior Honestly, whichever one works best for you There are two different ways of looking at the world one is not innately superior to another but The one you choose may allow you to do certain things more easily or or with greater difficulty So it's just two ways of looking at it now. Where does Zen fit into this as we said? This is a picture of the textbook type one And here's a picture of the Zen architecture Now we noticed that it's Largely the same as a type one, but there's something missing in the hypervisor layer itself We do not find device drivers and and device models How many people Have been around long enough to remember the good old days when you'd go to the Maybe it was the local computer show on a Saturday or something like this And you'd buy that board that you really wanted to get you'd come home You put it in your linux box and you boot it up and it's no driver and So you'd have to see whether you know if upgrading the operating system would do it Or maybe you had to go out for those of us who remember to source forage or to fresh meat and you had to get The driver you had to compile it. You had to link it in you had to build it up and see if it would work for you The bad old days. Yeah, the bad old days well when When the Zen project hypervisor was was defined they said you know what? We really don't need to go through that We don't want people to have we don't want to have to keep up with every device driver that comes along however Linux and the BSTs already do a really nice job of doing just that So what we'll do is we'll leverage their ability to give us device drivers and so enters this slight change this notion of a control domain or DOM zero the first the first VM and in there We will borrow We will leverage the the drivers that exist already in the control domain and the control domain could be Linux it could be free BST. It could be net BST. I think There's more in the hopper But we can leverage their device drivers So you don't have to worry about whether or not you have support Basically if you can load your favorite distribution up on your hardware, it'll be supported It's really you know a reasonably simple way of attacking it and what are the advantages of Working on the bare metal space. Well, one is density. It's very very thin It just sits right on top of the bare metal. You don't have an entire Operating system that has to be manipulated to give the hypervisor function. It has scalability It can really support a huge number of VMs and we'll be talking about this a little bit later it has a Nice security story since there is no host operating system to attack That that reduces some of the attack plane There is the scheduling Issues which we will talk about in just a few minutes where we can actually use custom schedulers and In a couple use cases. You'll see why that's really important and then there's the notion of power virtualization Let me just quickly explain that one Everyone understands hardware virtualization, right? You know you've got a VM and it thinks that it's running on a piece of hardware And if you look into that VM, you see, you know your any 2000 network driver or whatever the heck it is And it's sitting there Expecting that it's going to have this this hardware out there. Well, you know to pack up Say your network packet into the shape of an any 2000 and pass it off to the hypervisor Which doesn't have in any 2000 which then has to unpack it and then do something useful with the data That's that's wasting time Now it's fine if the VM doesn't know better if the VM doesn't know that it's being virtualized Then you can use hardware virtualization, but then Pioneer this notion of power virtualization if the VM it can be smart enough to realize. Oh, I'm a virtual machine Then what we'll do is we'll just open up very simple very fast pipes Basically, you just throw the data down the pipe you wait for the response to come back You don't have to worry about packing it up and you know doing all sorts of driver logic and all this other sort of stuff It's very very simple And there's an advantage to that and we'll talk about that in the use cases So let's just take a look about look at some of these things in detail Then automotive is a sub project of the Zen project Zen project is part of the Linux Foundation for people who don't know It's open source don't do not be confused by The commercial Zen based products that are out there in the world That is product. We are project. We are open source and So, you know always have been always will be and and it's a very straightforward there, but the Zen automotive work came along from a group called global logic and They wanted to do Infotainment systems in cars using a hypervisor Why would you want a hypervisor in a car? Well, if you think about it There's now a lot of rather discrete computers inside your average car. You've got that rear The rear view camera you've got the video that's streaming to the backseat for the kiddos You've got your on-star button or whatever on the dash. You've got all these computer-based things all of which have to be individually sourced Tested installed tested again That's a lot of hardware a lot of testing and they said well, why don't we have just a hub of Processing for the infotainment system. So we'll have a computer with a hypervisor and We will just stuff it full of those things that we need and then we'll hook up the devices that we need, you know the The screens in the back seat and all that sort of stuff that we need to control But we won't have to go through the testing and sourcing and Etc of all these discrete computer systems. We'll put it together in one lump And so they're doing this this happens to be an arm-based thing that they're working on Now there are challenges to this because we think about it things like, you know streaming video That's that's kind of real-time-ish, you know You don't want that thing hesitating and stopping and starting because something else wants the CPU You want a nice flowing picture? Well, that's a different sort of problem than your traditional virtualization So you've got soft real-time support has to be there for some things a hard real-time support needs to be there You have to have GPU virtualization, you know, how far we've come five years ago Something that had a hypervisor would never even have a GPU, you know most most servers were headless That's the way we ran. We didn't need graphics, but now not only do we have the graphics? We got to virtualize that too So that becomes tricky. We've got these other co-processors and stuff like this, which is rather unique to this use case We have to be able to get them supported. There's got to be need for certification You know cars are tricky things. You got to make sure that the passes all this stuff all sorts of driver support for weird things that don't exist on your average server and Even support for various operating systems that don't exist inside your normal server room So there are lots of challenges to this So one of the advantages of Zen inside this scenario is That it can help facilitate an extreme focus because it's not part of an operating system itself We can leverage the fact that it is flexible enough to be focused on exactly at the task at hand You can't just repurpose a server-based hypervisor and make this work. You have to do a lot of work for it So you don't like I said, you don't have to worry about a legacy hopes operating system And you can you can have the flexibility to do what needs to be done Now this leads us into the whole notion of real-time virtualization because that's actually used by the Zen automotive stuff and You've got efforts like RT Zen and there's another one the name of which escapes me But you know streaming video and stuff like this like we said you don't want that thing going Or it's pixelating it's got to be there. It's got to be there. It's got to be there So the rules for handling those VMs has to be very well laid out So you need a scheduler that makes sense for real-time operations well Zen being Type one it doesn't have a default scheduler that that goes along with the host operating system because there is no host operating system Zen actually allows you to use various Schedulers and in fact you can even have layers of granularity. So not only can you say well, I want to use a different Scheduler for this particular piece of hardware. You can say well, I want a one scheduler for these couple VMs I want another scheduler for these few VMs over here and so forth It allows a great deal of flexibility and that's the type of thing that when you're working with type 2 It's going to be much harder. So here's the types of schedulers that we have out there right now The default one that's in the hypervisors known as credit And it's a general purpose one. It works really well inside the data center and so forth the one that is in development that is Assumed to become the default as soon as it has all the all the pieces meet into it is called credit 2 it's currently in an experimental form and It is optimized for low latency and high VM density And we're going to talk about why high VM density is going to be really important in just a few minutes when we talk about Unicernals, but it doesn't even start there to stop there we've got Oh, come on We've got a mouse that wants to do things that I don't like There we go We've got other schedulers as well the soft and firm real-time scheduler For multi core operations or the hard real-time scheduler for singer single core So we can use different schedulers To go along with the workload. We're not limited In the way we attack the problem we can adapt to the problem There are also schedulers that have come We're used and are pretty much discarded now, but most of them are still out there And there's a list of some of them there and we've got a wiki page with all that sort of stuff on there By the way, these slides Should be up on the scale website soon. So if you see anything of interest, you don't have to Scurry to write anything down or so forth But this actually leads then into the whole arm based hypervisor because a lot of these things are leveraging the arm arm architecture, how many people are working with something that Eventually may be in the arm space. Do we have any people here? Okay. There's a few people already in that area so the The virtualization extensions that were added to arm v7 and beyond Opened up a lot of possibilities for us and architecturally The arm based hypervisor theory maps beautifully With zen and that means that we don't have to go through mode changes And no mode changes means greater speed greater security. Let's let's see what we mean about that here is the architecture For arm and we see there's a hypervisor mode. There's a kernel mode. There's a user mode and there's currently a device tree with with various device access Now how does zen play into this? Well, the hypervisor works directly in the hypervisor mode It doesn't cross over into user mode doesn't cross over into kernel mode It doesn't need to inflate privilege or open the door to some sort of You know possible exploit And then the user space and kernel space from the guest vms just maps into user and curl mode Hand and glove now we still have one area in the device tree That isn't covered the io space What goes there well Domain zero just like we said in theory handles the drivers for the hardware. That's exactly what's going to be here And pv being para virtualization like I said that wide open pipe that makes things very fast Since there is no legacy operating systems to worry about in the arm world We can just go straight to pv which is very efficient for moving data back and forth So it becomes a very efficient Very sweet picture architecturally Where will an arm hypervisor play? you name it At uh zen developer summit, I guess was about two years ago Someone stood up and said we want to put a hypervisor on one of these Like why would you put a hypervisor? on a cell phone Well, I said think about it in the world of bring your own device You go to work for big company and they say yeah, yeah, sure. You know get your get your email and stuff like this on your phone Of course when you leave us We reserve the right to issue the wipe command And suddenly the address book the picture of the wife the kids grandma's phone number All gone Because we need to protect our intellectual property But what would happen if we had multiple personalities on the phone? business you and personal you the wipe command could take business you and Remove all that stuff But the personal you It's just fine. It's intact. So as you're leaving the big company you can be on the phone to your grandma talking about the wonderful uh The wonderful things that you're about to do You see it inside embedded systems We talked a little bit about automotive We had talked from a guy at ge That's running hypervisors on trains Who to thunk you know It's actually a really interesting really interesting concept But you think about what's going on with trains. You've got this giant lumbering thing That ideally has to be able to look Miles ahead on the track to stop you know freight freight train You can't just hit the brake and not You know and come to a dead halt. It's not possible but if you had a if you had a computing plant in the engine of the train With multiple vms and one of which is to actually test and look ahead And receive data about what's on the track It could receive the signal that says wait a second There's an obstruction begin braking now two miles ahead So that you can come to a slow halt and not you know hammer the car or whatever else that's out there And this is already out there in ge You've got the things like the internet of things You know if you got lots and lots of little things Talking about millions billions trillions whatever. I mean that's huge basically anytime you have a button or a switch You can have something there that has to respond to it Well, if you got lots of little things then suddenly an arm Looks interesting because it's low power. It's small And it's ideal for a little responses And then of course, there's a whole notion of servers You know this one's been kind of coming and going we'll see whether it really catches on but arm based servers are a real possibility Now let's take a look for a moment about the unicolon And this is the thing that for me has been one of the most exciting things inside the past year The notion of super small VMs Very quick booting with enhanced security by design Easy deployment and enabling transient microservices Once again internet of things You don't want thousands and millions of Things sitting out there waiting for you to press The button or turn on the light switch You want when you hit the button something to say dispatch The thing that is going to respond to it respond and go away. That's a transient microservice It doesn't exist until the button is pressed Its lifetime may be a fraction of a second Where it's given birth It responds And it dies And things like unicernals make that sort of uh possibility happen You know in the world that we're used to if you've worked with the cloud everything right now Up until now in the cloud has been about orchestration We had this rough mean slow data center where everything Was static, you know and it used to be that you could wander through a data center And if you had the senior administrator, you know, he could you could Just cover his eyes and he could walk down the rows and say, okay, that's the hr server That's this that's the database. This is the and he didn't need to see it Because it was always in the same place at the same time In the cloud everything's got to be flexible And so the work has been On the cloud engine the orchestration layer open stack cloud stack, etc, etc And that makes perfect sense Except the stuff that we're putting in the cloud still looks like the stuff that we had in the data center Full images Stacks and stacks of software It's big. It's beefy. It takes up a lot of room. It's slow to start Takes up a lot of memory, etc, etc What if we could make it more efficient if we could make it small You know right now or at least in in the old days You were lucky if on your server if you could manage getting the double digits Of vms at one time going on your server. It's like, oh boy I got it, you know, I'm I'm something special. I managed to figure out how to do that But that's not what we want for the future in the future We want to look at these workloads We want them to be easy to deploy and manage. They should have a very small footprint Get rid of all that unnecessary duplication of code Fast startup have things like transient microservices Have higher levels of security. We got to remember in the cloud you know The the old status quo of security doesn't work If you don't believe me just check the news clippings for the past year How many security breaches have occurred? It's it's horrendous Government systems big company systems, etc. We need something better And ideally we'd like to have thousands of vms per hardware server That would be a whole lot more efficient Let me just stop right there it about I guess it's about 10 12 years ago now I was working with a fellow who was building a data center in southern virginia And this thing was coming together He only had cable running like half of it. He was starting to move the racks in And he said, you know this place is full Like what do you mean this place is full? You don't even have power in half this place. What do you mean? It's full. It's like no, no, you don't get me. I know where every machine is going to go In this building And when I finally get every cable run Every rack assembled every server installed This place Is full. I will not be able to put one more server in this building Why would you build a data center? With no room for expansion So because I went to the power company And I said how much power can you give me? They said that's it Not not a lot more So I built the data center to use that much power Because if we want anything more than that we got to go somewhere else and build another data center We think in the cloud. Oh, yeah, you know, I go out at amazon and if I need more I just say give me more It just keeps coming and coming and coming and coming But in reality power Is limited These servers have to be powered up. They have to be cooled, which takes even more power So you've got all these things that are physical limitations if we can make it more efficient Then that data center which was maxed out and couldn't do one more thing Suddenly Would have room to do all sorts of additional things He would have loved it But that was then this is now but that's where we're going with this now We've got docker and containers And they've done some tremendous stuff. They made the deployment so much easier. They got that smaller footprint together, which is really nice less memory You know less less disk Very fast startup time high numbers of vm pros really good stuff lots of steps in exactly the right direction But there's still some issues that are being worked on You know once again, you're sharing a kernel So you're not going to run your windows instance on your linux based Container system, you know, so you got to do stuff like that not a big deal There are potential limits to scalability, you know Linux is really really flexible That's not really You know it wasn't designed to be doing thousands of processes quite in the way that containers do but it gets the job done But security that's kind of the big issue And they're working on some really good security mechanisms But you know, I find myself thinking of every time That i've seen an administrator deploy a server with se linux installed on it And the first thing he does is to turn it off Because it takes too long To configure And the boss said get that server up now Oh, we'll get we'll I'll get to it. I'll get to it You know You'll get to it when that server ends up being you know thrown off the loading dock in a few years There's been so many that are like that because if if the Security is added on we got to make sure that it's as easy as the deployment model Ideally the security should be baked in to the pie So that you don't even have to think about it too much If at all Well, that's where the unicernals come in and do some very interesting things They're also very small very efficient very quick to boot and they've got a security element to them Which is unlike anything else that's in the industry right now Anyone that's ever tried to lobby for green technology Say all we have to do is invest so much money and then we'll get you know return on investment in nine months You know that that can be a really tough hill to climb But the interesting thing is that when once you get into stuff like this the return on investment tends to be almost immediate A lot easier Convince people to go that way Now there's a number of various unicernals out there. We will look at them extremely quickly So what really is a unicernal and I borrowed this from the mirage folks. Thank you richard um Unicernals are specialized virtual machine images compiled from the modular stack of application code system libraries and configuration Well, you know think well, that's just like any vm except that this is One executable One blob Not layer upon layer One blob single address space Let's take a look at how this plays out. This is uh, this is the development picture And during development, this is actually, you know, pretty pretty standard You take the configuration files application binary language runtime you compile that together And you create a blob that sits on top of a Linux and or unix like operating system and you test Well, you know, that's pretty straightforward. That's pretty much what we're doing now Nothing really knew about that But then you get into the full testing Uh Scenario and suddenly you're doing all Of these things through the kernel threads the user process of the file system network stack All of that comes together in one singular package And that is sitting on top of linux and or unix like operating system for testing purposes But then you get into the deployment scenario and the picture changes just a little again All that stuff is compiled together But now it sits directly on top of the hypervisor Which sits directly on top of the hardware What's missing? The operating system There is no operating system There are just the needed operating system functions That happen to be compiled into the unicernal ball And those ones aren't a full operating system. It is simply those individual pieces that are required to make that thing go Nothing more So here's kind of the concept. You want just enough to do the job You don't need multiple users Right? I mean if it's going to be some sort of database Uh system if it's going to be a web server or whatever You don't need something to be able to log in and walk around and have multiple There's no need for that So get rid of that You don't need a general purpose operating system. You don't need 10 000 utilities Why? You if you want it to be a web server let it be a web server. That's it Get rid of all the rest of that stuff Because you know that when the malicious hackers come that's what they use to try to bust you So we'll just Get rid of entirely we'll remove their toolkit And you don't even have a full set of operating system functions like I said You have only what's needed For that thing to exist So it's very lean very mean minimal waste very tiny size So how does uh You know this is like an embedded system theory We have any embedded programmers here or people who at least have been Hello Richard Okay, I'm back too. So um We're this is basically applying the concept of embedded programming Into the data center So it's not really a new concept But once again, it's that innovation of taking what's already existing and applying it to something To a scenario that's never had it before So you've got some some things that you have to be you have to count for one of which is that obviously You're not going to have a debugger sitting there You're not going to have all sorts of tools to debug things. You say, oh, you know, this is this is terrible How am I going to be able to do this? Well You know when you think about it when you're in production At least most of the production servers I've worked with The owner of that server server doesn't want you to be able to sit there Excuse me Doesn't want you to sit there And debug on his production system Says take the data take the logs bring it over your test box Find out what the problem is fix it And make sure it doesn't happen again So you're basically in the same scenario You just have to make sure that the stuff that you can pull off the production system Is enough that you can do the debugging elsewhere because you're probably not doing your debugging on your production box anyway So that's kind of a trade-off In order to get these ultralight images. What do the results look like? Well, here's some things from mirage a fully functional dns server for 449 kilobytes Kilobytes Some of us with the gray and the temples need to turn to some of the young people and explain what a kilobyte is Because it's not gigabytes. It's not terabytes or petabytes or kilobytes Most of us haven't said the word in 15 years you know a web server 674 kilobytes open-flow learning switch less than 400 kilobytes Ling another one of the uh unicernals Boot time to shell under 100 milliseconds. That's that's really kind of an old spec most of the time they talk about Uh 20 to 30 milliseconds Uh Erlang on zen.org which is where uh ling hangs out. They eat their own dog food. They've got A unicernal running that page and if you click up in the corner, they'll show you what they're using And when I clicked on it one day, I ended up saying it's seeing it's that it was using 8.7 megabytes of memory to do the job If you have a web server that's using 8.7 megabytes of memory Suddenly the notion of having thousands on a single host Not so far-fetched If it's small enough You can pack them in there Click os from the n e c people They have network devices that will do in excess of five million packets per second We had a really nice talk about from from some of the n e c from one of the n e c people Yesterday in the unicernal track um Once again average size these are also Uh somewhat older numbers. I think he's he said that right now it's about five megabytes of memory and closer to 20 milliseconds of boot time We're talking about very small very fast And functional It's a different world Now what about security? Type safe solution stack, you know when the first set of unicernals came out they were inside these type safe languages, you know, sort of esoteric things and it's like, you know, there's haskell and there's um camel and you know all these things that's like Who who uses these languages? But the thing is there's actually a need for that for some things maybe not for everything But suddenly you've got the possibility of a stack that can be certified About two years ago at zen project developer summit and it's available online at zen project org inside the videos section There was a talk given about software certification You know, you got like a fly by wire system something that has to be tested because it's life or death They said if you had a linux stack It would take millions and millions of dollars. It would take years upon years and by the time you were done It would all be useless You need something small Ideally with a type safe language But suddenly that becomes very possible in the unicernal world You also have image footprints. They're now unique to the image Not only do they lack The utilities and things like this for uh for malicious hackers to to do But you don't even know what the image footprint looks like You don't know what functions are but to do something then once if you do manage to break it, which Requires a lot more cleverness than it does now Is incredibly difficult task Because you have no utilities you have no blueprint you've got nothing to work with So you have no utilities. You have no shell You know, even know what libraries you can work with here's a very partial list Of some of the unicurls out there now Mirage os from the zen project incubator uh this uh This team uh a number of them Put together a company called unicernal systems, which you may have just read Was acquired by docker um So, uh, you know, we got some serious things going on there We had a very nice talk yesterday about Interfacing docker with unicernals. So I think there's a really nice Story that's brewing there and I think that that crew in particular is going to move that call that ball forward Hal vm from galwa ling from erlang on zen click os from nc osv from what was cloudius and now I forget its name now, but they're database people but You know, it's actually a really interesting thing But since it's an open source it doesn't matter whether the company's gone into a different direction or not It's open source. It's still very usable Rump run which is really cool from the from the rump kernel project Um, and that's just the beginning There's even a like about three more that I don't even I didn't put on here Basically every time I had to I've had to do this talk. I have to go back and check because the number of unicernals is just expanding People are working with it. They're coming up with new concepts So how does zen project enable this Well, because there isn't a host operating system. It's type one It means it's lean and mean the we don't have to make any assumptions about the way the hypervisor is going to behave So that so that vm can sit on a very thin hypervisor layer which can then be tailored To the needs of those vm's The attack surface is very very small from security standpoint You have great scale out support. Um, I think these numbers are a little old the last number I saw was that The goal is to try to get Of some of the engineers is to try to get, you know, two to three thousand vm's Uh being able to be deployed linearly With no degradation And the last number I saw was 600. I think it's up from that Basically, they're constantly going back Looking at the architecture of the hypervisor and saying how can we improve it to make thousands of vm's a reality In terms of absolute linear performance You know, that's one thing that is You know, it's one thing to have those the science experiment that says, oh, yeah, well, we can do it But it's another to make it really perform well And that's what that's what zen is doing We have the enhanced credit to scheduler. It's part of the reason why it exists out there as I said Part of its new life is to be able to facilitate potentially thousands of very small vm's And you want it to be able to work on arm as well So is this all in the innovation world with With zen? No, actually, there's tons of other stuff that I'm not even going to be able to touch on Virtualized GPUs. There's been a whole bunch of stuff in that area Enhanced enuma Uh colo, I think is still out there the notion of a coarse grain lock stepping of vm's for people who remember the old tandem systems Um Those of us with gray in the temples where where you had two systems that would that would sit there together One would die and within 30 milliseconds that second one was up with the exact state of its predecessor so basically Uninterrupted no loss of data That's what colo is trying to do And things like native vmdk support and we've gotten the innovations page out there for for people Want to look at it looked into it so here's kind of a review See there's these are some of the advantages that we see working in a bare mental state We can support Very high density We can support a very high number of scalability High scalability of vm's We don't have a host os that can be attacked We can rework the scheduling to go along with the need And we can use the simplified para virtualized Interface to make things easier that's especially interesting if you look at the history of the unicernals Most of them started on zen Why because zen had the para virtualization state? Because it's really easy to write a pv driver Open hole throw data down wait for data to come back You know if you have to sit there inside some other hypervisor that That's going to make you write a hardware driver. That's a lot more work And it's a lot less efficient. So, uh, that's it's been one of the one of the wins In this area So, you know tomorrow's workloads are not yesterday's workloads We tend to think of hypervisors as plumbing in the world of clouds now, right? You know, it's like oh every house has to have plumbing, but once you got it, you don't think about it Nothing to be further from the truth If your hypervisor Isn't keeping abreast of your needs If it's not moving where your data center is moving you're going to have a problem And that's one of the things that the project is very much trying to do We want to stay ahead of the curve. We want to implement some of these new concepts And make sure that when people need it, it's there So we're trying to get the next generation Uh together That's uh, that's the end of my presentation any uh any questions In the back Is there anything difference between any difference between zen rt and zen other than the scheduler? To my knowledge there it's the scheduler plus some other things, but i'm not i'm not an rt person But they've uh You know, they're looking at the full ball wax there So um, I think it is more than just the scheduler that's being addressed, but I can't swear to it. Uh, I don't I don't see Richard Well then on arm run on a tegra Um off hand, I don't know it's an a15 I'm not the right person to ask quite honestly There is a there is a wiki page that describes the various things that are supported right now. I just don't have it memorized. I'm sorry But um, did I answer your question or not? I'm trying Yeah, uh, there is also one of the interesting things inside the zen world particularly, uh, if you go to the wiki wiki.zenproject.org Um If you need to know anything just you know, if you want to know about zen rt plug that in chances are you're going to get some information out of it Um We try to keep stuff out there and if it's not out there We need to know so that we can work on it on zen rt is still and there's actually like two different real-time efforts And I don't know who's winning right now they're both Both design a little differently if you see larce curt who is here at the conference Um, I tended to work mostly with the user base and less with the developers. So I'm But he he has got his thumb on the development Stuff so uh, so he would know which ones are Coming along and and what they consist of sir again The the some of the people who were responsible for mirage formed a company called unicolonel systems Which has been purchased by docker So so there are a number of mirage related people who now work for docker And we had a talk yesterday in the unicolonel track about Uh, actually doing the demo showing Mirage running Under docker Being deployed and stuff like this Well no containers No containers Just using the docker apis the docker command set To the unicolonel directly. So we're not using containers at all inside that case Yeah, yeah, it's just it's just deploying unicolonels on a hypervisor But using the you know the familiar docker interface Well, if it's all using docker It shouldn't need to I mean it's it's actually uh, you know Yeah, it's it's it's it's all all working with the docker api It's just re implementing it so that the unicolonel is being dispatched instead of Instead of a container being dispatched. So if you have a docker based infrastructure in theory You know now this is still from what I gather, you know It's still sort of work in progress and stuff like this and sort of a proof of concept Um, I think it was displayed at docker con Uh, but uh, but they've showed that it can it's very possible It does work. Uh, let me get him and then you the back Okay, I'm not sure I quite get Each of these is actually is actually a vm. So each one can or may Have you know an ip address is uh associated with it If it has a network stack, which most of the time you kind of figure it would but That's up to you depends on what it's what's doing But uh, yeah, so I mean these are real honest to goodness vm's They're just really really small and they don't have a lot of gunk, you know, so the rules really haven't changed It's just uh, just the way it's uh Being put together that's uh, that's very different than before Ma'am Yeah, that's good. This could be Well from the hypervisor layer the question is all these what was that? Yeah, uh the question as I understand it has to do with there's all this new Hardware support that intel and others are putting into the servers can all of these things be leveraged in this scenario um Insofar as the hypervisor is using them. Yes Insofar as the vm is using them in most cases It's not it's not even using virtual memory There's no need for it Because it's a tiny static thing So it doesn't need all of these advanced functions now you say well, you know that that you know, why do we have it? Well, because there are going to be lots of things that are not going to be unicornals You know, you may have your big honking databases You've got all sorts of highly complex processing here sap things they they're going to need all of that stuff But Does your apache web server need it? Probably not You know So so that's the sort of thing that we're making a distinction between Those small manageable workloads that we can make nice and small and keep them small And then we can free up all that resource to be used for those big things that really do need that That bleeding edge technology and need masses and masses of memory and you know all that at high-end performance stuff So it's not that that none of that matters it's that We give to to who needs it And don't waste for things that don't need it you know So, you know, yeah, it's it's a once again since it's dealing with an embedded concept You know, you think about you know an embedded program in a device It doesn't need a fancy operating system. It doesn't need virtual memory It just needs to do what it's supposed to do Gets the data in throws the data out And that's it And there's a lot of stuff that we do in the computing center right now That do exactly that but they've got this gigantic infrastructure Let's get rid of the gigantic infrastructure Let's keep it simple And then we can take all that infrastructure that is now being wasted And put it towards those really big things that require it So it's a reallocation of resource basically Amazon Well, I know Amazon is you know become quite active inside the Zen developer community Exactly what they're doing in the unicolonial world. I'm not exactly sure but you know, I think it's safe to say that they're aware of what's going on so um You know, maybe it's not this month, but I would I would hesitate to say that it's it's never coming I mean, it is possible to deploy Unicolonials on Amazon. There's some papers out there right now Unfortunately, you end up having to use a lot bigger machine instance right now than you need to but you know Some of these ones if they come up with a unicolonial sized VM, you know, that that may may not be as much of a problem Yeah So do me a favor For raise your hand for questions. I'm going to try to get another mic That's a any more questions to me, it's That this works on arm I do a lot of embedded stuff Is it take does it take advantage of the memory protection? I I'm afraid I'm having a very hard time Just project to me and I'll see if I can't Okay, does it take advantage of memory protection on some arm processor Honestly, I don't know The developer guys are better at that than I am Like I said, I've tended to work with the user base, but If you come across Lars Kurt, he probably could answer that question Um If you want to help how do you get involved go to zen project.org That's probably the the best way Uh, you that's sort of the hub of the zen project universe You'll find information about the mailing lists that are out there. We've got two very, uh You know active lists zen users and zen devil You know you start with zen users and say, you know, this is this is what i'm looking to do And can I help and so forth but we have an extensive wiki out there as well with Information on how to do this or that and how to get involved with various things So, uh, you know spend a little time just sort of wandering the wiki and you will get a lot of information and ask questions on On zen users and you will get pushed in the right direction We also have, you know a couple irc channels and stuff like this all of that you go to zen project.org and uh Uh and uh Look at that. Uh, it will give you information Uh, are we what are we for time here? Okay, uh, do we have uh in the back How how does the unicolonial approach affect the storage infrastructure issues that what I that what I heard um Yeah, that's one of the things that's uh sort of under active development because some of these things. I mean There are there is the capability of using storage, you know, basically you compile in Just the file system that you want to use or whatever. It's probably the weakest Area right now You know may or may not be the best fit for uh for things as they sit at the moment But there's so much going on that Um, you know, I expect to see a whole lot more in that area Because you know, not everything requires extensive File stuff and sometimes a little rearchitecture Things that you'd write to disk instead if you're throwing it across to another vm which you know Records or whatever like this might make sense. It kind of kind of depends But there there is support inside a few unicolonials for writing things out Um, not always the most efficient in the world at the moment, but it's being worked on sir Well, there's always the hope of the question is how does rt Project converge with the main main one or how does it track with the main one? There's always the hope for convergence But like like I said, I think there's there's actually two different rt things I don't know if they're both active right now. Lars would know That are out there that have been working together They're they're both trying to solve problems And you know, the hope is that one of them is going to Is going to be the the you know sort of the preferred real-time methodology, but you know this being open source One of the beauties of open sources that is if you have an idea and if I have an idea We can both explore our ideas and let the best idea win, you know So that's one of the really nice things about this is that you know, we don't have to agree up front There is no company that's making a decision on the way it's going to be blah blah blah blah Let's go at it and and see who who comes up with something that that works best So, you know, is it going to converge? We hope so And there's a lot of work I'm sure going on in that area if you come across Lars Kurt I'm sure he can talk to that more sensibly than I can He is the community manager in particular. He is he deals with the developers which I've Generally don't do very much I apologize Going once Going twice. We're at the end of the hour. Thank you very much for spending the afternoon with him