 Okay, so welcome to my talk. My name is Jostus and I'm gonna talk a little bit about virtualization from the heart perspective and Let me just quickly introduce the heart to you because I believe a lot of people just know the stupid jokes about it never being done and released and First of all, what is the heart the heart is a general-purpose multi-servar microkernel based operating system and the GNU project intended it to be a replacement for the UNIX kernel and As we all know that didn't happen but I think there's still a lot of value in the heart because First and foremost it exists and this is highly compatible So we built on the GNU C library and we have a fat C library So most applications just need to be recompiled of course there are bugs and quirks and stuff like that But it's highly compatible Compatible also, you got Debbie and GNU heard so you can install it like Debbie and GNU Linux and You will feel at home. I promise It is also a great place to contribute to learn system programings and maybe to learn how to work in the project or in the GNU project and I think the most important thing for me was that it freed my mind from a really narrow perspective of what an operating system is and Can do for me and for users and Really the heart is about The first freedom the ability to run a program as the user wishes and it gives me freedom for example from my If I work at a on a system that I don't control I'm an unproven user I still want to do crazy stuff on that machine or even if I if it's my own machine, you know, I want to run any program in the environment that I want and I control to a Greater extent than it is possible on Linux say Hmm So what about virtualization? It's everywhere. That's That's that's clear and people do it for different reasons. So maybe they do it for development or for you know, the gun the cloud stuff is about maintenance mostly and Some people do it for security whatever the reason virtualization is everywhere and there are different Kinds of virtualization. So you might have whole system emulation or virtualization like box or QEMU or you might Want to virtualize just a tiny part of the system. So it differs in grain grain granularity, right? and Back in two in 2012 I was here and I watched a talk from this Italian professor and so Davoli if I mispronounce his name. I'm sorry and He came up or he provided a very general definition of what virtualization is and he said it's the ability to interpose any resource and I like that but I would Say it a bit differently. I want to Shape the execution environment of my my programs in any way. I like this is the point for me the important point so We do have a coarse-grained virtualization mechanism in heart and it supports and it's it's a bit about like I don't know zones or containers or LXG and What it does it gives you another well another view of the system and it you replace the whole system and It's tricky actually to do this on monolithic systems because they are you have everything or Every resource implemented in the kernel and if you want to do something like containers you have to introduce namespaces in the kernel and The Linux community has been doing that for like a decade now And I'm not even sure they they finished and even even if they did the security of the system Always depends on the implementation if there is a bug well you break out of of the namespaces I don't know environment the nice thing is that it's kind of trivial on multi server Operating systems because you just have to start another set of processes that implement all the resources So this is how it looks like to start a sub part you do We have this program called boot and you give it a device or it could even be a file and This file contains all this block device contains in X2 image, for example and then Boot mimics what grub and mark does When starting a real herd system, so it loads the initial servers and it executes this weird scripting language called boot script and you can see this here, so we load the file system sour and The pass is ploid a root Device to it. This is this is this thing that's different on the real hard and Then we load the exec server that executes processes and then the whole Server bootstrap just starts and what you see here is a screenshot of starting Debbie and new herd system that I installed using their bootstrap and I just tweaked it a tiny bit that's basically the the herd system runs unmodified inside and it's just the Debbie and bits that Don't quite cope with the sub herd environment, but it works to a large extent So how does it actually work? Well, we just start another set of servers like the proc server that implements processes and the file system servers and the exec servers. It's a completely separate set and It actually virtualizes very few things most kernel objects like tasks and I Don't know memory management. It's just used as is Up to herd 09 which we released in December last year We had to run sub parts from an privileged user. That is root basically and Under this model the virtualized resources were the console so that you could see and type stuff and the root device and the time device and to for the Mark device setup you you have this This device master port that you use to look up devices. So that was virtualized so an 09 we introduced Unprivileged top house and It was actually not that much to do so the only thing that was needed is to get task parts for the newly Created tasks tasks are the mark equivalent equivalent of processes So what was needed is a way to hand task parts to the unprivileged boot process and we added that and you know we know through a mechanism called task notifications and We added another privileged kernel port stop that we don't have because we are running unproven unprivileged and Yeah stuff like that it's actually not a big change and The nice thing about this boot pro program is that it's tiny. It's like 2600 lines of code which is for a C program not that big and it actually contains a lot of stop functions and This weird boot script puzzle thing. So it's quite simple to do cause grained virtualization on a microkernel multi-server based operating system But that's actually not the the interesting thing for me because I want to do fine grained virtualization and And One central design aspect of the herd is that every service is looked up through the virtual file system So we have the slash death Directory containing all the what Unix called device nodes and In the herd it's it's nodes and they are connected to servers So we've got definitely and stuff zero The the block devices and Then we have this set of herds servers under slash servers Like the crash server if a process dies it talks to the crash server and creates a core dump or whatever And we have the startup server that manages the system startup And we also got the server socket directory and it contains our network stack or network servers and We don't have a Central configuration on how to how this should looks but what we do is that every node in our file system can contain a translator record which is basically a Command line and on access Heard servers bond. This is a bit like socket activation, but more general and in a distributed way and done in the 90s, so the Linux forks a bit late and As an example, this is how our network stack looks like and show trends Is a program that can carry the translator records and we have this the dev net dde Which is linux device drivers running in user space and Then we have a simple program a deaf node for the other net device and It looks up the other net device through the net dde driver stack and then on top I layered The other net multiplexer, which is an ever net bridge and It is connected to the primary ever net device and then finally on top of all that we have the Linux TCP IP stack and It sits on this node and it talks to one of the nodes provided by the other net multiplexer So this is our network stack so the Heard uses a terminology We have a thing called translators and what is a translator a translator is a server and it exposes a certain interface and Why is it called a translator? It's called a translator because it translates between you know one domain and the set of Virtual file system operations and maybe more So the virtual file system operations Described in the file heard file system definitions. This is our our File system protocol the heart is a set of protocols and we have a reference implementation. So this is the file system protocol and There are operations like dear look up and you give it a note and a path and a set of flags and maybe a mode if you want to create a file and you get another mode node and Sometimes you want to extend this to allow for more operations because a bit like the Plan 9 we say everything is a file and then we don't want to communicate By streams on top of this file system interface, but we have structured data being passed around and we have RPC definitions to And we get to Client stops and so our stops created for it This is how it looks like if we trace a program. We do stat on etc hostname and what we see is we send a dear look up message to this This is the path and deflects and the mode and so and you get back another port port as a reference to a capability and as I said the design aspect is that the virtual file system is used for server look-ups and almost every server in the heart is accessible using the virtual file system and For me there is this set of underappreciated family of translators and It's the family of translators that just modify They map from the virtual file system to the virtual file system Using that kind of translator we can modify the view One process has on on the system as a whole so every process has a Working directory and a root directory similar to how it's implemented on other unixes and We can manipulate the root directory Using a set trans change route It's not change route in the unix sense, but it allows us to start a process with with the new root directory and we can provide a Server that provides this root directory to the To the service This is how it looks like so we do Set trans change route we want to execute this command and We want the root directory to be provided by the remap translator and we want to remap Etc hostname and we want to substitute it with my hostname some other file and Then this program is run and you see not the hostname But my file so I successfully Exchanged Global configuration file in the unix way with my own file and I strongly believe that this is something that I as a user of a computer should be allowed to do and Because every server in the herd is accessible through the virtual file system Using this very simple technique we can for example replace the server socket 2 which is the In the network translator for the internet family with my own server and Start some program using it This way if my TCP IP stack uses a VPN I get all of the network traffic routed through my VPN and this is on a per process base So this is a very powerful way of doing virtualization, so I played around with this idea and I came up with the simplest possible translator and It's the it is the identity translator It proxies all requests, but it doesn't alter them in any way and If it is used like a traditional translator in the herd You can actually mount File systems or stuff like that by attaching translators to some node and if you do this with the identity translator So I'm attaching to the node mount the identity translator and I feed it Some directory as the underlying node, which is a bad name for I would call it the input of the function and Now it behaves a bit like a sim link, so I just see the content of the directory But there is a very important difference because if I Do an LS on this And what is happening is that the the identity translator does the lookup for me so the lookup is executed with the privileges of the Identity translator, which is kind of a problem, but it's okay if I'm the one who is actually starting the identity translator This is one of the open problems how this kind of server interacts with the Hurt authentication model but I think it's okay for personal translators and I Can query the what translators attach the node using this function and it says it's the identity translator and If I use it with the change route Command it actually implements change route the the Unix change route and We have a change route on the herd and it's implemented in the file system servers and it's a bit like a hack And this is a nice way to implement a change route because I have an external server that provides a way View on the on some resources and I just start my program Like LS or shell or Firefox or whatever Using this as the root directory and it doesn't actually stop there because now we can go crazy and we can implement and TPG translator and by the way, these are actually implemented They are kind of prototypes, but it works as I show it here and I have this This translator Hurt GPG and I've a wrapper around it because it's cumbersome to use such Transchange routes to set it up But now I can just do a verify and this starts the the pgp translator and I do tar Go fetch this archive over ftp from the GNU archive and When it does that it actually goes ahead and verifies the signature that is distributed with this Tarball and only if the verification succeeds it allows me to open that file Likewise, it allows me to encrypt a tar ball So I say encrypt for some address and then create a tar ball And it actually creates an encrypted tar ball So this is stuff that you can do Finally, I believe that virtual machines or virtualization technology like VM there or box or stuff like that is popular because It's easy to explain If I fire up a virtual machine and it's just like a physical machine this is a good picture for the mind and It's not key. I think it's very important to explain the operating system to the users because it's important if you if we want to empower a user we need to Explain the system to the user how it is composed and this gets even more tricky if we start with the distributed system like the herd and I have a prototype for that that I wanted to demo and There are two aspects to it the first aspect is Explaining how the system looked like at the global level and the second is how to explain the relation between tasks Okay, wish me luck Let's see so Nice, huh? This is actually a tree and this is the root of the tree and then we have Slash death and it looks a bit like a snowflake it contains all the device nodes and then we have slash run which is a temporary file system and If I mouse over it, it shows me some additional information about this and We can see here a slash servers and we have Sockets and here Here is my TCP IP stack and the first try to do this with with generating static Static images using graphics But this completely failed because you get huge images that are hard to comprehend and hard to display So I opted for For an interactive Exploration tool and this this is my prototype And the other thing that you can do with it and Need to restart it. Oh No, okay So This is my process. This is the herd web UI and It's actually connected to other tasks like, you know the proc server that Implements the process the concept of a process of a politics process and For some reason it has a connection. Oh what you're seeing is a directed graph and We you see an edge if If this process Has a port or handle to an object implemented by the server so this edge says for some reason, I don't know the process opened deaf random or deaf you random, I don't know and So this is the proc server. This is the TCP IP stack the local network stack. This is the root file system. This is the authentication server and We can actually start from there to explore explore the system For example, this is the terminal And I can ask what other processes have opened the terminal by double clicking on this and I see That there's my shell that also has a part of the Terminal and sudo because I started the program with privileges because it has to be privileged to probe into all the other processes And we can ask, you know, what other kind of Programs are using the TCP IP stack a bunch. It's awesome. What the JavaScript world can do, huh? And we see actually here's the evernet multiplexer and And if I open it Works better with the bigger display. We see it has a connection to the Device driver, which is to be expected so to conclude from my point of view Being able to manipulate the virtual file system It's a way to do a fine-grained virtualization on the earth and It's actually surprisingly easy and fun to do on a multi server operating system and If you're interested you should actually come talk to us join us do some fun stuff with it and You can imagine all kinds of servers that do similar things to this. So maybe we can have a geeks translator because There are people working on porting new geeks to the hurt or the other way around or both and Manu is it's here and he's going to give a talk about that tomorrow. I hope that's up to date and If you haven't already you should go ahead and watch the previous talk because the previous talk was About the same thing Mostly but for the g-note operating system and it's interesting to see their perspective on this so if you're interested go ahead and watch it and If you want there are two fun papers both are called our virtual machine monitors micro kernels done, right? the one argues pro the other country and It's a nice introduction to you know different ways on the on the world and That's it. I'm happy to accept So the question is about the web server is actually running on the the hurt system this one By the way hurt is running on actually hardware not a new hardware, but on hotter And it's actually gathering the information from the system It's we have this this thing called port info which does the same thing, but not with the You know Jason interface Servers So the question was whether we can have The hurt running across different machines on an accurate and if it provides a single system image Kind of interface. The answer is no Mark was designed to allow that But it was lost or never really implemented and I know there's a guy working on net message that is able to send Mark messages across the network and I haven't tried it, but apparently it works to some degree I don't know so it's possible in theory, but it's not our focus. Oh, yeah Just a speculation, I don't know too much about it either, but when smtp support gets into hurt eventually in the future somewhere What would the impact on sub herds be in terms of of privileges and I mean a sub herd could potentially Hug its host right so the question is about SMP support and whether how it impacts sub herds So mark has been always SMP capable capable, but currently our version of mark only runs on 32-bit X into processors and we don't do SMP. Sorry and If you would that would be nice then again and the second part was about Whether one sub herd can hook resources like the processors Accounting we don't do accounting and that's a huge problem. And so we don't have a good answer for For Denial of service attacks basically If you need that you should probably look at g-note. It's possible to implement that in mark Everything is possible in theory, but it would require a huge amount of work to bring mark up to speed Oh So the question is whether we want to use hardware virtualization if you want to run a different Operating system and so if you want to do that we should part Qamo and stuff like that Someone would have to do it. I'm not that interested in that because I think this is more fun Through So the question was whether it's possible to implement that in the kernel or maybe also in user space a hypervisor I'm not sure we do device drivers in user space. So maybe it's possible to Implement some or most of that in your space, too. I don't know Questions to his colleague and probably co-worker with the words talk to the hand So this was a snarky remark about the papers and the one author The name 10 so the other one can say talk to the hand Lots of fun papers you should read them. It's about mostly about Zen and The first paper argues that a Zen is a better way to approach the problem Because of this and this and this and the other paper refutes this claims by interpreting The Zen architecture from a microkernel point of view and it's a fun read Yeah, the first one and This is from the same Then people and it's a pro and the other one was written later and it's contra All right any more questions. I guess not then. Thank you