 Hi, it's not every day that we get somebody to speak at DevConf, trying to get us to use something that's not there yet. So, well, I think Holger is a real candidate for that. He's always been disruptive among us. Please welcome Holger Lebsen, who's showing cubes always. What's this on here? So yeah, I will talk about cubes and my experiences with it in the last half year. I have considered it the first time, one and a half years ago, it took me some time to switch, and then it was actually quite easy. So, about me, I'm using Debian since 22 years this year. So, 1996, I don't remember whether I started then with Bust and Rex, because I was not using Linux fully, since 1997 when I'm exclusively using Linux. On my first laptop, I had Bo, then I switched to Hums, Link, and a potato I started contributing. Woody was my first DevConf in charge, I applied as AM, and in Edge, I was finally Edd. And everything, other release, up to Jesse, they were all running on my main computer. And I love Debian. And today, I still run mostly Stretch, but this machine, my main machine is now running cubes. Cubes is based on Fedora 23, so it's outdated and not supported since some time, but it's supported by cubes. And in this talk, I'll try to explain why. So, about you, who has heard about cubes or has a faint idea what it is? Okay, that's good. Who uses CubesOS? Yay, who tried CubesOS and didn't switch. Yay, why? We need to talk. And I know about one person who switched to cubes during Devcamp, which I found very nice. And he's happy with it. So, what is cubes? Wikipedia is useful. CubesOS is a security-focused desktop operating system that aims to provide security through isolation, and virtualization is performed by Sen. And within Sen, you can run any other operating systems. It's also a single user system, so it's not in the design that other people lock into the computer. It's just in your computer. And it's free software. It's mostly GPL. There's some libGPL stuff by, no, library GPL. And so the code lives on GitHub. So under GitHub user tools OS, very useful is this Cubestock, which has all the documentation as a Git repository. I really recommend to check it out. The reading is better done on the web page. And it really covers all the use cases and special cases and UB key and what not. So this is really the recommended source of information. And cubesOS is really old. So Joana Woodkova designed that in 2011 or earlier. So the following picture is six years old. And this is the way she describes doing stuff. So the red VMs are less trusted, the black ones are really trusted, and there are several internets on there, because you can use several ways to connect to the internet, and each VM connects up to each VM can connect independently to the network. And there's really nice, this blog post from 2011 is a really nice read, how she does whatever things, and then you can try to base your work on this or something else. So more about me, I don't really know Zen. I don't really know Fedora neither. And I don't have to, because I'm really just using this to use Debian to develop on Debian. And I'm only using cubes in six months. So what I will show is cubes 3.2, while last week the 4C release first release candidate was released. And it's a fairly unmodified cube, so I upgraded the kernel to better support the hardware. I installed i3, and I created a Debian 9 template. And that was it, and I upgraded some packages. And on my production system I've not done much more, I installed two more packages, and I configured i3, which I actually also done on this machine now, because I really prefer i3 over XFC. And all the graphical stuff I show you can also be done via command line tool, so you don't have to use the mouse. It's option cubes manager. I don't use this, there it is. So this is the cubes manager. And you see there, the VMs with the state with the red-green circle they are running. So the personal VM is running, and this net, this USB, and this firewall. And what the other things, the state is they arrow down, there are upgrades available, which I could install, and there's the memory usage and the net VM. And this is one thing. So DOM-0 doesn't have network. DOM-0 is insane, the hypervisor, here it also runs the graphical stuff, but DOM-0 doesn't have network. If I want to upgrade it, there's a special tool to upgrade it, which I will not explain here. The others, there's a network VM which has access to the network hardware, which is untrusted because the hardware is often easily exploitable, and then everything goes, this firewall is connected to this network, so the firewall VM is considered trusted, and all the other uses this firewall to go to the network, except that there's the su-nix, which I will explain in a second, so you can have several different net VMs. Oh yeah, starting Firefox, I've done it, but I'll show this other thing here. So this is the application menu of XFCE in this case, and I can start Firefox here, I could start Firefox here, I could start Firefox there. This is, I already explained a bit, but we have to connect this, we have to make this smaller probably, so we have the network, the firewall VM, which goes to the network, and which the application VMs are connected, and there's this su-nix VM, su-nix is a Debian-based distribution for Tor usage, so this su-nix is essentially a Tor gateway, so if I make this su-nix, the firewall VM of my application VM, all the traffic goes through Tor. And I can have also several su-nix, several firewalls, just disk storage is the limit. And then I switch the firewall now to Debian-9, because I've left it at Fedora, and that's really not nice. So one thing I need to shut down this VM, I don't want to shut this down, I want to remove the network for this personal VM, no more network, because I cannot shut down the firewall if something is connected to it, and now I can change the firewall, and now I can shut it down, the su-nix is still connected, okay, the firewall to be Debian-9 instead. Now I start it, and that was changing the operating system for my firewall. Pretty easy. I could also switch the firewall to completely, it doesn't have to be Linux as long as it runs Zen, so one thing, Mirage is a OCaml-based firewall, which I don't remember, one megabyte RAM requirements and a few more megabyte storage, so you can save resources there if you have other firewalls or if you just want to have more secure firewalls. And it's the same thing, I switch my personal VM to Debian-9, for this I will need to stop Firefox though, or everything, so this is, I shut down this fire personal VM, yes, switch this to Debian-9, I could use also Fedora, switch to Fedora on Firefox and Fedora, and I can also switch the network VM for this untrusted as a su-nix gateway, and now everything I do on this VM goes via Tor. And if I have a second UNIX VM, then it goes through a different Tor circuit, of course. So I could also make that for every app VM I can use a dedicated network VM if I want to. The next thing I want to show you involves actually switching to i3 because I don't really like XFCE, I use it also very long, if you have questions in between, just feel free to ask them. I actually need to switch to i3 to use it. You cannot see that. There's an upper right, the desktop environment switcher. So this is no cubes. Okay, and now I want to start, this is another way to start a VM, what was actually running. Now I start back Firefox. Modification I've done here is that I modified the status bar a bit with the boring Spanish clock and the battery usage, and these are the number of cubes. So currently there's two application cubes running, three system cubes, and DOM zero. And I have a total six application for system and five templates. And I'm using currently a gigabyte of memory. And the system. Oh yes, and the last thing I made, a simple clock, tada, to show the time. And this is really the only modification to standard cubes I really have. So with meta and enter, I open a shell in DOM zero, and with meta shift and enter, I can select on the bottom where I want to open the shell, the terminal, and if it's not running, then it will start a new VM. And the same is done with XFCE as well, but I like keyboard shortcuts. And the same with meta shift F, I can open Firefox somewhere. So we'll start the next one and start Firefox there. But it's there. The other modification I made for myself that I can turn on this place and turn them off. I really, really like I3. Anyway. And then there's one problem. You cannot, or one feature rather, you cannot copy stuff, copy and paste from one VM to the next one. So you need to put it into your copypair buffer with control C, or whatever there is. Then you need to use meta control C to copy it in the cubes buffer. Then you need to use meta control V to copy it in the cubes output buffer for the next VM and then put it in the final one. This works nicely. And it should not protect you, but should it enable you from accident in practice, I found that I actually more often now do copy and paste accidents than before. I'm not sure whether that's really the best thing. So if I should... If I should put it here. I will not show it. Yeah, the templating stuff. So, before I show this, let's go back to... So if I want to clone a VM, I can clone the whatever work VM. No, it's not wrong. So I go to VM here. Create new VM. So label is Montreal Airport. Airport is also nice. Then I can choose what I want. I want Debian 9. I want this fireball. And then I press OK and it will create a new VM. And it's reasonable fast because a new VM is created. And now I can open a shell in Montreal Airport VM. This is done because there's these templates VMs. These black ones on the left are all templates. And each application VM, that's this column, are based on a template. So if an application VM is started, they will use the templates file system with a copy and write an extra edition file system layer so that modifications of the update VM are lost after reboot. So if your system gets compromised, it's gone or you can install packages for testing and they will later be gone. The template VMs are configured. They don't have full internet access. They only have access to the up or young reports. So they cannot access normal sites on the web. And that's this part. And so you can also clone the Sysunix to make a Sysunix 2 or 3 or 4 if you want more tour gateways. I will not show this actually. And you can easily increase the disk size, but default they all have 5GB disk space in home. So if I want the personal VM to have 20GB, I can do this here while running. It's 2GB by default, so I just change this and now it's 20GB. Increasing is easy. Decreasing is not supported. I've already said this. One good thing is really I don't save on template anymore because I basically I'm aiming for every up VM as its own template. Sometimes I want some which are the same, but usually I have to get more and more templates. In the beginning I got more and more up VMs and now I'm collecting templates. There are about 5GB per template and I have a terabyte disk. And one mistake I made initially that I tried to use my user account in the VMs and that's not a good idea because Qubes is configured to use the user user for certain things I'm going to show and my recommendation is stick with the user user. Don't change it. And then I can also use Nautilus to copy files though I really never do this. Let's see how this is done. So I want the work VM files. Now I can use right mouse click and say copy to other up VM or move to other up VM and this way I can copy files. I usually do it over the command line because I don't like graphical stuff, but it's possible. And now I'm stopping with the demo part. There's also USB isolation. So there was this USB as well which has the USB controller attached. So dom zero is not exposed to them and so if you're attaching an USB stick it will be attached to the USB VM then you need to tell Qubes to reattach it to the VM you want to use the USB stick and before unplugging Qubes as well. Then you can have nice USB isolation for that. The same with PCI device isolation. You can the net VM is the only one which is access to the network devices. You can do this for any PCI device. I keep the network VM running with Fedora so that the modules in the VM and the firmware in the VM is the same as with the kernel. So maybe it will work test if it works with Debian and UVM. Sound is also working. It works via Pulse audio and just working out of the box so each VM has sound. Though this is also a bit of a security issue and in general the plan is to split this dom zero into dom zero and display VM and then also add a sound VM but this is not being done yet. That's hopefully in the next release after 4.0 and as you can see full screen applications including video are possible as well. So I use everything on the computer with Qubes now. There's more features. There are also disposable VMs which are created with a mouse click or with whatever and they're based on one template in 3.2 and the new version you can use other templates for it. So you can have disposable Debian 8 and 9 and Fedora. I've not used them this much yet because sometimes I really like to keep it. I rather create temporary VMs which I throw away every week or something. But you can open links and fires in disposable VMs and another thing is there's also an option to convert PDF into trustable VMs that's done. The PDF is opened in a disposable VM and there's a screenshot taken of each PDF page and then saved again as a PDF of screenshots and copied back to the original VM and then the disposable VM is destroyed and you do it with a mouse click so it's not any work. The same is there for images because also images can have exploit code in them and as I said there's many command line tools which are available in the VMs so it's converting image copied to VM MRO, entry I don't know DVM is the disposable VM and so this also means there's a cubes up repository which the cubes Debian VMs use and that would also be something which might be interesting to get in Debian proper one day and Domzero has a lot more commands these are for VMs and these are more for cubes manager and cubes database and stuff. Those are mostly if not all written in Python and then backup is included include cubes manager and it's good and bad I would say I use it but at first we suddenly have way more systems to back up it's not one system anymore but we have 20 or 40 or whatever and maybe three quarters of them are worse back up more or less I don't know and one issue it's image based backups only so it's the whole disk image which is back up which I don't like that much but as you have 30 systems to back up what's really nice one event recently I backed up my whole system then deleted some VMs locally set up another system with cubes restored Domzero and some VMs there and I had my system suddenly split into two computers and that was really really nice and then after the event I moved the VMs back to a single computer that's really cool and there are several custom improvements or custom script backup solutions if you don't like that one and again this backup restore is either you can use it with a mouse from the cubes manager or from the command line and then you can script it as you want and generally that's a problem repair recovery might be harder Domzero has no network and it's Fedora so if you don't know Fedora and GrubBrow or you switch from a Grub system to UEV system or whatever Fedora on Fedora you don't because it just works and memory so this system has well I'll start with this one 4GB is definitely good for testing I'm not sure whether you can test it with 2GB but I wouldn't recommend 4 is really good for testing 8GB is good for some type of works in my experience if you don't use a USB VM and don't have and the top is open to many VMs 8GB is fine Tauwa who started using it here has 6GB and he's happy he can work with it he said 12GB is definitely better than 8GB because of this wish VM but my system has now 16GB and I sometimes max it out and then you cannot start new VMs which is annoying so 32 is better you run Firefox you need RAM as much RAM as you need and if you want RAM you really want RAM 2TB RAM would be nice as well but seriously if you have 4, give it a try and 8 should be fine CPUs and SSD disk space one or two CPUs are fine more is better they need to be some support modern virtualization features which all systems from 2012 or 13 or something should support or not all i3, i5 is definitely fine there's a hardware compatibility list on the cubes web page which has models listed and what problems there are or not I found this laptop that's an X260 and it was crashing with cubes 3.2 like mad and it's kernel 4.9 which isn't upcoming for serial cubes release or there's also packages for 3.2 so this is stable now 60 to 100 GB SSD space is definitely fine for testing I've also done it with 50 GB for working depends what you need each template VM is 5 GB and an application VM is a few megabytes and what you put in there so it really depends on how you use the computer missing features I found you cannot hide I made with send encrypt on suspend doesn't work yet but there's work in progress for it so that forgets the disk encryption passphrase on suspend changes in a run file system to have the screen saver dialog and once you enter the disk phrase correctly it will then remount the file system again I really hope for this feature to be soon and I think it should also be in Debian somebody should do something and there's no IPv6 supporting cubes so if you need IPv6 that's a bummer because all you need to do the networking yourself I've not really looked into it I'm sure there's a ticket for it and how to but out of the box it just does IPv4 so my conclusions or preliminary conclusions totally work for me I'm using it since six months every day I installed it three days before FOSTA and I had hardware problems for five months just the months ago I switched to the X230 and before that resume was completely unreliable but it worked nicely without resuming and so I just didn't do that often so check your hardware I still don't like Fedora but I don't have to interact with it so I'm fine and partitioning one's digital life is really hard like this picture I've shown you from Yoana with all the VMs I've now I think I've got 35 application VMs and I still need more and it's tough question how to separate that the nice thing is I can still do this later and not lose anything and yeah and moving VMs from one cube system to another is really really nice super useful and it's super easy so if you need to travel and don't want to take data with you so my cubes to do use the password vault that's a special VM which doesn't have network where you keep your private key into the other keys into the other VMs I've decided I wanted this nice partition digital life but then I've just moved all my old computer into one application VM started using cubes and moved stuff out of this VM so I'm now using the computer more secure than I used with the VM but not as secure as I could with cubes and this is this password vault thing and I still not only use my Yubi key but also the password half having half the password in the Yubi key and half memory that works nicely because it's just attached to the USB VM but keeping GPG sub keys in the Yubi key then I will need to attach the Yubi key assign it to the app VM use it there so I will use this switch together with using the vault VM and then automate this so it is the configuration management system used by cubes internally to create all these different VMs and configure them so I want to build on that but I still haven't gotten any documentation how cubes use it and how I had some questions with different answers so I put this later on the to-do because I don't really configure the system much anyway but if you know Salt and can give me a short of that in general I'm very happy with cubes without having changed much more conclusions I really really like it, the separation is awesome I also like to be able to create new VMs so easily and this Hoonix and Hoonix itself operates great so there's this Hoonix is the store gateway and there's also Hoonix workstation which is in desktop with tour browser installed there's some other tour applications as well but that's really nice suspend, resume issues are annoying but with this with the newer kernel and the X230 they are gone sometimes there are some routing issues when the network is recreated which is also because of the network hardware so with different hardware it might not be the issue and so choose your hardware wisely and probably I'm using only one third or less of cubes potential but that really pays off already your mileage might vary try it for yourself and changing one's own habit is really hard but doing so is good so that's the talk thanks to the cubes people provide this nice operating system with offering help with it and thank you Debian people to provide a good operating system to run on cubes I have a couple of questions the first question is it looks like it's an administrator for virtual machines and not an OS so what would it take to run cubes on Debian instead of Fedora cubes has some 20 or so own packages but it also builds on top of Zen it has some tiny modifications I think to Linux as well but the most problem to create Debian based cubes OS would be the Zen part but on the other hand I don't I used to want to do that but I don't think it's sensible anymore because I think rather DOM0 should be a way smaller OS so I'll pine Linux or a leader based Linux system but not a system like Debian or Fedora which has too many libraries linked so that's the cubes people were also interested in Debian because Debian is closer to reproducible builds than Fedora so they were interested but now they switched to this aim of a smaller distro for DOM0 because what they have at the moment is the graphical interface is at the moment still running in DOM0 and that should be moved to a different VM as well and then you really need nothing in DOM0. Okay and then the other question when you moved the firewall from Fedora to Debian what was happening? Like you mentioned salt so maybe you didn't explain what was happening can you explain? I think I don't the firewall rules I don't really know how they are done there it's probably a package there but the operating system has switched and that will be solved in that case Right now my question is how did the firewall know how to set up the firewall in Fedora and in Debian I expect they are pretty different I think it's IP table in both cases I don't know My question is there's different ways the separation can be made more or less gradual say cubes is much more secure than having everything in the same context be more secure to have a laboratory of machines and both each one of your faces but I mean it's not implemented but I don't know if you looked into something that maybe would be more conservative, intelligent, resource wise that instead of virtual machines would use containers I know that would increase the likelihood of being able to escape a containment but it would be much more friendly towards smaller machines have you heard about the other approach for cubes is not to be friendly to resources but rather to provide the most secure isolation so that's why they choose send because they are the hyperbiover you can actually review and understand but the cubes architecture is made in a way that you can change this virtualization solution right I don't think it will happen really soon because there's not much interest in it but it should be possible and I think this general approach of separating your computing reams on a computer more and not do everything under one single user in one operating system just using two user accounts is better than using one it makes a lot of sense only that it would be nice to be able to do it more friendly something to add and a question I don't know if I missed it but one of the features of cubes and I don't know how well this comes out with i3 is when you're running the different VMs there the border is colored according to the specific VM where it is so like the one on the right is a blue so that looks like the work VM and if you're in one of these template or the DOM zero VMs it will be black so you have this nice visual cue that's very subtle but helpful for you to know okay I'm in this untrusted or I'm in this work or vault VM or whatever it's really nice I see the i3 so that's his personal VM there and just this small UI thing really helps you when you have a cluttered screen so you know that these two in the middle are personal on the right where the blue one is that's the work one I can't see very well from here so the question I had was if you have investigated any of the how secureboot works with cubes in any way do you know how that is integrated secureboot helps me as again like it yeah I mean does cubes or fedora have some way of doing secureboot so that you can they have a certificate for assigning the the way I do that would rather be to use heads coreboot based linux system which runs from the ROM uses the TPM to assert the nonchangeness of your hardware and software so it does a different implementation of secureboot essentially needs coreboot but that would be my approach and I've not yet anything about cubes using secureboot yeah secureboot they also recommend coreboot they don't recommend heads yet but my question is just sample who is the people behind this distribution and why we supposed to trust that people because one of the most important points to choose dvn is related with trust in my case so every time someone come with another technology I'm hoping to try but I always have this question about cubes is made by group of people now originally funded by Invisible Labs which is a company from Iwana Rotkowa in Warsaw in Poland I know Iwana from giving talks at the CCC congress since 10 years or something she made several security related talks there they seem to be reasonable paranoid people but yeah it's free software you can rebuild cubes and so far the image is not reproducible but hopefully it will be soon but yeah we'll trust how flexible is the firewall configuration have you got any ways banks I mean NET or masquerading things like that I'm doing NET and you cannot one VM cannot talk to the other I've not really looked into the firewall at all I just use the one which is there by default I'm happy with the nothing stuff so I cannot access the VM from the outside I can access the outside world and that was enough for me but I don't know but some people made this mirage based firewall I think it's quite very flexible I'm curious if if there's any movement by the cubes people to suspend in a VM so that you could sort of suspend it move it to a different cubes running and resume it there or if it's already possible I think this is supported by Xen so it's not impossible per se but I don't think it's working with cubes that way yet you know if there if it's a goal up there I don't know cubes are mostly made for running on personal computers so it's not so much the use case that you move it away from one machine or server to another there's also people some people want to use it as a server there's many things to do say thank you Holger and please take him together with me