 The channel is deppconf18-z on OFTC and this is On-demand unboxing both I have entirely forgotten already how I titled it which is entirely fine and basically Yeah, my goal I would say was to talk about how we can Provide a better experience with sandboxing applications and so on for users and developers and I guess I would love to start with who you all are and What you want to talk about and things like that. So I guess I already did introduce what I want to talk about But I'm Nico. I've been a DBN contributor for a couple of years. Hi and Have a seat, please So I was saying I'm Nico. I have been a DBN contributor for a couple of years Still not a DD because I'm particularly lazy about doing the paperwork thing, but at least I started the NM process again and Yeah, generally my interest is in this topic is Partly because I you know do not party particularly like Running code from the Internet or random code from the Internet on my laptop with all my permissions and partly because I Yeah, because I think it's a worthwhile problem to fix Hi, I'm Julian. I'm from the app team and My interest basically is The ability basically what I would like to see is to have Untrusted depths I can just install from the Internet and run Without having to trust it and without it having access to my data and my webcam or whatever I'm Paul. I'm Interested in converting the DBN archive into flatbacks I'm the Togatsugu Nocubi mainly Maintained some kind of Natural process natural language processing packages Especially for Japanese I'm Alex Doyle. I'm a build engineer at cumulus Networks and my interest in this was providing isolated build environments for developers to build in So currently we're using Shroots for that, but it takes a while to get in and out of them. We're looking at Docker containers as well as maybe an option For providing isolated build environments because we have a shared build systems We don't want developers stomping over each other every time they install new packages So I'm curious as to what sort of sandboxing technologies and options are out there I'm sure I'm a SRE We are using Docker and Kubernetes. So I'm curing if there's some other options I mean other than Docker And it's working for us. Thank you. Okay, so that's all Okay So you did not wish to introduce yourself or? I was looking to your friend behind you Suntai Lee, okay. Now, by the way, if you wish to speak, please use one of the microphones And I did that without speaking into the microphone. So yeah, see that quite a we have quite a few people interested some of which like some of you seem to be more heavily interested in like Build environments and of you are interested in having more less more desktop sandboxing kind of stuff Thankfully at least from my from my perspective a lot of the problems that are the same in the sense that we We need to provide clean environments with And we need to do this relatively fast Actually, how many people here are using as build or a CH route? Oh, sorry I was asking if you ever used as build or a CH route Hi, you just missed a round of introduction. So would you like to introduce yourself? My name is a big dust more pattern. I work for a calabra We currently one major project to assign with valve as part of which I'm working on runtime Shared library isolation technology. Yeah, we're saying Feel free to sit more in here if you want and also Feel free to introduce yourself from Carnotty many man maintain army L kernel Yeah, so the reason why I ask you how many people use as build and a CH route here is because a CH route is The closest thing we have to more or less and demand sandboxed environment in adibion currently, of course, it's It has two major problems. The first one is that it's not a particularly good security boundary in the sense that it's just a shoot and Well those days we have a bunch of We have a better Isolation features in the kernel, which should be taking advantage of like namespaces and second filtering and And possibly And possibly security modules like a palm or not know that we have one which is available by default Full disclaimer, I am on the apartment teams. I am slightly biased in favor of it But yeah, so generally like one of the questions I've been asking myself is oh To be entirely fair here it was prompted by in he recently pointed out that a see a truth does most of what I want just not It's not particularly fast at it because I mean, I don't know if you for those of you who an S build the first thing S build does is it Updates apt in a the see a truth it runs an install in the see a truth And that already takes by itself most of a minute and waiting a minute to start an application. It's not particularly good Don't waiting a minute to enter your development environment is also not particularly good So oh actually Guess I can just ask you a question Julian How terrible would it be if we did things like Not actually installing packages inside a sandbox as in not even running apt inside a sandbox and just extracting the depth in there Well It depends on the depth, right? So like a desktop application probably would crash if you don't run the maintainer scripts because it has stuff like g settings Would you need to update? like the We top the database for the So if you don't run the maintainer scripts these apps don't work you can't use them I think most of the time is not spent in running the maintainer scripts button running the unpacking and For the what what you should do probably inside one of these is to run the whole thing with eat my data And then do an sync at the end so you avoid these things in between Yeah, or just I guess you it would definitely be possible to just transfer It was a woolly matching temp FS and then eat my data doesn't matter because it's Yeah, I used when I use the S build I use the temp of s To build on and it was much nicer And overlay fs with temp of s at the upper layer So for people who are not super familiar with sch roots that's basically the equivalent of Using an overlay fs for that basically lets you reuse the same base image for all the shoots and as you spawn new shoots So it would be a bit like using the same from for docker build. I guess the main difference is that here's a The star as a second layer of your sandbox would only exist in temp FS So it's not actually written to disk Which is actually quite nice in terms of old fast state goes Yeah, of course it means that you use a bunch more RAM, but There are also ways we can deal with that Yeah, wanted to ask like as there are people here already doing that sort of Sandboxing for development environments or desktop applications and what do you already use currently? I typically have A few sch roots, but they're sort of long-lived because I'll go into them I need to isolate because from my work I need to Quite often try out some dodgy patches to lip see which can't really go into a normal development environment because you know That you can't break lip see the world catches fire So they're currently quite long-lived, but again, that's because they're slow to set up if they were fast to set up I wouldn't keep them lying around as long because you know I'd have a cleaner start every time and I wouldn't get assumptions baked in which does happen occasionally So yeah, I'm not doing anything clever at the moment, and I probably should And at cumulus we're using them as part of the build process. So for our release we have Trutes tagged for each release that will have a set version of packages Debian packages plus the stuff we're building So it's a consistent environment for the build and the developers will also use those they can log into them install their Development versions of their packages and test against what the actual release is So we're using them both as an automated build environment that gets run to build packages from source And there's also a development environment for developers to be able to play around without blowing anything else up They do tend to be fairly long-lived And then people will set them up install a bunch of dependencies and then continue to develop within them But at new releases they can always, you know close it out bring up a new one and re-initialize everything So that's maybe not the intended use, but that's how we found it actually works pretty nicely to use them You mentioned earlier that you had issues with developers kind of stomping on the one-on-us food by Because they were using the same environments. What did you mean by that? Right, so we use the the shrewds to avoid that happening So every time they log in there So we this also has the advantage of having the build systems be isolated from the shrewds They're running so somebody doesn't install, you know packages within them on the build system itself rather than within the shrewd So but one of the things we're looking at with that workflow is maybe using docker to Create a build environment where they can actually play around some and then nobody has access on the routes on the host systems apart from the sysadmins So there's still some configuration. I think to be done there, but it's worked The downside is that when you're loading a shrewd, it's a compressed file system So it has to decompress and then everything gets mounted And then they're in there and then when they exit it also has to recompress their personal copy of that So there's a bit of a hit getting in and out of it in and out of it like you've mentioned starting up your environment So with the compression, I think you Astrid I think it supports LC4. I'm not sure So that's at least relatively fast. I think so we all Normally it uses G sub I think and that's relatively slow, but if you use LC4 it's quite fast. I think sorry I Took a second to collect my thought Yeah, I think So we we heard a bit from people who are already using that sort of things I mentioned for instance. I'm already also using guess fruit Think I would like also to people to mention what I Quite hard as a main inconvenience you and Carter you mentioned that it's a bit slow to gets in and out of Isolated environment But we didn't really hear about anything else. So if anybody wants to take the mic So one of the things that I have to do Occasionally is sort of use the actual graphics hardware or sound hardware on the machine and you know, it can be sometimes it's a bit of a pain remembering to Mount all of the right things that a modern desktop environment needs into the truth So things are exposed and things don't immediately. So The thing that happened. I don't know if any of you were in my talk But one of my demos worked and the other one didn't and the second one didn't work because if you don't have everything set up Just right Pulse audio will go into a CPU eating deadlock as soon as you try to play sound through it Which is I think what broke my second demo because I had one missing thing that wasn't mounted in the cheroot But you can't keep everything mounted all the time because sometimes when you install a package It will do things and If it can actually communicate with the machine and like a service might get killed if you mount all of the run Directries and so forth. So you have to be careful. You can't keep them mounted all the time In case you install a package which does interact with it. It's a little bit different for me because the stuff that I'm doing is Things that need to interact with desktop type things all the time Which I suppose isn't necessarily what what people are always working with but if you are in that use case It's awkward. You need you need you need to conflicting things. You want to be able to interact with The real system in some ways, but at the same time, you really really don't want to touch certain services So that's kind of a pain point for me It's interesting because you said it's not necessarily what people are doing But it's very much what you need to be doing if if you want to run desktop things at all and Yes I mean since perhaps mentioned flatback It's one of the things I've been looking towards there because they have all those problems to solve They have a bunch of other problems that don't solve and actually I guess I will mention that as my Major complaint currently which is that it's actually there is actually no tooling to tell you to basically tell you Oh, your shoot image has pending security updates. You should update and things like that which is a problem both with flatback and snap but also with Docker wait can be I guess a bit more of a problem if you run that like in a production environment Because certainly well suddenly you just don't have updates for for your production environment well for snaps there's Notification service I think which Tells the developer who uploaded the snap to the snap store Tells them that they have security updates for the packages they include But if of course if they built other software from git or something they don't get notifications of that Okay, didn't know that snap already built something for that. It's actually super encouraging It seems quite nice okay, so If nobody else wants to mention anything about that, I thought we could perhaps mention talk about Well, at least brainstorm about what we can do to like solve these problems because so we mentioned that it takes a long time to set up environments That's it's difficult to actually get desktop stuff to run in there and Oh, yeah, and that we want or need Actual as actually the ability to know what is in store in terms of packages inside the environment so that we can Detect when there are updates available for it and I guess if nobody minds I wouldn't mind starting with what can we do to make like set up faster for those environments because I'm using P builder with cow builder and that's reasonably fast. It's There's a pre-built search route and it creates a new directory and I think it uses hard links to Set up the temporary root of us So you mean that co-builder avoids doing actual installations and Yeah You create the search route and that's the primary storage and then when you're building a package it Creates a hard linked copy of that. Oh, okay, and then Installs build dependencies and does a building as okay So that sounds fairly similar to what has built us if you use it with another layer first and tamper first or something like that Yeah, the cow builder thing is a bit of a hack that predates Overlap s so if you get the choice to use overlap s you should do that so I think another thing you could that could be considered maybe is like using Davion systems and build frameworks out of them For development or for running applications then stuff them into Austria or something Which enables you to manage these frameworks and share the contents and then you can just combine the frameworks together with your to build your own train route and Work in that or run the app in it basically like a flat pack works but with Davion packages and And Yeah, you can add an overlay on top of that and then install any remaining packages You're missing, but you get a speed up because you have most of the packages already installed Also, you mean basically putting more packages in surveys image and not just Not just a result of the bootstrap for instance Yeah, you could have the base image you could have then you could have an overlay like a gnome overlay or a Development overlay and then which has built essential installed and stuff And then you could just install the remaining built dependencies when you're building the package or when you're running an app You could just install the remaining different ends for that app And then you save time for the setup Yes, that's actually very fair that we could probably Have multiple overlays and be able to reuse more stuff Not just like the base image, but more things on top Yes, there's something I was Also somewhat wondering about is I Mean we can definitely cache environments like basically if we if I already built a given on environment with a set of packages and I decide I Want a similar environment again Probably because I'm running as built repeatedly or something I could definitely just Get a copy of the same one again without having to redo the wool To reduce a whole setup phase then I think that would be especially relevant if we are talking about sandboxing the stop apps and so on because It might be fine to wait 30 seconds or something when we are first starting Firefox or LibreOffice or whatever If I have to wait 30 seconds every time I open a LibreOffice document, I will be very very sad I think the desktop apps another approach We haven't talked about the desktop apps might be to just have normal depths and Install them all on the host system and then basically bind mount the host system into a container and then run the app in there with some isolation and Bind mount the stuff you need from the host system over there And maybe have some Obama or as a Linux rules to further tighten this Yeah, but then like all much of the whole system do we not inside serve quote isolated and quote environment. Yeah, so Optimally, we would just have like the user partition mounted there To have the whole OS, but that needs the whole user merge and handling of other stuff Maybe like ETC changes and var But it would be nice to have I think Because you can just run normal depths and yet the isolation you get with the snaps and flat packs But you get all the advantages of the normal depths with security review and stable updates and stuff Yeah, and I'm glad you mentioned that because It's very much at least for me a goal to have Be able to use normal depth packages at least At least so long as there is not a good consistent story for to have security maintenance of snap or flat back and so on So I've tried doing this with them bubble wrap the tool that flat pack uses to sandbox applications You can basically do it just with a few command line options I've only used command line apps within bubble wrap and it works fine But I haven't tried the graphical stuff I guess that would require some of the portal things that flat pack sets up Which I'm not sure how that works yet yeah, too as Useful information for people here you who might not know yet portals What are not as our security concept where basically instead of for instance, if you have evens in Sandbox evens is a PDF reader. So instead of giving it Access to all of slash home because you know, it might potentially read a PDF anywhere in there So instead of doing that because then it has access to all your user data and The sandboxing is a bit pointless if it can't just rifle through still this slash dot no PG or whatever else and So instead of doing that basically you give it access through a socket to a helper that runs outside the sandbox and basically when If you go file open for instance Evians will just ask that helper. Hey, please bring me a file open open dialogue so the The helpers and displays a file open dialogue from GTK for instance so from a user perspective there is no difference because it's the same graphical toolkit displaying the same window and So the user picks whatever files I want to open and then the helper sends back to the To the application Not a file path, but an already opened file descriptor. And so that's how you can basically move more or less Handles for a file through the sandbox without having to grant excessive so that's how it works for file mediation But there are portals for things like webcam webcam access and so on and so forth. I See that we have a bunch of people sitting in the back now If you are sitting in the back because you do not want to be on cameras, that's entirely fair and I'm very glad You feel comfortable coming in anyway If you want to if you are sitting in the back just because you arrived late I would invite you to actually come sit with us. So Yeah, and also at any time if you want to talk about something just ask for a mic Sorry, I'm no I entirely forgot what I was saying just before that Oh, yeah Yeah, we're saying so that's how file mediation works with portals But there are different portals for things like accessing the webcam and so on and those also give a fairly familiar User experience in the sense that it will do the same thing for instance that your web browser does which is Display you a tiny pop-up saying okay Application who wants to access your microphone and webcam and say are you just get a choice to just say no I do not want evens to access my webcam and yes, there Especially for sandboxing desktop application quite useful because It's the can achieve things security wise that would be extremely hard to do with a palm or because basically because Because there is no fixed set of file you want to be reading of writing to is your application Maybe this is a good point to introduce my idea for Converting Devin into flatbacks So I'm not sure how many people know how Ostr and flatbacks work but Ostr is kind of a git based file store for systems So you install the root of s into into a this git like directory structure and Flatpacks are built on that So there are multiple Parts of flat pack. There's the runtimes and then there's the applications And they're both Ostr both stored in you can store everything in one particular Ostr repository And so my thoughts were that because of a bi changes and stuff like this that happened between stable and testing and Packages get removed. And so if you keep the old versions installed, you can't upgrade your system anymore So those those are the two motivations for me for this idea I was basically to create some Ostr base systems like for each of the Debootstrap targets minbase build D And I think there's a couple of others and then on top of that build one Flatpack runtime for package and then build a dummy app that it's stored that depends on that runtime and has a few things over the top so that it works Without having to rebuild every single package with slash app instead of slash user so that's the difference between the Flatpack apps is they're installed in slash app instead of slash user And so By doing this fake dummy thing then we can get around that and build without having to build every package over again Which we already do on the buildings so yeah and Because it's because everything's OS tree it means that we would need we would Do well to also have OS tree images for the base system and pay me a few of their task cell Standard install options That's the general idea so Quick question if I understand correctly you mean you would want to Act when installing packages to install them in an in OS tree No, this would be pre-built OS tree slash flat pack repository, okay, so you could just do Go to the name software and install flat packs from devian Still the flat packs from flat hub or wherever else okay But then I I guess I'm missing the part where you go from having Debian packages to having flat OS tree packages basically So the way it works is basically you the bootstrap Import that into a flip flat pack. I mean into an OS tree repository and then as a second layer you do the run times and Import that into the OS tree repository and then for each app package you Upgrade I mean install the dependencies and the applications into those repositories and then import that into flap into OS tree so the advantage of using OS trees that The files aren't duplicated even if you have the same libraries in every flat pack because they're They're it's a content based store. So it's based on the shower one chart to 56 of the file Besides the motivation of easy back ports and installing obsolete remove packages This would also enable the sandboxing stuff that You could use Debian packages with Debian security updates Isolated in a flat pack environment using the portals and all that sort of stuff Yeah, that sounds kind of interesting because one of The problems that I see with the whole kind of we're going to ship applications in their little the little carry bubble of containerized stuff is that You just will to some extent we're in danger of Pushing the how do we do security updates problem to over there and people say like well distributions are hard to do Well, yes, they are hard to do The reason we put a lot of effort into them is that people don't have to do them over and over again and I like I like a lot of the things that the containerized stuff is doing but They say no, it's okay. Well, we'll ship the runtime. It's like well That kind of mutates into a distribution then at some point, you know You're just doing the work and now you're doing it for five different versions of the runtime instead of maybe two versions two releases of the distribution So that sounds like it might be an approach to That could address that sort of bridge the gap and hybridize the two approaches The biggest problem with this idea, I think is probably scalability I don't know how well post tree would scale with like thousands and thousands and thousands of apps I guess we you could reduce the set by just creating apps for each Package that contains an upstream file It should be easy to do I just wanted to ask Julian. I Know that the package now has a this dash dash root option. Do you know if that also supports that now? Well, I don't think we have added support for that But I guess you could probably work around it Now several options is at a root DFO app and you could pass the root option to deep package to By the apt option to pass options to the package So it might work, but I'm not sure how to configure it Yeah, since you mentioned the scalability issue Making a single OS tree repository with many many many packages in it Would it make sense to instead Have like regular that packages and on my machine the first time I say, please install Libre office in in a sandbox Basically build a local OS tree repository Put that work out. I think that could work, but it does mean that potentially thousands of client Machines are doing that instead of the work being done centrally in one place once So there's trade-offs So with regard to scalability my apologies if anyone from endless is already here But I think endless already use OS tree to deliver their operating system and it is Debian derived. So I Think it's Debian derived. I'm pretty sure it is. So I believe it does actually scale reasonably well to large numbers of packages No firsthand experience of that, but that's What I've heard talking to people from endless. Yeah, we'll definitely check it out then But if they have like the will Debian archive in OS trees and it's definitely will work for us the other thing about scalability and this idea is that Because I want to have obsolete things that have been removed from Debian that people might still rely on That don't necessarily need Security updates because maybe they're a game and they don't have don't use the internet. So it could end up being that there are a lot more packages In there from the history of from snapshot Debian org So, yeah, that could and because everything well the tool chain and library versions change a lot There will be a lot more files in there Well, I will really perhaps question should we move on to another topic? Okay, could someone remind me what was also two things we Wanted to discuss those How to reduce setup times for those Sandboxes and They forgot Entirely what was the other two? Oh Right As build as in How to use those environments for building packages or Okay, so there was a bit of earlier discussion about as built it was mostly as Like as built and I see it should assess Inspiration and possibly as like the tool we can extend to do what we need I Not depending on what we end up doing as there was not a lot of Discussion, I think about as build Specifically, but basically if you If you can if you give if you have the ability to set up an environment with And installed xx y packages and so on as well can definitely use that as a back end I don't know if that answers but what you had in mind But yeah, it's very much something yet Have an eye on because you know if we want to be able to for instance If you are a Debian mentor and routinely get packages from people who are maturing and built whose It's yeah, you probably might not want To have your old system system exposed to whatever is in Debian rules and what's not Should we talk about stuff like a pip install sent boxing like yes, please language package managers When you build the applications you download stuff from the internet and execute set up dot pi scripts and stuff Yes, please that's very relevant my to say oh, well, I wouldn't go as far as to says I have a plan because Having a plan is like far more organized than any of that stuff is at least as far as I've been working on it, but Basically is a kind of idea I had is if we can provide people with sandboxes You could I could do stuff like whenever I see the into double python my shell just dumps me instead in In my SCH root of whatever in my sandbox for Python stuff and they I have my own Python They I have people installed But if I people install whatever it can only touch my Python development environment for instance But I would love to hear if you have like more sorts about that things because I So fight was at least in my mind just Another application to confine and not anything really special. I don't know you might want to have an application in Python that you want to install with pip and Then you want the application to access your home directory later on But you don't want the build process to be able to access your home directory So you could like build a special sandbox that contains just your pip modules you have installed locally in your home directory and then find all that into the Sandbox and build the module in there and then just Exit the sandbox and run the application later on in the host system or sandbox it to somehow Okay, that's Very interesting and I didn't give any thought to that and is anybody else wants to react or Okay, I guess we're still like all kind of digesting your ideas Guess what I would be Kind of concerned about is It can give people perhaps a fake sense of security in the sense. I Don't know if all of you heard so I will guess summarized there was recently a fairly high-profile attack on npm, which is the JavaScript package manager and So basically what happened is that one developer got their account compromised and they had access to a very popular lintar and What's it? What's the attackers and did is basically publish a new version of the lintar that would automatically grab credentials for access for uploading to npm and sent to us over to base clear servers a controlled and I mean that sort of stuff could definitely happen like in other package manager like Cargo or paper, whatever it could even potentially happen in Debian like one Debian developer gets Compromised by the produce sent for people for lintian and since we all run lintian hopefully Then we are all compromised and Yes, that's a very big part of like why I organize this both and why I Not particularly this attack because it happened like really recently, but that sort of concern is more or less what motivates I think at least my need for Isolated development environments and applications and so yeah, if we are saying we should Build people applications for instance in in a sandbox, but makes them available like Unconfined I would be kind of concerned about the same thing happening and Just having a bad application running and confined Directly Yeah, that's true. Maybe you want to have them differently confined for the build and the runtime Okay, is that a Very a pretty good idea because there is no reason why PIP should have access to my webcam, but perhaps if I'm installing whatever Streaming application I might actually want the application to have access to my webcam and so on I Mean if we support thing up as things like as builds Should be possible to also support like yeah doing a PIP install whatever it's not just malicious things or other language package managers, so you know if Proprietary software developers provided dot-deb, you know quite often and I think mostly for reasons of inexperience They do some hilariously inadvisable things in their post-inscripts. Yes, so I usually Pull out the post-inscript and just have a look at it And quite often it's like you know what? No, you don't actually need to run for me to install this application I'll repack the deb without it but It would be nice to isolate so I've seen ones that Try to They do a few things the one is it removes any cached config for that application which was not Completely unreasonable, but from your own directory, which is like I don't think that should really be happening in the post-inscript The application should be taking care of it itself when it when it starts And it also tried to kill Anything with its own running name As that's Fully exciting. Yes. Yes. That was the reason that didn't get around. So that's what I meant with untrusted depths So I basically made great declarative maintainer scripts Would be the first step and then you could have the Dapps that you can safely install Without them doing anything evil to your system and then if you combine that with sandboxing for the actual app itself You could have a safe app basically you can install random third-party packages and use them with the same safety guarantees as you have We've containerized Package formats like snaps and flat packs Yes, that sounds That's not actually like something we very much would like to have Yeah, that's my vision like for The last two depth console sir. Oh We need to talk more than I think we're out of time. So already. Oh, yes We have even like five minutes over or something. So, yeah, I Well, thank you all very much and We'll probably try to Go through the recording at some point and kind of summarize the discussion Frighter and send the summary to depth conf discuss So if all of you are already in there will send it there if for some reason You are not on depth conf discuss. That's entirely fine. And if you want me to send you the summary anyway, please like let me know and I will just Be CCU to that email. So nobody has to know you are not on the conf discuss And Yes, thanks a lot again, I feel it was a fairly productive discussion and we have a new stuff to work on which Well, you know new things to work on a not it doesn't always feel positive in the sense that also to do looks got bigger But at least we know what we have to work on Makes that sort of stuff happen. I guess at least which directions we need to work So thanks again everyone