 Hello, everyone, we will start the next session. Please, if you have questions, leave it for after the talk. There will be time for questions. And floor is yours. OK. So morning, everyone. I'm Sandil Kumaran. I work for LENARO. And I'm also a Debian maintainer. So this talk is about LXC with Debian. But it's a generic thing where I want to introduce a specific use case in order to test Android devices. So to give a small introduction about me, I work as a Lava software engineer for the Lava project, which is nothing but a project which is used for automation within LENARO and elsewhere. So it's called LENARO Automated Validation Architecture. And you can see a lot of packages with respect to Lava, which is available in Debian. So installing Lava is just apt-get install Lava in Debian. So I'm one of the maintainers of Lava Star packages within Debian. And apart from that, I maintain some couple of Django-related packages, specifically Django Compat and Django Hijack. And I used to work for Subversion Project. And I'm one of the full committers of Apache Software Foundation contributing to Subversion Project. So with that, let's move on to the agenda of this talk. So basically, we'll see an introduction on the next containers. So we had a talk on Tuesday with respect to LXT, like Stephanie Grabber was delivering that. So this is the basics of something before LXT. So before LXT, we used to have LXT, which LXT tried to improve upon the user space tools which LXT was providing. So we'll have an introduction. And then we'll see the available packages in Debian and also a short demo on the installation, configuration, and basic usage of LXT. And then we'll go with, I have a Google Pixel phone over here. So I'll use this as a device in order to connect to see how we can attach this device within an LXT container in order to do things like fast-boot, communicate with fast-boot and ADB. So to define LXT containers, so it provides an isolated environment with all the processes combined into specific namespace to provide an operating system within the host machine. So you can have n number of containers within the host machine. And each one will be providing you an isolated space. So that is one stuff. All these containers will be limited to the same kernel which the host machine has been used. So that's to give a overview on what a container is. So it's basically an isolated operating system environment. We call this a system containers unlike the other container technology which exists like Docker, which are application containers. So a system container tries to contain the entire operating system within itself and provides an isolated space. So there are some differences from traditional virtualization. The containers are lightweight in the sense they don't have things like emulators in between in order to start up and there is no translation or interpretation which is happening in between the container and the underlying host machine. And there are other things like containers start quickly and also sometimes when you're moving a container from one host machine to another and if you're having the same kind of kernel with both these host machine, it may prove to be portable. So these are some of the advantages of using containers. And I mean, LXC is something which provides the user space tools in order to exploit some of the kernel features in order to provide a container. Some of the kernel features which enable containers are we have the namespaces, C groups, and all the mount file systems and everything. So all these things combine together. We have this LXC user space which provides certain commands in order to set up processes in isolation so that you get an operating system environment within the host environment. So some of the packages with respect to LXC which are available as part of Debian stretch release, I mean, there is a slight difference from JC to stretch. There are some packages which have been removed from JC, but these are the basic packages which are available currently in stretch. So you have the LXC which is, I mean, that's the main package which provides the LXC environment or LXC commands. So it's nothing but the, as I said before, it's the user space tools. And LibLXC is the one which enables us to use LXC. And we have different bindings like the Python LXC for the Python language and Lua LXC for the Lua bindings. And LXC-CTL is something where it again provides a command line tool in order to manage LXC containers. There are other packages like Vagrant LXC where you can create containers and manage containers with the help of Vagrant. So Vagrant LXC provides a plugin or a way in order to manage LXC containers with Vagrant environment. So these are the packages available. And then let's move on to the installation configuration. It is very simple as in any Debian package. So it's simple to install LXC. It's an apt install LXC or apt-get install or aptitude install LXC. So I'll just show you a short demo. So it's visible? There we go. OK. So for the purpose of this demo, what I'm going to do is like I'm going to run the installation of LXC within an LXC environment so that I have a clean mission where I can install packages. So it's basically a nested container. But this is for the demo purpose. Instead of doing it on my host mission because my host mission already has LXC installed. So think this as my host mission. So this is my host mission. Let me first update. Before trying to install my LXC, I want to get the latest which is available. It should be pretty new, but just in case I'm not getting the latest LXC packages. So as I said before, installing LXC is apt install LXC. So it installs a lot of packages. OK. So now we have LXC installed. So let me show you some basic config files which are required for LXC. So we have all the configuration. This is the default configuration for whatever container which you create with the help of LXC. So it applies to all the containers which gets created henceforth. So with respect to Debian, if you see there is a network type of empty. So we don't have any network which is pre-configured for us. This is not the same case when we are trying to install LXC or LXC containers with Ubuntu or any other distro, they have their own. For example, in the case of Ubuntu, you have app armor profiles. And also there is a basic networking which is set up using LibWord. So in case of Debian, Debian provides a pristine version of things. So everything should be configured by us, specifically in case of networking. Because networking becomes important in the case of using an LXC container effectively. Whether you want to install packages or you want to work with the container or you want to communicate with the external world, so we require networking. So there are different methods of installing network with respect to like, so one of the recommended or the easiest way to install networking for LXC containers is to use the default network which is provided by LibWordD. So I'll show you a sample configuration of how it may look like. So if you see here, this virtual bridge has been created by LibWord. So once you install LibWord, we can create a virtual bridge. It's one of the simplest things which we can do as far as configuring network. There are other configurations depending upon how comfortable you are with. In order to create bridges and play around with this, you can use whatever way you're comfortable with. So having said that, let's try to create a LXC container. Some of the default parts where LXC containers gets created are. So whenever we initialize a container, so everything goes inside varlib, like LXC. So the container root fs gets created inside this folder. And there is another folder called cache. So here, what happens is when you initialize a container for the first time, I'll just show you how to create a container, how to use the container, and then destroy it in some time. So before that, I just want to show you what are the different parts involved. The first thing is the configuration which we saw about, which is an ETC LXC default.conf that is applying for all the containers. The second thing is the cache. So whenever we try to install a specific operating system, so it gets cached inside this location. So for example, if you want to install Debian Stretch, the root fs gets cached here. And then if you want to create the second container, instead of fetching everything from the internet once again, it consults the cache first. If the cache is up to date, the root fs gets initialized directly from there for the new container. So let me show you. So let me create a container. So I'm trying to create a container named Stretch-demo. And I'll be using Stretch Release, and the template which I'll be using is Debian. So this is how I create a container. So before that, let's see what are the different templates are all about. So there are two types of templates. So one is a download template, and the other one is always specific template. So always specific templates are something like you have different templates. These are nothing but the always specific templates are like pure shell scripts, which is provided by the LXC package itself. So what happens is when you want to initialize, say, for example, a Fedoro-based container, so the specific OS-based template could be used. In this case, if you see, the download template is quite different from the always specific template, where the download template tries to fetch images from a server called images.linuxcontainers.org, or any other specific server depending upon how you have configured things. So this always specific template provides, I mean, bootstraps the operating system or the root FS within the same machine. For example, if we use Debian template in case of, I mean, in order to create a container, what Debian template does is it uses a Debian bootstrap in order to create the root FS on your same machine and then creates the container from that root FS. So there are some basic things. The first thing is you need to create a container, and once the container is created, you need to start the container, and then you can do whatever you want, play around within the container, or you can set that container up with different configurations which you want in order to run different kinds of software. And then that is, I mean, means to stop the container and then finally, if you don't want the container anymore, you can destroy it. So this is how the flow looks like. So I mean, I have some lines just cut off here. So here we have the destroy step. So if you see the container gets created and it'll be in stop state, once we issue like start, the container will be in starting state and then it can go to like running. Running is a state where the container runs and then you can use it as a separate operating system from different from your host machine. And then it can go to stopping and you can freeze or unfreeze the container. So these kind of freeze and unfreeze, I mean, states are being used within LXD in order to like do, I mean, snapshotting and different kind of, I mean, advanced stuff. So as far as LXD is concerned, like these are the basic states in which a container exists. So, I mean, I'll show you like, before getting into that, I'll show you like how to create a container. So this is a command LXD create and I'm using the Debian template and then like I'm giving the container name as stretch demo and the release I'm using is stretch. Since like I've already like tried installing this in this specific container, I mean, creating this container. I mean, already the root of this has been cached. So it should be quick enough. So the container got created. So let's try to like start the container which we created just now called stretch demo. So a container gets, I mean, it'll be started in the background as a demon. So if you want to like check what is the status of my container. So I can see with LXCLS command. So the name of my container stretch demo and state it is in this running. And I mean, I haven't like configured any kind of networking within this nested container. So like there is no IPv4 or IPv6 address that has been fetched yet. So in order to like the simplest way to like login or attach to a container is to like use the LXC attach command. So you can see like I'm right now inside the container which I've just created. So if you see like this is a stretch container. So, and then this is a kernel that has been used for this container. So let's like create another container with the help of, I mean, another container with the Ubuntu template. I'm trying to create a Zesty container based upon the Ubuntu template. Again, the root of this for this is also cached. I just want to like because it take quite a few minutes in order to like get the container bootstrapped. So I just cached it so that we can use it for the demo. So now our Ubuntu container is created. So let me try to start this container. So the start commands take the name takes the name of the container. Before that, I'll show you like whether what state this container is in. So a stretch demo container is running and the Ubuntu container which we just created is in a stop state. So I am starting this container. So if you see like Ubuntu container is running and let me attach to this container. So if you see like I have a Zesty container which is running. Okay, so again, stopping containers is simple and trivial. So in order to stop a container you issue Alexi stop. So if you see our Ubuntu containers again went to stop state and then like in order to like get rid of the container entirely from the host mission we can make use of Alexi destroy. So it'll completely destroy the container and it's an irreversible action. Okay, so let me give some background before I get into like how to like interact with Android device. For example, this phone which I have here. So we work for a project called linear automated validation architecture. So it looks the project interface looks like this. So if you see here like I mean this project is basically used in order to like do testing on different kind of devices and like validate, I mean the primary purpose of this is to automate tests and deploy tests on different boards and collect results. So we support like different arm based devices and also like x86 is possible. So we have like devices like the high key dragon board and all the 96 boards and QB trucks and different boats like this. So what happens is there is a dispatcher to which these boats are connected and that is a master server. So once you submit a test job, what happens is the master server takes this test job and then like gives it to a dispatcher. A dispatcher is nothing but a server to which the devices are connected to it. So once a dispatcher receives this job what happens is it tries to like run different tests on the device under test and all these things has to be automated without any manual intervention. So previously what we used to do is we used to like run tests directly by communicating with the dispatcher when it I mean to the device under test. So when it comes to Android devices we use basically use two like methods in order to communicate. One is with the help of ADB and the Android debug bridge and the other one is by using fast boot. So in order to like flash images to the Android devices we use fast boot and in order to like once the Android image has been booted on the device we use ADB in order to communicate with the device. So different tests which we run require different versions of software. For example, certain like there is a test suite called CTS, the compatibility test suite which has been provided by Android. So this test suite requires different versions of Java in order to like run on in order to test different versions of Android builds. So in order to like have different versions of Java I mean we had to like put those versions of Java in a lower dispatcher which is a production server which is running in order to like communicate with the devices. So this posed an important problem where we don't want to like maintain a lot of like soft different versions of software inside the server mission. So that becomes unmanageable at a certain point of time. So in order to like overcome that we try to like introduce the Linux containers. So what happens is there is this dispatcher and then there is a intermediate like transparent device called the Alexi container and then from the Alexi container we like communicate with the devices. So this is how the like Lava setup works with respect to like currently it works with respect to like testing like different devices with the in an automated way. So having said that like we have one Alexi I mean Alexi container will be created on the fly in order to like attach to device under test and then like we'll fire a test on based on that. So there is one important problem. Say for example, if a device gets attached to the host mission we need to like give access to this device to the container. So there are different, I mean says for example I'll show you like so this is the dmessage output on my host mission which is this laptop. So I have this pixel phone. So basically I've put it into fast boot mode so that like I can like attach it to the host mission or any container and then like I can like run fast boot commands on that. So let me like attach this phone via the USB port to the host mission. Okay, so you can see this the phone this is serial number of the phone which got attached right now. So if I try to like access this phone within my container so I'm trying to like create a container called a stretch demo and then on my like host mission and then I'm starting the container. Let me attach to it. So this my container and my host mission has the phone directly connected to it. So right now I want to like access this phone via fast boot within my container. So let me like install fast boot before that and also ADB. So these are like tools used in order to like communicate with Android devices. Okay, so right now we have fast boot installed. I'm trying to like access this phone from within the container. So it won't appear over there because like I need to like add this device to the container explicitly. Now, when trying to like do this within the Lava project, we were trying to like automate this entire process where a container gets created and there is a device attached to the host mission. So we want to like give access to this device, to the I mean to the device from within the container. So in order to like run the test. So in this case, like we like follow different methodologies in order to like identify when a device gets attached or it gets reanimated and things like that. So we've sorted to one like method right now in order like automatically like identify things, I mean when a device gets attached and gets detached from the host mission. Basically we are UW. So we've written, we have a UW rule which is similar to this. So what it does is like it sees add action within the UW domain. And if there is an add action with the serial number of the device which is FA6B whatever. So that is the serial number of the phone which I have. And then like it creates a sim link called stretch-demo, just a sim link. And then like it runs a script called Lava LXC device add. So this script could be named anything. So what this script contains is so it does a LXC device command and then like it gets the name of the container and then it adds the sim link of the device to the container. So once this rule, so what I'm, what I'll do is like I'll copy this rule to the UW rules.d directory. Okay, so now if you see like I have this rule as part of the UW rules directory. So once this is there, I'll reload the UW rules. So once the rules are reloaded, so what happens is let's watch the, so as you can see like my container stretch demo didn't show any fast forward devices right now. So what I'm trying to do is I'm just trying to like unplug the phone here. So you can see device number 22 has been disconnected and then like I replug it just to like trigger and add action in the UW as per the rule which we have written. So you can see like a new device has appeared. So that is a phone which got re-added. So now if I like try to do a fast forward devices, it should show up, but it doesn't. Okay, so let me like try to like run the command directly in order to attach this device to the container which I have. So basically like the command in order to do that is we do a LXC device and we give the container name and then like we say add. So this is basically in order to like convert the sim link into the proper hard link of the device. So maybe it is somewhere here. Okay, so I've added the device stretch to the stretch demo container which is nothing but here. So we do have the fast forward device over here. So let me like try to like start this device from fast boot so that it gets into like the Android operating system. And I'll just show you a quick way in order to communicate with the help of ADB. Okay, now the system is booted. So I have Android running on this like a phone. Let me try to access the device using ADB. Okay, so the device did appear with ADB. The only thing is like of course on the phone it is asking for the authorization in order to allow this LXC container. So I'm just authorizing that and giving it okay. So when we have a AOSP like build where we have like debug enabled by default, we don't want to like do this authorization manually. So that'll be helpful for authorization. So if you see like the ADB device is available over here and then I'm trying to like reboot to fast boot again. So you can see the phone rebooted into fast boot with the ADB command which I've provided. Let me try to like see, okay. So the device did appear in fast boot. So basically like I haven't done any LXC add, I mean LXC device add command for each of these things because the UW rule which I've written here, basically this rule is taking care of like the addition and like removal of the devices automatically. So this will be pretty useful when you want to like do Android testing from within a container and specifically when the Android testing requires different kinds of Java versions or different kind of tools that needs to be installed in your host mission. And as you can see, if it is a direct host mission, like UW takes care of things in order to like see when a device gets added and when a device gets deleted or removed. But in the case of container, you need to do that explicitly. If we have a UW rule in such a way so that it'll track when the device gets added or removed, it automatically like does that for the specific container which we have specified. We have a small script which we have over there. So this helps in order to like I mean, automate things, testing with respect to like Android devices. So this is a similar kind of setup which I was explaining. So there's a host mission and we have the LXC container and we communicated with fast boot and ADB to a Google Pixel devices which is connected from the host missions USB port. So that was about the demo. And that's what I wanted to like deliver as part of this talk. Thank you for watching. So if there are any questions I can take for we have another four minutes. You showed that you stopped the container with LXC stop, but that's like turning off the power of the machine, right? Yes. Is there some command to give it a soft reboot or something regular shutdown from the outside? I mean, that is not supported. We don't have any reboot or LXC reboot some kind of commands. The only way to do, I mean, reset or reboot is to stop and then do a start. So stop internally what it does is it does a kill of all the jobs which is running or at that point of time in the container. So there is no clean way of doing a LXC shutdown. But it's probably still more cleaner than shutting up, shutting off the power of a regular PC because it still can write the cached files and things like that. But the user space tools doesn't provide that. Maybe that could be an addition. And of course, I'm not one of the LXC developers, but I'm just a user of LXC. Maybe this could be a good feature to be added to LXC, use the space tools. In this case, you were showing us system containers, right? If I kill of every process in a system container, is it the same than stopping it? If I kill each process, system container, is it considered stopped? Yes, it is equivalent to a shop. So that, I mean, kill has to happen from the host mission. Right, yeah. I'm not sure whether the LXC LS status would show stopped if you manually kill all the processes inside the container. So you'll have the same sort of effect, but actually you should still use LXC stop to make sure that your own status of the container is right. You may find it problematic to destroy the container if you've manually killed all the processes and haven't actually run LXC stop. Just so that LXC has got its own status in order. More questions? And I'm wondering in answer to your question, the start and stop, it is approximately enough to do like a soft reboot of the container. Yeah, because it also start and stopping, it also takes away the devices that you've added, you will have to add them again afterwards. So it does do a clean sort of refresh of the container. And once stop is returned, there's nothing happening inside that container. So it is safe to destroy that and any fastest caches and things will have been flushed at that point by LXC. Hello, this is a new big question. Why do I want or was the use case of running Android phone attached to a container? So as I said, we used to do this from the dispatcher machine, which is nothing but a protection server. So if you want to have different versions of Java, in order to run different Android tests on the device, so we need to have those versions of Java installed in say, for example, Lava dispatcher server. So it becomes difficult to manage that. So we want to have a contained environment where it's nothing but a sandbox where we can put a lot of software and then scratch it off once the job is over or once the testing is over. So I think we are running out of time. So thanks for watching, thanks for listening to me. Okay, let's thank again our speaker and let's go grab a coffee or something.