 All right, thank you everybody for coming welcome Welcome to your after lunch talk. Hopefully you can stay awake and we can as well We'll try to help a little bit with that My name is Kevin Hillman and this is Patrick Titiano and out from Bay Libra We're going to be presenting the next talk called lab in a box introducing lab in a box so Little bit about us. We're gonna have embedded Linux consulting company Doing all sorts of things around embedded Linux. You can read kind of this as you like But actually I want to tell a small a funny story before I start because last time we did a talk like this where they cut the Ballroom and they split the ballroom in half I was just about to start talking and I opened my mouth and then the person from the next room Over start talking and they had missed they had mis-wired the sound So I opened my mouth and somebody else's voice started actually coming when I was talking It was really weird and it was like coming from above it was kind of it was pretty it was pretty funny But that didn't happen this time. So thanks to our audio team. They're they're on top of it. You sound okay today Yeah, it's on different Oops, I don't know what that was Anyway, this is a this is a little preview. So this is the lava box So if you if you really are more interested in somebody else's talk than our talk Basically what lava box is I'll tell you in one minute and you can go to the other talks if you want This is a this is a PC case with a bunch of embedded Linux boards jammed inside running lava and set up for automated testing So that's what the lava box the lab in a box is we're gonna go through all the details, but this is kind of the the preview and Tomorrow night that the technical showcase We'll have the lava box there if you want to come and see it you can see inside and and have a look for yourself as well So a little bit of background I Want to tell you a little bit about the colonel ci.org project because that's kind of the the framework or the environment that we've been working in And developing and contributing that made us kind of come to a point of wanting to develop this this lava box So if you haven't heard a colonel ci.org it's kind of a it's a distributed test framework kind of built towards built Focused on testing the upstream Linux kernel So we are testing the Linux kernel the latest Linux kernel from mainline from Linux next from all the stable trees and everything We're testing it on over 250 different boards So every day anytime there's a new commit and any of these trees mainline next stable a handful of developer trees It gets built and it gets boot tested on a whole bunch of platforms Mostly arm and arm 64, but also x86 We're starting to get some interest in MIPS and so we have some MIPS platforms that are being added right now Also risk 5 is under is in the conversation as well But that's that's basically what colonel ci.org is so it's its main thing is it's distributed So there are labs all over the world There are about 10 different labs that have a handful of boards each that kind of contribute to the colonel ci.org And the results are reported on the website There's also a mailing list where you can sign up for this so that kind of target audience has been colonel developers So they want to see if you make a change to some subsystem And you don't you can't test it on obviously all the hardware that Linux runs on it's a kind of a useful tool for Especially for maintainers and colonel developers to find out if the stuff that they just recently submitted actually is still working on the mainline On the Linux kernel has a test on a bunch of different boards So that's a little just a little background on colonel ci.org If you want to know more about this project the previous ELC's we've done talks on colonel ci.org project as a whole So there's stuff in the ELC archives on on this as well Just a little bit about the colonel ci loop so just standard ci loop There's there's you know, you check something in to get it gets built gets distributed to all the test equipment And then it goes back into the So this is kind of the colonel see that colonel ci.org piece here It's kind of contributed is the web UI and kind of the back end where all the results are stored in the database And what's interesting in this context the lab in a box contest is so we're talking me We're going to be talking mainly about this piece So how do you actually how the test get actually once the once the colonel gets built and is ready to run? How does it get distributed to the hardware that it runs on? So these these this box here is basically one example of a lab a board farm And we've done it kind of a simple board find crammed into a PC case And that'll be in just an example lab that can contribute to colonel ci.org so on a related project the Linux Foundation automotive grade Linux project is The ci at part this is continuous integration automated testing expert group inside of a GL is Also leveraging lava and colonel ci infrastructure as well So we've been working with a GL actually to to leverage colonel ci and expand the testing for a GL to a wider variety of hardware and especially more tests So it's it's also using this colonel ci infrastructure and it has a handful of boards that need to be tested and also be Distributed to different labs so different member companies from a GL for example could have lat the boards that they care about and Collect them all together and have a distributed test infrastructure so all that's just kind of background to Get to what this lab in a box is and that's kind of motivation that brought us to actually developing it So Patrick will tell you a little bit more about the lab in a box itself Thank You Kevin. Yeah, so in this context Let me try to explain you and develop what the lab in the box concept is and first I start with the motivations Why did we want to Design this lab in the box what it is? What's the starting point? So first or motivations Well, the number one motivation was this one. This is currently what a lab looks like in most of the cases Well, I think it's time to go pro. We have done enough of the Experimental phase and now it has to go pro and be a production grade thing This this one is my office by the way. Yeah, I'm not I'm not trying to shame anybody but myself exactly exactly And this is the on the left. This is the one we have in Belivere, France. So So that's the first motivation, but that's not the only one The other ones is that we really want to simplify the administration of the lava labs We have around the world Lava is clearly today the leading technology. It's a nice technology It's still evolving technology, but it is still difficult to get into it The installation process is known to be painful. It's getting better particularly thanks to the docker use the use of docker technology But it's still complicated We have to deal with we have to deal with many different configuration files like device types devices This is pretty difficult to set up particularly for new boards We also need to deal with the the serial console You may know that when you connect your serial USB cable, you need to look for the right TTY device It's not something that is automated and well for newcomers. It's always a pain All this to say that we believe that at some point Both farms users should not Actually be aware of all these internal technologies at the end we'd like someone to build to be able to build CI lab without having to deal with all that. It was just build and run and that's it So that's really what we want to achieve here Other motivation is be allow people to easily Deplicate or scale their labs in the first picture we show Clearly it will break at some point. It's okay if we have 10 20 30 boards eventually but Just think about hundred board. That's not possible. Just think also about corporations You are not gonna sell that solution to anyone So that's that's really one motivation here and in the end We can say that we want to accelerate The deployment of such labs because the more labs we have the more testing we can do the more branches We can test and so the The higher the quality of the Linux kernel care it can be based on those Motivations we We have listed some requirements Number one requirement being that we want to do an all-in-one solution Integrating everything so the lava instances the master the Dispatcher all the devices under tests so the boards All all the power supply units all the connectivity the wiring everything it has to be an all-in-one solution Because this is developed in the context of the agl project We will integrate also we want to integrate all the community and with reference boards for a year as usual The lower the cost is the better it is so we would like to keep the cost down We want something scalable and reproducible But also safe robust and maintainable Just imagine having those wires on the table Anything can happen. You can just stop by and remove a cable. You have no idea which device it was You can well, you know a cup of coffee or a Glass of water can fall into it and bam. It's gone And when you want to add or remove a board, it's always taking a time. It's always taking some time So we want to get rid of all that We are we also thought of all the home workers So again, no one we would like to have this in his apartment, but a PC case Well, you have your PC already, so it's just another one And last but not least it has to be well documented Which we know is a difficult point for open source projects Well, I should have actually we should have listed one one more requirement is that everything is open source And it is Okay, so for so there there will be challenges we knew there there will be challenges for Driving this concept and here are the two main ones We know that there will be a lot. There is a lot of stuff to integrate All together all the boards which has died or they all have different dimensions There's no standards into both sides. Eventually you have the business card size and couple of standard size But the the wiring the way the place the placement of all the connectors. It's all custom We need something But we call a power control unit. So something that will be able to power on power of boards dynamically All the connections so the wires we need a local networking we need some we need to connect all the USB the debug consoles and So for each boards that we will have to be under test into or Lava box concept we will need to have a cable for power for the debug console and for the Ethernet So ten boards 30 cables you can imagine The other challenges will be Given that there's a lot of integrate it it still has to be maintainable so to be easy for maintainers to add or remove boards if it's needed and Here it is. This is our lava box So everything packed into a PC case So there's been you can see that compared to the first pictures of the board farms we've shown Well, there's been some road taken But let's let's have a look inside So here you can see all the different pieces we have integrated into this PC case And I'm gonna try I will describe all the pieces all the parts But also explain why we have made this choice. So why a PC case? well The PCs you can have a PC for quite cheap price today and You already have everything you need for to run the Lava instances you have to know that running lava today is It's quite difficult on Regular arm-based development boards because there is some requirements on on processing and Also on storage and also on Run We have to download large tar files With all the the kernel and the root FS it has to be Decompressed so that's that's requiring some processing and some storage Also all the time so we cannot use something that is based on flash memory because it's going to be burnt into Burnt very quickly. Sorry. So that means more SSD kind of thing or hard drive Lava is based on Python Java and it runs web web server and everything so Ram usage is quite high and so in the end we decided to use a regular PC But it's an only only all in one sorry mini itx board with a fan less quad core CPU So it's fast. It has eight gigs of RAM and SSD drive for fast accesses The second interesting point of using a PC case is because there is a very powerful and cheap power supply unit so for like a few Tens of dollars you can get a 500 watt power supply Regulated sterilized protected you have 5 volt and 12 volts Outputs which are the standard voltage supplies for all development boards So and it has many connectors. So we could reuse that for to power all the boards Or I mentioned that we needed something you need to control the power to the boards and here we have decided to reuse or own Power measurement and power cycling solution. It's called ACME So it's based on a cape for the big open black and with power measurements and power switching probes So that's what we are using here to control the there is six boards integrated in this example There is obviously a USB hub to collect all the debug consoles There's a network switch To give the connectivity local connectivity to all the device on those tests So this generates a local internal network What boards we have integrated in this example is raspberry pi 3 a good old big old black brand new Love potato board. So this is running am logic s905 X Processor and belibra is actually in charge of doing the upstream support for this board. So that's why it's integrated here We have a dragon board from Qualcomm Based on a snapdragon 410c Since this is used for the AGL project we have the mandatory Orca air car m3 starter kit board from renaissance And then also we have integrated the cyber light imx6 board from nxp So here is a closer loop on the wiring and all the different parts we have integrated into the pc case Okay, so let's talk a little bit more about the internals of the the hardware and the and the software especially heard me Seems to be related to me. It seems to be related to me pushing the button on this then when it So maybe I'll use the maybe I'll use the yeah keyboard here Unless that's actually somebody else's voice that I'm from another Okay, so as Patrick mentioned there's Lots of USB so there's actually USB hub inside the case to connect all this together So primarily the USB is used we're using USB for USB serial converters for all the serial consoles for all the boards So because this is an embedded conference everybody knows about hooking up the serial to their to their board So that's no surprise One thing we did notice over the years of kind of maintaining lots of boards in a few labs is that I've or we've have USB serial dongles from just about every vendor on the planet whatever it happens to be cheap when we're buying and what we've noticed Over the years is the the cheaper ones just after a couple months you find them. They just they just flake out They just stop working you unplug it you physically unplug it you plug it back in boom. It's working again But obviously that's not very that's not useful for automation So what we found over the years is the FTDI ones are just you know They're a way more expensive, but they're also way more reliable and then in general the FTDI ones also have unique Serial numbers on them so like you'd have rules can be nice for unique You know identifying a cable that's put to a board because it's got a unique idea So that just it simplifies things cost a little more, but it greatly simplifies things And yeah, I have no idea why some of those some of those USB dongles just stop working They they're the TTY just disappears from Linux and you unplug and they come back. So I haven't actually debugged that but Anyway, so yeah, so we use FTDI for that So USB the serial consoles are one USB some boards also have another USB just for power So you can power that from the hub or you can So there's yet yet more USB serial cables also some boards that are driven over fast boot You sometimes there's a separate USB connection for the fast boot as well So you have one for the serial console one for the fast boot So this is why you don't just need a you don't need a single USB part per port per board. Sometimes you need to And also some of the boards that they don't have actually physically networking They might have USB gadget ethernet or something So there's another USB cable to do the so that they can do their networking over USB gadget for example So you end up with you know for just a few boards you end up with quite a few USB cables So you want to have a you know the reasonable size hub and a reliable hub as well And Yeah, so that's just all the different types of USB that are involved in this How big is the USB port? I think we only we have a 10 port USB in there right now for six boards Yeah, go ahead. How do you? Yeah, so the question is how do you secure the jumpers on some of these boards like a Raspberry Pi where there might not be good? Connections me you can answer that because you actually put it all together Yeah, we usually try to stick the cable so because yes, we know that the connection is Could be tricky and so we secure the cable itself so that it's tick it sticks everything together and it works we eat it The lava box took the plane a couple of times and it's still working. So that's our test You also have hot glue at the end if you really You eventually Yes Yes, so the question was can you can you buy it pre-assembled? Yes, and so power distribution So like Patrick said we're using just a standard PC power supply the nice thing about these ATX supplies There's five volts and 12 volts and there's also the standby power on the ATX supply That's the five volts so you could use that to run like the Beagle bone that's controlling the acne for example So there are there are ways you could use the the non-switched five volt as well And the other thing we learned the hard way the first build we did of this box in France We bought kind of a lower-end power supply and didn't realize until a couple months ago when we took the box to Japan That it actually didn't do 110 volts. It only did 220 volts For Europe and so we actually had to replace them We had to buy another power supply one that we could actually do 110 volts so that worked because the this lava box was traveling to Japan and then traveling here, so Yeah, if we learned that we didn't even we just kind of assumed that the power supply was going to do both But it turns out the cheaper ones may only mainly do one voltage show So for power switching there's a couple ways to do this the the cheap way there's lots of cheap kind of GPIO controlled relay boards out there, you know four of them or sixteen of them or whatever Those are pretty inexpensive you can drive them right from a Beagle bow and or from a Raspberry Pi or any sort of embedded board That's doing that as GPIO connectors So that's one way that we chose to do because we have this Bay Libre acne board We use that because we have them and they're easy for us to use And so that gives it a little bit more expensive But it also gives you the ability to not only switch power and power cycle But it also gives you to measure power at the at the board level So that's a useful feature and Because there's only we can only fit a finite number of boards in here that the acne actually does eight eight channels And so that's about what you could cram into a PC at this point At least and have it look reasonable so networking that you saw the switch in there so all the boards are doing networking because the The way that lava works typically is the board boots up into the boot loader Whether it's you boot or barebox or something and then it uses TFTP to actually fetch It's kernel and it's device tree and it's RAM disk and so on so all the boards are essentially net booting To get off the ground and so they all need to be connected on the network And we've done this with Docker internal networking So all the boards inside the PC are actually on kind of their own private network And then the lava server kind of exposes itself to the rest of the world But all the boards are kind of on an internal LAN Which is nice because sometimes boards especially running mainline kernels do interesting things to your network And if you just hook it up to your office LAN You might have some surprises when boards come up with hard-coded IP addresses that happen to match your router or something And it's in your it's in your office. You might have some problems So it's a way to just it's a nice way to keep things kind of isolated And the it doesn't need internet access if you want to hook it up to something like kernel CI because kernel CI is doing some centralized build and stuff and then can send jobs You can do push or pull from kernel CI So you need an internet connection if you want to hook it up to You know some external projects, but there's no reason you can't run this on an internal LAN and submit jobs locally as well So on to the software piece. We've been talking about lava a little bit. So I'll give a little bit of background on lava Similarly the ELC is over the years has had several talks on kind of other Aspects of lava and details So I'm going to give kind of a high-level picture of lava and particularly just the pieces that we are So you see how it's used in this project But lava is basically cut into two pieces. It's got a master and slave part So the the slave is also called a dispatcher in the lava documentation So the lava dispatcher is kind of the the piece that actually manages physically all that the physical connections to the boards And it provides all the services that the boards use so like I said when the board boots up it does net booting So it'll typically do a DHCP and to get an IP address and then TFTP It's kernel and device tree and RAM desk and sometimes it'll then mount a file system over NFS So the lava the lava dispatcher is actually what's serving the NFS file system or in some cases NBD file system and sometimes for running tests you the board will boot up And then it wants to go fetch some tests or set fetch some files to run tests And so the dispatcher can also provide a web service so that the boards can actually fetch things or test some web Web tools or whatever so the dispatcher basically provides kind of the interface between the boards and the rest of the world And also a bunch of services that it can use so those are all managed by the lava dispatcher It also manages the power control so when a when the lava gets a job for a given board It knows it just has to power cycle that board to start a booting so the dispatcher manages all that stuff So you have when you define a board you tell it what you know How basically you give it a command line or give it some some tool that it's going to use to actually do the power cycling So in this case it'll be a little command line that tells the ACME board to to switch the switch the power on a particular port or Something like that There's also this project called PDU daemon that's out there. That's that's pretty that's used in a lot of lava labs It's kind of a it's kind of way to abstract different types of PDU so it could do ACME it could do you know Rackmount APC type web web switchers and things like that. It's just an abstraction for PDUs. That's a it's a useful tool It's not a necessary tool, but But anyway, the point is that the it's the lava dispatcher that actually you configure for that for each device basically how to How to do the power cycling? It also knows about the serial consoles and where the you dev rules are for your nice Your nice FTDI cables with unique IDs that all gets configured on the lava dispatcher side And then basically how to connect to the consoles whether it's ser 2 net or some other You need to tell lava how it's going to connect to the serial console physics so they can start sending commands to you Wood and things like that So all that stuff and all the other USB connections fast boot and everything is managed by the the lava dispatcher And yeah, feel free to stop or raise your hand or yell anytime if you have if you have questions along the way No worries So the lava master side or the lava server side is kind of the The basically provides the web interface or provides a way that you interact with lava From the web to control things and stop things it also provides when you're sending lots of jobs to So you have one one particular type of board and you have Want to send a whole bunch of jobs to a beagle bone for example? I mean you only have one beagle bone the lava server is the thing that kind of gets all the jobs and schedules them Based on priority and kind of cues them all up So it has a it has a Priority type of thing where jobs can have different levels of priority and preempt each other and so on It also provides an XML RPC API for other tools non web type tools where you can drive lava From some Python scripts or other the lava project itself provides a few command line tools that actually interact directly with this API So you can do a lot of stuff by driving lava from the command line That's all interacting with the lava server and then it also provides kind of the the way that boards are actually described all the stuff about that I was saying about How boards are configured the serial port the power switches all that stuff that description the description files Actually, even though they're connected to the lava slave the description and configure files actually live on the lava master This is one of the more confusing things about about lava, but even though all the physical connections are on the slave You define the configuration files and store them on the master Anyway, and that these types of files are thing in lava the terminology is device type and device so device type is like a Board like a Beagle bone or a dragon board or something so it defines About a kind of a class of boards. So you might have you might have Ten Beagle bones. They'd all share the same device type, but each one of them has a unique Serial console and a unique way to power site with a cycle it so they're all of the same device type But everyone would have a separate device configuration file where you tell it where the serial console is and where the power is and stuff like that So that's kind of the device type and device is the lava terminology for terminology for making that difference and The important part here is what we've done is kind of taken a lava install and kind of dockerized it So we've made separate docker container for the the master separate docker container for the slave and we kind of put them together Using docker compose so you can't really read this YAML file probably very well But it doesn't really matter the the point is that it's it's kind of using this docker compose tool which allows you to define Basically applications that are made up of multiple containers and how the containers actually talk to each other So that's what we've done. So we've got the slave as a separate container The masters as a separate container and by default they'll run on the same machine But you could also run those you could run them on different machines as well if you wanted to run. Yeah, go ahead Pardon So right now in the in the current software that's available It's still totally plugged into the the lava master But we have a work in progress actually we have a development branch where we you can actually connect it to an either Have the the database itself be a separate container or just connect to some existing database. Yeah Sorry, I didn't repeat the question the question is what about the the post-grade database and you know Where is it and how is it containerized? So it's a yeah right now? It's it's still it's still into two integrated, but it is a it's been a request and it's in progress. Oh the other thing I The down here the the third container here is actually a squid It's a just a web cache because lava's Lava will when you give it a lava job you can tell it to download the kernel from here and download the RAM disk from here And so often it's running the same jobs over and over with the same kernel So we put a squid cache in here as well So if you're actually doing multiple jobs that are downloading the same images It's just a little bit of a speed up So all the stuff we're talking about here is just kind of the way we decided to put things together So this is just one example obviously We've had a lot of questions on well can I do can I add this and can I add that and can I change this and you know This is open source. This is this is an open world. You can basically do whatever you want So I just wanted to be clear that what we've done and put it all together is kind of our way of doing it our demo It's mainly a idea to demonstrate the concept You can pick your own boards. You can do whatever you want You can pick a large board and jam it in there any way you want We've done only small boards because we're using slideable basically disc drawers Get in there, but if you have a big board you want to get in there Well, you buy a bigger PC and cram it in there and do do whatever you want But the so yeah, it's obviously it's this is kind of DIY We have our example, but you know you can do you can do it However, you like and we've documented on the I didn't mention here on the The bottom of this this page here shows the Lava Docker project that the github for the Lava Docker project And the read me on that project actually goes through the documentation that the different pieces here and how we put it together And how you actually would configure if you are gonna The quick start for the Lava Docker project is a QMU only so there's no hardware But so the it'll show you how you set this up really quickly with just a QMU device And then it'll go through how to start adding boards One by one, but obviously the QMU is the easiest one to describe because there's no there's no physical connections to worry about Yeah, that's the way we're doing it now I believe I mean you could set up your you dev rules as well to create devices that have more more general access and you wouldn't Sorry, the question was do we need to run the Docker containers as privileged to access the the physical devices and stuff and We are doing it privilege at the moment, but it's not a requirement I think that's one of the things we have on the things to do is basically clean that up So it doesn't have to run with privileges I think so the other reason it needs to run with privileges if you want to expose if you want dockard Actually proxy some privileged ports and things like that you need privileges The question is do we have a bomb with a part list? Yeah, some kind of yeah, yeah, it's a text file count Yeah, yeah, yeah Yeah, to be honest we are a little bit behind on the documentation sign but we've been working mostly on the documentation for the software pieces of how it all fits together, but yeah We can work on the that should be relatively easy to put together a list of hardware pieces Okay, go ahead. Okay. Thanks Kevin. So Have we achieved our goal? Well, that's the question. I would say do you remember the first motivation? we started from that and We achieved that so I guess the number one motivation was achieved We we changed from a messy desk to a nice All-in-one solution in a PC case But that's not the only Achievement we believe so we have a fully functional lab. So it processes kernel CI jobs on Every board that's inside and it's operating table 24-7 so yeah, we have a fully functional lab from this perspective Everything has been integrated into a single PC case. So what's nice with this? prototype or concept is that you have this Case you just need to connect the power to plug in the the network. That's it obviously load the software and configure it to some extent but in terms of Wiring that's all So we think this is a great achievement We have no more right wiring no more boards on your on our desks or shelves It's okay to have it in an apartment for home workers It's under your desk. You just forget about it. It's It's fun. It's fun less actually so you don't you don't you don't hear anything. It could be fun less Sorry We believe it's a very good demonstrator. You will see tomorrow if you come at the technical showcase It's catching eyes and it shows everything while wired together all integrated. So To demonstrate it's possible to to do some Evangelizing of the need of CI while it's a good demonstrator We have put all the device on the test into a drive trace. So that means it's easy To maintain if we want to change a board whether because it's broken or we need another one We just slide out the tray and put the new board back in that's it Someone asked about the the the bomb and the cost so It's quite reasonable today. The cost is about 500 euros Everything included except the devices on the test This could be further reduced if you can recycle an old PC because any PC from More than five years is still okay. It's just it requires if you have four gigs or from that's it. It's okay So again, if you have some USB UPS already or some network switch, you can just reuse it There is no requirement on it except the number of boards obviously We have done so much from a software from software perspective we have done some automation For the regarding the installation, but it's still a work in progress to be honest We have containerized and scalable software. So that makes life more easy Still even if we achieved many many many of our goals. There are still some limitations The first one is that it's pretty tedious to build and quite difficult to mass produce today It's handmade everything is custom. So and it's packed into a PC case and to be honest We did not really realize the amount of work we we had to do to put all those cables inside There are some limitations with the dimensions of the boards you may integrate into this PC case So we have the three inch and five inch traditional base PC base Well, there are some limitations coming with that and also the but also the eight of the board if you have a board that has I don't know high Radiator or fan could be a problem to integrate eventually. That's not really a limitation, but each ATX connector from so from the PC has a maximum Current it can allow. So it's typically around four amps. So when you use a SATA or a molex connector You can get five amp or four amp of that Well, you need to balance that power among all the boards you have in your PC case Do not exceed this maximum power per cable Initially, we thought that well today we have we've used the standard tower case But if we put a larger one, we will integrate more boards. That's not exactly true Because of this internal wiring Actually, yeah, there might be more slots for the boards, but that's too many wiring and too many network Switch or so that doesn't scale very well in terms of PC case size It scales well because just okay. You have six in one with one PC case You can drive six six boards if you need 12 you just put two PC cases and and so on And the last limitation is that well that that's because it's all custom is that Every new board you have to integrate as a custom location For the connectors and so you have to deal with that. There's no nothing standard There's always things to improve of course here are a few ones First Well, it's not really improved, but it's a better choice. We could have made a better choice with the ATX power supply because we used The power itself is pretty sufficient. We used a 500 watt PSU and that's more than enough to drive all the boards But the fact is the larger PSU you use the more SATA and molex connectors connectors you get and that means that makes it easier To power the different device on the test typically we could if we have used a larger one We could have directly drive use one SATA connector per board and so we did not have to balance in terms of Development boards while we need to think about how to integrate larger ones so more than five inch In terms of administration, we haven't yet really provided any Control panel that's I think we think it's something that that is key and missing in many of the tools we use today if we had a panel that we would just use to You know use menu and click some box text some checkbox To configure all the things that makes life very easy. So that's where we want to go to some extent and also in terms of complexity and pricing if we you wanted to use such a solution just for a single or two or dual boards Lab, it's actually pretty difficult and expensive and in terms of Documentation as I said previously we are a little bit behind But it's a work in progress So what's next? Well or lab in the box concept was a first experimentation To validate the concept actually So we wanted something really low-cost as much as possible to target individuals and small groups and Next we want to address more use cases. So the one one board lab So what we call it the mini lava box. So you just have one board That's the board you are using every day and you just want to run CI on it So how to address that? We want to also address a more professional grade solution where we would use racks the standard racks instead of PC cases That's to address corporations We really want to enable more software automation both in terms of Installation and in administration perspectives We need to address the connectivity side of it You may have noticed that it's all wired Connectivity networking there's no Wi-Fi or Bluetooth and we know that what today in the IOT world It's pretty mandatory We'd like then once I would say the CI thing is running We'd like to start working on start-on jobs standard jobs that would run on the boards So again to help people not have to develop the own test and we reinvent the wheel every time and Of course works work on the documentation. So by the way, we have put on this slide The documentation available today and that's it And if you do if you do play with the github projects You'll feel free to raise issues or questions just using the github, you know issue tracker or whatever And we'll get back to you on that if you want to play with it or if you find stuff in the documentation That's not clear or whatever let us let us know it's it's a quick start is available now But the more full stuff is still being worked on. So Any any other questions? Yeah, go ahead So the question is that we given any thought to adding additional boards like GPIO connected things or iSquared C connected things Yeah, we have in the in our kind of regular lab We have more setups like that where there's additional hardware But I mean we haven't thought about it in in the context of this in the sense other than yeah It just complicates the wiring, but it doesn't it doesn't make anything harder except for finding space in the PC case is really working. Yeah There's not yeah, if it if it's small enough to fit in a bay, okay, you may have noticed we can go back There's one board So it may not be possible for all the boards, but yeah This For this one You see you have more space. So It's already possible and you may reuse here. You have two base two five inch base You could fit more more peripheral But it's true that because it's leveraging the kernel CI dot org project currently. It's a kernel booting and some Kernel testing, but there's not much yet peripheral testing This this has been already discussed and potentially we could try to find a way to Provide connectors for outside like the box is still the box and with Internal periphery devices, but also a low external Device in the test to be connected to it. That's exactly Exactly, so that that's an idea any other questions or comments. Yeah, you might have to be a little bit louder Right. So how much custom hardware is in there? So the only thing custom is basically the the acne cape that it's a it's a standard Beaglebone black cape that we built at Bailey Bryn we sell as a product But you could also just put in you could put in any sort of relays to switch power in there that you like But that's the only thing that's not really well It's off the shelf in a sense. You can go to our website and buy it, but That there's nothing else custom in there other than you know Yeah, actually custom cutting of the wires to the right length so they're not you know a bunch of loops a cable around Yeah, there's a comment on the install page about you need to have good soldering skills and stuff so the yes Yeah, and so speak to that a little bit. Yes. So yeah in terms of tinkering skills and soldering skills. So We have done a lot of integration in this work So this acne solution is the standard one So you buy it to the store and use you use it as it no problem It's more about the mechanics and putting all together. That was all handmade and custom. So cutting wires Also the The all the connections from the PSU so the ATX PSU to the different To to to drive the power to the to the different boards. We had to cut all the more X connectors and fine user All things to Cable that together. So it's not really shown here the the rhythm. Yeah, it's about finding a mechanical solution To to have is have it something fixed in the in the case The other soldering might be that if you're some of these boards the connection between the power switch The relay or the ACME and your board You know you buy a cut you have barrel connector And you might just you know when you make that custom cable just the right length you solder the connections on yeah It's little little things like that. We're not doing any so we didn't modify any boards Yeah, like a little soldering on boards or nothing like in terms of soldering. It's really making custom cables That's it everything by the way. Everything was ordered on Amazon. So it's It's very easy to get any other question anything else. Yeah Yeah, yeah, thanks. Thank you. Okay. Yeah. Yeah, thanks Yes Yeah, so there seems to be a lot of custom cut acrylic also for multi-multing devices the plexiglass. Yes. Yes. Yes Yes, so again. Yes, we we just about a large large pieces of plexiglass and we cut it ourselves and Make all the holes that we needed. So yeah, that's the mechanical and tinkering skills. I was referring to Yeah, there's nothing standard here. Yeah That would that would make sense if we want if every lava box was gonna have exactly the same boards in it But I think in this in this environment I think people are putting different boards in different and every board has a slightly different form factor and different mounting holes and stuff So it's it's not as yet. So yes for these ones For these guys that would be very helpful because it's standard. That's standard But that for these guys on the the right-hand side. It's It's gonna be custom every time almost Okay, I think we're out of time here is ten ten till yeah, so I think we're out of time. So thank you very much Thanks for coming