 So we have the Ninaro Connect and so what is that LKFT? LKFT stands for Linux Kernel Functional Testing. So the project started last year in Budapest. We were asked to run a set of functional tests against long-term stable kernels. So who are you? I'm I'm I started this project I'm leading the team that's behind it. Sort of? Sort of. So yeah this idea was floating around for quite some time but yeah in Budapest we were asked by members to actually do that in a more formal way and we were granted more more time and more resources. Because the Ninaro for a long time did a lava right? Yes it is using so but lava is just a tool inside so lava is used for executing the test but the LKFT is a combination of a few tools so we have Jenkins doing the builds, we have lava executing the test, we build a dashboard that connects it all together. Is this the drawing here? Yeah the drawing here. Where are you in this? Oh you're here. So what are you doing? Are you this guy? No we build everything so this is our dashboard. Somebody presents the results and sends emails. This part is lava executes the tests on the boards using the builds that we do here in Jenkins. So it could be like Android, open embedded, some other things? Yeah so we build, we push the tree, we build Android, we build open embedded images for the boards like this. This is my key and this is where it is just controlling the relay. So they run open embedded? This one runs open embedded yes, this is a file we just made. So there's a problem with the phone no, it's something special? I'm not sure if anything about this is special, it's just a phone that we got. So we are going backwards, we push to the git tree with the build, build with Android or open embedded, then those builds are sent to lava and all the test execution goes here, that's Jenkins building stuff. And then once we run all the tests we pull the test results from lava to the dashboard and dashboard does the whole processing sends emails, checks if there are regressions and this kind of thing. So why is it important? To ensure that we don't regress in the progress internal. So Greg Crow Hartman sends a release or release candidate for long-term stable roughly once, sometimes twice a week and all on each of those pushes we run our test. So we first run the build test, then try to boot the boards and run functional tests and report back to Greg whether we regressed or not, like whether there were some problems with the code. Yeah it's working, so it's been working for three or four months now? Did you already find a lot of amazing things? Like let's say, in the past I'm guessing, maybe the Linux developers, they worked with some bugs and new releases, they wouldn't know and now there's a better way to know or not? No, I guess we didn't find anything big because we were running very simple tests. We found a few problems with builds recently, I guess a handful of tests that regressed, so we sent all those reports back upstream and they were fixed. So that was to hold an extra minute? I hope. This is a tool for getting an extra test? Yeah, so we have regular weekly meetings with Greg and as he referred, it helped him to cut down the release time, so he was a bit, yeah, so he has a bit more confidence in what he does. I think he must be a fan of continue. I hope. No, I'm not for him, so that was the primary goal. Also, Google had questions, not questions, they requested that we do the same thing for Android common trees, so that's what we do as well. That's the AOS key part of the image. Yes, Android common is a Linux kernel tree that derives from long-term stable. So it's the stuff people do in the AOS key? Yes, so this is the base for building kernels for Android, for mobile devices. So how do you use the lava ecosystem for that? As I said, lava is our test executor. Is it a big hardware thing? Yeah, so it controls the hardware, it controls flashing the images to the board, it controls the test execution and collection of test results. With all these tools that are connected to the boards and stuff? Yeah, so with the relays? We have relays that control the OTG board on this board and they control the power on and off the boards. Is this just somewhere in Cambridge or somewhere? Yeah, we have the lab in Cambridge. In Cambridge it has many, many boards and many things testing on that. Yes, so many, many, many is not that many, many right now, but we're hoping to get a few more. Dozens or so? Well, right now, from 96 boards we have high key, we have dragon boards, I think five or six of them of each. Then we have Juno boards, this is a big Greek development board from ARM. We have X15 from, this is Beagle board X15 from TI, I think also five or six of those. But if people want to add their boards, is better they join 86 boards or they can just ask other boards? So the service is primarily targeting our members, linear members, yes. But if there is a unique board, so that's something that we don't have, a unique architecture or something that we don't have yet in the lab that we will accept. We have pretty strict requirements of what boards have to do in order to be accepted. So what did the community think thus far about this? Are they happy? Is it working? Is it impressive? You had a presentation, right? I had a presentation and I also attended FOSDEM earlier this year. I had a few chats with folks and yeah, general feedback is positive. We didn't have any, we were not told anything special so far, except for what I quoted from Greg that he was very happy about it. They're probably saying good job, right? Yes. I'm really excited, you want to know how to use it? We didn't advertise it too much yet, so that's probably why it didn't get much publicity. Okay, but after this, what do you think the problem is going to be? You're going to get too much demand, requests. People can put their requests in the system or how does it work? You give priority to members? Yeah, we give priority to members but I guess we can... Do we have any... I'm not sure if we have any mailing lists set up for this. Well, we do have mailing lists but I'm not sure if it's open. Okay, we'll see.