 I'm Ian Jackson. I'm, relevantly, the primary author and maintainer of EarthSesTest, which is the testing system we're currently using for upstream Zen. So I'm going to give a brief overview of EarthSesTest, mostly from the point of view of somebody who might want to add tests to it to prove our test coverage. And please do interrupt me with questions. So I know that it is now possible to install this yourself. It has a sort of standalone mode and I'll get on to how you might get to grips with that towards the end. And several people have managed to do that, although they have obviously tripped over some things. So the production of EarthSesTest instance is run continuously on a dedicated pool of test machines at Citrix. And its main function is to provide the push gate for Zen. So changes made by the maintainers are to committers to Zen.get are pushed firstly to the staging branches. These are automatically tested and if they don't appear to contain regressions the contents are automatically pushed to the master branch. If the regressions are found the push is blocked and also then EarthSesTest attempts to bisect the intervening changes to try to identify where the problem was. And this can take place across the multiple trees which combine to make the source code for Zen's deliverables. So for example the bisector is able to look at and blame changes in QMU as well as changes in Zen.get by observing noticing that QMU was updated here and it condenses the revision graph into a single revision graph for testing. And we also run a number of non-push gate tests on other interesting code branches and the reports of that are posted to Zen develop. So we test for example Linus's upstream kernel and a number of other kernel versions. So I did mention it has a standalone mode. The standalone mode doesn't depend on our production infrastructure and obviously the associated database. So you can kind of run it locally on your own machine and you don't have to have an automatic power cycle or it prints up a message saying please turn the machine on now. And so forth. So here on the slide you can see one of the report output webpages. You may be familiar with their link to from the reports on Zen develop. So each test run is called a flight and the flight typically has a set of source code inputs that go into a build and some idea about which tests ought to be run. But mostly it consists of a series of essentially independent jobs. So each job carries out a predefined series of preparation testing steps, some of which might run in parallel but in general they're sequential, a bit like this NLT thing. And each step is the execution of a separate script. It's an actual program which can succeed or fail or do various more complicated things. The jobs that you see labeled test on the right they refer to the build outputs from the build jobs in the same flight. So I've got a slide here about the data flow which is kind of helpful if you're run. This is mostly how it works if you're running it in standalone mode. So the usual way of running standalone mode you run standalone setup which sets up a little local SQLite database. So it initializes some state. You probably have to write a little config file for that. And then you can run SG run job to run the individual steps from one of these test columns. So you can see here for example build and D64 allocates a host and in standalone mode that's a no op. Installs the build operating system on it so that you have a known good thing and standalone mode has a way of disabling that if you don't want your test box wiped. Host build prep installs some actual build utilities and then Zen build essentially runs make and sees whether it works. Collects the resulting output. So where was I? Right. So you can also invoke the individual test scripts. You see there TS host install TS runs some test whatever. You can invoke those directly from the command line. So you're not restricted to having the whole job run at once. And this is quite helpful for hacking about when you're testing things. Most of them are Perl scripts. Most of the thing is done with Perl. So SG run job is responsible as you can see. So the blue lines here are execution of a sub program and the sort of orangey color is the database, which in this case is SQLite. So SG run job is responsible for invoking the scripts in the right order and it's got embedded in it in a sort of semi declarative syntax, the knowledge of what the right steps are for each kind of job. And each of these test scripts update and refer to a set of things most in the database. Mostly these are things called run vars which are test variables describing which tools that to use what architecture we're dealing with and also the outputs and inputs. And the run vars are shared between all the steps. So that's how they mostly communicate with each other. For test jobs, there's normally some run vars that refer to other jobs that refer to build jobs. So that references which builds should be installed, for example. So if you do this in the most full on way, you'll run SG run job on a build job or maybe several build jobs to build Zen and an appropriate kernel from scratch. And then you'll run SG run job on a test job to run some test. But you could start halfway through and skip the Zen installation if you like and just run whichever test you're interested in directly. So the test jobs know how to do things like installing guests, which is quite convenient. The test scripts, they all use a Perl module called testsupport.pm, which contains a lot of common functions and provide access to the run vars know how to install, do the sort of core parts of installing guests. So just to give you a flavour of this, here's one of the simpler test scripts, which I'll just, this is the one which tests migration from one host to another. We have a local host migration as well. So you can see it's very short. This is because most of the important stuff is done by the library. First off is just boilerplate. Includes some Perl modules. In the middle section we have some set up time out and the standard call, which you can't quite see. Let me see if I can do less than that. It's not really helping, is it? If I put this over here, does it actually make any difference? Smaller numbers. Yeah, let's try 140 maybe. There we go. I'll probably be able to read that now. Right, that's somewhat better. You can actually see what's going on now. Yeah, so there's TS read config call, just initialise the library. It's a piece of boilerplate and call it. It does some stuff. We then parse the three arguments, which are the two hosts that we're going to be migrating between and the identifier for the guest. The meet is this migration function, which, so guest check running is a quick check using a TCP port on the guest. And guest check up does some more serious checks. I'm not sure I need to go through that in too much more detail. The main program here is fairly self-explanatory too. So I just wanted to give you a kind of flavour of how easy it is to add stuff to this and reassure you that we're very interested in your contributions. The main problem we're having, I think, is that we don't have enough interesting tests. A lot of important features that people care about aren't being tested at the moment. And if you'd like to help fix that, please come to my BOF workshop, which will be very soon. And I'll help you out. And then you can help us out. We're able to take questions? Yeah. Anybody, any questions? Is this like the pearl version of the ZNRT? No. No. No, it's quite different to ZNRT. It's much less, the software architecture is much less centralised. So each of these individual things here are individual programs. And in principle there's no particular reason why you couldn't write individual test scripts in Python if you wanted to. You might have to write a bit of library to help you get at the state variables and things, but that's not really a difficulty. That makes it a bit easier to play around within a standalone mode. And also it means that the different things that are running on the system at the same time are not really interacting with each other so much. It's easy to update the code, for example. So I can run that the existing system is capable of running test flights using different versions of the testing software on the same test pool simultaneously. Because there's no central process that is running all of this code. Other questions? I can't see the hand. Somebody pointed over there. No? No. All right, so if you want to know more and learn how to do that, Ian's just doing a boff slash hands on session next door. Thank you.