 So, I'm Jan. I'm from Fedora QA team and our job is to make sure that nothing goes wrong with Fedora and what is yeah, when and when it happens we just sit there and laugh and pretty much Do nothing. Yeah, and Open QA is something we've discovered a year and a half ago Which help us mostly with the first part of our job So, why do we need Open QA? Some of you may know what it is. It's an installation matrix for every compose that is automated for testing Used to be called test compose Metrists like this gets created. There are dozens, dozens of different tests and we had to go through them manually and We had to go through them and test each manually So that means that we have to try with Fedora installs and the disk whether installs Full disk and can delete all partitions and it was tedious work and it took a lot of time We only could do that Several times per week and then we just discovered Open QA Open QA fits our workflow Miraculously, it's designed to work on something like this So what actually is Open QA? This is the main page of Open QA. Open QA is a Service for OS level testing It's best suited for testing installations. That means Anaconda for Fedora Yeah It is developed by CUZ, Open CUZ or Open CUZ. I don't know how to pronounce it. It's collaborative effort Which basically means that they are developing it and we are using it and Yeah, we're using it for More more than more than a year and a half. It's developed in parallel So hopefully it doesn't Doesn't make problem for you How Open QA works? it spawns QE move virtual machines It connects to them via a VNC and it simulates user input, which means mouse clicking, moving Peace sending keystrokes. It takes screenshots of the whole screen and It does the image recognition and image comparing and uses this for Doing like a process like the user would do it Which means that we can use unmodified ISOs For example with Doctail we had to Modify the ISOs but correct programs here and so on With OpenQA we can use the same ISOs that are which are in the trees and Just put them and test them as whole Yeah, which fits again, which fits our use case because we want to test what we are developed Delivering to our users This is architecture OpenQA. OpenQA consists of two parts. One is This part is actually called OpenQA This other part is called OS Auto Inst, but the whole project is called OpenQA This is the application There you can review tests so you can Develop new tests at least from the screenshot side You can see progress of running tests and so on and It spawns, it uses workers, you can use as many workers as you want so it scales pretty well It sends jobs to workers and here runs project called OS Auto Inst Which again spawns virtual machines and connects to them via VNC And then it sends results back to the web application What this isn't suited for We are We don't want to use it for Something that that needs a human intervention For example, we have a test that you can submit a bug to bugzilla from Anaconda And then you have to go to bugzilla by yourself and close the bug There's this is something we don't want to do Automatically because there then we would have both spawning or bugzilla And also it would be pretty hard to Know the exact URL where bug was created We are not using it for non-installation related tasks Check sums or something like that. This is suited mainly for installation, but we have also some desktop desktop and Also server tests in OpenQA it is Yeah, it works well even for those things and Another thing for what OpenQA isn't suited for is Where you need some specialized hardware. It uses QEMU. It doesn't have to but Main use case is that it uses QEMU So you have to have all your hardware virtualized in QEMU So if you want to test a firmware firmware right or something like that, you can't because you would have to Get firmware right into OpenQEmo. We are testing for example ARM with it. We can emulate ARM We are testing UFI But not for example rate What is spinning the ass with OpenQA There are lots of Font changes for example, this is fabricated example This this didn't happen, but we have a lot of fun changes from changes last year It's subtle. It's something that user don't really see but Even though that OpenQA uses Only partial or you can you can set level how similar the images have to be but even even then Sometimes tests failed only because Button has moved or has a different different color or or different font or something like that And then you have to go through all your screenshots and rework them We have Adam Williamson in our team So this doesn't pose problem for us because every time this happens Adam just goes through all our tests All our tests and all our screenshots and and do this. I don't know how he's doing, but yeah I'm gonna show you How developing tests for OpenQA works So first time we are we are going we are gonna develop tests that tests Federal installation. It's basic test that federal installs in default configuration and then that it boots into login prompt First you have to think what you want to what do you want to code what do you want to test? Video so this is yeah Let up. Yeah So this is something testers should know this is virtual machine with Fedora 22 running and You have to Yeah, you have to go through the installation by yourself and and consider What what do you want to what do you want to achieve with test? So first federal is the boot Here you will wait Until it shows language selection screen. Yeah, here we go then notice