 Okay, well thank you for coming. I know some of you are hungry and thirsty and so I won't take up much of your time I'm not a technical expert So if you do have any technical questions, I can take them down and get you to the right person or get an answer back with you Okay, so My name is Douglas DeMio. I work for open Suza. I've been there for about a year. I'm fairly new to Linux. I Have a background in military So and my wife's German, so I don't have much of a sense of humor. So bear with me. All right So this is a basic overview of what we'll talk about today with open QA Open Suza, if you're unfamiliar with it is a community project that's sponsored by Suza It has two distributions and it has multiple tools A lot of people will ask, you know, what is the difference between Suza and open Suza? Open Suza is a community version and Suza is the enterprise version So why automated testing well the answer should be fairly obvious, right? Maybe that's why there's not too many people in here, but you know, we're talking about innovation. We're talking about progress. We're talking about getting Driving technology forward so This is just a brief summary of The changes that have taken place Using open QA in the past two years. You can see how many changes have taken place in Slee 12 You're talking 600,000 packages added and nearly 2080 removed, right? Same with open Suza 13.2 Very similar numbers there and then our latest release open Suza lead 42.1 Also with very similar numbers. Some might ask what's the difference between Jumping from 13 to to 42.1. I kind of just want to clarify that because I'm sure some of you might be thinking Why that happened so Suza or open Suza in its latest release used a lot of source code from Suza Linux Enterprise 12 service pack one and They use that as sort of the base and then community built Packages on top of that. So it's it's a little different than that CentOS. It's more dynamic there's more of an effort from community in that base so and Suza has a long history of starting something new with 42 reference to the meaning of life, yeah, so Yast was its first version was point four two and Suza's first version technically was that they released was four point two so I Hope that answers the questions that you might have in your mind. All right So, yeah change is good, right? Change is good because you know you you you fix security issues or you Enhance performance you you fix bugs You have newer hardware that you can accommodate newer features You know upstream upstream is very fast-paced and and so you want to be moving just as fast as them But change is also bad, right? So you get new bugs and you get new security issues You get new functionality, you know, some people understand change some people can adapt others Sort of live in the past, right? okay, so Audit made a testing helps open Suza develop fast and you will see that in the in the slides that come a little bit later with Timberweed, I'll give you some basic numbers and you can see how fast. We're actually moving with upstream What are the problems with the other testing tools? So how many are you how many of you are actually using? Some testing tools that are out there right now solarium anyone solarium, okay Cucumber No, Jenkins. I hear a lot of okay Jenkins. I heard a lot of people use that So so what are some of the issues? Yeah, well This is just the basic list of it, right? So some don't render well some are entirely web interface some focus strictly on packages They lack a gooey Really, there's nothing out there that compares to open QA So open QA to the rescue a little bit of background this started out as a hack week project in 2009 by A guy by the name of Bernhardt vitamin and it's developed over time That was something that they continually focused on on the hack weeks that followed. So We eventually got to a point where Where we could start to integrate it and when we looked at integrating it, you know, this is sort of the dev ops issue and Discussion the person in here previously was talking about is it's really a cultural change, right? So you need to Look at I guess your lease with open Suza. We looked across the board. We saw how how our processes worked and We thought well, how can we integrate this in and and so that's what we've done is we've We've taken this tool that we've developed and added it to create and enhance our dev ops This is sort of an overview. So testing testing testing. You'll see the submission come in gets an automatic review goes through a Pre-integration tests a manual reviews and then it goes into what we call factory and within factory You're looking at multiple staging levels of tests. So this is where some of our processes has have been enhanced there's a huge flow chart in my co-worker's office and it it If you look at it, it just it doesn't make a lot of sense I guess when if you don't really know the whole thing But you have the yes-no answers and it all kind of cycles around and you could start it Or you could be at the very end and get a no and it'll shift all the way back to the very beginning to run through that process but as you go through some of these testing phases where we use open QA and within factory in the state in certain staging areas When we get to it to where we can get to the final test and everything's worked out and it gets QA approved You get a release Or an alpha or a beta or whatever you want to say whatever point you're at in your in your development cycle so a little bit of overview is about the About open QA. It doesn't really touch the software. It really acts like someone is actually Doing the testing Manually and so this is this is a great benefit. You're talking about opening up all different programs You're talking about testing the console a variety of things. You're actually you can You can create a test with open QA to basically do what you want So it's it's an installation Yeah, installation testing. It's it's from beginning to end but really When it hits a failure doesn't need to go forward, right? So it just stops right there. However, you could Potentially if you wanted to in the scripting ask it to continue if you'd like So it gives you a graphical interface you can see on the left there you get the green Means it's past and it works on to the next level right One thing I would point out with this So in in your development process, you know when when when we have something like our rolling release Tumbleweed you get a lot of version changes, right? So so you might wonder how do you account for that in your testing scenarios? Well, basically you have these green boxes that you can Focus on certain areas that you say hey, I need this to be a hundred percent that this this area cannot change However, you have version numbers that might change you can give a percentage of acceptance of change So you can account for those little things that would happen and obviously if you're doing routine cycles, right? You know those little things would matter and so that was thought of so All right, so you can see the script running too. That's another good thing below this If you actually go to the open QA website You can track a live test that's taking place. It's in you'll see a little yellow button And if you click on that you can actually see it See it moving and and see it the text the test taking place and see the script below it and how it's actually being tested So this screen as I said, I'm not a technician So to some of you you might understand what this is but this is supposed to be from what I understand an output of the previous screen So I'll give you a chance to look at that So this is sort of the architecture the way it's set up We have various workers it goes into a pool and It gets I guess it's call up and and sort of waits in line because you'll have multiple tests taking place And they're happening every day. I mean there's there's there's tests every day going on over the over the day over the night and they all kind of wait in a queue and then eventually when it's their time they they go through testing and You can you can adjust that too to if you need something that you need to rush through You can I guess subvert the other ones and go to the top of the list if that is what you need to do So it gives you a quick emulator It it supports so with ours it supports multiple architect architectures Which is Very helpful for a lot of people right? Because because they're working on on several different architectures or we're working with so many people Yeah, so we we have to make sure that our stuff is running on a variety of Revot rev variety of hardware And it does run on bare metal so Most of it is done tested it most of it is tested in a VM, but you can do it on bare metal This is sort of the dashboard that you have from it where you can see all your tests and It basically this is in the development cycle of a distribution you could you could see that I Guess years past right you get six month development cycle Fairly fairly quick at that time I guess But you might have introduced something into it as you're building as you're building your distribution And you're toward the end of it and and then you introduce something else in there And then you find some bug or some issue and and and in the past it was kind of difficult because You go where did this happen? You know it might have happened like four months ago If you got a track and go back to all that well the benefit is you know you get to keep all this information In your servers you have all that data you can go back you can figure it out fairly quickly if you need to But a lot of the testing you'll see it right away You'll see the issues right So there's a failure that you can see right there on the left. You'll get that You'll get that red check mark and basically it stops from there So who is who's using open QA? Well, obviously open SUSE is using it We use it for tumbleweed and how many people here are familiar with tumbleweed How many people here are familiar with rawhide? Okay So tumbleweed is What some of the people that I work with believe is actually the future of Linux distributions and that is That the latest software is being tested and coming out very quickly, so It's it's the latest packages the latest applications it's it's just that's what tumbleweed is you you have the latest and Greatest and it's stable and it's tested and that's that's the most important thing That I'd like you to take away from from that idea of tumbleweed. It's tested a lot of people will compare it to arch and You know arch is good, you know, but you have to compile arch right and there's a lot of stuff you have to do with tumbleweed You don't tumbleweed you get snapshots and it's tested and it works We used it in a leap and that was that was quite a Difficult thing right to bring two source codes together But we used it for validation testing system testing Susa's using it they're using it for a Susa Linux enterprise 12 they used it for 11 I believe 11 for was their first one. They actually started using it a little bit And red hats using it The person who wrote this Who's our QA engineer Richard Brown and the open Susa chairman? one day he was a little bored and so he Thought well, let me let me see how how easy this is to make a test and so he wrote a test for fedora and In doing that test he learned writing that test and creating it he ran it with Fedora and he didn't do it to kind of Yeah like I guess what what's the word just Not be confrontational he didn't do it with any intention like that he wanted to take Distribution he knew but not not so well and just write something for him and he did and In that test it found some bugs that fedora had and of course He knew that the release manager for fedora and he said hey I Just wrote this for you. I don't know if you want to take want to take a look at it and the release manager was very Yeah, thankful and he said okay So they started talking about it and they started collaborating and now you have red hat using it and red hat is using it And they're contributing back to it Which is the most important right because if open Suza has some has some issues and fedora has issues You know you have certain bugs a lot of ways right there. They were late. So we're kind of working together and we're We're helping each other and that's I think that's a positive good news story for for the community and for Lennox so and You can see right there in the bottom there. That's the release manager and I did that I took this screenshot just a Couple hours ago, but they so they used it five days ago and they're using it with rawhide You know that you might see that progress develop a little bit further Based on Based on them how they integrate it into their processes, I don't think they're there yet But they I'm sure they will get there soon So open QA and leap I've briefly explained a little bit A little bit about this earlier, you know, you're taking source code from Susa Linux Enterprise And then what we had in open Suza is sort of the development going forward and really bringing those two together It goes through it went through extensive integrated integration challenges, right and You're looking at over 25 different installation upgrade scenarios and Open QA Allowed us to do that. You know, you have an open QA engineer. He's twice as productive, right? He's running his tests He's doing his manual testing. It's open QA is just a really really beneficial for everyone and As I mentioned earlier, we have it in our processes. It is an integral part to to open Suza development and Really it's it's allowed us to to be very fast in our releases so The very bottom there testing testing testing New tumbleweed releases as I said a rolling release. They're expected about One every two or three days when we have the when we have the Building of of our regular release it kind of it kind of slows down Right because because areas are focused in other other places But it still does move forward during that time It's just we have to dedicate more resources to to the development of our regular release so Here's an example of a quiet week for tumbleweed This is something I guess has happened back in April or something like that. So they had three snapshots 146 package updates 15 new new packages packages on the DVD and We're also removing that some of those packages, right? So we're cleaning up after ourselves And one new kernel All tested in 118 different installation and upgrades This is this next one is a little more interesting, right? So and this is what we're talking about with development. You have five snapshots 298 package updates 47 new packages on the DVD 42 packages removed two new kernels and And it was sort of just another week as someone described it when that when they did this and this was again this was Back in spring, but we did the same thing last week. So we had five snapshots within tumbleweed last week. Yes I'm not exactly sure on that number. I mean I could explain the top ones, but that one. I'm not exactly sure do you Know George. Do you know anything? So yeah, as I mentioned earlier The first one the first time they started using open QA Slee did or sorry, Suza did was in Slee 11 sp4 and they integrated that into their later processes with Slee 12 which was released last year and Slee sp1, which was just recently released. So you How you incorporated into it as you do it with some pre pre-validation and staging so And every part I guess of the development you're kind of looking at opens open QA and you're using it to your advantage So it's inside our processes inside Suza's processes and They don't with Suza. They They don't really show their their stuff You know, it's sort of Hidden in the back, but but with open Suza, you know, we do make those tests available for everyone So over 20 over a hundred different Validation scenario test. I think I would think that would kind of be along the same lines as your question earlier You're using it for we're using it for alphas betas Different different scenarios matter of fact when we released When we released Leap, I think we just went straight to a beta I Think that's what we did and you can use it for post-validation as well So Go back open QA and you So if you're if you're thinking about using Q open QA, I basically there basically here some some information for you on how you can either contribute or or learn about it or Contact someone that can help so The documentation is available the bug reporting is available at that website And it's located the main project is located on get-up as I mentioned earlier You know open Suza's transparent. They're they're making all their test test publicly available for everyone so everyone can see it and The best way to start and think about how you would describe the steps for someone someone else to do it So if you're writing a test you want to think well, how am I gonna get how how? How am I gonna use this right and so you want to write those steps down? Assume that that person doesn't know how to do anything and Then use those steps to help create your test code write the test code for each step and You also want to think about what you can do with open QA to enhance or develop your processes To aid with devops transformation if if that's something that you're here and thinking about with this presentation This is some basic background sort of I guess advisement whatnot Yeah Organization I it's a British guy that did this so this this There's no there's no z in it. Yeah, so I Just left it the way it was. I don't know if that if that was if that's a necessary And here's some other Coding actions for you, I guess here's a Example test console it's for the console this next one's for the graphic and so As as we're looking at at that we see how we're open using open QA With their distributions, but it has a lot more potential than that right so here are the Some of the things that we are actually looking at Doing the thoughts the thoughts have already there, but the they're actually working toward this right and My understanding is that possibly they might be looking at salt stack, too So and that was pretty much all I had for this presentation If you want to find out more You can go to that website And you can contact either Richard Brown at Susa comm or hop on the IRC opens who's a factory and All those guys will will be able to answer any question you have pretty much white right away They're they're on there all the time and And they'll get back with you on any sort of help you would need Richard Richard's very helpful He can he can answer any questions you have he does it as his day job, so What's that? He's the guy that made the slides and yeah, he's the British guy the slides No, I don't think they are I I did send them to scale But I did makes Just a couple of changes not not much. I mean the content is For what he did was was there so So does anyone have any questions about open QA is it being used to test anything other than the installations I Believe it is. I think that was actually in the slide Let me see if I can find that I I I did see that somewhere You can use it to test You can use it to test network cards. You can use it for basically a Lot of different things Open CV. Yeah, let's start bring that up to read the actual screen output and compare the predefined needles That isn't the question. Okay? Okay, I Couldn't answer that question for you. I wouldn't I wouldn't be able to well I Mean I don't see why you couldn't make a test for that. I mean if it's I can't tell you if we do that but But judging from what I can see in the documentation and and the way that they've discussed it It's you can test for pretty much anything you want. It's just a matter of actually writing it, right? and that that Brings up a valid point if they hadn't thought about that, but I'm sure they probably have But who knows? Yeah, I can pull up. I Can pull up IRC and ask if they would if they would know that I mean probably everyone would on that Yeah, yeah, oh That's that's actually a test. Sorry Yeah, that's that just No, that that was something I had running in the room before Before people showed up so they could actually Zone out or look at it. It wasn't there was nothing No intention behind that other than just giving something to look at rather It is. Yeah, did you want to see it? Let's see I Don't know if it would be in doc again. I'm not a technical expert. I'll go to the documentation and see if there's anything that Would be Needed to install and set up tools General operate as probably in that starter guide I would think I mean, I Really, yeah, I couldn't really tell you I know we have our security team. So that I know they look at stuff As far as like incorporating security tests in it I Don't I don't I Don't really see anything But it's an interesting thing to bring up and I can ask Richard and see I give you my card You can reach out to me and I mean those that's a good idea, right? Why not? I've heard of it Yeah, yeah, that's I'm sure that that would I'm sure bringing it up. We'll probably make someone do that So yeah Any other questions? All right. Thank you