 Well, thank you everyone for coming glad to see you all here We're doing a presentation on how to contribute to open stack It's for newbies and we're trying to keep it light-hearted. So feel free to laugh join in See if you see anything you recognize go for it So we actually work with AT&T. We're on the integrated cloud and we work on the open stack community team We are doing a little bit of upstreaming for a test suite called patrol and we're also doing a little bit of work on Yeah, we actually do a little bit of work on the security side of things now And so we're looking at different things like Barbican Keystone and other projects with an open sec So a little introduction Hi everyone, so my name is J. Woo It's a little bit of a background on my open stack journey I started programming and research in academia specifically in medical research and then that kind of led into a lot of application development opportunities So that transitioned me into an applications developer role with an AT&T And then I kind of came across this black box called the cloud and was really curious about How the the cloud really worked so transitioned over into a cloud of structure team at AT&T and Probably wondering fun fact. I studied self-driving cars on my free time So I do a lot of stuff about deep learning and sensor fusion So my name is Nicholas Hulgeson and I actually have a degree in plant virology, which is a little strange I know But I actually discovered programming because I went into the field and I didn't really like it very much So I actually discovered a program called Udacity Some of you may know it Through them I did a full stack now degree and got an internship with AT&T last summer that's actually where I met J-Woo and ever since then I Go while I got switched to a full-time role with them and ever since I was working with AT&T on open stack and before I ever did that I had no idea what open stack was so another kind of interesting thing from my perspective is that I found it very interesting the similarities between how Viruses interact with the body and interact with plants and the similarities between that and the way code interacts with things like servers and viruses and code and things like that So a little bit of our background we are web app developers That's where we came from. He was born the full stack or front-end. I was born the full stack back-end stuff We know things like Python and Frameworks like flask and angular and things like that. So when it came to open stack when it came to open source We had very little experience Open stack was a black box. It was magic to us. We just the little we did know about it was you know You kind of throw your app at it and it makes it elastic it makes it load balanced and just Takes off from there So now that you know a little bit about our background We think you would benefit from this presentation if you're you have development experience somewhere else And you're transitioning over into open stack or if you're a new developer just starting your journey out And then you happen to just come across the open stack out of luck or you're managing a team And you want your developers to contribute more and be active in the open source community open stack community, sorry So a little bit of an agenda so We'll kind of address the problems that we faced when we were starting out with open stack and the method that we kind of took and Think that it really worked out well and think that can work out for everyone else That's starting it to learn open stack and then we'll be doing a very brief introduction into Dev stack You guys probably already have experience with that suck But since it's a new beguide we thought it was Necessary to kind of do a little bit of an introduction and then we'll be talking about tempest Which is the most main point the testing suite of open stack That we'll be talking about and then we'll do a slight demo on using tempest and PDB and what you can kind of learn from there So in the end you'll have a better understanding of how open stack works in general so Going back to our background When we transitioned over into cloud cloud application cloud infrastructure development we're actually very thrilled at the opportunity because We finally got the chance to learn about the black box that we had no idea how it worked, right? So we started learning about open stack in general before we died into a specific project like other people Kind of suggest that that we do and we kind of came across a couple of problems So why do we have a problem when we're first starting out learning open stack? It was you can kind of relate it back to the fact that we were very coupled to using frameworks and libraries that was Development toolkit, so we're very used to using these frameworks and libraries that were meant to be used to develop Applications, and it was very unlike open stack So these Development toolkits had a very low area various to entry and the user base was doubtably larger and more diverse and had a plethora of tutorials training material and step-by-step guidelines and it was easier To make a sandbox setting and play around with a couple of prototypes before you could push it up into a production environment Open stack was a little bit different The documentation and the community was is fantastic. However, it's not as obvious as these development tools that we were familiar with So not all the challenges the errors have been replicated on stack overflow And you just can't go and copy and paste and pray that things work out So it's a very challenging environment for newbies and When we are starting out we started out with the low hanging fruit like other people suggested And this was a common advice given by our co-workers and other people in the community as well however when we looked through launchpad, it was a little bit difficult to find a chewable bug because There the easier ones would get picked up easily because there is a lot of open stock contributors trying to Become I guess major contributors and it was hard to explore different projects to find something that we could actually work on So it was confusing, but eventually we found something and submitted a patch for that Although we had to go through the troubles of setting up DevStack under a company proxy that we had But hooray now we're officially open stack contributors, right? So looking back, what did we learn from that bug fixing experience that low hanging bug fix experience? It taught us a lot about the Garrett workflow workflow It taught us a lot about how the code review processes It taught us it gave us a refresher on several get commands that we've never used before in the past frankly and We it also taught us how to interact with the community and how to do code views and things like that But we still had a fundamental problem that wasn't really solved So what was that problem? Where do you go from there? What's next after fixing that first low hanging bug, right? So where do you go from there? More bug fixes you go you go on stack overflow or excuse me you go on launchpad and You find all these bugs you find a bug like no G upgrade from Newton to a cut of fails Well when I first started I had no idea what any of those mean and frankly I still don't know how to fix this bug I don't even know where to start on that So read the docs, you know who wants to read the docs Have you seen the documentation has anyone actually taken a look at all the different levels? There are no way Unit testing so there was another common look area that people would suggest to go ahead and start with but we ran into the same problems So we said, you know, well, which project do I start on? Where do I start on unit tests? You know, what do I test? so we had problems and Like J. We were saying, you know, I did my first bug. It was a low hanging fruit. It was a one-line change What did I learn? I didn't really discover anything about that service. I didn't know what it was doing Frankly, the one line change was pretty cool because I was an open-sat contributor, but that was about it I mean we were web developers. We weren't in for rock stars and we just Hope that our way is going to help you where we didn't have any help So we're not saying that documentation is bad, right? It's wonderful. If you know what you're looking for It's very specific in that sense. You can go in and learn what you need to and get out However, if you're starting new you can kind of get lost It feels like a rabbit hole because every link leads to another and you go from one page to a wiki back to one page And then a wiki you kind of get the idea, right? So you can get lost if you're just starting out. So what is a less of a didactic approach? So we very much agree that starting it with unit test is a good idea However, as Nicholas mentioned, it's really hard to find out where to start So we think you should get a general understanding of opensack first before you dive into a specific project and start Getting to know that project by looking at unit tests So we'll be learning at how different components work by looking at tests and tempest Which I'll explain later on and dig deeper down into the API tests and scenario tests and specific that tempest offers So Since this is a new session we'll now go into a little bit of an introduction to DevStack if you already know please spread through it so Like we said newbie guide So we figured it was a good time to start right at the beginning How to get into DevStack up and running Ubuntu that kind of thing So there are lots and lots of resources online on how to do this. They're very very good So I'm just going to go through it really quickly Obviously you're going to start with a virtual machine We picked Ubuntu. It's just what we liked. We've Well because that's what everybody on our team was doing and that's that's an excellent point. Thank you And to that point I've I've made maybe 20 different virtual machines and broken every one of them I have one running right now, which is good And you guys will see that one very soon So you would want to go to GitHub find DevStack clone it set up the local config file This was a particular pain point for us because working in AT&T we were working behind proxies This was a lot of work to set up and actually get things stacked So here's an example of that said local.com file. This one's very very simple It's what you get when you copy over the sample file that exists on DevStack With a few things redacted and So the reason this is important is because this allows you to Designate what branches of what services you're going to use in your DevStack So from here, let's do a little example just to see what DevStack looks like So right now I have a Ubuntu running and The first time it stacks You get something that looks like this. So you have your IP address of your locally running horizon your login name and passwords which are set up in your COMP file and Then if you type screen dash R you're rejoining the processes that are running in the background So you have access to a text-based Access so you can do something like open stack user list And you get a list of users If this doesn't work for you off the bat Then what you need to do is you need to source and open an RC file and add admin at the end that gives you admin rights Another way of accessing this kind of data is you log in. This is the tempest or excuse me Ranking and this is the horizon GUI. Thank you And you can log in with your login name and password and you get the same information here so That's just a quick look at Dev stack running so here's just a Quick overview of what I just showed you so screen R which is apparently being depreciated or deprecated and So once you're in screen, there's an important code control shift a gives you a list of all the running Processes but obviously that's not important anymore So Tempest is our main focus for today as I talked about in the agenda, right? So what exactly is tempest? It's an integration test suite to be run against a live open stack cluster Whether it be a one-node dev stack that we talked about or a thousand node cloud So it strives to cover all of open stack So it has as I said before API tests and scenario tests among other things and API tests cover Validating the API's that that services and open stack use to communicate with each other and the scenario tests cover common scenarios that demonstrate a working cloud and So why exactly is tempest an entry appealing point of entry to new developers? It holds a lot of valuable information about open stack in general and that's why we think it's a very valuable entry point You can for the API test you can view specific API's and compare that with the documentation So that you get a better understanding of the requests and responses of that specific API which would generally give you a better understanding of that service and open stack in general and if you look at scenario tests, you can view common scenarios that happen in open stack and Kind of educate yourself about what an operator might go through when they're dealing with open stack So by looking at tempest you're learning more about the API and common scenarios that happen in open stack and You can kind of get a better sense of how different services work independently and kind of see how they interact together by looking at the scenarios Does this sound like a general good overview of open stack? Adam abuse I guess Cool, so how do you access the tests in general and tempest They're among many ways You can ask you can look at the director as tempest itself But that is a problem in itself because tempest allows plug-ins So it won't let you discover all the tests available So another method is you can use test the test our engine and perform a list tests and Nicholas will be showing that in a little bit and you can run a specific test by using test our run and Calling that specific test So running the tempest us there's various ways of running them So the primary method of running all the tempest test is by using tempest run However, we think that using Python test tools is a best Educated way of doing things and Nicholas will explain why So the first things first Test star is not compatible with PDB So we're going to be using PDB in our tests to try and dig deeper into them So if you try to add a PDB break and run it with test are you're going to get this error So you have to use a Python test tools. So for those of you who aren't familiar with PDB. Here are some general keys and codes that we're going to use commands that we're going to use to walk through the code so basically we're just going to be able to look at the code that's around us step through that code and Finish it if we're done with it So now we have the tools that we need so we're going to go deeper into the code So what PDB helps us find is Basically, it gives us an opportunity to look past the functions that we see in the Just looking at the code just looking at the test We're able to see the code that it's firing. It's going to Lead us all the way down the chain to the point where it's going to make an API call and it gives us better a better Understanding of the service and what that service is doing and how we can go further and then and test that So I'm going to go through an example of that. So if you have any questions go ahead and stop me While I'm doing it and then I can show you from there So we're going to go back to my virtual machine Where I am running So the first thing we're going to do is we're going to take a look at the code. So what I'm going to show you here is So this is actually this is patrol. So this is a tempest plug-in and it runs our back testing on Tempest So what I've done here is I've added an import for PDB and then I'm just going to do something simple I create user and I've added a set trace here So that this is where the code is going to stop So once you now go to the terminal run the code so we're going to be running the code out of The Opstack tempest folder. So that's where every all the codes being run from So the first thing we need to do is list the tests in order to find them. So we're going to do a test our list tests So this is going to give us a very long list. I forgot to grab it So the thing we're going to have to do next is rep for the Users so test our No, this is a shortcut that I set myself. So Here we have test create user so we're going to do the Python dash M test tools So here we can see that PDB has stopped us So we're going to take a look at where we're stopped and we can see that we're stopped exactly where I put my set trace So we're going to step through the code a little bit and Step into where it's going to create the user so that we can see where it's doing Yes, that's good sandy tech. Yeah, okay. Thank you So we're stepping through the code we found a place where it's been It's being called So we're going to step through that too So we can take a look at it and we see that it's being called right here in The user's client So that's where we're going to try to get to and then see right here We've switched from the patrol folder to the tempest folder So now we're running directly from tempest and we're running directly from the user's client. So if we take a look at that We can see that this is where it's being called from and if you didn't know about this Then it's a very enlightening fact that we now understand where the API is being called from and now We can go through the code and find that API and change it there Another thing that this does is it allows us to write more tests because we understand how the test is set up We understand where the test where the code is coming from Another really cool thing about PDB if you're unfamiliar with it Is that I can now do a print statement for something like the response from the server And we can get in more information that way and that really helps us later when we're trying to debug the code So from here, what do we do? We want to find a way to contribute So we now understand more about what's going on behind the scenes, but where does that really help us? Where does that leave us? so there are lots and lots and lots of tests that are necessary to keep up with stack running smoothly and Not all the APIs are tested some of the tests are passed with errors in them As much as we try to avoid that And other tests are just testing the wrong thing I actually almost posted one test that was testing the completely wrong thing, but it was still passing the test so mistakes happen and what you can do is you can go through that code and find where these tests are missing and Contribute your own code and what that is on what that really does for us is it Removes us from the necessity of other people finding bugs It removes us from the necessity of other people telling us what tests need to be done And it really gives you the freedom to write your own tests to contribute on your own time in your own way So I was going to do one one last example for you guys On how you can write find and write your own tests so From here We're going to go to the tempest folder And one of the things that I discovered recently was that in So running on the tempest server itself This is the same area that's being tested by patrol Which is the test that I was just showing you and this is the v3 version 3 Identity tests and as you can see here, this is the for the users There is no actual test for create user So this could be an oversight it could be intentional The best way of knowing is to submit a bug So you can ask the community or you can go on the IRC platform ask if there is a this was intentional or not So one of the things another way you can check is on v2. There is a test for test create user So this is great for us because that means that there's already a template for a create user And it's really easy to just go ahead and copy paste that into v3 tests now you're gonna have to make some edits because obviously v3 is not v2 So I did that for you already here So basically I just went through made sure that it was using the v3 client instead of v2 Put it in a few parameters and the cool part about it that here is that if we go back to the tempest and List the tests again We'll find it there test our list tests So here we have the that's patrol v3 test users test create user So we can just run a tempest test on that and we can see that it will run so Sounds very cool, but I guess it works Cool. So our methods not really perfect. You can see a lot of flaws as so in our presentation too, but You'll learn a thing or two if you if you start looking at the tempest and really looking at the different tests that a specific service has If you look at just create users as Nicholas talked about there's a lot more to it than just a bunch of us are equals So you you'll learn a lot about Keystone if you will if you look at the keystone tests or whatever service you choose So this wasn't really a step-by-step guideline Although we kind of wanted to do do it in that terms because we're kind of limited by the 40-minute time frame, right? But we really think if you look at tempest and kind of teach yourself open stock that way it can be a step-by-step guideline and You do have to remember when you're looking at one specific service Project service in tempest because every component in Opus stack is set up slightly differently So there might be some discrepancies there But that's the case of Opus or software and you kind of have to live with that with knowing that fact So that's what they meant when they said start by looking at unit tests And that's a soft wrap for our presentation. Do you guys have any questions for us? That's the VM? Yeah Or oh, but that's our okay Yeah But if you do the same Thank you Go to those By writing Thank you. Does anyone else have a little bit more simple questions on our process well today I learned developers actually talk to people Kidding But I to expand on as play I actually did look at our PDB when I was actually working on Keystone because you can't really Do a PDB setrace on an Apache server. So but I had some trouble setting up. So I think that's why I kind of Put that notion aside. Yeah Another thing is that now open source is just the nature of it everything's different and what works for Keystone doesn't necessarily work for Noah It's just a matter of exploring use PDB and that's kind of what we're trying to convey for you guys Does anyone have going back to the questions? Does anyone have simpler questions for us? We are going we are actually on the security team, so I'll be looking at security related matters But yeah, that's our next journey. Yeah, so we're still in the process of upstreaming upgrading a few things For AT&T Okay So before we finish off, we would like to mention that AT&T is proud to be a member of the LCO community I'd see the two guys cringing back there Taking a picture But on behalf of everyone working on Opstack as well as us we'd like to thank you guys for coming out and spending time With us today Here on this presentation and we try to muster together The common pain points that a new developer onboarding Opstack kind of would come across and kind of our Opinions on how that can be improved. We hope that a lot of new developers can benefit from this and Kind of improve upon the process of this new onboarding procedure as things change in Opstack, which is pretty fast Thank you everyone. Thank you