 So I think you're ready to go. Oyan is going to be talking about hypotheses with PyTest, so I think we're interested. So it's all yours. Good luck. OK. First of all, thanks to everybody for coming. This is my first talk ever. So please be gentle. I'll be going over the django testing, especially using PyTest and hypothesis for test generation. We're going to see how we can make thousands of tests with relatively small code. Now, my name is Bojan Miletic. I'm the founder of Terrific. I've been working as Python developer for more than a decade, and I absolutely love talking about Python. So you can connect with me on LinkedIn or on email. Sorry, but I'm not very active on other social networks. So during this presentation, we're going to cover PyTest, PyTest django plugin. And after that, we're going to move to using PyTest for optimization to generate tests. After that, we are going to add hypothesis into the mix and see how far we can go. The main point to mention is that we're going to be focusing on PyTest and hypothesis. But this is something you can apply to basically everything. Any kind of Python script function, we are talking about in the context of django, but it's pretty much usable everywhere. Since you only have half an hour, we are not going to go much into the dev because PyTest, PyTest django, hypothesis, good practices and stuff like that on its own will take much more time. So I'm just going to give you a brief overview of the things and hopefully get you interested to start discovering more things on your own. If you have any questions later, feel free to ask me. I'll be on the channel and let's get started. For this example, I decided to make a website for keeping track of Unicorns. Basically, we're going to add new Unicorn and list the Unicorns. We're going to focus basically just one endpoint. And I believe we can generate tons of huge number of tests for that endpoint. Before testing some of the good practices, that you know what you're building. Sometimes when you're a programmer, you're going to get a specification and clearly defining puts, outputs, and that's perfectly fine. But when you're doing project on your own, take some more time before jumping into the code. It can save you quite a lot of time. Also, since in this project, we are going to be working as REST API, you want to write API specification and then consult your front-end team or some other person. When writing API, it's quite helpful to have multiple persons. And once we have API specification set, then you know what we need to do. As you can see, here's the example of the endpoint we're going to be using. It uses just two parameters, color and the name for the Unicorn. And it returns status code 201 when it has successfully added Unicorn to the database or 400 if we made something up. What we're going to do is make sure to generate as much as possible 400 responses. Basically, try not to crash the server and get some 500 error. For a Unicorn model, it's very, very simple. I try to focus mostly on the testing side which made my jungle state a bit weaker. We have one item for the colors and then some limitation, pretty simple. So the name must be at least two letters long and it marks 30 letters long. Now, when writing tests, big, big number of tests is going to be located on the domain limitations of specific value. For example, in the name, if there is some kind of error, it's going to happen either on a character that is two characters, the text that is two characters long, one character long, 30 characters long and 31 characters long. Also, we're gonna test the empty strings. Those are some extremes and they cover 99% of the errors that happen. It's quite unlikely that we are going to have some sort of error with the string that is length of 10. So always focus on the extremes and now we can start with our simple test. In this first test, we're sending just empty data for that to expect our backend API to return us a simple error and tells we're not doing fine, error 400 invalid request. Ideally, when you're returning requests, you wanna provide some text in the body of the request. Basically tell your front-end friends what they need to do to fix this. But since we are doing as minimalistically as possible for this, we're just returning status code, nothing more. And the next one is basically checking if the client is, if the unicorn is successfully added. Now, one thing I will mention, you can see that we have some nice fixtures in PyTest. That's the client and client DB. Now, that's all thanks to PyTest Django plugin. Client will allow us to basically use Django as we would use the normal request library or something like that. As you can see, I'm not calling the view function directly but using the URL, which is quite handy. And the database fixture in the second text test is basically telling us that this test will need access to database, otherwise it's going to crash. So those are the two fixtures that you're probably going to use the most. On to the next things. And here is our simple view. Basically, I created a class based view since in normal situation, we're not going to have just post. We're going to have get delete options and a bunch of other stuff. But for now, just post. If you can see my validator validation here it's basically just checking if dictionary is empty. If yes, return HTTP 400 by the request, which is perfectly fine because my test here is passing. And afterwards I'm just saying it and it's all good. When doing TDD, it's very important to not get ahead of yourself. I love writing tests that fail because when you write a test that passes right away you don't know if your test has any errors in it. If you're testing with drone or if everything's perfectly fine. As much as it is to the back when you test crashes then you fix the code, okay? And now we're going to use PyTest Parallel Trace. It works quite simple. Basically just type this, give it the name of the parameter which is going to be parameterize. Here it is key and the list of values. Now, as you can see in this example here it's very, very basic. But the idea is we don't want to repeat the code. We could easily split this into two tests. One that has only name in the request and one that has only color in the request. Like this, we only have one test. It's quite trivial in this example but when you have a much more complex request structure it's going to save you quite a lot of time. Almost regarding the PyTest Mark parameterize you can supply multiple parameters that you wish. Behind the key as a first parameter you can just add comma and list the second parameter. And instead of the list of values it's going to be a list of tuples that are the same length as the number of parameters. Okay. So next thing we're going to do is once we edit test that validate structure of the request is basically try to send some junk data to our endpoint. Because also a good tip when testing is that majority of your tests are going to be located at the borders of your system. That means you're going to have to extensively test your API very receiving response requests from the user. And the third party libraries that you're using your sending request and receiving responses. Since in majority of the cases we're going to get the world stuff from our frontend team which would be using this API. But they might get something wrong or some kind of malicious user is going to send us some crazy stuff that we're not expecting. For that we need to try throwing everything at the API and hopefully getting just error 400 which is user request is invalid. What we don't want to do is get error 500 which is a series of errors that correspond to server crashes and stuff like that. Basically we want to be able to graceful handle every exception possible. Now as you can see we got from here three tests above from this we got two tests. So that's five tests and we just wrote two. And it's time for hypothesis. Now hypothesis is a property based testing library. It uses Haskell as a starting point and it did quite a lot of good stuff that we can use. Basically in hypothesis you specify your strategy. Here we are saying name is going to be a text and the text is going to be a minimum size two and max size 30. The same thing we got when we defined the field for the name. Now usually what's gonna happen is we could write for example in PyTest parametrize just pass the range of values as the string length from two to 30 and basically multiply character A or B by that number and get the string length which is good and which is going to work but like almost every single web input that requires text, we're gonna get Unicode and hypothesis is gonna provide us with a bunch of crazy stuff, ASCII text, Unicode text, characters, involuntary formatted skeinputs and what we want to make sure is that we get status code 211 that everything is set in the database as it should be. If you're doing some preprocessing lots of time this is where we're gonna catch Unicode errors with the text. Okay, now that we have successfully tested this case we wanna test all the invalid cases as well. For example, we can set up one strategy for text that is using maximal size of string one. It basically go into zero and one. It's not as quite complex in this case but I'm just asking for a little bit of abstraction because you can apply this to pretty much anything. Also, okay. Next thing, we are also going to add tests that are outside of maximum size. That will allow us to test also bunch of crazy stuff. Now, the reason we are limiting the size, we're not putting 10,000 or some other stuff is because in SQL when you define a field it has a fixed size in memory. Okay, we did some basic, basic overview of hypothesis by this and by this parameterize. Now, I'm open to the questions. Feel free to ask me anything and that way I can go more into depth regarding concepts that interest you. So for anyone who wanted to ask questions there is a Q&A bottom and they need to click there and ask the question. It's also possible if you click in the right hand and the kind name will allow you for the question. So if anyone want to ask, that's, now is the time. So thank you very much. Okay. Okay, any questions? My presentation was so good that nobody had any additional questions. You were, seems that you were really, really good. We have a couple of questions. We have two now, we have two now. So one person is asking what is the given decorator? Oh, yes, I can explain that. That's a very good question. Given decorator is provided by hypothesis you're basically telling here, like we did here with PyParameterize we're telling this function. We're gonna supply multiple values for the name. And here we are saying, okay, hypothesis is going to supply you multiple values for the name and generated by default. This is going to supply you 100 different kinds of texts that correspond to these parameters. There's, you can use another decorator about this to set it to a higher or lower number. So this is the hypothesis decorator. Okay. So the other question is, what's your feeling or what's your thought about PyTest versus PyUnit, if you have any opinion? Well, when it's a difficult question for me because when I started, I was using a unit testing and it was pretty much similar like in other frameworks for me. But when I started using PyTest, it felt more petonic to me. Tests are much prettier. And I liked the fixtures and bunch of other stuff that gets provided here. So I prefer using PyTest mostly because it's my kind of the style of programming. I don't think you can get along with unit tests as well. An important thing is to have tests. Find the framework that works best for you. One plus side for PyTest is that it has plugins for pretty much anything. I'm not sure what's the situation with the unit tests. Okay. So next question is, who do you define the number of cases the hypothesis has to generate? Oh yeah. Above given, you have another decorator which I don't know on top of my head. I'll type it in the chat after this. You can set it's at settings, I believe, and you define number of how much you want. When running PyTest, you can provide additional argument. Dash dash show hypothesis, the script statistics and it will show you how many tests it has generated for you. So please write me in the chat channel and I'll send you the code examples and we can discuss further if you like. Okay. So I was just going to repeat that. There is a talk Django testing and I sure that that's the channel that you can follow the discussion. So one more question. Robert is asking, do you have any recommendations for applying tests to an existing code base? Yes. Now, this is something that happens quite often. You get the code base that has absolutely zero tests. First of all, before you even touch the code, you are not going to start with tests like this. You're going to have to start with functional testing. Basically start large, test entire system and outputs to make sure you don't break anything, large functionality. And once you get that down, then you can start the moving code around. Then you modify a little bit of code, write the test for that. See if anything breaks down. Don't start with the functions, start big. That's the way I did it and it usually works. Because that way you discover, oh, in my local setup, I'm missing this sort of library. They did something crazy in the production. I also need that. So first start with the large functional tests. I would even go so far to recommend Selenium for that. And then get smaller and smaller. It's when you usually get code base that doesn't have any tests. Coding standards are not quite high. So you might get a function that has, I don't know, one tons of code. The max that I saw was 5,000 lines of code and 45 input parameters. Stuff like that, you cannot start to break apart until you have dozens of tests that test existing behavior. And only that, you can start slowly breaking down into smaller pieces, okay? Cool, thank you. So I have one question more from Discord. Javier is asking, in case you need to test with more complex input, like a self-made object, how hypothesis handle this? If it's a Django model, hypothesis has a very nice support for that. You basically just tell it to use this model and it will generate a random values for that model and supply it. You can also make your own strategies for generation, link multiple strategies together. For example, if I wanted to add more unicorns here, I would use, I could use a name and color as input parameters for hypothesis. Okay, good. So we have more questions, so I go to continue because we have a few minutes. Jennifer is in, okay, seems that she missed some part because she was a three-year-old person interrupting. So how did you generate the 400 tests? Here, well, when you say given, that decorator basically tells, okay, I'm gonna turn this, provide, I'm gonna use this strategy to generate various values for the name. And by default, it's 100 values. So this piece of the code is currently on the screen. That is 100 tests. Yeah, you can set it up above and it will generate 400 various tests. Now the text field, which I really like in the strategies for hypothesis, will try to break your code in ways you didn't think it was possible because it will throw a unique code, asky characters that are invisible, stuff like that. Also in our reformatted unique code, basically all the crazy stuff you can expect from your users. I hope that answers it. I think it does. Okay, last one, last one because we already like 10. Daniel is asking, he always has the impression that testing something a lot of times doesn't do much more than be the few careful thought like with the standard normal test. And he's asking if you have an example when hypothesis was catching an error. So when hypothesis has code edge cases, like because you were doing like a really big set of tests. So I think that's a question. Sorry, it was like maybe changing it a little bit. Yeah, basically, I kind of agree with his approach when you're doing testing inside your own system because then you are already, my philosophy is this, you have user input, you have one giant wall, you're gonna do lots of data validation and stuff like that and that's going to be most of your tests that are going to be concentrated. Inside your system, you can trust it to provide the valid inputs and stuff like that. So there you can use carefully designed tests and not treat it as a black box. If you can see here, I'm never calling the unicorn view. I'm treating it completely as a black box. So from my standpoint of view, if I didn't use PyTest.junk, this could be a flask application or something in Java, anything at all. Does that answer the question? Yeah, I think it does, I think it does. Okay, thank you very much. That was a really nice talk. So I need to play some because. Whoa! Thank you.