 So I'm Emma and I'm a developer advocate and I really enjoy sharing my knowledge with the community As well as learning from fellow developers My talk is called property-based testing the Python way and this is an introductory talk to To a more effective approach to testing, which is called property-based testing So it's not expected that you should know about property-based testing Like in order to attend my like talk as well, but as well as No prior knowledge about testing Is required, but you can have some basic knowledge about just traditional testing about PyTest I think that will be very helpful So I hope that you will get curious about this topic and Let's start So this is the agenda So I'll talk about what is property-based testing its advantages and disadvantages The tool hypothesis Which is used for property-based testing And the takeaways from my talk I'll conclude with that So here's the problem that we have our problem is a game which is called the game of hog and the problem is So the game is the following in hog We have two players, but this is the simplified version that we will actually use it as a problem and Here we have just one player who rolls two dice And the score for the turn is the Some of the Like outcomes of two dice Um, so there are also for example if it rolls like two and three then the total score for that turn is five Uh, and there are two rules that apply to this game The first one is if any of the dice outcomes is one, uh, then the total score For that turn is one. So if it rolls like one and four the total is not five, but it's one or if it rolls and The values are like the outcomes are four and one again. The total score is one And the second rule is that uh, if after rolling the player's score is a prime number, uh, then they gain enough plans That that like their score, uh increases to the next prime number And here we're assuming that the player starts at zero So they have zero score at the moment and then they're rolling the two dice And if they get prime number as uh as a score for their turn, for example, if they roll two and three and they get five Then their score Automatically becomes increased to the next prime number and it becomes seven So, uh, our problem is actually to uh, we have the implementation in python, uh for this like small game and our, um, Like task or let's see the the problem is to actually test these Functions that we have and test tested and check it for their like, uh expected outcomes So this is the implementation in python. We have two functions. So the first one is Is the function for uh, the the turn, uh, where The the function calculates the total score for the turn and the second one, uh calculates, um the next prime number, uh, that is Here we have because uh, we were also like assuming that the dice, uh, this is our dice are like, um Six cited, uh, so the the outcomes that we can have, uh, in terms of like prime numbers are one two three five like seven and 11 Uh, and uh, these are the cases. So now we our task is to actually test these functions, right test for these functions um So, uh, if we let's see write it in py test like using py test, uh, we can actually, uh, Write like separate, uh functions for each test cases. Let's say, uh, Let's let's check if the outcome is two and three. Uh, is it five or not the expected outcome if the input is a list of two Consisting of two, uh, and three, uh, then let's check for another two so we can write, uh Like separate functions for each of those cases, but it will be very like, uh Like time consuming task, uh, and uh, the alternative is, uh with py test is, uh Parameterized approach of parameterized testing Uh, so py test has this function called, uh, parameterized And py test parameterize makes test writing in py test more powerful because it helps cover more Scenarios by passing test data as parameters, uh to our test function and then a certain expected outcome um So in this, um parameterized case, uh What we have let's uh, let's see. So, um, for if you want to try this for yourselves You need to have, uh, like python py test and hypothesis installed on your machine uh, so firstly we're importing py test and, uh, we're, uh, using the function parameterized from py test uh, we have our, uh, like, uh function that we want to test. It's called test score and, uh You, we write this function and then it comes like, uh after, um The py test decorator which, uh, or from which we actually call the parameterized function And so the approach here is that, uh, we're giving some fixed input input data and we're expecting some, uh fixed output Here we're giving, uh for, uh Like a table, uh, a list of four and one and we're expecting one Uh another case the test case, uh two and four. We're expecting six, et cetera We can have many test cases in one function So, uh, this is a better approach than just writing like separate functions for each test case because it's either like shorter and Also faster But if you imagine that, uh, if we also are like other like test cases as well It will become just like a huge like long list of, uh, like test cases and it won't be readable at all Uh, also this has other costs, um as well, um and, um The the concert that, uh, for example if this test fails, uh for one like test case, um, then we will, um, claim that we'll like, um conclude from this that, uh Our claim is false, but if all the test tests, uh, we can say that our, um, claim is Like correct. We can just say that all the test tests are our like function work Actually, we're making what we're doing here. We're just making a strong claim. We're saying that is using specific like examples and concrete scenarios. We're, um Making the general claim about our like test function and about the behavior of our software And so this is not very like precise and, um, you also see like there's some like details, uh in the test and so This is uh, this is a column of this like approach. Um, and, um It's not very effective to test this way to actually generalize from a specific example and From a concrete scenario, uh about the behavior of the software because we we may go like long Also, uh, here as you can see, um another like, uh, conus that, uh, a lot depends on us on humans. So we should actually, um, like, um do this like manual task and, uh, Think of all these like test cases and, uh, write all of these and we might, um We miss very important like edge cases, for example and so, uh, we can't actually conclude from this that, um like everything or, um, maybe, uh, the most important things about the behavior of our software and system And so, um, this is this has, uh, like many, uh, like job ads, um, this approach Um, so in this case our function, uh, will execute execute, um for given like values and inputs And this is the traditional approach that most of us use, uh, but as I said that, uh, along with, uh, like all these cons, um There is also another thing that, uh, the code is long and it's also a difficult to debug and so, um Here, uh, comes another like a better and more effective approach to testing, uh, which is called property based testing So property based, uh, tests, they check the truthfulness of the property, um, of the software using concrete scenarios and they generalize them abstracting the way the details So they allow us to design cleaner tests, uh, that actually specify our software's behavior better um, and, uh property based testing has, uh property, which is a statement that, uh, we want to, uh, check us, um, that we, uh, actually claim, uh, to be true, to hold, uh, to always hold true And we're interested in, uh, actually testing and like checking, uh, for, uh, for, um, for a property of the software And we're actually interested in the truthfulness of that claim So, uh, if you compare this approach to, uh, to the previous approach, like the traditional approach, uh, we will see that here we're actually, uh, doing the opposite. We're, um, focusing on the general claim And this is more effective because, uh, when we're focusing on the general claim, uh, we can better understand the system And how the system works and how the software behaves If we actually want to, uh, uh, like, uh, check for for our general, uh, claim and, um, So, uh, when we're checking for, uh, for elements of a property that should always be true, uh, rather than actually writing, uh, like, um, each, uh, test, like, um, each, each value, uh, that we want to test, uh, for, um, we allow the program, uh, and, uh, to test, uh, for a variety of input values in a single test Uh, so in property-based, uh, testing, uh, we are actually generating the input data. We're not specific. We're not giving us fixed, uh, data, test data. So, uh, when we're, um, Generating this data, it's very, like, uh, random, but at the same time it's predictable and we'll see how Um, and so property-based, uh, testing, um, Uh, if we, like, compare it to, to the previous one, like, example-based, uh, testing, it, uh, as I said, focuses on the general, uh, claim, uh, and when it, uh, abstracts away the details, uh, it becomes even more powerful, uh, than, uh, the parameterized one, uh, and, um, here we can, uh, see, uh, the advantages. So, uh, the focus on general claim, uh, as I said, it helps, uh, it helps, uh, debug, uh, the code, um, like, more easily. Uh, it also has better implementation. It helps us understand, uh, the requirements better. Uh, in parameterized, uh, approach, uh, we're not able to understand the requirements better because we are actually, uh, specific examples that we want to test. And when we have here a better implementation, it helps us, like, uncover the assumptions, uh, the hidden assumptions, and it helps us understand the requirements better because we're focusing on the general claim. We generate the input data. So, when we generate the input data, we're testing for a variety of inputs in one single test. Uh, we're actually, uh, covering, more domain space. And in that case, we're also covering the edge cases, like, automatically, uh, versus in, uh, like, parameterized cases might, might miss some, like, important edge cases. And it has less amount of code, so we have a cleaner case to it. So, uh, actually, the main, I think, about property-based testing is, uh, automation. Automation of the process of, uh, specifying every single input, uh, for, for an expected output that we humans do. And we're, uh, just automating the process with property-based testing. So, we're letting the computer, uh, do that test for us, and we're focusing on more, like, creative, uh, part of, um, understanding the software behavior and understanding, um, all of these, uh, like, uh, like, uh, the, the details and why the test failed or, um, all these, like, edge cases, um, like, because the, the, the automation helps us, uh, and, uh, again, we'll see how, in more details, um, later. Um, so, um, uh, as I said, um, uh, so, when we have, like, cleaner tests and, uh, when we, uh, better understand the software behavior, um, in that case, uh, we can actually, uh, like, suggest, uh, better, like, uh, suggest better, um, like, solutions for our, like, uh, functions for our, like, problem, and, um, we, we want to do that, uh, with a tool, uh, which is called hypothesis. Actually, you can do property-based testing if you've ever, like, automated, uh, testing, uh, like, process, uh, you've done property-based testing, uh, but, um, the tool hypothesis really, uh, helps, uh, helps in that process. Uh, so you don't, you don't have to do it with a tool, but a hypothesis, um, as a tool is really helpful, and, uh, the focus of my talk is actually hypothesis. Uh, hypothesis is, uh, this paper library, designed to help you write property-based tests, and you can pick and store hypothesis, and you can, uh, write, write the test and run the test, uh, with pipest. A hypothesis, uh, doesn't have, um, it, like, isn't a test runner. It runs in, um, in a, uh, testing, uh, framework. Uh, you can do it in your, like, testing framework. It can be pipest. It can be unit test or other. Uh, we will do it in pipest, um, so you can actually run it with pipest. Uh, hypothesis is a very good tool, and we'll see, uh, what, uh, advantage is, um, it has, um, and, uh, so it is, uh, like, for the python implementation, if you wanted to do property-based testing in python, uh, you, you can use hypothesis as a tool. Um, so, uh, here's the code snippet for our, uh, like, first, uh, function that we had for, uh, when we actually want to calculate the total score, uh, for the turn. Uh, so what, uh, what, uh, we need to do, we need to, uh, import from, uh, uh, hypothesis, uh, the, the, the things that we need, that those are, like, the given strategies, and I'll explain, uh, those, uh, later. Uh, so, um, yeah, I want to, I want to focus on, uh, the, actually, uh, the things that hypothesis offer, uh, offers, and those are, uh, the first things are, uh, the strategies. So, as I said, in property-based testing, we're, um, uh, generating the input data. So, uh, with hypothesis, you can easily do that, uh, because it has strategies, uh, where you can actually call this, uh, which, uh, helps to generate, uh, the sort of data that you want. For example, if you, uh, want integers, you can, uh, call this function from strategies, and it will generate a range of, like, integers for you, and you can specify the range. Uh, if you, if you want a list, you can do lists, there, there's also, like, text, um, et cetera, uh, booleans, et cetera. Um, so, uh, once you have, uh, like, the hypothesis, um, once you have, uh, installed and imported all these things, then you use given, which is the decorator. Uh, you use it to generate the data, uh, in the brackets, uh, you call this, um, like, uh, functions. Uh, here, we're generating lists of integers. And, um, then you write the test function, uh, the function that you want to test. And so, if you were, uh, writing the test core, we, we're interested to, uh, test that, uh, function, uh, which calculates the total score. Uh, so, as you can see, when we're running this test, uh, we are getting a falsifying, uh, like, example, an error, assertion error, it's written. So, so, the, the test actually failed. So, it's written falsifying example, l equals 11. So, what does this mean? Actually, what hypothesis did, is, so we're, uh, we have a claim here. The claim is that we want to assert that, uh, that, uh, where we, like, call this function, like, take turn, that it calculates, uh, the, the total score, it should equal to the sum, yes, of the, uh, the, the, uh, outcomes of, uh, of the test. And, uh, it just failed, because, uh, we didn't, um, actually cover this, like, hk11, where we have one, it's not, it shouldn't be, like, two, yes, it should be one. Uh, the interesting thing about hypothesis is that, if, um, if we run it, like, once, uh, and, uh, the, uh, the, the, the test, uh, passes, then, um, uh, if we run it again, it might get, uh, it may be in, like, pass, uh, and if we run, uh, one more claim, uh, it may, like, pass, but, uh, that doesn't mean that, because it automatically generates lots, like, uh, various, like, uh, data, like, every time different, uh, like, uh, like, a range of data, uh, that doesn't actually mean that it's not going to fail at some point. And, uh, we, we do it, uh, over and over. And for example, when it fails, then if we run it again, we run it, it will fail again. And, um, uh, it will fail again, and it will do the same error. Uh, and the, the thing is that hypothesis actually starts this, like, error in a local database. So when we run it again, it gives the same error. Uh, it gives this, and it will do the same error until we fix it. And so this is, uh, some repeatability, which gives some predictability. And so, uh, yes, hypothesis is random because we're generating random data, uh, but, uh, giving us an, we give, uh, as an input random, uh, data every time in one test. But, uh, when it fails, it actually, uh, lets us know that, uh, we, this is, this is an important, like, case two, two, to fix. Uh, and so, uh, we, we know that if, uh, if we don't fix it, we will get the same error. So it gives the same error. And also, uh, as you can see, um, this process is also called, like, simplification. Uh, it, it gave one error, but it kind of simplified the process. There are so many others like as well, because it's like randomly generating, um, inputs, uh, but it simplified the error and, uh, it reported us. So, uh, the reporting simplification and, uh, repeatability are also important, uh, things about hypotheses that, uh, the tool is, that make the tool, uh, more powerful. Um, and so, uh, yeah, why choose hypothesis? Uh, as I said, easy to generate data with strategies and tools. So, um, hypothesis has the strategies that you can use to, uh, you can generate the input data. You can also, it also has tools that you can use to, uh, actually create your own strategies. It isn't too light. Uh, it would be, yeah, as you saw, that it's like, um, less, less line of code, uh, compared to, uh, parameterized one, um, uh, as the property says, but also, like, what is nice and easy, you just use the decorator, you give the input data, and then you, under it, you, uh, you have your test function. Uh, simplification process, as I said, uh, bars. It helps us to actually, um, think about our, like, general queen and understand the truthfulness, um, uh, of it. Um, so, uh, another is, uh, that it is robust to finding edge cases. So, if only edge cases, one, one, it can, uh, it can be like, um, an empty string or an empty value, et cetera. These are the edge cases, and it's very robust at finding those and letting us know that, uh, you know, this is, this is an error, you should fix it. This is important. And so here, compared to, like, the traditional approach, we're actually, um, uh, it's, it's, it's, it's actually, um, helping us to understand what, what is more important and what is less important, uh, because we don't have too many details. Um, but, um, the thing is also, uh, that, um, it has, uh, some calls as well, like property-based testing. Uh, I do think that it's way better than the traditional one because it has so many, like, advantages. But, uh, there are also calls about it. Uh, the first one is that it's more difficult to debug, uh, when you have a rent, a rent, uh, a huge rented inputs. In any case, it's, it's, it's hard to debug because it's, it's randomized. It's predictable. You can understand what errors is going to, you're going to get, which like high probability, but it's still, um, it's a bit difficult. Uh, it's difficult to design, uh, because it's a lot, uh, the creative part is, um, is like, like, the creative part is up to us and depends on us. Uh, uh, the automation is, um, for the computer. Uh, it's, uh, slower to run. Um, and it's not always the best choice. Uh, of course, uh, you can't use property-based testing in any, um, in any case. And therefore, uh, like every problem, uh, in some cases, when you just need to put, um, some like very small, like fixed, uh, uh, like input, uh, for your test function, uh, maybe, uh, the traditional purchase pattern, but, uh, property-based, uh, testing, um, can be used for many, like, uh, problems and, uh, many, like, functions. Um, and so it's not always the best choice, but it's highly, like, recommended. Um, um, thank you very much. Uh, and if you are interested in this topic, in hypothesis, if you're interested in adapting it to your, uh, like code base, uh, or if you're interested in exploring, uh, this topic, uh, just feel free to reach out to me. Uh, I will be very happy to get in touch with you and, um, and connect, uh, because I'm interested in, uh, further, uh, researching this topic, uh, and I hope you got curious about it and you enjoyed my talk. Uh, and if you have questions, um, feel free to ask. I'll be happy to answer. Thank you very much. Okay. Does anyone have a question? If so, please could you go to the front and line up? Okay. Thank you. Yeah. Thanks for the talk. Really good to see something like this in action. Um, what I'm wondering about is when you design your tests, what's your thinking and like approaching the tests for like something that, that's probably fairly complicated even. So you're testing a complicated piece of business logic and now you want to write a test for it. That's generic. Obviously the go to or the pitfall is I could now write my business logic again implement the same thing in my test. Uh, what's your approach there to make sure that doesn't happen and how do I test instead? Uh, thanks for the question. Uh, as I said, it's a bit difficult to write in this like approach. Um, um, I think that, uh, the most, uh, I think that the tool, uh, tool actually helps in, uh, you know, like, uh, well, like when you test, uh, for, uh, for that, like when you write a test and then, uh, like actually check, uh, for the truthfulness and you, uh, you, you get, uh, like you get some, uh, like, uh, outputs then based on that you understand how to further improve it. Uh, I wouldn't say that it actually hurts the business logic. I say it actually helps, uh, to better, uh, like to, to improve, to improve it, to better understand, uh, the, uh, software's behavior. And my approach is, uh, I do, I do think it's, it's a bit difficult, uh, to, uh, to actually test a general claim like focus on the general claim. Uh, but I think that if you kind of start like small and, uh, test like some small, uh, like pieces, uh, of the, uh, like business logic, then, uh, then, um, expand from that, uh, it will, it will be like less circle. Um, I don't have a one, like, uh, specific strategy, um, for it. Uh, I just, um, try to, uh, better understand, uh, different, like, approaches and, um, like about this, uh, actually testing, uh, method. Um, yeah, I think that, um, probably, uh, if I answered, I hope I answered, um, yeah, thank you. Yeah, sure. Do we have a question remotely? No. Okay. Then go on. Hi. Uh, Ghostwriter is, yeah, um, it's been a part of, uh, hypothesis for, for quite a while. And Ghostwriter tries to write tests for you. Have you ever tried Ghost, Ghostwriter? What are your experience with it? Uh, I haven't tried it. So, looking at the input generation part of using hypothesis, it seems to me that this approach is a lot more applicable if you want to test functions that operate on numerical quantities and maybe output quantities. But if I had, uh, a routine that processes, I don't know, dictionaries or strings, and if I wanted to, I guess, declare how those types of inputs are generated, it wouldn't be as scalable. That's my impression. Is that fair? Or maybe I'm missing something. Yes, uh, you can. Of course, strategies, uh, do have, like, uh, different, like, sorts of data. You can generate different sorts of data, not just, like, integers. But, uh, I think that, like, uh, probably for, like, integers, it's more, like, common as well as, um, I mean, probably, like, easier. But you, you, you, you can also, um, do for other, like, sorts of data for any sorts of data. Hypothesis allows it. Okay. Thank you. We have time for one more question. Thank you for the talk. Uh, quick question. How do you reconcile the random nature of the space exploration of the input space exploration with the, with continuous integrations need to have stability? Um, can you repeat the question? Um, you have, uh, when you run hypothesis, I understand you randomly explore the input space, uh, which is fine for me as a developer locally, but because, as you say, you have a database where it stores the space that I can reproduce it. But what about on continuous integration? What if I'm running this where I'm, you know, in the cloud, CI where I don't have that database persisted between runs? Is there a way to stabilize it so that I don't get a test failure today and then same test passes tomorrow and then again fails the day after that? Yeah, this is a good question. And yes, you can. Uh, it allows, like, you can adapt it to your code base and any kind of, and, you know, it allows for continuous integration. And you can do that. That's possible. Uh, but that, and that's also a part of, like, I further want to, like, research that, that how to adapt to your code base and, uh, yeah, how to actually deal with that problem. But you can, that's possible. Thank you. Okay. Thank you. Um, here's no more question. And otherwise, if there are more questions, um, will you stay in the breakout room for more questions? Yes, sure. Perfect. Then, thank you for your talk and, uh, let's give her a warm applause to the end and.