 Hi, good morning So the first session today is going to be introduction to pie test with Andrea's film who is also the maintainer of the Pytest Django library, so Yeah, thank you Hi, so I usually I work at a place called personal column we build a time tracking and the seller application and In order to manage that we write a lot of tests and we use pie test so today I'm gonna try to show you some Good things with pie test and how you can make use of that when you write your tests So these are kind of the topics I want to cover today I'm gonna show you what pie test is I'm gonna show you how to how you can write tests with less boilerplate code I'm gonna show some useful plugins and the plug-in system of pie test And I'm also gonna discuss discuss how the test discovery is done And then I'm gonna show a feature of pie test is called fixtures Which is really cool? And then I'm gonna discuss how you can port your existing projects to make use of pie test in an easy way So first testing the kind of testing that I'm gonna talk about today is Automated software tests that verify that your software works correctly And this can be anything from small fast-running unit tests to more high-level tests that Test your entire system and or anything in between there So before we get started how many of you are familiar with the unit test module in the standard Python The chips with Python Yeah, you've used it you can yeah, you know roughly how it works. Okay, good And how many of you are already pie test users also or have used pie test Okay, let's go so Pi test is a full-featured testing tool for Python. So that means it does everything from test collection to run the test to give you the output on whether which test failed and passed and And it also has some other nice feature that help you maintain a test suit and help you organize it in a good way and You can run anything from those short unit tests to more system level testing with pie test It has it scales like in both directions there You can also use it if you do a TDD fits very nice or And it's easy to integrate with the continuous integration systems such as Jenkins that works out of the box So here are some people that are happy with pie test and it's used in some Also some bigger organizations in both commercial and open source projects so One important one thing that I find really nice about type pie test is that it allows you to write tests with less boilerplate code And to show you what I mean by this I'm gonna show you a little demo So, let's see. So I have a test file here Which contains a very very simple test and it uses the can make it bigger Yes, it uses unit tests. So this is probably familiar to most of you so we basically asked Make sure that the the return value of this function is what we expected to be And I can run this with By invoking the unit test module and it collects the test and runs it So if I instead use pie test what we do is We invoke the test shoot from the pie dot test command So if I run that it finds the test and it runs it The output is output is slightly different, but it's the same thing So I want you to notice that pie test can run existing unit tests Just as they are But we're gonna change this test to make it To show some features of hi-test so the first thing I'm gonna change is The assertion instead of using assert equal. I'm just gonna use the plain assert statement. That's built into Python So I'm changing this and Just use the equal operator to make sure that the output is the expected one And the next next thing is that we don't actually longer need to subclass from the unit test test case So we can simply ask to move that And in fact, we don't need to we're not anymore required to wrap or all our test cases in Class so we can just remove the class too Since we don't have the test case anymore, we can just remove the unit testing port. So this is what we're left with So let's just save this Run it So it works the same I just want to show you what happens if we get a failure So it's really important that you get useful feedback on your failures when you when your test does not pass So let's assume we introduce a bug here. Just add some garbage to the output and we save and we Run the test again so what you can see here is that You get a very nice detailed output and you even get some colors in them That shows you exactly what the wrong value was and where it failed So this was the code before and this is the after code So what I did was I used a search statement instead of search equal And pie test can handle pretty much anything you throw at the search statement is it's very smart it can You if for instance if you check for a dictionary key or something like that and it doesn't exist It will give you a very nice output. So It's hard to find a case where it doesn't work. Well Also sub we did not have to subclass from test case anymore and we didn't have to Put our tests in a class at all but we can still do that if we like That could be can be kind of nice to to group your tests and We invoke the test suit by running the pie dot test command which collects and runs the tests so one thing about pie test is it has a very powerful plug-in system, so In your own project you have a lot of hooks into pie test that you can use to customize how tests are collected How they are picked up and how they are invoked And you have the same kind of hooks into pie test in your own project that Third-party plug-ins has access to so these are some plugins that extend the capabilities of pie test There are a lot more, but these are the probably the most popular ones The pie test exist Plug-in it provides distributed testing so you can actually run your tests on directly on have them distributed to remote machines or you can run them locally on your own machines in Parallel processes to have some nice speed-ups I'll come back to X this later And there's the pie test Django plug-in which helps to integrate with with Django and with Django test suits and Make them run directly There's also Support for running twisted plug-in twisted tests And there's a lot lots of other plugins in for instance if you do log capturing you can install the capture log plugin so How to actually run the tests I showed you in the demo by just invoking pie to test you run all the tests and pie test will recursively search all directories To find test files, so this this is usually what you do to run your entire test suit But then you can also limit the test you run in one invocation By just specifying the file names of the test files that you want to run And this of course works very nice with the standard like tab completion in in your shell So it's very convenient to use You can also match you can also select tests based on the test's name so K stands for keyword and This can be used to find a specific test if you don't want to type the directory so as I told you pie test Recurses into directories and finds test files and by default it looks into Files that matches these patterns and you can of course change that if you like if your project structure looks different But that's a kind of same default There is another way of Organizing your tests with pie test you can use the marking functionality So you can make up arbitrary marks. I just made up this slow mark For instance, if you want to mark your all the tests that are very slow And then you may not want to run them or you may want to run them You may want to filter them out you can put any you can put multiple marks on one test If you like So this is how you run only the tests with that specific mark and In this case, this might actually be more useful because you may not want to run them that as often There are also markers that are built into pie test. For instance the skip if marker So you can also annotate some extra data within the marker for each test So in this case this test will simply be skipped if you run it on Mac OS But you can implement you can implement your own Logic with markers by yourself. It's in in your project specific things to your project so I'd like to take some time to go a bit into a feature in pie test that is called fixtures So fixtures is a mechanism for injecting Objects that you need for your tests in a in a very structured way You probably have no idea what I'm talking about right now, so I'm gonna try to to give a demo of this so We have two tests here which use as any do you know what selenium is by the way Yeah, it's so it's it's a Way to remote control a web browser and that can be very useful for tests So I'm using that on us as an example here So I have two tests. They will navigate to these websites They will look at the title of the websites and verify that it That nix is in the nix OS nix package manager website and that pie testing is in the pie test website So I'll walk you through what happens here. So when pie test finds this test with the web driver argument in the argument list it will Try to construct this a value the value for this argument So in order to do that it looks for any fixtures that are named web driver So it works by looking on the name of the argument itself And in order to register how these fixtures are constructed you use you create a function with the same name and Then you annotate that function with the the pie test fixture decorator so The this function should return the object that we want to have passed into the test So in this instance, I just create a firefox web driver instance And I also add Some code that will run In the processor of the tear down when the tests are finished how to clean up after this test in so I add this Driver quit it's a method that will be invoked after the test and then simply return the driver so This is what happens for the first test and then for the next test the same thing will happen It will be constructed once again and passed in and then the test will run and then the web driver will quit Okay, so let's let's just run this and see what happens and the hope for that the wireless are with me Okay, so we can see Okay, so it seemed to work. Well it was very fast so and We had firefox start up close down and start up for the other test. So that's what that's what we expected one very important point of fixtures is that You move the the dependency construction out of the test case themselves so the test case is not concerned about how the the Test fixtures are set up and this has some nice Implications so we can for instance You saw that firefox started twice Let's say we want to we want to use the same instance on firefox for both of our tests That is really simple and actually everything we have to do is to add We had have to tell pie test how this Fixtures should be should be scoped or how it should be cached So we just add this and pie test will then cash this fixture object for the entire session test session So this means that if we try to run it again It might be very fast, but we should only see one One firefox window showing up. Okay. Oh, yeah, it was very fast, but you see it it It can certainly speed your tests up Very nice For in a lot of cases for instance creating a database Let's imagine your tests Need a database with the schemas and everything in them then you can just create that once before all tests run and then have it reused for every test and That the other thing with when we separate the creation of the fixtures is that These tests don't need to be concerned of what kind of browser we use we can for instance change this to chrome and We'll have it just work and we can have even generate More tests in vocations. We can generate new tests by parametra parametrizing this fixture So we can actually trigger each test to run in multiple browsers just by changing this fixture definition So another thing I want to talk about I mentioned the pie test x this plug-in earlier It is a way to run your tests in parallel So that can be that can give you a very nice speed up because most people have multi-core machines these days so The good thing is it's really easy to use it. So if I want to if we want to speed up these tests We can just run them in parallel So I already install the pie test x this plug-in So if we specify dash n we can Instruct by test to run these parallel So I just start to process now and it's what this will probably be with that very fast But let's see if we can yeah, there are these so there are two Firefox Browsers at the same time. So these tests now ran in parallel So That's a very easy way to speed up the runtime of your test suit You just install pie test x this and you run it with dash M to for instance So my own test suit it usually takes about two minutes to run and when I run it on my machine with In four processes. It's about 30 seconds. So it can give you very nice speed ups So once again that it's the dependence injection and it's based on the name of the argument and the name of the fixture function That's how how they map together and how they are found Yes Yes, so so in this case you you if you want to have both Firefox and Chrome run You specify the implementation of this fixture itself. So You instruct I have I don't have time to show you that now, but you can Tell pie test to for each Each time a test needs this fixture invoke it twice once with Firefox and one once with Chrome And You can also you can also use multiple fixtures just add more more fixtures to the argument list of other You can you might want to have a database. You might want to have access to Something else to or just another object. So you can you can use multiple fixtures in the argument list and you can also So the web driver fixture for in in in this case it can depend on other fixtures, too But I want to go more into detail about fixtures. This is just it's very powerful and it can be used in a lot of different scenarios Yes, yes, I'm I'm gonna tell you about it so I also showed the the exist. It's it Doesn't really have anything to do with fixtures, but it's a Nice way to speed up your tests So and this is how easy it is to get started with it just install it and then Run with the end command line and you can all do all sorts of Remote distributed tests with it, too, but this is the simple case So I want to talk a bit about how you can port if you want to switch to pie test How how you can a good strategy strategy for doing that So I showed you that the unit test was just the test case was just picked up by pie test And and there there is also actually support for no style tests So if you use nose currently, you can probably just switch to pie test and it should mostly work out of out of the box And if you install the pie test Django plug-in your pie test your Django test suits should also run as they are So the in that way you can just start using the pie test command to run your tests and then you can gradually write new tests in this lighter style and you can make use of fixtures and then you can change your existing tests or time or you can just leave them as they are But even even unit tests and nose tests. They all get benefits such as the parallelization from exist So you don't have to go and throw all your tests away and start over in order to start using pie test. It's you can switch gradually Yes, so I highly recommend that if you want to know more about fixtures that you go visit florist talk this afternoon about fixtures So I just scratched on the surface on what you can do with fixtures and he will show show you more advanced uses of them So don't don't miss that talk And also there will be a training with Holger Krekkel who is the main author of pie test So that will be on Friday. I'm not sure if it's still possible to sign up for the trainings, but Check that out if you're interested and I will do some sprinting on the pie test Django Bindings and the plug-in on in the sprint. So feel free to to join in there if you like to get started with it or See what it's all about So, yeah, feel free to just come and talk to me if you have any questions after or I guess we might have time for some questions Five minutes. Okay, cool Where you started to discuss fixtures where does the request argument come from? Oh, yeah That's a good good question. So the request argument it is a special fixture That you can use to get information you can introspect the surroundings You can get the for instance if you need access to a command line argument from within a fixture It's available on the request On the request object. So you can leave it out in the fixture definition if you lie if you don't have any use for it But it provides access to To pie test basically so that you can communicate with it. So it's a special fixture that gets passed into to this fixture Right I'm not very convinced. I must say so my question is what features of pie test you would say you use every day and You wouldn't find in other things like unit test or nose and stuff like that because Just not having a class and inheriting from unit test or test case is not a very strong argument for me At least I don't know I would say that the fixtures is the really unique feature And it's it's hard to to show you How powerful they are in this short time? But they you can combine them in different way and combine the caching and the scoping of them and you can parametrize them and you can really Have your have a very good structure of how your tests are invoked. So I would say that that is the absolutely biggest Like feature of pie test when if you compare it to two other frameworks Hi I have question how much more difficult it gets to debug your tests with pie test So in unit test you need to do much more work So you don't have this nice features, but you do everything explicitly and here you have a lot of features That are nice, but they hide So the work it is done by itself, but You use it every day. So is it gets more difficult to find problems in your tests and Well, I've used it for a while, so but I Wouldn't say it's harder to debug it and there are all sorts of You can you can run like a pie dot test collect only and it will show you how it collects the tests and you can have All sorts of different outputs. You can have very verbose outputs if you really You can get a lot of information about the failure failures So in that sense, I guess you get can get more information out of pie tests then With standard unit tests, but I don't see it as a problem. I I don't really See that it makes it harder to debug anything Thanks Any more questions So I've used pie test a little bit recently and one of the problems I've found with it is the test discovery So it's one of the advantages that you don't need them a lot of kind of info in your class for it to find the test but if there's something wrong like one of the examples is I had a Mock dot patch on the test, but I missed out the tea and patch and it couldn't find the test So it wasn't running and I didn't realize that it wasn't running Is there anything where you can find out tests that haven't run? Well, it it should run tests that are mocked use the mock decorator or It was just because I had a mistake in it though. So I had misspelled patch. So it hadn't found it But I didn't realize that it hadn't run Okay You should I think you should have Got an exception them or narrow if you misspelled the patch decorator something that that it should have failed During the collection phase. Yeah, there was nothing. Yeah, just it didn't even try to run. Okay. I Am not sure in that particular case, but if there are errors during like in Importing things or standard errors like test They should be shown. Yeah. Oh So you a flag to show the skipped tests or yeah, so the report skipped A command line argument, I guess. Yeah, then you can show all the skip tests. Thanks any more Okay, thank you very much