 Okay, so let's welcome Andrei Komann, and he'll talk to us about testing the web application with Selenium. Is this working? Yeah. Cool. So effectively testing, effectively test your web app with Python and Selenium. That's a mouthful, right? So what's this talk actually about? I've been working on a project for a pretty long time now, and I got to see, like from its beginnings, how the project itself evolved, like the code base, and how the test around it evolved, because we reached some conclusions at some points in time, and we tried to improve what we had. And that's what I want to share with you guys, like what we learned from that, and hoping that you don't make the same mistakes that we did. Cool. Okay. So my name is Andrei Komann. I'm from Timisoara, that's a city in Romania, in the west of Romania. I work at Threepeller for PBS. That's the public broadcast system in the U.S. You can find me at friends in my code on GitHub, or follow me on Twitter, that's cool. How about you guys? How many of you have been working with Selenium? Well, quite a lot. How many of you are, like, QA, software engineering tests, something? Okay, a lot less. That's not a bad thing. So, well, let's get cracking then, right? We kind of have, like, I like to think of it like the way the test evolved in three phases, and we're going to take a look at each phase. You can see what we did right in each phase, what we did not so good, and how we tried to iterate over that in the next phase. Okay? Cool. So, this is how a test looked like in, like, the first phase. Bear in mind that this has been, like, in the beginnings of the project where the QA team didn't have that much support for development because we were, like, crunching out to get features out. So this is more like what the test looked like. This created a resource, think of a resource like a blog post or something. It gave it a title, right, pre-incent. And then we had a test that kind of used that particular resource. Nothing out of the ordinary till now. So let's take a look at how the page object model for Selenium looked like. So we were good kids, and we separated the Selenium interaction from the test itself. So we have, like, this page object model classes that model the page. And at the time, we were using XPath. So to get the title, we were doing, like, getting the div and going by specific ID and, you know, go a level up and starts to get a little bit confusing, right? So what this test was hiding, actually, that at the time we were using NOS tests as our test runner, and NOS tests, when it starts collecting your tests by default, it runs them in alphabetical order. So if C goes before V, you can use your test kind of like means to create the fixture and the state for the test on the server. And then test that in a different test. You can see how that kind of doesn't scale and contradicts some good practices of keeping tests separated and independent. And, yep, this is the kind of the problem with the way we use Selenium. So we used the page object model. That was cool. But we had this really long XPath that makes my head hurt when I try to read it. And whenever a developer came in unknowingly and changed the way the HTML was structured for this specific page, well, that XPath, that test got broken inevitably. So all these tests were running, like, on environment. We were giving it, hey, run this on production, run this on QA, run this on staging. But, yeah, you can imagine that if the tests were failing like midway, it's not that cool that you left leftovers on production. Okay. So, yeah, as I said, we wanted something better. We wanted to stop using tests as fixture generators. We want to move to something more robust in terms of identifying a Selenium element and interacting with it, getting it, getting its value or interacting with that element, like if it's a button, click on it or something. So I think at the time, we upgraded to Django 1.4, and then we had live server test case. So we said, yeah, let's give that a shot, why not? And then all the problems with setting up fixtures kind of magically went away because we were using the Django ORN to create, like, the state we were going to test, and using tests to just do a targeted test to see, yeah, this is the title of the page where I can click on a specific element and the video plays or stuff like that. And we weren't stuck in that end-to-end test where you need to create something, maybe using the admin and then test it using the published state. So this was like a better world for us. We can run tests independently, we can parallelize, which we couldn't do without many headaches before. So this is good, right? So we also took note of how we were handling Selenium integration. And we started moving to something more specific. So we started hooking into IDs or CSS selectors, and that gave us also more readability into the code. Like, you know, it's going to be an ID called title, and it was also less brittle because once you moved, like, sections of HTML in your templates, it's not as likely to break the tests with that. Cool. So what's the problem with this kind of approach? It's not testing the real environment. Now, when you get that email in the middle of the day saying that, hey, we are a team that you're a client of, so, like, we have a team that manages RR image ingestion and resizing, right? And if they make a deploy, they kind of send us an email, hey, run some tests to make sure your production environment still works. Well, you can't really use these tests for that kind of thing because these tests spin up like a really bare-bones environment. You just create just enough data to do your test in them, and afterwards you throw them away. And they're not really designed to test the real environment, whereas in the previous approach, we were actually testing, like, QA production, so on. Cool. So this is a good thing that in the page object model, we started using IDs, but then, again, we could do a bit better here, too, okay? So what we had, what we wanted to do better after all these two phases were, like, separating long-running tests from short tests, is still like to have, like, end-to-end tests that go maybe to multiple parts of your application and test the workflow. But maybe something like if you have a blog and you just check that the specific page works, that's good enough for, like, a smoke test. Google does this. So how Google tests software, they have, like, a section where they kind of chunk tests in medium or small, large, extra-large, and small tests are the unit test the developer runs on his machine. The large tests are more like these integration tests, and an extra-large would be, like, an end-to-end test. So it's important to put a distinction between these and put time boxes in which a test with, like, all my large tests must run under five minutes. Running them independently, that's kind of a conflict between the first phase and second phase. So in the first phase, we couldn't run them independently, but we could use an environment. In the second phase, we ran them independently, but we weren't hitting any real environment. So we started looking around. I mentioned we were using those tests, but PyTest was like the new kid on a block and made sense to try to look at what that has to offer. We put a limit on our Selenium suite, like, you know, it has to run in under five minutes, five or ten minutes. Previously, our whole Selenium was running in, I think, 45 minutes, and in the first iteration, so in the first phase, it was really hard to debug a test. So if you wanted to, if it failed, like, in the view part, you had to go through all the creation, and it was a very cumbersome and hard effort to debug. So also, what we wanted to make an emphasis on is decoupling the test from the HTML structure of the page, and for this, we went a step further from hooking, so identifying elements from existing CSS selectors or, you know, IDs in the page to setting up a convention between the development team and the QA team, saying, hey, this is a prefix which all the identifiers, so if you have a class like Selenium-something, that's reserved for testing, so for allowing testers to hook up to that, identify that element, and do stuff with it. It's important not to tie any CSS or JavaScript functionality to that, because you're just going to go loophole. It's important to keep that those test hooks focused only on that. So how does a test look like nowadays, right? The page is pretty much the same, except that we have this stuff like Selenium prefix, everything. So you don't go in and rely on IDs or CSS classes that will put in on developers and maybe make that specific element red or pop out or something. And also, we kept the same page object model, and that's pretty much it. And test-wise, because we switched to PyTest, we started using a couple of interesting plugins from the Mazele foundation. They open sourced a few plugins that I'm going to showcase a bit later in the presentation, but they really helped us to ramp up on a new test suite. So one of the first plugins that I tried out was PyTest variables. This is something that you can put in to your project and keep your fixtures or your credentials or something like that in a JSON file, and then pass that into the test. It kind of separates your fixture data, your credentials from the code itself. And it has a pretty simple interface. Pretty cool for the Mazele guys to open source this. Otherwise, there's PyTest HTML. It's a really cool plugin that is not necessarily tied to the Selenium integration, but it provides you with a test report saying, hey, this test failed here, and these were the values that it failed at. And if you're using it in conjunction with the PyTest HTML, the PyTest Selenium plugin from the Mazele guys, it also hooks in and puts you a screenshot from when the test failed. That's pretty cool. PyTest Selenium, this is the PyTest plugin that the Mazele Foundation open sourced. It's pretty cool. It has a lot of Chrome drivers, a lot of web drivers, sorry, Chrome, Firefox. A fun fact, we initially ran a lot of tests on Firefox, and after switching to Chrome, we saw a significant performance improvement. And it was pretty cool that this plugin allowed us just to change a parameter somewhere, and then all our tests were, like, running on Chrome instead of that. And also, they have support for connecting with cloud-based testing services like Sauce Labs or Browser Stack. Sorry? Yeah. Yeah. But I think for Chrome, you need to install the Chrome driver. Other than that, you need to install the Selenium package in your environment. So what I was thinking for you guys to take away from this talk is, and what we're planning to do on the project going forward is leveraging APIs to create data fixtures. So if you have, like, a REST API to create stuff and remove stuff from your app, try to leverage that, you know, like, creating a blog post and deleting it for test purposes, adding additional metadata into your page. So these are that convention between the development of the QA team that everything prefixed with something Selenium, it could be whatever, banana or whatever you want. And last but not least, defining, like, these test classes. So I have large tests, I have small tests and put time boxes around them. And when that specific class exceeds that time box, then you really should look at failing that whole class and looking at why, all of a sudden, my whole large test cases, like, maybe that's a smoke test, so it's running slower than before. That was about it. Thank you for the time. Do you have any questions? Do you know of an easy way to get video recording of these test runs? Not really. I haven't toyed around with that yet. That's an interesting point. I'll try to check up with the Mozilla program. Maybe they offer that. If not, maybe it's something worth contributing to open source. I know in the past, we were at a previous company, we were using BNC to FLB to record test runs and see why it failed. To answer the previous question, I know Mozilla has something because they use videos in PyTest HTML. So maybe just check the documentation for that. I hope it's documented if not an open issue and then maybe it will be. Any other questions? Have you looked into Splinter at all and PyTest Splinter plugin, are you? I haven't. It's a convenience wrapper library in Python for Selenium which helps you write better asserts without going into the markup. So it's like, instead of putting .sl markup everywhere, it helps you to write better tests that are less brittle. So maybe just a hint, it makes it easier to write nicer tests. Cool. I'm curious to talk to you about that afterwards. Thanks. How did you version your page objects if you were making changes in the application? How did you deal with having different versions of regression testing with Selenium? Good one. So what we're trying now is trying to get an element. And if you can't get to that hook element, you skip the tests and market the skip and see, say, hey, this test will skip until that code reaches the environment you're trying to test. Okay. Any other questions? Okay. If that's all? Let's say thank you to Andre. Thank you.