 Thank you very much. So yeah, this talk is titled Test on how to keep it simple. So just a little bit of background about me, first of all, and what the talk's going to be about. So I was going to say, first of all, I want to thank the Python community for developing Python as a tool. I think I've learned an awful lot just looking at Python and playing with the Python tools and the fact that they're all developed in an open source that makes it a lot easier for people like me to come along and work out what I think you should or shouldn't do. And also to say, it's a real pleasure to come here and speak at PyLond idiom and to thanks for the organizers in Bloomberg for putting it together and putting in the effort. So yeah, if you want a little bit of background against me, that's basically the best place to find me is on LinkedIn at the minute. I was playing with my web server and I crashed it all. So there's nothing there to look at at the minute. This is a talk about testing. And originally, I was going to talk about something slightly different. I was going to talk about a very complex thing to do with hypothesis and how you can generate a lot of exhaustive input data for your user test case scenarios. The reason I'm not talking about that is primarily because of the time limits. And also, it's probably one of the more complex things that you could possibly do with the robot framework. And as I was trying to put together that talk, what I found was I had to keep going to go back and explain to people or explain to myself what is it that I'm talking about when I'm saying that I'm doing testing. And so this is really a very initial introduction to what I think are the most important things. And then after that, you can begin to look at more complex tools. So initially, I think the things I'm going to talk about, I'm going to say, is testing in any way actually different from development? Sometimes I think people think there is a difference, and I don't think there is. Why do we test things? Why are we even trying to test at the start? I mean, a lot of developers I come across don't even think that you should test. What is testing versus what is verification? Is there a difference? Do you know what you're doing when you say test? Because half the time, I think people believe that the testing is something slightly different than what I'm going to say it is. And then the initial golden rule of whenever you're going to try and test something, you have to really start at the start, and you have to try and reduce the complexity of the domain. And if you're finding that you're adopting a complex test approach, you might feel like that's going to benefit you, but I think initially it's a very bad thing to do. So that's why I've titled the talk Keep It Simple. So last week I looked at Stack Overflow, not just for this, but I looked for quite a number of reasons. So I looked at just a number of jobs and the developer jobs, and I found that there's roughly 205 jobs were listed for Python. So this fluctuates quite a lot if you go and sample. If you sample today, you'll find different numbers. But then I checked what jobs are Python and also say test in them, and that was 100 jobs. So roughly 50% of Python developer jobs, just looking at this very unempirical study, but roughly somewhere around 50% of jobs explicitly mentioned testing to do with Python. And then when I looked at the specific role of a test developer in QA, there were 134 jobs and 38 of them mentioned Python. So there are a lot of people who are using Python to do test automation and to try and use it as a tool to test their products. So I think it's just a really big part of the community. And whenever you, if you're going to apply to work as a Python developer, there's quite a big chance that you'll come across testing. But at the same time, I feel like testing is often not a very popular topic. It's not a popular topic for a number of reasons. It's testing is often quite a painful issue. People come across testing because of bugs. They're driven to adopt testing methods because they have had issues with the things that they've released. And they've realized subsequently that actually we need to do more to make sure that we're developing the correct features. And in an ideal world, I think we even wouldn't have tests. I think most of the time, whenever we start at the very start, whenever we learn to become a developer, we learn about development. And then we learn things like bugs exist. And then we learn about testing. So I think that often in the lifecycle of when people are learning about development, they find that testing is something that is kind of an inconvenient truth that you have to deal with. And often there's quite negative attitudes towards bugs. So I remember someone saying to me that if someone writes a bug, then we should make them wear a dunce's hat. I think that is an indication of the sort of commercial reality that people often don't understand the role that developers play in an organization. And typically, developers don't run organizations like management and capitalists run organizations in the UK. And they believe that bugs are a sign that you're doing something wrong, which I don't agree with. So is there a difference between testing and development? So I think that's a thing that some people think there is. And certainly lots of people segment their companies and their functions like that. And I've certainly met developers who don't really write any tests. For months and months, they'll not write any tests. They're not even write unit tests. And so I think that's a shame. But I'm not going to tell you that testing and development are the same thing. I think it's just a question that's worth thinking about. Like, is there actually a difference between testing and development? In reality, tests and tests have a lower priority than feature development. So we don't really even sell code. We sell features. And we definitely don't sell tests. So typically, most organizations look at tests as a cost area, whereas they look at features as a source of profit. And so development and development features is seen by a lot of people as a way to make profit. And tests is a thing that you need to shrink, reduce, and not expend more money on. So that might be intuitive to a lot of people who are running organizations. But when it comes to the practice of delivering software, actually, I think that's a really counter-intuitive. I think it's counter-intuitive to understand that actually testing needs to have the same priority. It needs to have generally the same amount of effort put into it. But a lot of people will disagree with that. So I tend to think that actually a really good analogy for what a bug is is radioactivity. Basically, you can't predict when a bug will or won't occur. But you know it does occur, right? And in different circumstances, bugs occur at greater rates. And sometimes bugs are really bad. Sometimes they don't really matter at all. So it's quite like radioactivity in that that's kind of the way they work. And there's a sort of the upside to radioactivity is it can kind of kill you. But at the same time, it can also provide you with like a lot of par if you know how to harness it and handle it. But if you don't, you're going to get yourself into trouble. So that's essentially why people start to write tests is because they start to get burned. They start to get burned by features they ship that just don't work. But in reality, what are the, in my opinion, what are the four reasons that I've actually seen that people actually write tests for? And I've put this in rank order of things that I believe they do. And number one is they simply write tests as a way to protect themselves whenever things go wrong. If your job is to develop code, at some point you know it's going to go wrong, and at some point you're going to look like a fool. But what you don't want to look like is such a fool that you could be accused of being negligent. And so people begin to write tests as a way to say that actually, look, I'm trying to do my job in a quality-orientated manner. So I actually think that's the number one driver that a lot of people initially write tests for. The second thing that people write tests for is to find out the product's requirements. So this is where you get into BDD and test-driven development, right? That's all about discovering the requirements correctly and developing the correct feature. So I think that's a good approach, but it's also sort of a mature approach that a lot of people don't take. And then the third thing is an emotional reason. Not just an emotional reason, but can I release the product? So whenever you come up to product releases, people look to tests to say, am I confident to release? And you can get up to very complex situations if you look at Bing. Bing claims to basically continuously release. Netflix claims to continuously release. I do believe those things, but that type of level of testing that they've developed to get to that situation is very complex. In a lot of organizations, you're never going to get that type of maturity in your testing model and the link between development and test. And so most, a lot of people, especially in smaller organizations, are going to be looking at tests just simply to reassure themselves whenever the release cycle comes up that we can release. And I actually think that's an emotional decision. It's often not really a fact-based decision. So yeah, and then the fourth reason we test is simply to catch bugs. But typically, you'll only really catch bugs at the start. And then afterwards, your tests probably won't catch bugs, even if you're running them all the time. So I'm just going to run through some quick tips. And then I'm going to hopefully show you some Python code. First of all, I'm not going to call it automated testing. People think automated means free, and it's not free. And it's not free from human involvement. It's better just to call it programmatic testing. Otherwise, you're likely to create misrepresentations as to what you're doing. Testing is in verification. So if you're dealing with significant systems, safety-critical systems, I basically put it on a scale of significance. And the other scale is who you're talking about. But basically, if you're dealing with an insignificant product, so an example of an insignificant product is perhaps an app or a game that if it crashes, no one really cares because they can pick up a different one. Obviously, the developers of that app care about it. But it's not a safety-critical system. Testing is in verification because it doesn't prove that your product's free of bugs. That's what you would need. You would need verification to prove that your product's free from bugs. So testing is just going to show that your product can work. And it's not really appropriate to only use testing techniques if you're dealing with a significant type of product. You would need to do a lot more analysis around your code. So yeah, the main aim, I think, whenever you're looking for test tools is to just try and keep it simple. So if your test methods increase the complexity of your project, then you're basically doing it wrong. Testing is all about trying to understand your complexity and reduce it. And that feeds into how would you choose your tool. So I suggest that don't pick up a tool and use it unless you can sort of understand it in a single afternoon. So that basically excludes most test runners in Python. So pie test, you can't understand in a single afternoon unit test you may, but probably won't. Nose you won't. Soap you won't. Robot framework, certainly you won't. Any web server you're not going to understand. There are some small exceptions. So a good web server is Bottle. It's 2,000 lines long. It's written in one module. It's a way to reduce your complexity. If you're introducing complexity by the tools that you choose, whenever you go to test things, you're actually just probably introducing more bugs. And you probably aren't helping yourself to understand the domain you're trying to test. So the things I would suggest in Python is look for little small libraries. XPEX is a really nice library. It helps you write better exceptions. I'm going to quickly show you that. Bottle is a nice example that we replace really heavyweight tools like Cherrypie. So we do have an exception, I think, which is Python. Python clearly takes more than an afternoon to learn. Python is a really complex tool. I think this is the one area that you're allowed to say no to is that you're allowed to say, I can pick up Python because it lets me do lots and lots and lots of things in nice ways. But after that, I think most of your tests should be written in pure Python. I think there's a number of advantages to that, but I'm not going to explain all of those. The reason I say this is because you're a part as a tester, if you really know how to use your language and you're a good programmer. Test and time is generally very tight. It's a low priority activity. It comes up after the development work happens. It's whatever release is coming. You often just don't have time to pick up new tools. And so Python is also a really great way to make really unreadable code. Python is really easy to write really bad code with. Because of how it works as a duct type language and interpretation, you can do a whole lot with it. That's just brutal for anyone else to maintain. And actually, testers are often considered sort of a second-class developer citizen in a lot of organizations because they think, well, if I can't code, then maybe I can become a tester. And if you're going to use programmatic test methods, you should be as good a developer or preferably a better developer than most of the feature developers. OK, so show me the code. So I'm going to quickly show you what I mean. OK, so can we all see that? Is it readable? So yeah, this is just a really trivial example of a unit test. So we've got an odd function, which really isn't particularly a valid function. And then I've got two functions that simply test it. And this is basically how every single test in Python that's programmatic that doesn't require human analysis kind of relies upon, which is simply an assertion. An assertion in Python is just a special type of kind of return value. So in some ways, you could write this pure of pure functions and not with assertions, but pretty much every test it will really just provide you with the way to write assertions. And that's a trivial example of, yeah, so I'll show you a run-in. I presume everyone here knows how to write unit tests. Maybe that's a bad assumption, but OK. So it just says everything run, OK. So and this is just a trivially incorrect program. Instead of addon, I'm using a minus. So here it's just through the assertion error. So built-ins, assertions in Python obviously aren't very helpful, which is why a lot of the frameworks will try to give you a lot of login tools and stuff to tell you what the variables are going on. But half the time, I think that's a waste of your time to use those frameworks. Not exclusively a waste of your time. That's a very broad sweeping statement. But I think you're generally better to just write your own into your Python. So I wrote my own in Python, which is I called it Donal's stupidly simple test framework. And I wrote it in just two hours. And this is essentially what you can pretty much replace all the test runners that you really use, like PyTest and Doze. Like you can get that basic functionality just by 70 lines of code or something like that. And really, all the test frameworks do for you in Python is this one thing here, which is execute a test. You just provide it with a function. You try the function. It just catches all exceptions apart from the three that are outside the normal exception hierarchy, which is like, you can stop the test. It prints you some information. It prints you some information about the assertion error. And then it just tells you whether or not a pasta failed. And it's sort of at the end, it just prints that. So if I look at this, to me, this is basically what PyTest. This is what I use PyTest every day for, is actually just to get that little level of output. But PyTest itself is a tool that most testers probably don't initially have time to actually look at. You can just write it in pure Python. And the flexibility is that you're only getting what you need if you write it yourself. I've often had issues with some frameworks where I've spent more time just trying to work out what the tool is doing than actually trying to debug the test. So a lot of the time, I think it's just better to write it yourself if you can. So yeah, and so how would I make this slightly better? So I've just, for my unit test, what I've done is I've just imported a really simple library called XPEX. And it just helps you write slightly better assertions. And so with this, I can then run. And so XPEX just gives me a slightly better assertion, which is expected minus 1 to equal 3. And so that's kind of the power and flexibility of actually just writing it yourself, is that you'll understand your tool and you'll not introduce additional complexity that will likely hinder your test effort. Nope. So yeah, so just before I stop, there's just some things I'm not really saying is that I'm not saying that the things that I've written are good examples. I'm not saying that the framework's complete. I wrote it really quickly. But actually, with what I wrote, that's basically what I used whenever I'm going to write any unit test. That's really what I need. I don't really need anything else. And if I need it, I can probably just quickly make it. So you have to ask yourself, any time you introduce a tool whenever you go to test, you have to ask yourself, why am I introducing this? Do I actually need this? Do I need all the complexity of this tool? Because I'm just predicting that you're going to trip yourself up at some point. And I'm not saying that you can't use other tools. I think there's just a real cost-benefit analysis you have to do. And you often aren't going to know that. And that's why I think you should just always guess and say, if I can't understand or write it myself in an afternoon, then it's probably not appropriate. But there are quite a lot of circumstances which might change that. But the good rule of thumb, I think, is write it yourself or use something trivially simple. OK, so that's really thanks for listening. That's probably the very first thing I've publicly sort of said about testing, which is why I didn't go into the full background of how you could use hypothesis. Because I just found that my methodology and approaches is sort of, to me, different than what I often see recommended online. But there's a bunch of things that people say different things about me and say different things. And these are some of the links I think you can have a look at. And you'll find people who say quite different things. And it may be worth you having a look at. OK, so that's really all I wanted to say. So thanks for your attention. I hope I didn't speak too fast because I have a massive habit of doing that. But if you want to ask any questions or just totally disagree with me, then I'm very happy to hear. Thanks, Donald, for making it simple. So we have some time for questions. Hi. I just want to say thanks for the talk. It was a very interesting take. I've looked at a lot of different stuff. I've looked at TDD, BDD, all sorts of interesting stuff. One question I'd have, though, is if you're going to roll your own tests to your test framework, how does that work if you're working in a team? So you've got a team of 10, 15 other people. Would you end? Well, what would you do then if you were doing that? Yeah, so I think it's a good question. I think most of the test tools have appeared. So thanks for the question. It's a good question. Most of the test tools have appeared by people who have worked together and collaborated and decided that these are tools they want to use. I would say that if you're at the start of a test project, it's better for you to collaborate and create your own tools as a team. I think that's going to actually serve you a lot better in the long run. And I think if you write really nice, clean Python code, you're going to understand your domain better. You're not going to introduce unnecessary complexity. And you're going to learn quite a lot about the issues. But I think it's still a valid trade-off. But I don't know. Oh, yeah, sorry. Just want to disagree with a few things you said. The start of the slide just said that you shouldn't really write tests if possible. But I kind of disagree with that. Tests are really important. So as a developer, you should write tests to prove that code is correct. But then also the tests that are in place, if another developer comes to refactor code or makes changes, if they want to know that their chest hasn't broken anything, then if someone else has done somewhere they've written tests, then if those tests fails, then that's an instant sort of ability for them to find out that something is broken. And that leads on to things like continuous integration. So if you have a framework set up, so when you check your code in, you can have CI working to run this test automatically. And then if you're a huge team and someone's making a change and you've written code, test code there, that test verifies that no one else will accidentally break things and so on. And things like unit test, that comes as part of standard Python. So that's something that every developer should at least learn what unit test the package is all about. If not, PyTest, PyTest takes it to the next level and so on. And then if you're a Python developer, there's things like PyCharm. So in terms of you can look at coverage. So PyCharm has tools to code coverage. So when you look at your code or highlights, bits of code are doing red or green. So things that are doing red show you that there's no tests being written that cover that line of code that they execute. So these sort of tools are really important, I think, as a developer to really understand and get your teeth into to produce much better high quality code. Because obviously, if you have tests, then that reduces the amount of debugging you have. And if you're a developer, you've ever been in production, someone says to me, this is broken. If you catch bugs before it goes to production, that's where testing is powerful. And just one thing to finish off as well. I think people know about Instagram. There's a big move, obviously, to move to Python 2 to Python 3. I believe Instagram team, they moved, ported their code within a week because they had such a high test coverage. They moved from Python 2 to Python 3 within a week, or something really incredible. I think that's some talks on that online, which I would recommend people to have a look at. Yeah, so maybe I didn't. I thought I did. I said it's an ideal that I think people come to development and they realize afterwards that they need a test. So I don't say that you shouldn't test. I say that it's an ideal scenario. You wouldn't have to write tests. In fact, in an ideal scenario, I think you would just describe the computer what you want to produce. And it would do the development down the testing for you. So yeah. For many people who are starting programming, the idea of tests is really difficult to grasp. Have you found any good ways to encourage people into that that doesn't seem completely alien and doesn't seem to get in the way of what they're trying to do, which is, as you say, even a new programmer wants to make features, not they want to make their code go forward. Have you found any good ways to encourage that? Well, I mean, there's ways. I think basically most tests initially are just broken down by just think of specific, simple examples of what you want your program to do. And from that, you can iterate and create more and more complex scenarios. And so that teaching someone the method of how you come up with test cases is probably the biggest mental barrier that they're going to have a specific to testing. Because a lot of things about testing are primarily about just doing good development work. And that isn't, obviously, the developers, that's not a specific test, I think. But teaching people the methodology of thinking about simple examples and expanding upon them until they're kind of happy that they've got a nice orthogonal description of the problem, that's the method that I would suggest. Which is like TDD. I think that's all the time we have. So we have the coffee break just outside. Thanks, Donald. OK, thanks very much.