 They will test the Untestable with Andrew Burroughs. Let's give him a hand. Cool. Hello. Welcome. My name is Andy Burroughs. That's my Twitter. A little bit about me and what I do. I work for AHL. I've been there for nearly 10 years. AHL is a systematic quantum hedge fund based in London. We're in the business of using computer algorithms to invest on behalf of our clients and we do that entirely in Python. If you've got a code base written in Python or any other language, managing billions of dollars and trading around the clock and around the globe, then you're going to want to have some good tests. And for us, we find that mocks and mocking play a part in providing good tests. Hence the talk. A bit more about my AHL. We, a little plug, we host the PyData London talks. We run a coding competition encouraging students to get into Python. You may have seen Charlie plugging that in a lightning talk. We do a whole lot of open source stuff. We use lots of open source. We make our own stuff open source, check out our GitHub. And my boss would love it if you were to follow us on Twitter. He's even put it on my back. I think it's in case I get lost. If you see me roaming around Bilbao and you can tweet my boss and say you saw me. Cool. A bit about the talk. The talk is going to be really example-based. There's going to be some theory and definitions and some of my own opinions. I'm not going to go into all the deep workings of Mark and the full richness of the API. Helen gave a really good talk yesterday, which went deep. This is a beginner's guide. If you want more, meet. You should watch that on YouTube. It was really good. I'm going to introduce Mark and give you enough to get started if you're not already using it. All my examples are in Python 3, but that shouldn't be a stumbling block, really. You would hardly notice if you're not already using those technologies, and all my examples are available on my GitHub, so you can get them yourself and run them. Cool. So why are you here? Hopefully you're not like the guy with the beard and you are actually writing some tests, but if you're not writing tests, I'm hoping to give you one less excuse for writing unit tests. The excuse that mocking is somehow mysterious, hard, or not for you, then I hope to spell that myth. If you are writing tests, I hope to give you the tools to make testing complex systems both easy and fun. Cool. So I said the code, the talk would all be very example driven. Here's our first example. I was trying to find some common grounds, and I figure after all being locked in this conference venue for five days together, the one thing we have in common is a deep expertise in coding conferences. So all my examples are based on a mythical system that models coding conferences. Here's the first example. It's conference speaker, and this class does just like what I did. Comes out, welcomes his or her audience, and introduces her to handle. And with this, we get our first definition, system under test. You may see this in the documentation when you read around the topic. It means exactly what you think it means. It is the code that we're testing. So this class, and specifically this greet function, is our system under test. Cool. So how do we test it? It's not very long, and it looks like it should be easy to test. We can just create an instance of a conference delegate. We've probably got a class for that lying around our code base already. Pass it in, see if it does the right kind of thing. Easy peasy. Unfortunately, just like in real life, all our conference delegates are Twitter-enabled, tweeting continuously. Just for reference, when I put up a loader code, and you need to look at a bit of it, I'm going to highlight the bit that's most important in green. So feel free to read the whole lot, but if you want to save time, just skip to the green bit. So if we were to use this class in our tests, then we would... One of two things would happen. Either our tests would fail because either whether running doesn't have access to the insect, maybe it's behind a firewall, or it doesn't have the connection details it needs, the passwords, keys to access the API it's calling, or the test would pass and we would spam all our loyal followers with test tweets. So either way, we don't want that to happen. But this is Python, right? So it's easy. We can just make something that looks like a conference delegate. If it cracks like a conference delegate, it's probably a duck. So we can just make a test delegate class, make an instance of that, pass it into our system of the test, make sure the right kind of thing happens. And that totally works. And you can go home and do that, and we can stop the talk right now. But the talk's meant to be about Mock, so maybe there's a better way. And it seems like it could be a lot of work if we had to do this every time we wanted to essentially mock out the clues in the name, a class or an object in our code base. So we can use the Mock library. If you're in Python 3.3.0, you can import it from unit test.mock. Previous to that, you can pip install mock to rolling backport and import from Mock. We're able to create an instance of the Mock object, pass that in to our greet function, and then we're able to assert the right kind of thing happens. And we'll look at that in a bit more depth in a bit. Cool. So let's have a quick detour. And actually look at what we've got with these Mock objects. The most important thing you need to know about Mock objects is that everything on a Mock object is another Mock object. Every attribute is a Mock object. Every method is a Mock object. The return value of calling it is a Mock object. The return value of the methods are Mock objects. The attributes of the return value is the methods of Mock objects. It's Mocks all the way down until you hit the turtles. But what are we talking about as well with this Mocks? When I first started doing this, I was quite confused. I worked in a team of people who used a whole load of words that all seemed a bit interchangeable, but had some subtly different meanings. So before we go any further, I just want to define some terms in case you're coming to Python from another language or you routinely use more than one language. Or like I was, you work in a team of people who just use completely random terms all the time if you're trying to understand what's going on. The first term is a test double. Think of this like a stunt double in a film. It's a really general term. It just means any pretend object used in testing. And certainly in Python, we would use Mocks. And Mocks are definitely test doubles. So there's a first translation. Fakes. Mocks aren't fakes. When you fake something, you're using a real implementation, but it's taking some sort of shortcut. So maybe you're using an in-memory database instead of the real thing. So that's not what we're talking about, but you might hear it. Dummy values. In Python, dummy values are sentinels, and we'll cover those in a bit. We use these to pad out argument lists to trace the flow of data through our code, and you'll see that used. Mocks, now that's what we are talking about. We've just seen one in action. It has no real kind of implementation or any pretend implementation, but it records everything that happens to it, and you get to a certain that later if you want to. And then closely associated, and certainly in Python, we use the same object, but we use Mocks for stubs. Stubs in the kind of wider world are talking about this. A stub has more implementation behind it. It makes more of an effort to pretend to the system of the test that it's the real deal. And we can implement a stub in Python using a Mock, and we can err with a side effect. We'll see that. And for spies, spies are Mocks. As far as we're concerned, that's not air, split hairs. Cool, so we've kind of all that kind of theory behind us. Let's go back to our example. In Python, we use action assertion mechanics, which is a slightly fancy term to mean that first we arrange our test environment, then we cause our system of the test to act on it, and then we assert that the Mocks saw the behavior we expected. And it's really important to notice that what we're doing here with Mocks is we're actually asserting on the behavior. So we're asserting on what got called not what the state of the final system was. Here, I use one of many asserts that are built into Mock. I use assert called once fifth. There's a whole range of them. I'm not going to go into all of them here. You should check out the docs. The docs are really good. Or Helen's talk yesterday went for a good number of them and provided a good overview. I'm just going to show you my two favorites. Slightly opposite ends of the spectrum. Assert called once fifth does exactly what it says on the tin. We assert this Mock was called once with these arguments. And if it's called less than one, so more than once, or with different arguments, it fails. At the other end of the spectrum, the kind of superpower of these assertions is Mock calls. What Mock calls does is it records every call to your Mock and to all child Mocks and it records the order of them. So in the example at the bottom, I'm just in an interactive shell. I make a Mock. I make various calls to it and its children. And then I get to see what the Mock calls are and I could assert on that. And you notice that it's found everything I did and it knows the order they happened in. So that can be really powerful and there's pretty much nothing you can't test with that. Another flip side, another down side, I suppose, of it being Mocks all the way down is that if you pass a Mock object into your system to test and maybe there's a typo in there, so instead of calling speak to, it's calling spoke to or it's just calling a method that you've not implemented yet, it doesn't exist. The test may still pass because Mock will create those methods, those attributes on the fly for you. If you want to limit your Mocks to only having the same interface as your real code, you can do that by specifying a spec in the constructor of the Mock. And this is the interface, essentially, you want for your Mock. Yeah, in the example, I at the bottom, I just interactively create a Mock that has the same spec as a conference delegate and then try to get it to snore loudly in the session and it fails because you definitely can't do that. All right, so let's have a look at the hard example. Before we saw our conference delegate class and we highlighted that we couldn't use it on our other tests because it was tweeting. What if we just mocked it out completely? What if we want to actually test this class? So we've got the same problem, it's going to tweet all the same problems. So how can we do it? Well, we're going to use a Mock, of course, but last time, we were able to just pass our Mock objects into the system of the test. This time, we can't do it. The thing we want to mock, which is almost certainly simple Twitter, isn't passed in, it's imported. So how are we going to get our Mocks into this code? We do it using Patch, Patch is a great tool. It, what it does is it replaces the specified object with a Mock and then at the end of the Patch, it puts it back to normal, so it completely covers its tracks. So what we're able to do is we're able to patch simple Twitter.tweet, call the system of the test and then assert as before and that Mock gets created and injected into the right part of the code. The one gotcha is that you have to get that string, that specification of what you want to patch correct and it has to match how the code is used in the system of the test. So if, for example, instead of importing simple Twitter, we did from simple Twitter import tweet, then not only would the call site change, but so also would the patching. In my experience, working with a number of people, introducing them to Mocks, this is a thing that people always struggle to get their head around. It's worth taking a moment to think about why it happens. Essentially, when we're doing from simple Twitter import tweet, we're creating a new variable in this module called tweet that is a reference to the code in the simple Twitter module. But then if we were to patch simple Twitter.tweet, we are replacing the code in simple Twitter but our reference, our tweet variable in this module still points to the old code, so we're patching the wrong thing. So we have to get it right. You don't need to get it wrong many times before you kind of spot what you're doing and learn, but it's worth looking out for and worth thinking about. Another thing to notice with this example that I was showing is that I'm using a sentinel. I mentioned before that sentinels are the kind of Python word for dummies and what we do here is we pass a sentinel into the initializer and we check that it comes out in the call to tweets. It's a bit like kind of dropping balls on the top of some sort of machine and seeing which hole at the bottom they come out of and can be really handy. Another use for them is just a pad argument list to cause when the values aren't important to the code you're testing. I really love sentinels, by the way. Brilliant, you should use them, they're great. Okay, let's dig a bit further into our fictional application and look at the wrapper simple treater that we've built around our Twitter API. What it's doing is it's adding just a simple retry functionality on top of the underlying API. The underlying API would return false if it failed to send a message, whereas you can see this code is picking up on that and trying five times to send a message regardless. We can write some tests for this. We want to, I've said before that the return value of every call to a mock is another mock, but you can override that and you can actually set the return value. So here I set the return value to false and that means every call to a tweet will fail and then I can assert that it tries, but fails and ultimately gives up. Also notice in this example that I've been using patch this time as a decorator instead of a context manager as it before. These two things are completely interchangeable. You can use them, they work exactly the same. One reason you might use patch as a decorator, especially if you're patching more than one thing, is if you're using it as context manager, your code would slow, as you had more and more patches, your code would slowly disappear off the side of the screen, whereas of course stacking them up as decorators, you don't have that problem. But they're interchangeable, you can do both, they work the same way. Here's a lot of kittens. Oh, don't you love kittens? Cool. All right, what if we want to test this retry functionality? We don't want it to always fail. And we want it to succeed sometimes. So we want it to succeed on the third attempt. We can't do that return value, what we can do is side effect. So I set the side effect to a list, and the first call is gonna get a failure, the second call is gonna, failure, third call is gonna be true success. Side effects are like return values, but they're basically just more magical. So there's three types of magic supported. There's the sequence we've seen, where subsequent calls to the mock return different values from the list in turn. You can set the side effect to an exception. So every call to the mock raises an exception. That can be really handy if you want to test failure scenarios without having to actually, you know, manufacture failure somehow. And the final and most powerful form is setting the side effect to be equal to a functional lambda, or some other callable. And the return value of that function becomes a return value of the mock. I like the first two a lot, I would know caveat. The third one I do like and is powerful, but I just want to kind of raise a little red flag that if you're doing this a lot and you might want to just look at your code and just make sure that there's not some a better way to structure it to make it easier to test. It seems like you're having to use some quite, if you're doing this in all your tests all the time, maybe you're, you know, maybe there's some light up, you can construct a code differently so it's easier to test. Have you said that? It is useful and powerful, so let's have an example of it in action. Let's give our conference delegates the ability to chat to each other over coffee. So here we've got some sort of implementation where our delegates can hold a conversation and do whatever they like. And we can test this by creating essentially a stub. I mentioned stubs before, they're kind of mocks, but with more faked up implementation. We create a function, strange speak to, which holds one side of the conversation, we set that as a side effect for a stranger mock, although now it's probably a stub, doesn't really matter in Python, we just call them mocks. And we pass that into our test. And we can test that code if we want to. But this can look a bit, I mean it's not the prettiest test, I would argue that it looks like, you know, you have this quite nice sort of set up, run it, assert, but then in the middle of it, you've got this big chunk of code and what's that doing there? So a little plug for my own library mock extras, it's built entirely on top of mock, it doesn't implement certain thing, it just makes it kind of prettier and nicer to create side effects on your mocks. So it gives it this kind of fluent interface, so you can say when this particular mock is called with these particular values, then return this. And these are all quite simple examples, it supports them quite. Complex things, if it looks nice to you, check it out, it's on our GitHub, it's on Read the Docs. I mean to me it just makes the test read a bit more like a story and makes the test slightly better documentation for the code for testing. But, you know, your mileage may vary. Cool. So we've now learned about mocks and patching sentinels and we're ready to go and save the universe. But before we can do a warning, beware the dark side of over-mocking. What do I mean? So I think most people agree, it's uncontroversial, we will mock third-party APIs, we'll mock out course mail servers, web services, we'll mock out things that make our code undeterministic like randomness and time. And we won't mock out stuff that is built into the language, we're not gonna mock out numbers and lists, tupples, strings, and stuff that in your world is as good as built-in. So we're not gonna, if you're doing a lot of NumPy, pandas, don't mock it out, leave it in there. And there's probably, in whatever you do, there might be other types, which are like the oxygen you breathe and you shouldn't be taking them out. But it still leaves a lot of other code and what do we do with it? If you read the internet, you'll find people ranting on all sorts of things. As you know, two schools of thought on this particular problem. There's like a classical TDD approach which says that we should only mock all stub or fake objects when we really have to and we should try to use the real objects whenever we can. And there's a mockest approach which would value having unit tests, testing a single unit has been the most important thing. So they would mock everything. Where do I stand kind of pragmatically towards the mockest end without being crazy about it? But with a real eye on not wanting to make my tests over mocked. Over mock tests are brittle to changes in our code, expensive to maintain, but tempting because they're easy to write and boost coverage statistics. A quick example, we're gonna add a feature to our conference delegate which is the ability to rate talks. Rating talks is good, it's good for speakers, it's good for conference organisers. You should rate this talk, but be kind, it's my first ever one. You should maybe consider using the ISO standard for talk rating, which as we all know, is the number of kitten pictures in the talk multiplied by the sum of how useful it was and the clarity of the presentation. So how do we test this code? We could go crazy and we could write, we could put mocks in instead of our numbers. This is a contrived horrible example, you would never do this. But to show how bad over mocking could be, let's look at the worst ever case. So we pass mocks in and then our asserts end up looking like some kind of weird twisted inside out representation of the original code. It's a horror to behold. It'd be much better to test our code in this case with meaningful examples and edge cases and you should do that when mocks don't fit. Okay, to summarize, do write tests, do use mocks, they're easy and fun. Patch is brilliant, love sentinels like I do because they're awesome. Use functional side effects as an occasional treat. They are powerful, but don't overdo it. Never over mark, love each other and thank you very much. I believe that we have some time for some questions, so do you have any? Oh yes, the disclaimer is it, oh the question was don't, I have a large disclaimer and the answer is yes I do. It's a side effect of working in the finance industry I'm afraid. But basically to summarize it, it says if you use this talk as investment advice, don't. But do read it, don't waste all my summary. I don't want to undermine the small print. Okay, any other questions? Okay. Maybe it's an easy one, but I was just wondering what's the real difference between magic mock and mock? Oh yeah, I kind of skipped over that. So here I use magic mocks. Magic mocks are just the same as mocks, but they also mock most of the so-called magic or Dunder methods, so they're kind of the operator overloading in Python, so because here I wanted to test that things were multiplied by each other and added to each other, and that's the Dunder mole and Dunder add methods, I need to use a magic mock. As a quick aside, patch always puts magic mocks in, but really you don't ever need to worry about the difference, they just kind of work. Okay, any other questions? Okay, if you don't have any questions, this talk will be followed by the lightning talks, so in 15 minutes. So let's thank once more Andre. Thank you. Thank you.