 So welcome everyone, this year we will be having a talk about test levels and we will be also talking about some type of creators along the way like doppelgangers. For this talk I will do it with Frank, who is a former student of the Paul Sabaté University in Toulouse and he contributed to Maouk and K-people and now nowadays he is one of the main contributors on Zenshin, which as you know is kind of my pet project, and is the resident software classman at K-Lab. As for me, I'm Kevin Notence, I'm also a former student of the Paul Sabaté University and I keep teaching there part-time. I've been working for the KD community since 2003, so third of my life, and I got a PhD in artificial intelligence in 2007. I'm also a resident software classman and a trainer at K-Lab. So as you know for a few years now we've been banging the drum of TTD, so test-reven development, but really when you say that it's just a label, right, that's like a brand, we're selling soap or something. Let's still ask the question of how you write those tests properly, and there are several challenges with that, one of them and the main one being really how you insulate parts of your design during the test, and so we need solutions for that, those solutions, that's what we call test-dubbles. The name you have to think about movies, you know, we've stunt-dubbles where you have one person looking like the real actor who actually takes his place during a stunt was the same here, but that will be the double of a class which will take its place during a test. There are several types of test-dubbles which exist and that's what we are going to explore in this talk. Before that we'll have a definition which I expected from Wikipedia on what they are, so let's say it's automated unit testing, it may be necessary to use objects or procedure that look and behave like they're really intended counterparts but are actually simplified versions that reduce the complexity and facilitate testing. The emphasis is my own in that definition, I think that's really the three main characteristics of test-dubbles, they have to look like the original, they have to behave almost like the original, and then it's a question of complexity inside of your test which will guide you into picking one of the different types of test-dubbles. There's roughly four types of test-dubbles, it's kind of a continuum so you can't really categorize them crudely. On the frontiers it's kind of fusive and you get from one to the other type but that's mostly so dummies, thubs, mocks and fakes. You can sort them on a complexity axis because you generally pick one depending on the level of complexity you have in your test and they are basically tool to move that complexity outside of your test inside of the test-dubble so it has to match in some way and so all the questions for picking that which amounts of complexity I want to move from my test, which design aspect I want to validate with my test is that just verifying the state of my code under test or its behavior. And so in the title we've been talking about doppelganger so we still have to carry on with that. So what are doppelgangers? In fiction and folklore doppelganger is a lookalike or a doppel of a living person sometimes portrayed as a paranormal phenomenon. Well test-dubbles are deep like that, right? They are someone doppelgangers of a living class instead of being doppelgangers of a living person and they can be of different types of paranormal phenomenon depending on the complexity you have to deal with. So now we are going to see the different categories. First one being the name is so that the simplest form of test-dubble you can have in a test anything that basically is an idea where anything goes, right? You will pass something it's totally passive and opaque and no one really uses it that's more to satisfy some interface. So if we look at our paranormal phenomenon that would be something like the voodoo doll, right? You have something which is totally passive you can just pin it, you can torture it because you can't really torture the original and that won't really have any consequence, right? Well except if you have some beliefs and so on. So the definition of a dummy that would be this one, dummies are used when a parameter is needed for the testing method or constructor and so on but without actually needing to use the parameter, okay? Maybe when you have your test running, okay, you pass a parameter but you know that that particular method in that particular case is not going to use it. Then you just pass some dummy value instead and something which will be opaque and not really reactive or passive. That could be a new pointer or something equivalent like a dummy identifier because you know it won't be used. So I've been talking quite a bit already and so I will leave the mic to Frank so that he can show you an example of dummy. Okay, so can you hear me well? Okay, so I need the pointer. Okay, so I'm not going to show Zenshin but you know it's a to-do application to keep it simple. So we're going to mostly look at code and test code basically which is always having an explicit name and a structure with a given which is basically an initialization when which is the test and then the test checking making sure everything went right. Okay, so in order to see the dummy we're going to see the should add task because we're managing tasks in Zenshin. So we have a given part to initialize stuff. In the test we're going to basically add a task here you can see. Just look at the selection I'm doing. Just don't try to reach to read every line. It's not going to be helpful. So in order to do so we need to initialize an entity and here you can see we pass a dummy because in order to create a task we don't need a note entity. It's totally unrelated and will not be used. It will be passive during the test. So instead of passing a real object what we do here is we create a dummy as Kevin said and basically it's a new pointer, a Qshad pointer but initialize as null. So this example is quite trivial and after it will be way more complex and useful. So that's our first case. So all dummies they have pros and cons, right? So the good thing about dummies is extremely simple to use, right? As we've seen well basically you've got a pointer. That's it. And you probably already do that without even knowing. And that's a mean for in your test you avoid creating a lot of collaborator objects for nothing, right? Because you know they won't be used. On the other hand they just cut dependencies and that's it, right? They won't help you to enforce anything or any type of checking in your test and your code under test has to be able to cope with the fact that there is a dummy, right? It has to not die just because you passed a new pointer so you need to be ready for that. So they won't go everywhere. So that's why we have more complex cases like stubs and that's because now you're in a case where for you see with the dummy you basically just ignore it, right? You just pass it around and that's it. Now you need something that you can beat, okay? That you can beat and which will react always in the same way every time you pocket it, right? So that's a bit like a zombie, right? You can beat the hell out of it and it will always carry on asking for more brains, right? That's the only thing it does. It's totally predictable in its behavior and that's what we're after here. So stubs are used for providing the tested code with indirect input because very often in your tested code you will call a method and you pass parameters to that method, okay? So that's a direct input for your code under test. But generally when you start to have a system, well some of the input you pass directly and some of the input of the code under test will be coming from some other collaborator object, okay? That would be a stub and that's a way for you to inject values inside of your code under test if that's not a direct parameter of what you're manipulating. The simplest of stubs is basically you have something which returns always the same value that can be a bit more refined. You could have something which returns value a then b then c always in the same way and predictable order. Sometimes I might record b2 then we get slightly into the mocking territory that we see later on and they are meant for state checking, okay? The idea is that you create your object which will be your code under test, you create your stubs, link everyone together, you manipulate the code under test and then you check the state of that code, the state of the different stubs and you declare that was fine or not fine. Okay, to make that a bit more concrete we'll have another example with friend. Okay, so we're going to look at this piece of code so again just follow the selection, otherwise you're going to have a headache. Okay, so the then she is using an editor basically when you select a task you can have the detail of a task like the title, the description and some test, some start date, is it done and everything. So the editor is basically a widget and we use a model so here we're going to try to test the editor but we're not going to use a real model so we're going to stub it, that's why you can see here that the editor model is stub. In order to do so what we want is to set it up so the test is about verifying that the task which is delegated to someone will display a certain label, well some text basically. So, get back to the stub. In the initialization what we do is create the stub and we set it up the way we want so basically we made the task being available we set up the task that will need to be displayed later and after we will really do the test by putting, setting up the model into the editor and we want to make sure after that that the text is displayed so it's visible here the text is visible here. So what we do basically as Kevin said is we set up a really predictable object a counterpart of a real object which is the editor model stub we set it up with this value, we do the right test here we pass basically the stub, not the real object and back there we make sure that everything went well and it is because it's a really simple version of an existing entity because you made it predictable he just said the indirect output Kevin was speaking about which is the state that's all. So all stubs they will allow us to factor out some of the test logic and they will help us to make the setup and state checking of all tests more reliable in general. That's something here we use our own class for stubs, okay that's something you could do by using a mocking tool as well if you use it partially, but you're basically you turn to a mocking tool instead of making your own class you're basically trading readability with the code size. If you use a mocking tool you have less code to write to create your stub, but then there's much more setup appearing again, so you kind of lose in readability with your own class that means that you will have to maintain this stub class if one of your interface changes for instance but then that's generally way more readable the problem you might have with the stubs is the fact that you have no clue how you got to the result because the only thing we know when we use stub is that we queries the code under test to check okay what your state now and we will verify that the state we expected we have no idea how that code under test got to the conclusion of that particular state which is sometimes what you might need. If that's what you need then we want you a real mox. That's when you need something which starts to be really more clever and basically that's more for behavior. So we want something which will confront you every time you have a wrong behavior, so that's a bit like the Frankenstein creator it basically will pursue you until the end of the world for you to realize that you've behaved wrongly in the past right something went wrong in your test. So mox are used for verifying the indirect output and not input anymore the indirect output of the tested code. With the stub we basically injected value inside or for code under test with mox we are looking at what the code under test is pushing out toward collaborator objects so that's different direction of the data that we are checking and so we are checking both the actual value and what got called on the collaborator objects. That's why for stuff we don't know how we got to the conclusion why for mox we know because we can examine which method got called exactly. Know that in the literature there's two types of mox. There's what's called mox. That's the one where you define the expectations of the behavior of your code under test before you run the test. There's files where you set the expectations after the execution of the tested code. Nowadays these distinctions tend to disappear and people just talk about mox. That's kind more of a technicality which is not necessarily very relevant nowadays. And because of that that's clearly for behavior checking and before you fall asleep we'll have yet another example with mox. I'm going to try to take a bit of time because it's more verbal than we're using a framework which basically you need to understand a little bit before starting. I'm going to grow up as well a little bit. Okay so this test is about creating a new item. New item into the engine. In order to do so we use the only backend of the engine is icon ID. So if you see icon ID all around it's normal. Okay so in order to do so you look at the one section what happened we try to create a task basically and in order to do that this whole bunch of initialization I'll go through right after. And after that there is some checking like always. Okay so let's have a look at the initialization part. So the point is to create an item and in order to do so we need to initialize the repository with a bunch of objects and we're not going to pass real object. As we said we're going to pass mox because we want to make some more in-depth testing than with the steps before. So here you can see it's mocking, mocking and some pointer we don't care about. Okay so I'll be mocking the engine. We use moquito framework. So basically let's have a look at the storage moch. Storage moch is basically declared here using moquito functions. And right after that what we can define is how it's going to be, what's the impute, what's the state of the moch. So here what this line is meaning, the one I'm going to try to select is when you call the default task collection on the moch which is mocking the interface, when you call it the default task collection function with this parameter which is void basically, then it returns this which is something you just prepared right before. So you set up a state here. You do the same with another function to create item when you pass with this parameter and this parameter it's supposed to give back this which is made before like again everything is fake, everything is mocked. So you just have a lot of control against the entity your test is picking with basically. Okay so you tell me like okay that's like a step you just prepare a state. Okay that's fair enough. So when you arrive here when you make the test to create task this is going to call basically all this method with just moch. And the point of the framework is to crash it and just replace with what you pushed before. So it's going to be really predictable as a step and everything is going fine. Okay so now what's the difference with the step we were speaking before is if you look at the Zen section what we're going to do is we're going to make sure it behaves as we wanted which means that we want this function, okay this one is the we are speaking about the storage so let's keep it. Okay so we make sure that this function is called with this parameter exactly one. So we're making sure of the code pass under the test you know like how it not only just the result how it has behave internally and same with create item we make sure that it called with this item exactly and with this collection exactly, exactly one. So that's more in-depth testing that's why it's more complex and study and more verbose. Okay so what's nice is that we can completely simulate all the collaborator with our code under test and that's also a way for us to verify hidden protocols. Okay because then we start to see protocols appearing between our collaborator objects and so we make sure that we make a certain number of calls of that particular method behind the scene with that those particular parameters. So that allows to get the code under test in a vacuum. Okay it's completely in a simulated environment which pretends to be the real thing. But as we just seen it's generally very verbose to set up right because you have to declare every method where you expect a call with particular value and then you have to do all the checks to say okay that's the number of time was expected to be called with those parameters and so on. The other big problem here is that it's making refactoring more difficult later on because by doing that you're coupling your test, the code of your test with the actual implementation of the class right. Okay so you later down the line you will have more problem okay you will be less able to react to change because then you have to adjust all the tests just because you have an internal behavior would change which maybe led you to the same result okay but just it's done different reason your test will fail. Okay so sometimes we might be in a situation where mocks or steps are not enough. Generally the sign of that that's when you start to have not one mock in your test but two mocks, three mocks, four mocks okay then you start to have something like 40 lines of code just to prepare all the states for your mocks and all the checking well basically what you have at that point is you're trying to simulate a subsystem, a complete subsystem and not just a few collaborator objects so in that case if you want to simulate a subsystem you need to create a fake one and so that's especially interesting when you have big external dependencies and you wish to cut them for the time of the test so you need something almost exactly like the original system but not quite. So that's where we get the real doppelganger right that's the clone so you almost can't tell apart which one is real and it's basically running at you trying to steal your life. So fakes are used as simple implementation of a system so for instance in-memory database so instead of the real dependency which could be for instance a database you plug something else which will be equivalent feature wise but you lose something else which is not relevant for your test. So in-memory databases in my opinion are a good example here in that definition because I find that fakes that's very often related to storage because if you're having something which end up on database and you have to write plenty of tests just because of the runtime of your tests with the database that would be way too expensive and you want to cut on that and having it in-memory would just go faster. So that's very often linked to that so storage would go to the system you're running on. So that's why on Libsolid which we brought like years ago there's a fake back-end which allows you if you're a user of Libsolid to run your test on a virtual machine right Libsolid will pretend being on the machine with some particular devices that's how you can simulate that. So if you're in pain of having me to talk so now we're having our last example with Frank so enjoy it. Okay, so the nice thing is it's fresh new fake just being done like last week or two weeks ago. So we will, not all tests have been ported so we will be able to compare this one with the other tests right underneath. Okay, so the keep looking at the selection. The test is about reporting all tasks so basically having them all if you look at how we do that we just have a task query entity which is basically doing a find all to retrieve them all and we're going to have a look at the initialization which is done through a fake because in order to initialize the query first we pass a lot of things. Before here it was mock I don't know if you remember and here now it's some fake entity with like you can see data is a fake and create storage is a fake as well. So let's go to the initialization part. Okay, so the whole point here is just to see that the call is way simpler than before we just initialize a fake create some data okay here create two top level collection create a task create, well just read the comment and you will understand it's quite easier to read there is not all this framework, mock it or mocking stuff that you which is way more verbose so it's basically having like the real code the thing is it's not like the real entity but it's really well imitating it okay so if you compare with this initialization with the one right under it which is more or less same kind of idea well you can see that here you have way more initialization when you initialize the objective in a really different manner you still set the mock which is very verbose as well you set each function of the mock so your code gets way more simpler counter part of that is when you go into the akonadi fake data you can see it's full of functions that need to be implemented to mimic the real entity like akonadi storage you're faking which is complex. Okay, so just to give you an idea about that because we see that there's a small blob of code for the initialization with the fake and the big one with the mock I started to port more of the test to use the fake because that's the most recent stuff we have and very often that stuff which was something like 40 to 60 line of codes to initialize all the mock so that they are consistent between themselves as well compressing something like half a dozen or maybe 10 line of code and done. So that's the main point of the fake it will simplify your test code quite a lot and that's because also your closer to actual production code just initialize a state in the fake and then all the rest is at the same usual code you would have and often then the fake is way faster than the original especially if they are storage but that means potentially you have a lot of code to write and maintain because that means you have to create a fake with the relevant features so you have to write the test code for your fake and then you can use that in the test so that's potentially quite some code for the fake the in-numory icon that we have right now in the engine that 2,600 line of codes of course that's something you end up doing if the technology you're using has no ready-made equivalent which was the case for something like a code but if you use database just swap I don't know might equal for something in memory and done so in those cases that's much less work okay so we've seen several types of test doubles and now you might be wondering okay which one do we pick is there's one we should pick in every cases where we've seen that no there's no silver bullet because they all have pros and cons and none of them are ideal so instead we will leave you with kind of a proposed approach and that's broad guideline that we are not using when we have to deal with test double and of course that's a proposal and your mileage might vary so our opinion is that well you should use the means whenever you can okay that won't hurt just make sure you're not killing your code under test by adding exceptions everywhere like verifying okay that's a new pointer I have to deal with it but if you feel that's natural for your class I mean that's easy to actually deal with a new pointer or something just go for it and then use dummies so whenever you can but in practice they have a limited use so that whenever you can won't be that often so otherwise if you can't go for a dummy release at the stub that you should prefer in most cases 90% of the cases the most common testable you should have in your test code at the end of the day that's just small helper class that you really use throughout your tests right so that's reusable code between your tests that's generally a good investment because that's cheap enough and then you get quite an impact if really you want to spare one then you can use the mocking tool to create a stub as we've seen you just do the state creation you don't do the checking at the end that's basically using mock tool to make a stub but then you have to be very careful because it becomes very easy to unwillingly move away from that's just a stub to that becoming a mock what matters here that's not the actual tool that's the intent of the code you're writing as for mocks you have to use them temporarily so they're really great when you're guiding the design the early design phase of some part of your code that's what I call last year emergent design because you start working on one class which needs collaborators which don't exist yet right so you create the mock and as you make the mock more complete to be able to make your test you're basically in the mock what you're writing that's the beginning of the specification of the next class you will have to implement right so that guides you doing that before you have the actual class so that's very nice for that but beware of the verbosity and coupling to implementation you provide so that's why you should do that so that you get to the point of that specification and then throw them away when you can then you just remove them and favor or stub or build objects whenever possible as for fakes there clearly are an investment so that's stuff to keep for the frontier of your system your external dependencies because they might get very expensive if you don't have a ready made solution as we've just discussed before you jump on them only when you actually settled on those dependencies if you change very often then that's wasted time and that's most useful for integration tests or acceptance tests which are full task testing where you want to speed them up and simplify and that's it for us thank you for your attention and I'm not sure we have the time for questions thank you we will have the group photo so one question and then we go out towards Munch following the balloons and at some point we will be stopped and wait for a second to smile and then have lunch so everybody please after the question go on and follow the balloons and take a quick picture so one single question and then we need to be on our way we're not inherent from anything so actually this doesn't work with better methods does this mean this works with templates instead? yeah that might not be clear when you see on easy using site of the code but I can only affect data as in two methods I can only affect data that's like having the naked data that you can manipulate and so on so that just wrap around ashes and stuff like that to set your data okay so it doesn't really need to inherit from anything but then it has two it has two methods one which is create storage and one which is create monitor okay and that's the one where we return an object with the right interface where it's all virtual and everything okay and then in that one the implementation talk to the data that you just set in the effect data so grab them on the way to lunch after lunch I'm sure thank you so much you short of memories