 Okay. Thank you. My name is Alexandre and we are here to speak about integration tests. And I don't know, this morning I woke up and I was quite surprised to see, you know, on this app for your Python that you can rate, you know, the talk that you are going to see. And some people already rate in my talk, it was crazy. Like I already have four stars. And people didn't see it yet. And then I went on the web and I saw that other people wrote about me. And so I am very glad to tell you that today you will see the best talk ever of the year. Look at this. You will not believe it. It's really amazing. So thank you so much for being there today. But yeah, let's be a bit more serious now. So I live in Berlin. I am French. So if you don't understand my accent, it's normal. But I live in Berlin since two years now. And I work with really cool folks that are just sitting here at the front. And I have to say that, you know, often when you apply to a company, they tell you, yeah, we have ping pong tables, we have bars, we have, you can play tennis, you can play kickers, but no other company has a espresso machine like that. And so that's why you should ask your boss to have this kind of stuff. It's the best coffee in Tonit Berlin. And you can travel for free to Italy by driving a quick espresso. But let's go back to integration test. I will need this microphone to be turned on. That is good. So I will need in the room everybody to raise their hand. Only one or two. So this is really all techniques that Greeks people use to communicate with others. And usually, people ask questions at the end. But what I noticed is that often when you are sitting and watching a talk, then you forget your question or you're afraid to ask it at the end. So I will make some stops during the talk. And please, if you have any questions, just raise your hand and scream your questions. Or just go maybe on the front to ask your question. So we don't need to wait until the end. I think it will be more interactive. I want really to have your feedback and I think it will be better. And I will start right now by making this kind of question. So this talk is about integration test, but let's talk about test in general. Who in this room likes to write tests? Honestly, if you answer honestly. Quite a few people. Quite a few. I will go around. So who didn't raise, you didn't raise your hand, I think. You did. So why do you like to write tests? So I know what I'm doing. Did you raise your hand? Why don't you like to write tests? Because it takes a lot of time. And I know I should write a test, but I only postpone it in the time to later, always. So yeah, fair enough. And that's something I noticed when I was in my different jobs. And that's quite often people know they have to write tests. But most of the time, they don't like to do it. And they do it just because you know, you write a code, you make a pull request. And then your colleagues will ask you to write tests to be sure everything is working. But people don't do it because they like it. They just do it because they have to do it. And this is, yeah, something really, I will not say problematic, but I hope that after this talk, you will maybe have some, I'm sure you will head test even more, but maybe you will understand how to do it maybe a bit better. And the main reason why people don't like to write tests, I think is that as developers, we are a bit lazy, but it's not really, I will not say something bad because because we are lazy, we like to automate a maximum of things. So that's really something good. But the drawback is that maybe we don't spend enough time on reviewing tests as we will do for real code. And I think that's why because we are lazy, we like to use mocks when it's about writing integration tests. So let's see quickly what is a mock. So most of you already know what is a mock, I'm sure. But let's try to do a quick live session very quickly. So basically, let's see, I am a panda and I like to eat bamboo. So I eat bamboo, for example, and I eat some number of bamboo. And because it's quite hot this week, I only eat one. And now I will write a test. So let's test this function. Let's say I eat 10 bamboo. I have 10 bamboo, but I eat only one. So the result should be this one. The result should be 9. So now maybe we can try to run the test. Let's try to run that. I hope there is no syntax error, Python test. So there is no, there is no problem. Like the test was working. It's great. So basically that's how we write a test in general. We have our function. We have our function here. We write a test. We assert the result. And everything is good. Problem now, let's say this function was a bit more complicated. We will use a mock. So let's say I will mock this function. So what's how we have the mock? In fact, if we would have to write a very simple mock, it would be something like that. So taking a function in parameter, having a wrapper, taking some hugs. And then in the mock itself, we will assert to return a very fixed result. We will not execute the function itself anymore. So we will return 9 in this case. And then we return the initial function. So this test should still work normally. Yeah. So still no problem. But now let's imagine that I am the developer of a library. And I provide you this function, idbombo. And because it's making very complex stuff, you decided to mock it. But one day, I will release a new version of this library. And let's say, for example, I change the implementation and return to. So let's execute the test again. The test still works. There is no error. But I change the implementation. But you don't see it because you use the mock. I can change it again. And still it will be the same result. I can even raise an exception. You will not see it because you use the mock. Let's deactivate the mock and let's see what's happening. You got the exception now. And that's the main problem of mocks. So they are very simple to use. And we like to use them because it's very straightforward. But the problem is that, suddenly, it's hiding the behavior of the function. It's hiding the implementation. And you imagine that this function should work on that. But if me as a library developer, I decide to change the implementation, then your test, we still work. But when you run in prod, it will fail. And that's one of the major drawbacks of using mocks. And that's why I would really recommend you to never use them again. And we will see what kind of alternative techniques we could use later. But when I tell you never use them again, it's a bit extreme. Of course, sometimes when you run, for example, to simulate complex system, mocks are very handy. Or let's say if you want to simulate some network disconnection, you will not go to the data center to disconnect a cable. So you will use a mock. But there are many other alternatives. So please don't try mocks anymore. And instead, let's see what we can do. So basically, integration tests are based on two ideas, dependency injection and interface testing. Who already heard about dependency injection? Okay. And interface testing. Okay. So a lot of people heard about dependency injection, but not that much about interface testing. So let's see, again, how does it look? So let's make a quick file. So let's say, for example, I have a function. Let's see. Let's say I want to scrap the web. And so I am using the request library, for example. So I do something like for, I will take an URL. I will do four links found in URL. Then do something. For example. So the problem here is that if you write a test, test, test. So here, I said, scrap the web. Here. The problem is that in your tests, now your function here, oh yeah, I forgot that. Sorry. Request.get URL. So now, this function explicitly depends on the request library. And dependency injection, the concept behind is that you will say, instead, I want to give a client myself, like, let's say request by default. And then in your tests, you can give any client you want. You can say, okay, I want to fake client, something like that. And then you can give it to your test. So this is the idea behind dependency injection, that instead of having a function depending explicitly on an external library, you give the choice to the developer using this function to use the client that he wants. Now, if we go, if we have a quick look about interface testing, what does it mean? Well, it's very simple. Here, I have a fake client. And let's say I want my fake client to behave like the request library. So maybe my fake client will look something like that. It takes a URL and it does something. So now, I want to use this fake client in my unit test. Because I don't want to make real request to the web. And so I don't want to use the request library. But I want to be sure that my fake clients still have the same behaviors and the original request library. So let's say test client and for client in request and fake client, I compare that. I said that client dot get something equal something. So interface testing is just about checking that a fake implementation of a real library will return the same result as the original one. And this is the main idea behind real integration tests. So usually what you will do in your test base is that you will use a fake implementation in your unit test because you don't want to interact with the outside world. But from time to time, you want to be sure that your fake implementation still behaves the same as the original one. So let's say one per week or one per month, you will have a test interface testing to check that the two behave the same. So this is way more work than mocks, in fact. But the advantage here is that now your test don't depend anymore on the internal implementation. So if you remember here before it was doing something like that. For example, it was hard coded that we are using the request library in the main function. And normally if you were using mock, you would have said something like mock dot asset, mock like that, request dot get, and asset result something. So here the main problem is that your test is dependent on the implementation of your function. And if one day you want to refactor your code and use another client, you will have also to update the test. So you will have twice more work to do. And this is one of the real disadvantage of using mock. At first it's very simple to use. And so we do it a lot. But on the long term it's both the test and the code become bound together. And if you want to refactor, you will have even more work. So now that we saw a quick comparison between mocks and interface testing and dependency injection, let's see a real use case. So in our use case, let's say we have a static website. And we want to upload it to the cloud. And we want to use OpenStack. If we go on OpenStack website, we'll have access to the API. So the API of the OpenStack object store looks like that. So you can create a container and then you upload objects. So an object can be HTML page. And now we don't want to upload real, we don't want to use a real OpenStack system in our test because it's very complex to set up. So we decide to instead write a fake implementation of it. So how does it look like? So for example, if I want to create a container first, this is a definition of the method. And then we can create objects here. And let's look to the fake implementation. So a fake implementation could look like that. So I have a container. I have a fake object store. So if we go back here, we can see, so this is a real API. So we have an object store here and we just replicate here some behavior. But instead of storing some objects, I don't know on the web, we'll store just in a list or in a dict. And then we replicate the behavior of the real API. So we have to create containers, delete, etc. So it's quite a lot of work when you look like that. And now how to use it in the test? So this is an interface testing. So it will use both the real implementation and the fake implementation and compare that the two behave the same. So we can even try to execute them in live. Okay. So I will first execute the test just for the fake implementation. So it's running very fast. And now let's execute them for both implementation. So we can see here I'm using PyTest. And for each test, PyTest will execute them both. So here is the fake implementation. Here is the real one. So in the first case, I did not run the test for the real one. And now I am executing the test for both. And we can see it's taking a lot of time. And that's why we don't want to use real implementation in our, in, in test because it's taking too much time. And we can even have a look online when the tests are running. So let's say I will, I will edit a quick, a quick test, test cloud. And here I will just put a PDB. And I will execute the test again, just to see if it's really running on live. So now I am inside my interface testing. I can see, I can see that my cloud object, it's a fake one, it's a stub. And my container is also a fake one, it's a stub. Now, so this was just a test for the fake implementation. Now it will execute the test online. Let's have a look. So this one is coming from workspace. So it's a real object. And the container also, it's a real OpenStack object. And we can see here that the container is named test. So now let's have a look online if we created a test container. And we'll see that it will appear on the, on the dashboard. So here. So we can see that the test is really running online. And we are comparing our fake implementations just throwing objects in a dict. And the one running online has the same behavior with interface testing. Yep. So now, can we have a look on more, yeah, maybe some other implementations of dependency injection in other use cases. Because this one is a very traditional one, like you just put send objects somewhere, get a result. And sometimes you want to simulate complex systems. For example, if you interact with the web API doing a lot of stuff in the background, you don't want to reimplement a fake doing the same, the same job because it will be too much work. So what you can do instead is to write some kind of mock, but not really a mock and then inject it in your test. And how could it look like? For example, let's have a look again to a quick live coding session. So let's say we are using, okay. So in a test, let's say we have a, we are testing a web API and it's really doing some complex stuff. So we don't want to use again the request library. We want to use a fake. So we have a fake, for example, fake client. But in our fake client, we don't want to do to reimplement the, the world logic behind. So what we do instead is, for example, we can do something like that. This endpoint, I want to assign this result. So this is a bit like a mock because we will assign a very static response. But then this client, we can let's say we have this function do something. Client, we can inject it like that. So in, in real world, what we can do is that we don't have to always write complete fake implementation. But we can take the best of, of both what like marks and dependence in the interface testing. So in this case, we just out called the result. But the main advantage here is that we inject the dependency. So our test still remains independent of the implementation of this, this function, for example. And so in the future, if you want to refactor it, we just have to switch to another client. It's still easily possible. Another UK's, let's imagine I want to covert Blu-ray to MKV. So of course, not some movies you downloaded illegally, but some holiday movies, let's say. And one of the problem you have in this case is that a Blu-ray is very large. So you cannot write a usual test. In usual tests, you will compare an exact result. So asset something equal something. But in, in, in this case, the problem is that if I provide this library to someone else, we'll not have the same Blu-ray on our computer. So we cannot check that we'll have the same result at the end. So what can we do with interface testing with that? In fact, we'll not check the exact result, but we'll just run some the code and check that there is no syntax errors. For example, here, we don't assert the exact result. We just assert there is something coming back. So this is to get playlist on the Blu-ray, for example. This is to get some tracks, audio tracks. We don't check the exact result. We just check there is a result. Because in this case, it's not possible to do real interface testing because we don't know the result that someone else will have on his computer. And so everything is very, very flexible. We don't have to have something very strict in how we implement dependency injection, how we implement interface testing. But when you use them, you will see that your code will be easier to refactor. If it's only one thing you have to remember, please stop to use more because it's simpler at first to use them because you assign a very fixed result. And everybody, yeah, it's very simple. As we saw, implementing dependency injection and interface testing requires a lot of code. And mocks, on the other hand, are very easy to use from the beginning. But then your test will become dependent on your implementation because you will mark a specific library. And if you want to change this library, then in your code, you will have also to refactor your test. So you'll have twice more work. That's why dependency injection in this case can really help you to have a better API in your code because you will say, okay, I want to use this client, for example, or this one or this one, you can easily switch, you can easily change. But you should always ask yourself if you really need 100% test coverage because as we saw, it's a lot of work and sometimes you don't need to test perfectly your code. You only need to test something 100% a small part. So you can easily choose what strategy you want if you want to use mock or if you want to use interface testing. Yeah. And that's what's the main idea behind the talk. I also know that in my talk when I wrote it, I saw that the buzzword of this year will be docker compose. So I prepared a slide about docker compose. And in fact, I am completely wrong because the buzzword of this year is async IO. But you can still maybe get some ideas for this. I think interface testing and all this stuff is very boring. You will never find a job with that. But if you have docker compose on your resume, you will easily find a job. And that's a standard workflow that we are using at work. We use docker compose in this way. So we launch docker, in local compose, we have different containers. Let's say database here. Yeah, several database. We don't care. And in the main container, we have the source code and we run talks and pie tests. We made a workshop on Monday. So if you want to have some idea on how it's looking, you can click on the slides and have a look. But basically, we have one docker file for production. And then we have another docker file, depending on the production one, just foreign in text. And when I discussed with some people around, I saw that it was not the only strategy possible. I saw some people doing this kind of stuff. Like they were directly implementing pie test fixture, for example, and running like a DB like that. They need a DB and they will docker dot run. I don't know progress. I don't know the real API of the docker library. But then in the test, they will do something like that. And once the test needs to use the DB in pie test, it will run a container aside. One drawback I will see with this strategy is that now your tests are dependent of your environment. Where are your tests are running? Your tests are dependent on docker. But imagine you want to run your tests. I don't know on Kubernetes on GitLab CI. On GitLab CI, for example, you cannot by default run docker in docker because it will be very slow with the layered file systems. And this workflow, on the other hand, makes the test very independent of the whole stack. And I will really recommend you to use this one. If you want to use integration tests, in addition of interfacing and all this stuff, you can have a look to this GitHub project and get some ideas. So that's what holds for this talk. I completely forgot to ask questions during my talk. But if you have some questions at the end, please just come here. And I will help you to answer. Thanks.