 So anyway, let's go on, I have three parts. It's the introduction, the case study and the results. So there was this URL and you could visit that URL and get some data, but don't try it out right now because it doesn't work. We have refactored the URL since, but at that time it was four years ago. If you visited the URL, it was giving you some data and this was the code that was running behind the scenes and it would give you the data. So for some time this worked until eventually one day a user got an internal server error and Django emailed me the trace back and the error message and it indicated that the problem was on this line. So who can tell me what the problem was? Hey, it's an easy one, no volunteers? Yeah? It can't be converted to an integer. Yeah, okay, that's a good try and this is also what I would have said, but that was not the error. But okay, I'm making it difficult for you because Django emailed me the trace back and the error message. So and the error message was time series does not exist. So you see that with all this, the diagnosis was trivial and for people who have some experience with Django, the fix is also trivial. Yeah, what is funny? Anyway. The solution for the decaying. Yeah, so that issue that you found it would have caused a value error and I don't know why we don't check for this here. Either it has been already checked elsewhere by the time we reached this point or we didn't notice it, but this code is absolutely anyway. So the thing is that it was a very easy fix. It took possibly less than five minutes. Then we had to test it because we have the rule that we pretty much always write tests for our fixes. And so we wrote this test and that's an easy one. But if you have done this, you probably know the frustration when you're writing main code and then you need to write a test, you need to context switch your brain and you need to figure out the details. So which file is the test going to be in and is it going to be a new test case or is it going to be in an existing test case? And do you need test data and how are you going to create it or if does it already exist? Where are you going to import it from? So even this test has some tricky parts. It had a decorator that I have removed for this presentation. And it's also an unusually easy case which I gave as a first assignment to a programmer who had no experience writing tests. And now some people will tell me that we should write the test first and verify it fails and then write the fix and verify that the test passes. And I agree with that. The point is that whether you do it before or whether you do it afterwards, you have a trivial fix that takes you five minutes and maybe you need 30 minutes or one hour to write the test for it. So for some years I have been wondering am I spending too much time writing tests and how much is it? Is it maybe two times for the main code am I writing twice as much in order to write tests or only as much? So am I worth doing it? Am I too slow because of this? And so I measured it and actually when I submitted the application for this talk, I hadn't measured it yet. And I didn't know what the results would be and I can tell you I was surprised by several things. And one of the surprises had to do with this example, this particular example because trying to find such a simple example I searched and searched and I searched and I found a repository history of several pieces of software and I thought I would find many examples of a trivial fix requiring difficult tests and I ended up showing you some obsolete code because this doesn't happen nearly as often as I thought. It just stays in your memory because it's very vivid. And except for that, even trivial fixes often need some refactoring if you want to keep the code clean. So even for this, which is as trivial as it gets, some people will tell you that you should refactor this function because it starts getting too complicated. It's more than five lines. Some people say no more than five lines in the function. And so although diagnosing and fixing took maybe less than five minutes, if you want to refactor the code it is going to be another 10 perhaps. And so the testing as a percentage of the total time is going to decrease if you take refactoring into account. And of course you practically cannot refactor the code without good tests. So now I'll show you the case study. We have this software which is called anydris and it's free software and all the code I'll show you is available here. And there are meteorological stations on the map and you can go to the detail page for a station and you can see the variables it measures. And if you click on a variable you can also find out what the measurements are and you can download them. And so it's that simple to begin with. As usual, the devil is in the details so you can't rewrite it overnight. So the problem we had in this particular case is how to update the data. You have the meteorological stations. They have sensors, they measure. You have to somehow get these measurements into the database. And meteorological stations are often installed in remote areas. They have no connection to the power network or to the fixed telephone network. And they are usually powered with a battery and a solar panel. They also have some means of communication and at the other end of this communication there's usually a computer which usually runs Windows and runs some kind of software that is provided by the manufacturer of the meteorological station and it downloads the data. It stores it in files usually. So what we've done is we have written some accompanying command line software and we run it with Windows scheduler or sometimes this computer is Linux so we run it with Chrome. In one case it is a Raspberry. So we write a configuration file for it and this configuration file tells it, it's in an e-file format. I don't know if you can read it because I made it quite small but it doesn't really matter. It tells you which is the URL to which the data has to be uploaded, the credentials and which file contains the data and what is the correspondence of the columns in that file to the time series IDs in the database and some details about how exactly the file is formatted. So that's how it's done. Now this way of communication has been changing in the last few years and more and more often the data is made available with an API. So in some setups there's a specialized computer which has the size of the ADSL router you probably have at home and this downloads the data and makes it available with an API. In some other cases it's very similar to how it used to be except that the software that the manufacturer runs on this Windows computer is automatically uploaded to a service which in turn makes the data available with an API and so it's going to be more reliable and simpler for users if we just give them a form where they can configure API access and have any trace which is our software occasionally connect to the API and pull the data and this would be much simpler than having software that must be installed elsewhere and push the data to any trace and actually another problem is that you know this INE configuration file it's not rocket science to write it but the people who deal with the meteorological status and all that they are usually less brain damaged than you and me and so they have some trouble with these configuration files. So last November I decided to add this feature to any trace and this shows you how it ended up. So a station administrators or managers if you prefer got this little button which they click and then they get a form and because different manufacturers have different APIs the first question they get is what is the API? So far I've implemented only one and it's MeteorView2 that's its name and this form also asks some details about the time zone of the data or how often it should fetch the data and after that you get forms that are different from each API so this is going to ask you for the credentials in order to log on to that API and then it logs on, it finds out what data is available and it asks you to it asks you to tell it which time series on the database corresponds to which time series on the API. So I don't know if you understand this clearly it doesn't really matter if you understand this clearly or not because the point in the end is that I had a specific problem to solve, I had a specific functionality to develop and I also had to write tests for this new functionality. So are we going to use test-driven development or not? Who thinks you would use test-driven development for this? You would, okay, I'm very glad about it, you're smarter than I am. So test-driven development, I'll go through it very quickly. It has three rules. The first rule is that you may not write a main code before you have written a failing test. You write a test, it fails. The second rule says that this test that you're writing must be the simplest possible and should test just a single thing. And the third rule says that you should not write any more main code than is necessary to make the test pass. So it's really hard to grasp the implications of these three rules until you actually follow them. So try doing this for a week and see how different the way you develop becomes. And so as you can understand, I think that it is a good idea and I often try it whenever I can, but the problem is that it works when I have a clear idea of what code I'm going to write. But very often I don't really know how I'm going to approach the problem and I experiment a lot. So in this case, for example, I needed to develop a system that supports many different APIs and I use the word driver for this functionality. And so I didn't know how this would work. So would the driver be a subclass of some base class and what would the base class be like? Or maybe would the driver be a file that offers an entry point or maybe it would be a module so there are all these details that I couldn't figure out. What I did is I made some design and I wrote it down. So this is part of the finished documentation. It's not what I initially wrote. It's the finished one because I wrote something which was similar to this but as soon as I wrote three lines of code, maybe four, maybe two, then I realized it's not going to work. So I changed this immediately and then I changed it again and again and again and as I was writing the code, it was changing again and again. So I can't imagine how I could have done test-driven development in such a heavily experimental process. If you can, I expect you to tell me in the next break. Thank you. So what I did is I didn't write any tests at all. I wrote the base stuff. I wrote the first driver at the same time and when I finished and it was working, I hadn't yet written the unit tests and then I wanted to submit a talk for this conference and I thought let me submit this one. And it was a very good trick because actually I didn't have any deadlines for this or I didn't have any other pressure. So the fact that I had this talk to give made me sit down and actually write these tests. I wouldn't have written them yet. So as I told you, the results surprised me. I spent about a third of my time testing and all the other two thirds were all the rest together. Coding, designing, redesigning, documenting, everything. So I thought that testing would be more than the rest or maybe as much as the rest but it was only half as much and I think that this is a real bargain. And testing was pretty much exhaustive. The coverage is 100% and I did the next best thing to test the driven development. So what I did was I commented out all my code and then I followed the three rules of the poor man's test driven development which are exactly the same as the test driven development except that I changed two words. They were right and I made them uncomment. It's not as good as the real thing but it's better than nothing. Now there are some provisors, not provisors there. Who is a native English speaker? Native English speaker? There must be one. So how did you pronounce this word? Is it caveat? Yeah. Yeah, C-A-V-E-A-T. Caveat. Caveat. Thank you, okay. And another word I have trouble with is is it integer or integer? Integer. Integer, okay. I've been wondering for this for 40 years. I vast again but I forgot. So next year I ask you again, it's certain. So, there are some caveats and the first one is that every case is different. So this was the test writing cost for this particular case and I won't be surprised if it's very different in other cases. And the second caveat has to do with how experienced the programmers are. So I think I'm experienced enough but some programmers are relatively inexperienced in testing, they might write the main code faster and the tests slower. And I also have an example for myself. I used to have some difficulty with mocking so I understood the concepts of mocking quite well but somehow when I was writing mocks for my code they weren't working and I was trying again and again to make them work. I was re-reading the documentation for hours and the worst thing is that years passed by and it wasn't becoming any better. And I settled with the fact that I will always suck at mocking and I was particularly afraid of this case, of this project because it's about testing an external API so I had lots of mocking to do. But, and this is the third surprise. My mocks worked reasonably easy. I did get some error messages and I said, okay, this mock needs a comma there and this mock needs a fix there and then they worked. But my 2019 self would have had much more trouble. I would create the mock, it wouldn't work. I would re-read all the mocking documentation. I would insert breakpoints trying to find out why it doesn't do what I expected to do. Then it would get on my nerves then I would start breaking things, the usual stuff. But it seems maybe at last time becoming better at mocking just after I had lost hope. But mocking is yet another thing the programmer has to learn. So I expect this result, this result, I expect it to vary not only depending on the case but also on the abilities and the experience of each programmer. But, what is the alternative? So can we do without tests? I mean, we had this function and it became like this. And you know that soon afterwards it's going to become like that. And this is a real function in that code in the new telemetry code. There is this function, it's the worst of all. And now that I have the tests, I'm going to refactor this. In fact, I started refactoring it already and I found out that, okay, now it's readable. It's something like seven or eight functions. But then I thought that seven or eight methods is too much and maybe it's going to become a different class because you see this patch of this patch, other step. If there's the first step and the next steps. So this dispatches the next steps, all steps except for the first one. And then I realized that this wants to become a class that is going to be called step or something. Or other step, I don't know. And even this is readable, it's not too bad. If you read it carefully, you will understand it. But if you don't refactor it, eventually it will become impossible to change anything in it. Thank you. So I have a few hours of work in order to refactor this and two other functions. And this has been enabled by the 20 or so hours I spent writing those tests. So my conclusion is, let me show you the last. Okay, so my conclusion is, write tests, it's a bargain. Thanks a lot. Thank you very much for your talk. Is there anyone who has a question to answer? There you go, just line up and go ahead. Well, hi, that was a pretty fascinating talk. Which testing library do you prefer to use, like unit testing or high test or anything else? Yeah, okay. You know, we have this kind of contact here that says that we should respect others and be nice. So I think it's a very good idea as long as the other people don't use by test. So once we murder these people, I'm okay. Anyway, okay, of course I'm joking. So I use unit test. I think by test is very smart. And I think it's too smart. So my opinion is that if these tools like PEP8 or I don't know how we call the other one that checks the quality of our code. Sorry? Yeah, by Lint. So if it understood what by test is doing, it would throw a mess of warnings. So I don't like the magic of by test. I think it's too much magic. But other people like it. It's okay. I use unit test. My experience software development comes, really gets fun if you have a test coverage about 90, 95% what is your take on it? My take on... Yeah, when does software development get fun? High test coverage or if you just... Yeah, yeah, that's a very difficult question. And you know, there's a joke about it. It says master, do you know it? Unfortunately, we don't have enough time for me to say it. But I'll tell, so I'll go straight to the result. The thing is, I read a very good book. I think it's called Testing or Test Driven Development or something like that in Vue.js. So the author there says that the purpose of testing is to make software cheaper, to make it faster. So sometimes when I fix, it takes me one minute to fix because there's just a comma missing and I need two hours or so to test because I can't see it. Okay, this is a very hard thing to fix, to test. And I don't think that this is going to cause any problem in the future or that it is very unlikely for that bug to go back in there. Then sometimes I don't test. So my experience is that most of my code has between 95 and 100% of coverage and usually it doesn't go beyond that. It's a matter of experience, however. Other people might be doing it differently. Hello and thank you for the talk. And I had a relatively short question. Do you consider testing the mocks and counting coverage for them as well? How do you know that mocks do what they're supposed to do actually? Thanks. So have you practiced test-driven development? So if you do test-driven development, I think that everything you do just works because of the sequence of how you do things. But as you refactor code and then you refactor tests, so the code, some things fall behind and the things that fall behind in the main code if you test them, but the things that fall behind in the testing code, maybe there's some obsolete stuff there that is not being used or maybe there are some tests that don't work. They just always reply, okay. That's your question. Very good question. I have no idea how, I wonder the same thing myself. I think that testing is a relatively new thing. Very people knew it 20 years ago and I started testing about 10, 12 years ago. So it's relatively new. Even the books are new. You can't find enough books and most say you should do this thing, the others. So we are still learning. I think we are collectively learning how to do this correctly. Okay, thank you very much. Antonis for the talk was very interesting. I enjoyed it a lot. It was fun listening to you. I think we've joked a lot. So thank you and give him a round of applause. Thank you.