 So I make it very quick because we are five minutes past nine already. This is Emily Bache from Sweden. She's a long-term testing expert using test-driven development for at least 10 years, has given various talks and conferences about it. She uses several languages ranging from Python where I met her in 2004 to Scala, Ruby, and all kinds of things. She's currently working with a company in Sweden. And one thing that is interesting is that she's also doing coding dojos and has written the coding dojo handbook, which is a special format where you actually sit together and you have people coding in front and everybody else commenting. Isn't that great? And so I'm very much looking forward to Emily's insights about testing and if we all will still get jobs in a few years from now. Emily. Thank you Holger. I'm very pleased to be invited to be here talking on this topic. So when we're predicting the future, it's good to have a model. So this is a model of disruptive technology adoption that Geoffrey Moore came up with in his book Crossing the Chasm, like in the 90s. And he noticed that the kind of attributes of a technology that made early adopters flock to it were different from the kind of attributes that made the broad mainstream, the biggest part of the market, come to a new technology. And this idea of the chasm between that technologies can be very successful amongst early adopters and then not make the transition to the mainstream. So I found this quite an interesting idea because well I don't know about you but I'm personally a bit of a serial early adopter. New shiny technologies. I'm sure there's many people here who can beat this but I did start programming Python in 1999, which was fairly early maybe. I also wanted more evidence that I'm an early adopter. This is a letter I got from Reid Hoffman in 2011, congratulating me on being one of the first million members of LinkedIn when they reached 100 million members. So I was on LinkedIn when it was new and cool. But about this time, probably about 2011, I noticed they started running these little adverts next to your profile page. They would find some company that some of your contacts worked for and put up this little advert and say, picture yourself. You could have a new job working for this company. The thing is, the company that they happened to pick was one that I was thinking, ah, that's my worst nightmare. That is a job I absolutely do not want. The thing is, if I can characterize bureaucratic cooperation, which is of course a figment of my imagination, doesn't actually exist. But this company, I know that they have this group of architects who don't write code anymore, and they make all the decisions about what tools you're allowed to use. So if it's not on their approved list, you can't have it. In fact, you can't even download it, because the machine is totally locked down and you can't install anything that they haven't approved. So basically this company is pretty much guaranteed to use only zombie technologies. So when I'm talking about whether I'll be able to get a job in 2024, I'm really saying, will I be able to get a job I actually want, or will I only be able to get a job where everyone is a technology laggard? Laggard. I don't want to work there. So is test-driven development going to be very important to get a good job in 2024? Well, I noticed another commentator, Alan Kelly, made this prediction already. In fact, he said by 2022 that it will be the case that programmers who don't practice test-driven development will be unemployable. It will not be acceptable to question TDD or claim that it is unproven, and if you can't use TDD in a job interview exercise, you will not get the job. So I thought this was fairly stern stuff, to be honest. I mean, I'm a bit of a fan of TDD as you may have gathered, but I thought, is this right? Could it be that TDD is going to be absolutely ubiquitous in less than 10 years? Because as I see it, TDD is a disruptive technology and is subject to this problem of crossing the chasm. And I think it's disruptive because it is a very profoundly different way to develop software compared with the way that most people have been doing it so far. And at this point, I think I need to probably define a little bit more closely what I mean by test-driven development. So I'm going to go for a narrow definition here of TDD, where you have to write a test to drive pretty much every line of production code. The units that your tests are testing are completely isolated, and that might mean using mock objects and so on. So if your design in tiny increments, you build up the code very, very gradually using a lot of refactoring to pull together the design. And you end up with hundreds of tests that will run in less than a second. And when they fail, they're pinpointing exactly where the failures lie. So this is a very narrow definition of TDD. And last but not least, when you get this to work, it feels so good. You feel so free to develop and be productive. And this is the kind of TDD that you might have seen demonstrated by Kent Beck or Robert Martin or Kevin Henney. This is the kind of thing that gets shown at conferences in short demos. And many developers do use this. And I think if I look at this very narrow definition of TDD, I see early adopters using this. But I don't see the wide majority and mainstream developer developing this way most of the time. So what about in 10 years' time? I think I might need a bit of evidence to back up my placing this in the early adopters still. And I found a quote from Kent Beck in 2009 when he was developing JUnit Max, which is a tool which is only useful if you're really doing that kind of minute-by-minutes very tight feedback TDD. And he decided to stop developing that tool at that point because he estimated the global markets of Java programmers doing this kind of TDD was only a few thousand people. So that was five years ago, but I still see this kind of very narrow kind of TDD as not being very widespread. So where are we going to be in 10 years? So this is another normal distribution, which I found a picture of on the web, which I thought was kind of interesting. If you're colour-blind, you might not be able to see the amount of women on this graph, but I assure you that's a problem for the rest of us too. But anyway, I was at a conference. It wasn't in the US actually, it was in Europe, and the distribution of people at the conference, I'm afraid, was fairly similar. And I was chatting with this guy, and he was really interesting, and he was talking about all these cool technologies and stuff, and I happened to glance down at his feet, and I was a bit surprised at his shoes. At this point, I'd actually never really seen a pair of shoes like this before. I don't know if you've seen them with the little toes. They look quite odd, so I asked him, of course, about the shoes. And it turns out that he wasn't just an early adopter of all kinds of programming technologies. He was also an early adopter of running shoe technology. This is a pair of advanced running shoes. So the thing is, at the time, I was running around when I go jogging in a pair of shoes like the one on the right here with a really thick sole. And the shop assistant, when I bought them, assured me that this was the best kind of running shoe, and it was going to correct my pronation and my running style, and it was going to be preventing injury and be great. And then this guy I met had a pair of these minimal five-finger shoes, which are completely different. And he was clearly an early adopter of this, and he explained with great enthusiasm all the benefits of these shoes. So he was like saying, oh, it's got no padding on the heel whatsoever, and that's a good thing, because it changes the way you run and makes you don't shock your knees, and it's got this really thin sole so you can feel what you're running on and adapt your stride. Your toes can spread out because they've each got their own little toe and gives you better lift off. They're really light, they don't slow you down, and, of course, the killer feature. People will stare at your feet. So this is what it's about for an early adopter. So I listened to all this, and I did some research on the Internet, and I became convinced. So, of course, I bought some new running shoes. I'm an early adopter, right? So I was really happy with this pair of shoes, but I had to learn how to run again, to some extent. So I had to start going on really quite short runs to adapt my stride to these new shoes, because I was used to the kind of the guy on the left. You see he's about to strike the ground with his heel first. And when you've got these really thick soles on your running shoes, that feels really, really good. I was doing that before. But with these really thin shoes, you have to run like the guy on the right. You have to put the whole foot down at once. And this is a bit of a change of style, and it took, you know, a few weeks to get used to it. But actually, eventually, when I got used to the shoes, I was really happy. Running around the forest in my usual route, enjoying the light-footedness and being able to feel what I was running on. It was great. The thing is, it didn't last. After a few months, I think... Well, the technical term for it, I think, is autumn. Swedish autumn. It turns out that when your shoes are practically invisible and it just consists of very light fabric separating every toe into a kind of a waterlogged toe mass, you get really cold feet. And this was really difficult, running around my forest track in Swedish autumn, having freezing toes. So what do I do? Of course, I look to the internet and I do some research, and I find the latest cool technology. And I buy some new running shoes. So here I am with my new running shoes that have no heel padding. They have a thin sole. My toes can still spread out, although there's actually no separated toes on these ones. They're still really light. And of course, they are waterproof. So this is the killer feature for Swedish autumn running. But unfortunately, people no longer stand at my feet. So this is a story about disruptive technology adoption. And my points from this is basically two points. Early adopters do like to stick out and have people ask them about, what's that cool thing you've got there? And early adopters will also try out this new technology without very much proof that it has any advantage. I mean, I just took this guy's word for it really and some guys on the internet look cool. So when you're looking at a technology going to the mainstream, they actually are a little bit more conservative, probably don't want to stick out quite so much. And they probably want a little bit more proof that there is actually advantage in this technology before they'll try it. So now I want to relate this to test driven development. Because of course I was an early adopter of test driven development. And I remember this feeling of enjoying sticking out because I just wanted to relate an incident where I was sitting coding and a guy comes to ask me something. He's from the next team and he says, ask me something and then he looks at my screen and he does a double take. Are you doing test driven development? I've never seen anyone do that in production code. Do you mind if I watch? Well, okay. So test driven development is, I think, disruptive technology. And I think to reach the mainstream, these more conservative types, it might need to adapt slightly, become more practical, the equivalence of losing the toes on the shoes. But it has to do that without losing the essential benefits, all the other things that we need TDD for. Just like you want to keep the thin soles. But maybe the toes were not the most crucial feature. That was just helping them to stick out and drive early adopters to use it. So when I'm looking at TDD, I think there are some very essential benefits to it. When you're doing this narrow definition of TDD, you get a lot of feedback from the tests on the design and how the design is progressing. Are you managing to have isolated units? Are you managing to have a decoupled design? You also get this effect of self-testing code. That is code where you can push a button and it will check that it's still working. It will run itself tests and give you that feedback about whether you've broken it. And that's what you need to enable you to refactor that code. And of course, as I mentioned before, TDD feels really good. You feel really productive when you get into that state of just concentrating first on writing the tests. I've got all my attention on that. And then I get to write the implementation. All my brain power is just focused on how to do it. What I want to do, how I should do it. So this kind of essential benefits of TDD, I'm not sure how many people really have experienced TDD in the narrow sense. And I would encourage you that it wouldn't take you very long practicing a code carter to experience this. Now, I should explain a code carter. This is an idea that Dave Thomas came up with, watching his son doing karate, where he was practicing these karate exercises where you take a sequence of moves that you repeat until you can do them flawlessly. And this puts those moves into your muscle memory so that when you're sparring in a more random situation, these moves are just come naturally and you can do, spar better. So here like in this, well, in coding, perhaps we could do small coding problems and repeat them until we could do them flawlessly. And then those moves of test driven development would come naturally in a more production code situation. So an example of a code carter, which I've done a bit, this is a very simple one, write a function that returns true or false depending on whether the input integer is a leap year or not. And a leap year is divisible by four, but not otherwise divisible by 100 but it's also a divisible by 400. There's a rule. You have to code it. Helpfully, it comes with examples. And these examples are your test cases. So for a beginner at TDD, this is a very easy function to write in any programmer, really. But you should be able to write this with TDD. There are your test cases, one by one, and get how that loop feels, how I want you to feel that freedom and productivity. And this is a very simple code carter. It will teach you how to test drive a simple function that returns a boolean. And then, of course, you need to probably do another carter which would help you to learn to test drive a more complex function or a class or some cooperating classes. There's easier and harder ones of these, but they will help you to experience TDD in ideal conditions, TDD when it's really working, running shoes with toes in the Swedish summer when it really works. So this is a great idea, but, of course, doing a code carter by yourself is not half as much fun as getting together with others in a coding dojo, just like in karate, you go to the dojo and you practice your karate with other people. You go to the coding dojo to learn about coding in a group, and it's much more fun. It's these two French guys who came up with the idea. So it's basically, you try and do this fairly semi-regularly, and you get together maybe your team from work or your Python user group in your town or maybe at a conference, just some random people. And, of course, you write some code, you do some code carters, you practice TDD, you collaborate, discuss, get feedback, and it can be really fun. And I don't actually want to go into a lot of detail now about the coding dojo. I've written a book that Holger mentioned that enumerates exactly what is different about a coding dojo compared with any other kind of meeting where coders get together and hack, which is also fun, but it's a little different. But one thing I wanted to pick out is the way that you could use a tool to help you to get feedback. You've got the people at the dojo giving you feedback on how well you're doing TDD, but you can also get a tool like Cyberdojo. And this, actually, every time you run the tests, it records whether they passed, green, failed, red, or there was a syntax error and it couldn't actually execute the tests. And then you can see how often you're running the tests and whether you can see those cycles, TDD cycles. And you see this is a bit irregular and there's not very clear cycles, but after you've, you know, this is an example of a real pair, this is not me, this is some beginners I had in one of my coding dojos. And then after six dojos, a similar pair, and you can see that it's changed. They're running the tests more often. There's a lot more green, and you can start to see those cycles. So that's something that you can do to fight, you know, get some extra feedback. Am I doing TDD now? Do I understand the essential benefits of it? Because that's really what I want you to know, to experience what TDD feels like when it's working and to be able to recognize problems suited to it. So when you're working in your production code and you realize, oh, I've got a writer function that's going to return a boolean. I know how to test drive one of those. And this gives you the tools you need in your production code. And then you'll recognize how to adapt TDD to your local conditions. Because this kind of TDD in the narrowest sense, I think actually is only really appropriate in quite narrow conditions. And actually in a lot of production code, this is really quite hard to do. And you can end up really writing some very bad tests. So I think you need to be able to adapt to your local conditions. And that might be you've got a web layer, you've got databases, you've got widgets and GUIs and rich clients and asynchronous code and everything else where it gets harder to do this kind of pure TDD. So I start thinking about TDD in a much more general sense of, well, actually what are the essential parts of it? You've got to retain that feeling of productivity. That's kind of essential. You've got to learn to work incrementally. That's one of the big parts of it. And incrementally might not be quite such small steps as in the narrow version of TDD. It's important to have testable units, but they might not be quite as small and related as in that really pure TDD. You need to think about designing APIs separately from designing the implementations. And probably beforehand, but maybe not quite, maybe not unit tests, maybe more granular tests. And this idea of self-testing code, having code that you can ask it to all the test paths after my latest change. And you can have self-testing code without those tests being fast enough to run in that really tight minute-by-minute loop. And that can still be really useful, even if your tests take ten minutes or an hour or overnight. It's test self-testing code, and that is worth something. So I kind of wanted to go to talk about not just this narrow definition of TDD, but what are the essential benefits that you can get without actually doing that bang-on. And then I've got to plug Harry's book here because I've been reviewing this book, Test Driven Development with Python. And he's done a good job, I think, of showing you, demonstrating with a lot of code how to do TDD when your local conditions are Django and you've got JavaScript and all of that stuff. So I do recommend that. Then I wanted to take this a step further. What if your conditions are not just Swedish autumn? What if we're talking winter? And I'm looking at my running track and thinking, why are my cross-country skis? This is a much better idea. Is there a situation where TDD just isn't the right approach? Because I think there are some areas where TDD just doesn't really do it for me. And I've got three of them here I wanted to talk about. The first one is the problem of requirements. I don't actually know what the user needs. I could write a bunch of tests expressing what I think they need, but I don't actually know. Or I could write the tests, but I have no idea how to implement this. Or I've got some spaghetti legacy code with no tests. How do I do that? So these three problems I'm just going to look at a bit more. So this problem of requirements, I've built this beautiful teapot that I think is exactly what the user needs. Unfortunately I made it with chocolate. And it doesn't work. So this is the problem of not knowing what the user actually needs and this is only a problem if you are not actually the user yourself. So a lot of commercial software falls into this category. So the TDD loop here with unit tests has been, some people have suggested really for a long time that you should augment this with a failing acceptance test. So this inner loop is you go around in the normal minute-by-minute coding activity and this outer loop is the scale of hours or days to get a whole feature working and that test passes when the whole feature works. So the point with these acceptance tests is that you can't write them in isolation of the users or at least somebody who has talked to the users. You have to talk to somebody else. There are various methods. Behavior-driven development is the one I've got here with the three amigos discussions or you've got specification by example or you've got just acceptance test-driven development. And these are all ways to extend TDD to not just be a thing that developers do but a thing that teams do. And I think that's kind of useful when the user is not the same as yourself. So when I look at all those, these are all the benefits of TDD that I put up before. I think Double Duke TDD has all the same benefits because you're fundamentally doing TDD at the inner level but if you have the additional benefit you're hopefully building something that the user needs. So this is a useful extension to TDD. The next situation I had was when you realize you have no idea how to implement this. And I've put up a Sudoku board here because there's a very famous example Ron Jeffries, one of the originators of Extreme Programming wrote some articles in his blog where he was showing how he was going to test drive a solution, a Sudoku solver and he writes several articles and he writes lots of tests but the code is just nowhere. He's getting nowhere solving this problem. He clearly has no idea how to solve this problem or at least if he does he hasn't done a very good job of showing it. And then there's this other example of Peter Norvig doing a Sudoku Solve in Python and the code is absolutely beautiful. I don't know if you've seen it but it's really spot on and he's using concepts like constraint propagation and backtracking search. He understands the computer science of this and he does a really good job without test driven development. So the point is really if you don't know how to solve the problem TDD is not going to lead you to the solution. In my opinion you need to probably go and find somebody who knows how to solve this kind of a problem and get some training or mentoring research on the internet but one of the concepts that's in XP from the start which I think Ron Jeffries should have known about is the spike solution and this is the by analogy with woodworking. If you've got a piece of wood that it's a kind that you've never worked with before and you really understand you just get a spike and you start hammering at the wood and playing with it and trying to understand how this piece of wood works and then you put that bit to one side and you take some more of this wood and you build your cabinet once you've learned how the thing works. So this in the same way with code if you've never built something like this before don't try and do test driven development. Just hack about, play with things try out algorithms get to know your tools and when you know enough to build the thing properly put what you've hacked together to one side, start over and then start doing TDD. So that's the classic TDD XP approach. More recently some guy Dan North and Liz Keo and others have been talking about spike and stabilize. So you start just as with a spike solution and you get to know your tools you try out algorithms you read up on backtracking search and constraint propagation and try and understand them but instead of then throwing away your work you refactor it into something that can be tested with unit tests and add the tests. So this is this is not test driven development because you start from code that works and add tests afterwards. With the essential benefits of TDD you're getting most of them you're getting these testable units because you do write the tests eventually but you're not forced to design your API before your implementation. So it's a different slightly different but of course it has this benefit that you might go faster and be more productive than if you started over from scratch. So this is spike and stabilize the third situation where a classic TDD maybe doesn't work so well is when you have spaghetti code and you can't find any units to extract and write unit tests for and this is a hard problem and Michael Feathers wrote this book several, about a decade ago in fact and he has lots of helpful advice in this book about how to get this code under test when it doesn't have any. The thing is the title of this book I think is kind of misleading what he really should have called this book and some guys on the internet agreed with me was this book should be called Unit Test Anything a guide for Java C++ and C because all the examples in the book are in those languages and it really is focused on adding unit tests and that is really quite hard work because adding unit tests you have to take risks the book is all about minimizing that but you still have to take some risks when you haven't got any tests you have to make some refactorings to extract some units to put unit tests round and as you're doing that you might make the wrong choices and just start to bake in units that are there in the code but actually are really poor abstractions because this is legacy code and make it harder for yourself to refactor towards better abstractions plus of course if you're adding unit tests there may be integration errors between the units that you've extracted and that your unit tests won't pick up on so although this is a good approach this is hard to do well and I would start with some full system or integration tests instead basically that would be my starting point get the self testing code even if those self tests take hours that's a more stable and safe situation to be in so a useful technique I find for this kind of full system tests is approval testing now this is something that you may not have heard of because this is a case where I'm not actually an early adopter I'm one of the innovators for this technique so I really wanted to talk about this so I'm going to talk to you about it so let's say I've got a little program a command line program that I'm calling Mind Sweeper and it takes on standard input a text file well ASCII art representing the positions of some mines on the minefield that's the stars and produces the standard output a similar ASCII art augmented with these little clue numbers how many mines are next to this space on the minefield now some of you may remember a certain operating system that came with a game somewhat like this a few years ago that was kind of addictive but you'll notice that these kind of clues could be useful in that game although this isn't that game this is a much, much simpler version which is much easier to test with this technique hence why I chose it so how do you test this how are we going to check that this little command line tool is working with the unit testing approach you come up with a little minefield and you do some assertions about certain positions on the output that the clues were correct in those positions so you maybe have some code this is Java, sorry in approval testing you don't write assert statements we rely on gathering the actual output so the approval testing approach is saying well okay this is the input data when I feed in this to standard input I get this on standard output and I will inspect that the first time I see it and check that it's correct and I will approve it and say okay now I have a test case this input produces this output and so long as it keeps doing that everything is fine of course at some point this test will fail the new output will be different and then the test will fail with a diff and show you your programmers you've seen a diff tool before this is very clear there's some zeros there that weren't there before the approved version it's not matching the actual version but is this a bug or a feature well no we can't tell so let's say this is a bug and this output is not wanted and we've got this test failure now and I can go and fix the problem until the approved output matches the actual output notice if I had to write assert statements I might not have remembered to write assert statements for all of these kind of edge cases and maybe my unit tests would have just passed and not noticed this so this is actually a real effect of testing I recently often find defects that I didn't anticipate when I wrote the tests and defects that I wouldn't have found with my unit tests so that's an advantage of course the other scenario is that this is a feature and now I have a test that needs updating I want to approve the new version of the file and that's fairly with a good tool that should be pretty straightforward okay I approve this new version test now passing this is very straightforward just with if you just got one test failing you could imagine doing this with a unit testing tool you could imagine taking that new output and pasting it into your test case that works until you've got about three tests and then you suddenly start thinking oh this is boring pasting this text all the time you need tool support approval testing really only works with tool support that helps you with this managing the behavior of a whole test suite with many tests potentially failing in similar ways and this is the tool that I'm a contributor for which is text test and approval testing so just to summarize it's about diffing the actual output against the approved output updating approved versions and finding defects you didn't anticipate so if I'm looking at the benefits of TDD I mentioned before you don't get all of them with this kind of testing you don't have to design any APIs before you implement them you don't have to design testable units you just have to have testable programs and these this is different from TDD but it has the advantage that you can test spaghetti code these tests don't care what's on the inside how well designed the thing is inside it just cares about the outputs and the inputs so that's approval testing back to the essential question of this talk having gone off on a bit of a tangent there so I said before that TDD in the narrowest sense has not yet crossed the chasm and where will it be in ten years time well I think you've probably gathered by now that I think it's more useful to talk about TDD in a more general sense this one that I showed you before with these other advantages similar advantages little different so when I look at this more general version of TDD I can pick out self testing code as something which I think actually has already crossed the chasm I think most teams I go to now have self tests for their code they're probably too slow to use in the TDD loop and they're probably not testing isolated tests and all of that but it's still useful and I think by ten years time this will be pretty much ubiquitous that every reasonably sized code base will ship with self tests so another aspect of more general TDD is this thing of working incrementally running the tests often and then the tests do have to run a bit faster you can't tolerate more than a couple of minutes for the tests to take while you're developing and this is really much more widespread I think than narrow TDD but still this is probably not across the chasm yet but I think it will get there I think the benefits of working incrementally are really good and people realise that I think even the conservative majority will realise that working incrementally is a better way of working than only running your tests once a day the computers yes so that will help good point tests will run more quickly the hardware will be faster so this other thing this thing that I think characterises TDD in the narrowest sense is about using tests to drive your design and get design feedback this is not across the chasm and having said what I've said this is not going to cross to the majority this is a very specific way of doing design that suits some people and not others and some people like spike and stabilise does not have using the tests to drive the design approval testing does not have the tests driving the design and I think that maybe this is like the toes on the running shoes it works really well in some situations but actually for the majority it's just a little bit too extreme and that people will continue to do design in the ways that they've been taught and learnt to do design and maybe not in this really tight TDD loop so maybe that's controversial I don't know so will you still get a job in 2024 at all? well of course if you're going to a coding dojo and you're improving your skills this could really help because then you're learning about how to do TDD and that's got to be a good thing and in an interview situation if you can do a code carter that's got to be good so of course it's not just about TDD you can learn all sorts of other programming skills so this is my last slide I wanted to just summarise what I've said will I still be able to get a job see if you can try learn TDD in the narrowest sense see if you can experience that get a tool like cyber dojo get some feedback from peers so you understand the essential benefits of this technology and can apply that to your local conditions I've talked about double loop test driven development I've talked about spike and stabilise kind of design and approval testing these are techniques which may be up and coming extending TDD so that it works in more conditions and to answer the question I think of course you'll get a job in 2024 if you can write self testing code if you can work incrementally and you can do design even if you don't do your design with tests first thank you thank you very much Emily we have two microphones if you have any please walk up to the microphones you can queue there and there we are going to have I think not much more than 5 minutes hello Emily I just would like to in addition to the audience there is not just coding dojo but also code retreat and this is a special event form where the people in Germany especially in my home town Bielefeld meet regularly at weekends sometimes a year and in other towns too and so there we just do coding dojo in a special form to the game of life yes I've led code retreats it can be a really fun day also recommended over there I had a question about approval testing you had the slide up of approval testing and all the benefits that came with it and one of them was working incrementally but at least with your example it's not really working incrementally can you explain how approval testing and incremental development work there because I didn't get that from I had glossed over that you're quite right the thing is if you've got 100 tests with all the different variations of inputs and outputs you'll find that when you get a when you change something in the code you get if you change too much at once all of those tests fail in so many ways it's really hard to clean up the mess so you find yourself forcing to work in just making one change at a time so that you can approve the new results across all those tests and it's really clear that it's just that change you didn't introduce something unexpected so it's not it's not obvious you're quite right but my experience doing approval testing is that you do have to work incrementally or it doesn't work but how do you even get to the point where you can run the test without having written the entire program that's I guess that was the core of the question so you start approving the results when they're incomplete so you write a program first that just returns this so you start with the mind sweeper the first iteration just returns the same on output and you approve that and then you write something that will just write ones around all of the minds and you approve that and then you write the code that will put 2s and 3s and things and then you approve that so you do work incrementally next question please thank you again for this wonderful talk I have three things actually but we will be really quick so first thing is approval testing I got reminded of something it's not that completely new because when I think of a software like Gaussian that's a chemistry program that calculates some energies they did this in the 70s because they had to make sure that when you install it on your computer it calculates the same values than I did before so they put all the files and then when you run the test suite it actually makes a difference in the end comparing the actual result with some file that I just prepared and beforehand so but I mean doing it in this kind of approach is kind of new let's say second thing and there's another scenario that I wanted to point out where you don't use TDD really and this is in my opinion when you do rapid prototyping because it will slow you down you just want to get your code out so that didn't come up in your three examples that you had another question I have is to convince my employer to do it I mean I'm kind of new to this testing field but it will take more time to write your program and it will be more expensive and I think that's one of the reasons why the majority hasn't adopted yet so what do you say to that? You're right the majority needs more proven benefits before they'll adopt a technology and I think the benefits of self testing code are obvious and I think most people will accept that and so that's why self testing code I think really already has across the chasm the thing with working incrementally that only becomes clear when you realize how much more effective a team can work when they're sharing code more often yeah you have to I don't know I'm an early adopter I just try stuff what's going to convince the majority I'm not so sure on perhaps you just have to pick the right employer one second we have one more question but please could you just for one minute still reduce your chatting because that's a bit hard to concentrate thanks so one more questions over there hello yeah thank you for the talk can we just make a quick show of hands who is already using TDD in their everyday job yeah I should say this is the people you meet at conferences the proportion of early adopters is much much higher that's why I love going to conferences one of the reasons thank you so thanks again Emily and she's going to be around for a bit