 OK, so my title of my talk, which you'll see any moment now, software development in the 21st century, is a completely meaningless title. It's a title I give to all of my talks that I give wherever I go because it's a nicely vague phrase and I can actually talk about whatever I like in the course of a talk. And in fact, I'm not actually going to give you one talk. I'm going to break it down into separate talks because I think that rather than listening to one long boring talk, you can listen instead to free short boring talks instead. And meanwhile, we're having some projector problems. I've got something. It's up. We're in action. OK, so instead of one long boring talk with a long period of fiddling around with the thing at the beginning, I'm going to give you free short boring talks. The first of which is about agile in general. A talk I call Essence and Fluency. And I give this talk because agile's in a kind of a curious state in many ways at the moment. When we started with agile back all those years ago, we were very much working against the tide. I remember in the early days, in the early zeros, when we were working at Fortworks on agile projects, we had to be very wary about mentioning we were using these weird agile techniques. Clients would come to us and say, we really like how you do your software development process. But you're not doing any of that agile stuff, are you? But now, of course, agile is suddenly very mainstream. Big companies who poo-pooed a lot of the agile stuff 10 years ago are now coming out with their great agile programs and agile methodologies and certifications. You know you're in a bad state when certifications are dominating the landscape. So I think it's important to remind ourselves what is the essence of agile software development. And to do that, we need to go back to where we were when all this started, which was back in the 1990s, where we saw lots of projects that had a great deal of problems. And the problems to some extent, and well to a large extent, are still there in the industry. I think they've been attenuated a good bit, particularly by effective teams, but they were very much in people's minds in the 90s. And at that time, people also felt they knew what the answer was V. They knew what the cure needed to be for these kinds of problems. And that was the big plan-driven processes. Things like CMM, heavy methodology, detailed planning up front before you begin doing your work, all of that kind of thing. That was seen as the professional way to develop software. We're not only wanting anybody not doing this stuff because they're clearly cowboys, incompetent, et cetera, et cetera. That was seen as a future. That was the world as it was seen in the 90s. But then you get this unusual bunch of people who got together and said, we have a different approach. We've been building software, we've had a good deal of success, and we've done it with a very strikingly different attitude to how we do this. To me, the important thing about the agile movement and the successor to agile manifesto, it's not that I want to see agile methods dominating the world and everybody doing everything agile. What I want to see is those teams that want to work in an agile fashion should be able to do so. And not seen as some kind of unprofessional people just because they work in a different way. So in many ways, agile has had a bigger impact than certainly I was hoping or expecting. But at the heart of what this shift is, this difference between the plan-driven approach and the agile approach, I think are a couple of very simple core ideas. And these are ideas that I sort of worked through in my own mind when I was writing an essay that I called The New Methodology. Still out there on my website, still a very popular essay, and I wrote it because I was very much steeped in one of these agile approaches, although it wasn't called agile at the time, and that was extreme programming. I worked with Kent Beck and Ron Jeffries, I wanted the very first, well, I put the birth extreme programming project, and I really liked what it was done. And I looked around and I saw Alastair Coburn was saying, I saw what the scrum people were saying, and I thought, there's a lot of similarities. There's something very much in common between extreme programming and these other approaches. What do those boil down to? And I thought to myself, well, what we're seeing is a contrast between these new methodologies, the ones that would become called agile, and the traditional approach that everybody said was the way of the future, which was the plan-driven approaches. And I boiled down to a different attitude towards planning and a different attitude towards the relationship of people and process. And to me, these two differences are the essence of agile thinking. And I'll go into a little bit of detail on each one, starting with the planning approach. If we look at the traditional approach to planning, it's seen as something you do before execution. You wanna work things through, come up with a detailed plan, and then execute on that plan. The model is often taken from traditional engineering, particularly things like civil engineering. You wanna build a building or build a road, you first have the engineering people come up with great detail, exactly how the building is to fit together, do all the figuring out of the structure and all the rest of it, and figure out how the building should be built, what pieces should be built in what order, and then you take that plan and you hand it over to a construction company, but actually does the work of building it. That is the model of the plan-driven approach to planning. Plan first, then execute. I think of this as a predictive planning approach, because what you're doing in the plan is you're coming up with your prediction of how things are gonna go in the execution phase. And that is also a crucial factor in judging success, because if you're successful, then things go according to plan. In fact, the whole notion, whenever you hear people saying, oh yes, our project was a success, it was on time and on budget, what that's saying is they're thinking of a predictive planning mindset, a mindset that says, success is things went according to plan. In an agile context, that actually doesn't make much sense, as we'll see. Now, this engineering approach was very appealing to people because of the fact that they saw all the problems with software development, and they felt this is how we become professional, this is how we become organized. But what it means is if you're gonna come up with a plan, you need to be understanding, very clear understanding of what your requirements are gonna be. You have to have a good set, well understood, stable requirements, because otherwise, your plan is built on sand. And that of course means that because the plan is such a central part of your software development approach, it means your entire software development notion depends on having stable requirements. At this point, I have to ask you a question to see if you've actually woken up yet this morning. How many people on their last project that they did saw significant changes in the requirements for that project? Exactly, a large amount of people. We have to question, can we actually get stable requirements? And to be fair, the plan driven people understand this and they talk about the dangers of requirements churn. And they said, we have to come up with stabilization techniques, but we can use to try and prevent that churn. We want to prevent that change. We have to stabilize the requirements. I remember someone saying, if your requirements churn by more than 15%, you're doomed. There's no way that you can be successful in software development. So you have to stabilize and you really wanna get it down to less than 5%. Now, the agile thinking looks at this combination and says, hang on, what happens when we in software have a really awkward dependency? What do we need to do? We need to get rid of that dependency. And so what we do instead is we come up with a process that's tolerant to changes in requirements. And that's the essence of things like, for instance, taking iterations and building things in small pieces, looking for feedback, actually expecting that requirements will change and giving us a way to handle that in a controlled manner. That's the central attitude, the little shift in planning with the agile thinking. And it goes even further. I'd summed up really well in this phrase by Mary Poppendick. Change, like changes, are actually a good thing. It allows us to do things more quickly than our competitors do. We actually not just respond comfortably to changes, we actually expect them and we take advantage of them. And that's an important mind shift that is an underpinning one to agile thinking. So that's a shift from adaptive planning to some predictive planning to adaptive planning. It says, we don't register success on the basis of things going according to plan. We look at other measures, such as have we made a positive impact on the business or things of that kind? Because in the end, if you things haven't gone according to plan, that could just mean your plan's wrong, right? I mean, it doesn't necessarily mean that what you've actually done is bad. There are plenty of projects that went according to plan but still didn't yield any business benefits. Now, the other part of this, you have a shift, is that of the shift to, in the way between people and process. And if we look at where the attitude to process in the traditional world begins, it really begins with this guy. Anybody recognize him? Yeah, there's always some Agilists who do. This is one of the most influential thinkers, men who's had the biggest influence upon human society in the 20th century. His guys, the name's Frederick Taylor, he was involved in America, involved in manufacturing, and he is the one who came up with this lovely little phrase. He looked around and he saw what was going on and he said, at the moment, the problem with manufacturing in the late 19th century was that the workers were deciding how they should do their work. And he felt that was very inefficient because workers were lazy and stupid. And instead what you needed to have was professional planners who came up with a system and then the workers would follow that system. It's an approach called scientific management and it underpins a great deal of industrial and administrative processes that you see in the world in the 20th century and indeed through to today. His influence has been enormous, much greater than sort of famous politicians or general's or things of that kind. He's had a huge impact on the way we work. And he had this impact on software development as well. So when people talked about how should we do this professional software development stuff, they would do it by typically coming up with some process of various activities that need to be done in certain order with various deliverables that are handed off from process to process. Now you look at any of these process things and the sure sign of this kind of thinking is we have this process diagram. And then if you want to use that process in practice, you look around at the people that you've got and you say, well, I have a bunch of people here. I have five programmers, three testers, two analysts and a project manager in a pear tree. And I take these people and I look at my process and I just slot them in. And that's how I go about planning out how we take people through this route. The process is first. Now, of course, I have seen some people do talk about agile processes this way. Say, hi, we have an agile process. Here's the various steps and the arrows and the diagrams connecting them. You know that somebody doesn't get agile when they show you that kind of thing, right? Because when you have some clever little diagram like that, you're missing the point. Anything that talks about steps and deliverables and stuff. I mean, that's off the thinking because what the thinking behind agile is about is to say, well, actually, the people that develop software, they're not lazy, venal, 19th century manufacturing workers toiling in factories. They're actually programmers, analysts, user experience people, but they're actually pretty intelligent because I mean, they wouldn't be doing this if they weren't pretty intelligent. And they need to come up with their own process. So we reject the idea that put the process together first. And we say, no, it's most important that the team of people themselves decide what process they follow. Now, it may be that they'll go out for inspiration to the processes and things that have been published. They might look at extreme programming or Scrum or something like that and say, oh, that's some good ideas. They might even, and it's often, can be a good thing to say, we're actually gonna do extreme programming by the book at the beginning just to learn how it works and to get a feel of how it operates. But very rapidly, they should start kicking in and saying, well, this isn't quite working for us. Let's try that and see how this works. And the process evolves over time. And the people in the team direct that. They choose how they wanna work and they choose how they're gonna move forwards. And the important corollary to this, of course, is if you've got a team of people who don't want to work in an agile fashion, it's pointless to try and impose it on top of them. The team has to decide they want to work in that way. I mean, if there are, can be external pressures that will kind of push you in that direction. For instance, the increasing desire to have regular delivery of software into production tends to push you this way. But in the end, it has to be the team's choice how they work. And this really boils down to the observation made by all of us in the agile community that when we saw the difference between successful projects and unsuccessful projects, it had nothing to do with a process drawn on a board. It all had to do with how good were the people on the team and how well did they work together in a human level? And that's why the very first value statement in the agile manifesto says, individuals and interactions are more important than processes and tools. And remember that was written by a bunch of people who were either process weenies or tool weenies. I mean, that was what we did. We got involved in processes or tools, all of us. But we knew that the processes and tools we were advocating were second order. What matters is a good group of people who work well together. And if you've got that, then they have to decide what process they're gonna follow. So that's the essence of agile. I wanna now move on to talk about a model of agile called the agile fluency model. It was developed by Diana Larson and James Shaw a couple of years ago. And they send it around to a bunch of people for comment in an article. And I saw it and I really liked it. And I said, I would love to publish this on martinfowler.com and they went for it. And so it's on my site, but it's their model. I'm just a publisher. And what they did is they said, well, in our observation, we've noticed that teams operate at a certain levels of fluency. Fluency meaning what teams do when they're under pressure and they have to just operate. They were very careful to avoid saying maturity model here, although it does kind of feel like it in a way. And they had these four star levels that talk about to work through. And I said, I find this a very helpful way of looking at it. The first level that teams typically operate this is really can be, I think explained by saying, it's focusing on the management, organizational practices of agile. It's things like doing things in iterations or sprints or it may be using a Kanban board. Particularly importantly, the idea of using retrospectives to be able to tune and adapt your process. The benefit of this first star level is you begin to have a team now that's able to redirect, be able to change direction, to absorb change. Also gives you a great deal of visibility because you're thinking now about how do I focus on business outcomes rather than just following plan points. And that's that shifting core metric. Most teams that they saw operate at this kind of level. And it's relatively easy to get here. It takes relatively small amount of time, matter of a few months. But many teams actually never get beyond this point. And that's not necessarily a bad thing. I mean, they're doing better often than they were doing before because they have greater visibility, greater ability to change direction, but they aren't really getting the advantages that agile has to bring. Because although the management practices are valuable, in order to really become effective at agile software development, you have to bring in technical practices, the engineering practices, things like refactoring and continuous delivery and think about how to design software in an evolutionary way. And these, of course, are the kinds of techniques that I'm familiar from because extreme programming did a really great job, I think, of combining both these management and technical practices. In the early days of agile, extreme programming was the dominant, most visible technique. Over time, it has become much less visible because strung became more visible. But I always have a strong fondness for extreme programming, not just because it's where I started, but because of the fact that it very firmly combined management and technical stuff together. Now, when you do that, what you get is you get the high productivity and low defects that really begin to see a lot of the benefits of this approach. Because the problem with only using the management techniques and neglecting the technical stuff is you tend to have a poorly designed code base that's not able to react well. You're not able to change direction as quickly as you might do because you've got a poor code base to work on. With good technical practices, however, you can begin to respond to change very effectively. So this is really, really valuable to try and reach this point. The disadvantage, though, is it's a big investment to get there. It takes many months to learn these technical skills, to learn how to do test-driven development, learn how to refactor, to get a good continuous delivery pipeline up and going. There's a significant investment in time to be able to do that. And that's, of course, why significantly less teams tend to operate at this level than at the earlier level that we saw before. Because it's more of an investment to get here. It is one, I think, that is worthwhile for most outfits, but it's important to remember to do this. And you need to begin that technical practice stuff at the beginning. One of the comments that James and Diana say is you need to decide where you wanna end up on that star scale and start planning for that from the beginning. So if you want to aim at two-star, start with the technical practices from the very beginning. Don't begin with the management stuff and add the technical stuff later. Do both at the beginning. So what we see here as we shift, as we adopt these things, is the first level is about shifting the culture of a team to think about how do we do this visible approach. The second is a big shift in the skills of a team to be able to take on these technical practices. So what does the third star bring us? The third star is a much bigger shift. I mean, again, look at the amount of time it takes teams to get there. Look at the very small amount of teams that they've observed that operate in this way. But teams that are doing this are shifting in terms of not necessarily just responding to what people are asking them to build and building it very quickly and effectively, which is kind of the essence of the two-star approach, but actually taking a more proactive role. I'll talk about this a good bit later on. And in that they're actually deciding what should be built. What are the best things to do? And they're taking a much more directive role in where they're going. And that means the team has to become much more knowledgeable about the underlying domain that they're working with to bring in the business expertise. And they're taking much more of a directed approach to what we're gonna be doing. It means we're shifting from just a change in the team's perspective to a change in the organization's perspective. And that's one of the reasons this is hard to pull off because you can't do this just within a team's bubble. The organization has to change in order to let the team take this amount of direction on. And that's of course part of the reason why it's so rare. But this is where you can really see some very significant effects. And this is where a lot of the stuff around lean product, lean enterprise and things, it's this three-star kind of level. Now they have a fourth-star level as well, which is a much more broader use of the three-star level, but it's not one that they say that they've actually seen very much. And so it's really a bit speculative. Have a look at the paper, it's worth reading, but I'm not gonna actually talk about that. So that's a quick sense of where we are with how I look at the agile world. It's essence and I think a good model for thinking about how fluent you can be within agile projects. And that's the end of the first talk. And now I'm gonna move on to one of these technical practices that I mentioned earlier on. So now I'll be talking about continuous integration and delivery. So I always like to think about this by thinking about the very first software project that I ever saw. I was a student doing a small industrial placement in between university. And I went to this industrial complex in North of England and I was taken to showing this big software project. It was a huge warehouse, probably about four times the size of this room, full of cubicles. And the people said, oh yes, we're doing this critical project for the future of our company. They've been working on it for two years. They ought to be done, well actually they should have delivered a few months ago, that was when the plan was, but they spent a lot longer in their current phase, which is integration. They've been integrating for several months now. And you know, to be honest, I don't know that they know when they're gonna be done, because it's really hard, this integration stuff. And the lesson was integrating big software projects or even small projects is quite difficult and therefore you don't wanna do it very frequently. And we see this notion appear both at large scales and at small scales. So a common statement of how you should work if you've got say two or more developers on a team is something like this. Well let's imagine we've got a two person team with Professor Plum and Reverend Green and they're away working on their pieces of work. And they do so in isolated branches of the code, isolated code lines. And maybe from time to time they'll bring in any bug fixes from the main code line, but they keep themselves relatively isolated. This is an approach called feature branching or topic branching. You say if you're gonna build a feature, you go off, you make your own branch and you work on that over time. And that time can be considerable. It's not uncommon when we've come in, gone into organizations to see people doing feature branches that last weeks or months in this area of isolation. And the nice thing is you work on your isolated piece and you don't have to worry about what the other guy's doing. Until of course you need to combine both of those pieces of work into the main project. And then you have this ugly nasty integration to do because you've got two people who've been working separately and they've made all sorts of assumptions that aren't compatible and now you've gotta make them compatible so things get tough. Now some people say well this is actually no longer a problem because we have powerful merging tools. Git can do wonderful merges. It can do all sorts of different things and figure out how they fit together. And that is true but it misses the point that merging has got two parts. There is the textual part of merging bits of code base but there's also the semantic part. If I'm working on my code and I decide all this core library isn't really organized in the way that I need it. Perhaps I need to change some names of methods or organize things in a slightly different way and Jesse's working away on his branch and making similar changes. They could merge quite cleanly at the textual level but the whole program doesn't work on a semantic level and that's where things get more difficult. Now testing can help us spot these problems but we still have to go through and solve them. So again the notion that we have is integration is difficult so we don't do it very frequently. Now I think at the heart of all this is a misconception. And I think of it like this. I'd like to draw this little pseudograph and say let's have a look if we plotted the amount of pain involved in integrating versus the time between integrations we end up with some kind of plot like this. The longer we leave integration it grows exponentially harder for us to be able to merge. And if this is the case which my observation in experience says that it is it leads to a very intriguing cororally. It says that if it hurts you should actually do things more frequently because if you do a very rapid frequent integrations then they don't hurt so much and the overall aggregate becomes much much smaller. And that is the principle behind continuous integration where at every change that you make you integrate it into the main line of the project. So if we compare the two here you see that the continuous integration has more frequent integrations but they're all small. And as a result the overall amount of pain is kept smaller. And of course you have a really bad thing about the delayed integrations that you see in feature branches is not just you get more pain but you get it at the end when you're under the most pressure. Now getting rid of complicated nasty merges is one advantage of continuous integration but there's a potentially more important one. And that is if you wanna keep your code base healthy you need to constantly refactor the code base so that as your understanding and your design changes you can work with that. If you're working with feature branches though and you do an incompatible refactoring on the two branches when are you gonna find out that you've done that? The only time you really know is during that big horrible merge which is why it's a big horrible merge. The important thing about continuous integration is that if you make an incompatible change you find out right away because you're both integrating all the time. And well, again the observation we seem to have is that generally in general teams don't refactor enough but in particular feature branching tends to discourage refactoring because you say, oh I don't wanna change that bit of common code because that's gonna lead to merge problems. So I'll work around it in my code that I'm adding in my branch. And so as a result, the software structure tends to degrade. For refactoring to really be effective you have to do the continuous integration so you can spot the anomalies as soon as possible. So that's about a very brief introduction to continuous integration and why it matters. There's a paper on my website, you can find out more. There's a book, you can find out more about that. But as well as continuous integration I also want to talk to you about how we move on from that and talk about continuous delivery. No, I can't do it. I mean I can't talk about continuous delivery when you offer the book. He's right on the front row of the audience. So Jess, you come up and take over.