 Okay, then it's, 12 o'clock, so you can start on time, and anyone that's late in just has to catch up with whereabouts. So I'll start off to say welcome to the sessions on what I've called the Software Poverty Trap. My name is David Kitchen. I'm a Software Engineering Manager at Johnson & Johnson. Johnson & Johnson. I've been involved in Drupal for nearly 10 years now and I'm currently working on our adoption of Drupal 8 at Johnson & Johnson. Turn this on first. So I have to make some declaimers first just to say that these are my own views and not those of Johnson & Johnson. The choice of character is based on role and apologies for a few British cultural references. So to set the scene I want to tell a story about poverty and debt. Let's make us think about what that means and I'm going to introduce Jim who's our character in this story. Jim is breaking out of a poverty trap. So from his great-grandfather, his family were all in manual jobs and you might think of some Dickensian story of deprivation that the family has been in. So poverty traps are quite well defined. Something that keeps people stuck where they can't generate the income to live on. Jim's broken out of this poverty trap by getting some education. It's normally well known as the best way of getting out of a poverty trap. Poverty traps are something that don't just affect individuals. Countries and companies can be affected as well. Many effects that cause poverty traps but for a company we might think about the culture and the retention of people in the company or monopolies that are affecting the way that company can work or bad governance within the company. So going back to our story, Jim is off to work now with his new education. He's got a job as a junior developer in a company but he's only contracting so he gets paid every two weeks when he's done his work and all of his money he's getting every two weeks he's going on all of those things. He's got his bills to pay, his living cost, his rent, his travel to work. But one day he's on his way to work and his car breaks down. So he can't get to work because his car's broken down and if he can't get to work he can't earn any money. So he's not got much options of how to continue. So off the car goes to the mechanic and he gets it repaired and then the invoice comes to get that repair work done and anyone that's been a car owner probably can sympathise with Jim at this point as he gets that bill and what does he do to continue so he has to use the credit card to pay that bill. And this is where it starts to be a problem. So he's got this bit of debt now that he's got on the credit card for paying off that invoice for the car work. He's continuing to go to work so he's getting his money every two weeks still but he's only still paying all of his living costs and that credit card bill coming every month interest going on there but he's only able to pay off his minimum payments every month and the debt's just getting bigger and bigger. And that's what we might see as the start of a debt spiral which again something well known, ever increasing levels of debt not just individuals or people. We see these stories in newspapers of short term payday loans that we see in the UK as well but companies again fall into this and even countries. We've seen this maybe thinking of somewhere like Greece might have been in a debt spiral back in 2015. So that's what we think of as those two classic things of poverty trap and debt spiral. But I think there's a similar story that affects us in software development as well. So this time our story is about Rosie and she's a product owner and she's done all the work, got the stories into a backlog, everything's ready and our team has started working, they're doing their sprints, building this software. They've started, first release, they've done. It's out and they've built this nice shippable product. They continue working through their sprints, second or third release comes out and it's all going really nicely. But then Rosie's boss Mr Toothle here comes and he's got some new stories that have got to be done ASAP. They need to be done now. Can you just push that into the backlog for this sprint? Anyone else experience this? So what's Rosie's options? Well she's going to use the credit card and ask the team can you just get this done? Can you do it? We need to get this done. I know that we might get some technical debt just find a way of getting it done and the story is sounding familiar, there's a bit of debt there. It's hanging on. Now we're talking about what we're planning and the product, we've got goals. Boss Mr Toothle wants everything to be delivered, all the features but back in that corner even if it's on the backlog it's down at the bottom to fix that debt. And there we go, the team's happy they've delivered version one here but underneath it is those debt bombs waiting to go off at some point and they're going to throw everything off. And in fact yesterday in a session building high performance teams that was given by Duncan Andrew he called this the snowball effect. He actually talked about this problem as well. So this isn't just my opinion or your experience actually academic research has been published demonstrating that teams who have adopted Scrum have seen a accumulation of huge amounts of technical debt. So my question to you is how do you feel about technical debt on your current project? Do you feel like this is the technical debt coming at you from behind or maybe you're one of these three, closed blind eye, you're just ignoring it, you don't want to hear about it. So we need to find new ways of thinking about an understanding technical debt. So can we bring in some sort of financial thinking? It's debt after all. We can think of other ways of thinking about debt. So in 2016 a group of academics derived what they call the 16-16-2 definition of technical debt. So this is an official academic research definition of technical debt. Something that's expedient to do in the short term that's going to cause debt or something that's going to have an ongoing additional cost of it. So we can look at debt and think about it as having two parts, the principle and the interest. The principle, again, two bits of that, the initial principle. So that's our saving, possibly that we gained by doing something quicker in the short term. Or the current principle. What is it now? So it's something we did six months, two years. I mean, we could even be talking about in software where it's ten years ago that this was done. So the current principle is going to be what would it cost now to fix this problem. Then let's think about the interest. So that's the cost of not fixing this technical debt. You've got the first one, the recurring interest. The additional cost incurred because of the presence of that technical debt. This could be performance, reduced velocity of the team, because it's taking them longer to do the work, reduced productivity, induced defects. So, and long-term maintainability of the product or project. And then our accruing interest. So, I'm sure you might all have experienced this as well. You know there's something in your software that other things that you're building are relying on that bit that's got some technical debt in. And you're building things on top of it. And each of these things that you build on top of it has got that debt built into it. So you're building on top of not quite right software. And this is again taken from that definition of technical debt. So I think we can split technical debt into three types of debt. Thinking about the debt that we're used to. So, our mortgage debt. This is things like bad code formatting or code smells. There's a low rate of interest on these. They're not causing necessarily cost. And the cost of repairing them or fixing them is not growing as the software project goes on. But there could be a big principle. It could take a lot of work to go through your project if you've got lots of lines of code that are badly formatted. Our next category alone, the type of loan that you take out for a car that you're going to buy. These are going to be maybe might have a smaller principle or still could be quite big cost of repairing. But they've got a higher rate of interest. Something like a performance issue that's costing you recurring interest every month because you've got to pay. So could fixing that performance issue half the cost of your service? Because you're going to reduce the need that you've got. And how does that principle, the cost of fixing that performance issue, compare with the extra cost of your service? So it's not just necessarily about the code and the extra cost of development. It's what's the wider picture of how does that debt affect the whole company as a wide on that project. Documentation might be that type of debt that's causing a problem on building on top of the not quite right software. You've not got the right documentation of the software that you've got. So when you start developing new software, you're going to make mistakes. It's going to take you longer to build that software because the documentation is not there for the developers. Particularly on long lived projects. I know that we've experienced this on projects where it's something that was built three years ago. You're now starting to do some improvement work to that project and you're not finding the information. And you've got to guess how it was built because no one's kept the documentation for it. And then the final one, credit cards. You might have something that's going to be interest free for a short period of time. 30 days interest free on your credit cards. But then if you don't pay it off in time, you're suddenly going to see huge rates of interest on it. And there might be times when it's right to use these different types of debt. An example where you might use the credit card debt is a quick fix or a business opportunity. Say you've got an e-commerce sites. You want to get some functionality out for Black Friday for some sale. Some special offer that you need to get your site doing. So you're going to build something. It's going to get out. You're running it for Black Friday. You know you didn't build it right. You know it's going to cost you more to run the servers because the performance is not as good. But the opportunity cost of missing out on the extra business on that sale is going to cover or be much bigger than the interest that you're paying on that technical debt. So there are times when debt is right to take on and right to manage. And that even includes that mortgage debt. How do you manage that in a long term big project that you've picked up? So that's our story about technical debt and how we get into those situations. But how do we think about better managing and measuring the technical debt in our projects? Because without knowing and understanding what's there, we can't really keep it going. So anyone doing scrum here? They think they're doing proper scrum. Anyone use burn up or burn down charts like this that show each sprint. We've delivered some value. We've delivered 10 story points to sprint. We're showing we've increased our velocity. Each sprint we're delivering a few more story points. I think there's a real big problem with this report that's missing a really big important thing. If you think about companies and how they report their accounts and finances, this is a bit like just looking at this company balance sheet here. This looks all right. We've got some assets at this company here, but this isn't the full story because we're not including the liabilities. This is what a balance sheet looks like when you look at a company, and this company's got a problem. It's in negative equity. We're not looking at that for our software projects. So what if we were looking at a report that looked more like this? This is our velocity and our story points at the top. We're delivering these story points, but there's a problem here with this project because it's just getting more and more debt each sprint that it's delivering. But we weren't seeing that before on our previous reports. If doing this sort of thing and this information isn't getting to the senior management that's looking at this project, they're really missing out on understanding what's going on in the project. They want to see this value being delivered, but it's a fallacy if they're not seeing the true cost that's being built into this project that's going to have all of that interest on it. So this is a sort of report that you want to see for your project. This is a project that's managing its technical debt. You can see here during sprint two there was some credit card debt went in there. There was something needed to be done. They built up the debt, but back in sprint three they paid it off. So we didn't see any velocity. We didn't see anything delivered, but they've paid that debt off. And if we were showing an average balance sheet across there we'd be still seeing a constant improvement, a constant level of increasing value in this project. But we're now understanding where the debt and the assets of that project are fitting in. So what can we do to better manage the finances in our projects? We need to report both the value delivered and the debt incurred. And there's a really important difference there about it being the value delivered. Because again on those reports we're saying how much effort we used to build things. But that might not equate to the value of what we've delivered. So there's some things might have low value, but take a lot of effort to deliver. So what's the actual value of the software that we're delivering to the business and how does that compare to the debt? So maybe you're asking how, but how do we actually calculate that debt and show it? We've got some things that we have about story points. That's our way of calculating effort. But how do we measure both these things? The value, and that needs to be shown as business value, something that the product owner should be doing is that the value they should be assigning to each of their stories that they're creating. What is the actual value that this is going to deliver to the business? And there's something wider there about understanding what the end goal is for the business for this project and how that fits into the value. But in terms of debt, we've got those component parts working out what the principle is of the debt that we have and what the interest is that's being incurred on it. Some things we can actually do quite easily. We can work out how much effort's involved in fixing some stuff and what that recurring cost is. We think we can reduce our server costs by 30% if we spend this much fixing this performance problem. One of the options, a tool that we can use for that is SonaCube. Is anyone here using SonaCube? I've got someone else down there using it. This is a static code analysis, open source software. There's some other SaaS products that will do this for you or linked into GitHub and Bitbucket. Every time you do a push, it will track and show you. It's doing code analysis in terms of looking at how well your code is commented, any bad formatting. It can pick up on some classic mistakes that are made in code. Really, this is something that's providing a continuous tracking and reporting that you might have used Coder as the Drupal module to report on how well your module's written. This is going to also try and work out how long it thinks it's going to take you to fix those problems. It will give you a value of the debt in days but it thinks it's going to cost to fix it. This shows you the principal cost, the cost of repairing debt but doesn't really help you figuring out what the interest is. How much is this actually costing and increasing in value? You might see that if you're tracking over a report you can see a graph in the background there. You can track how your costs, your interests might be being affected by this level of cost. A real difficult one is finding those costs that are caused by bugs or unknown issues. That's the real unknown that lots of us have on projects is where a bug is hidden in the system and what it's going to cause. It's where having good testing and things like that comes into keeping that debt as low as possible. The next step in... That was our reporting any debt that we've incurred but of course we want to make sure that we don't fall in to a debt problem. That's all about making sure that you have a good budget. I'm sure that there are other sessions, there have certainly been sessions at Drupalcons in the past about how should we think about our projects in terms of agile, using budgets and not using estimates and trying to think about different ways that we work working out the cost of a project. This budgeting needs to be both on the project-wide level but on an individual sprint level. It needs to be the responsibility of the team in the sprint to say we need to allocate this much of our time to paying off debt. Also, we want this much of our time to training and improving our education as a team because if your team is not including education in their time and everything's... This is a goal of getting everyone working on delivering code all the time. There's often a problem and again it was mentioned in the session yesterday. You're falling into that poverty trap issue that the team isn't able to deliver or improve because it's missing out on that education. Everyone in the team needs to understand the consequences of debt and this goes to all levels. I think developers and people working on the project experiencing very closely the consequences of technical debt but is everyone in the project the end-business users, the management, the project sponsor? Do they really understand the consequences of that debt that they're incurring? Accept the ownership for that debt as well. Back to that ticket that's come in, the ASAP ticket that needs to be done now. Can you just squeeze it in? Whoever's asking for that needs to accept the responsibility that they're going to generate technical debt and understand that they're going to have to pay that technical debt off at some point in the future. Something else that I think is important here is depreciation. If debt incurs interest, the value that we've delivered also is depreciating. This might be competition if you're building a product. Your competitor has built a new feature that's made your feature obsolete. A sudden big depreciation or changing needs from either your business, what the requirements are, or the end-users' expectations. Or even things like keeping it up-to-date with the platform that it's running on, update to iOS. The value of your delivered product is depreciated because the user interface expectations have reduced. There's different things out there that are showing a better option. That goes back to our report. We can't just think of value being delivered. We've got to think about how that value of what we've delivered in the past is changing. That depreciation of the value and the combination of our technical debt leads to something called the sunk cost fallacy. I don't know if any of you have experienced this, you think, on a project where your project and the whole project has become so big that the decisions being made about it are based on... We've already spent, we've already done this much. It's just that little bit further to get it there, to finish it and deliver it. But actually, the cost of getting that delivery might be more than, say, it's time to just go bankrupt on this project. There are projects where that's happened and lots of money has been wasted. Having those reports about where you are and where your debt sits compared to your value that you've delivered early on and throughout the project helped to make this decision much earlier in your project. Anyone in the UK might be aware that there was a project to deliver a patient record system for the National Health Service. This spent years and years in development. It was never delivered and it was a classic example of something like this, where the sunk cost was the problem and in the end the project effectively went bankrupt. So there's a moral to this story. Don't let technical debt kick you in the ass. It really will if you've got a project particularly. There's a little bit of one here. There was a story I was telling you earlier on about Rosie and Jim. Well, as with all good stories, they live happily ever after. Any questions? Anyone has. So there's a lot of ongoing discussion about that, but that's basically where the next questions are. How can we actually find some way of standardising, measuring technical debt? And it's along the same as the things like story points and value. We all know that you can't compare the Scrum teams based on how many story points they're delivering because the story points is just a measure of effort required that that team has set on some experience of what the project is required. You might have some one team that's delivering 90 points sprint and another one that's delivering 15. But for those two projects, the one that's delivering 15 might be delivering more value of the way that they're pointing their items. The same goes for value and the debt. Now, there are some ways that we can see debt as a monetary value, the cost of hosting a website. Maybe you can equate your performance to the cost of servers and say if we measure this, we're costing us extra $20,000 a year because we've got a performance problem that might only cost us $5,000 to fix because we just need to allocate the time for someone to spend a week fixing it. So there are some things like that. If I move on to the further information slide I've got here, so one place is later, the start of next year is actually the first an inaugural conference just on technical debt that's taking part as part of the international conference on software engineering. This is mainly academics from universities who are researching software engineering practices, how agile has changed the software engineering and also industry participants as well bringing in information. The basic question at this conference next year is what you just asked. How can we find standardised ways of measuring technical debts across different projects? And there's a great resource. Previously they were single day workshops held as parts of other conferences and there's an archive of the seven years history of those. These slides, of course, will be on the session page and these are linked through to where they go through. There's also a link there to a blog post that I read that originally inspired me to go on to investigate this sort of issue on software poverty and how this happens to projects and think about how we can fix it as well. So... Yeah. Is it that individual development is creating technical debt or is it more... I'd say going back to that, everyone in the team needs to be taking ownership. But yeah, you've got a question that the developers need to question. They're going to need to come back and say if we do it this way, you need to know that we're going to incur some technical debt and we need to add that on our debt register. But it might be, as I said, that you're doing that because you need to get this out. You know, as you say, it's quick. We can hard code this. We can get this out tomorrow if we hard code it. But it's going to take us three days to write a settings page and UI test that and make sure it all works. So it's the balance of understanding the value that it delivers to get that out tomorrow because you hard coded it and the debt that you're incurring to possibly have to go and fix it in the future. So actually, when I was discussing this, someone else that's recently joined... Johnson & Johnson in America had written previously a blog post categorising three people that caused technical debt or three views on technical debt. One is you've got the person, almost normally the developer who wants to go. I want to do this right. I want to make sure that we don't generate technical debt. We need to make sure we're doing the right thing. The next level is the broker. This is probably your product owner or your scrum master is the broker in this situation. And then you've got the demand. I want this done now as soon as possible. We need to get this done straight away. And that's usually someone from the business. But they're all responsible for managing and either stopping or understanding how they're creating technical debt. There's a question in here. You might have chosen an architecture and later might prove to be sometimes hard to predict. And I think agile projects make them a bit harder to make by architectural decisions because you are horizonist. I think it falls into the principle, the initial principle of your debt. You've made a decision that we're going to use this architecture. But you don't know, as you say, you don't know whether that's the right decision until later on. So you don't really know if it's actually a debt or not until later on in the project. And particularly, as you say, that scrum process leads to the higher probability of a debt incurring. And one of the presentations that was at one of these workshops was about the interest having probability in it as well. It's like a risk register as well. You've got some things that aren't incurring any interest. And your architecture might be one of those. It's like the mortgage type debt. It's low interest now. But then suddenly you discover it was the wrong choice. And it wasn't incurring any interest. And now there's something, a risk has changed. It's changed the state and suddenly it's now incurring a lot more interest. If we want to compare our models again, your fixed rate mortgage has finished. You're in the middle of an economic crash and now you had 1% mortgage and now it's an 8% interest rate on your mortgage. So I think these financial models can really help. We think about it like a debt. I should be doing all right on time. I think we have five more minutes. Any more questions or thoughts that anyone wants to say as an experience of technical debt and whether you think this sort of idea is really going to help your project? I think the biggest thing is actually when you... when a team is doing the backlog refinement or the story pointing, you've got to balance that. It needs more than just a point saying that this is the effort that's required. It needs those three combination of things. This is the value that this story is going to deliver to the business. This is the story points the effort is going to take to deliver it. And this is the principle debt that we're going to incur by delivering it this way. Or you might have three options. There's going to be this much effort that's going to balance against this much debt. Or we can deliver with no debt for this much effort. And it needs to be a business decision about where it fits with the value that's being delivered and the debt that's being incurred. Because some debt is right to take on. And I don't think we'll ever have any debt-free projects. We have too many reliances and we might have made that bad architecture a decision early on. I don't think we can ever avoid it, but it's the management of debt. Just as you would do. And the ongoing, having a budget allocation each sprint, we're going to spend 10% of every sprint paying off some technical debt. Making sure that it's allocated to that as well. All right, well, thanks very much. Please remember to review the session. My name is David Kitchen. If you want to get in touch, David Kitchen is where you can. Or any questions you've got and once we ask directly afterwards, I'll be here for a while before we'll go for lunch. Thanks very much.