 Good morning. It's a privilege to be here speaking at Agile Singapore. I think it's a good time to put our hands together, acknowledge the work of Stanley Forth and the amazing number of people who put this conference together. Having my mistake being all myself involved and doing a lot of technical conferences, I know how much work goes on and how much time it takes for a few people who care about their community to make a big difference. So thank you for the opportunity of sharing this wonderful event with you. My slides probably have the wrong background for this room so they'll all be available. I'm a fine example of how not to do PowerPoint, but I found when speaking in international audiences that people value the fact that there's more text on the slides so they can take them away later. So I'm not planning to go through every bullet on the screen. I'll probably just talk about the title line and a few other things and you'll have the slides to refer to it for detail. So my background is really building products. So I'm a products first person and occasionally I get sucked in to do consulting and I sort of apologize for being involved with the initial Agile Alliance because I actually thought it was going in a different direction than it's gone. I think there's a lot of nodding heads in this room and there's been a lot of polite phrases, Agile next, beyond Agile. Josh's talk yesterday, modern Agile. I think it's time to learn what we have and move on. So I want to start with kind of where I tend to get involved, not always with a CEO, but this is sort of the... Most CEOs eventually get sucked into Agile because they believe and they're looking for Agility and there's enough letters that look the same and of course the Agilista, when they want to do it so badly, they actually say when the executive says, well this will bring me like productivity and Agility and they know that's not really in the Agile contract but they kind of nod and the executive signs off the check and after that there's a massive transformation that goes on for years until people die of exhaustion. So this executive is very supportive but basically his problem or her problem is that they really need things to go faster and they need to get things done and so the mission is basically to turn up the dials on productivity to use Kent Beck's word. He wasn't talking about productivity particularly, he was talking about cutting code or features quickly and so really, and I know this is something that is really irritating for people, better, faster, cheaper. That's really, in all the years I've been doing software, the only thing that really matters to the business, the only thing that really matters to the customer. So it's basically, you know, the CEO is basically saying we need to raise the bar, we need to focus on value, oh yeah we always do that business value. All your stories have great business value on them I'm sure, along with acceptance criteria. We need to change our practices. Well in the Agile community we've changed our practices, right? We've been doing OO for 40 years. 40 years. What other generation has sat in the same technology for this time? You know, we're now competing successfully with COBOL, right? But unlike COBOL you can, which is maintainable, the Java and OO stuff is so complicated that you can't refactor it and you can't fix it. We need to refactor our organizations and we need to explore and experiment and I think one theme you heard throughout this conference is experimentation. Experiments are very important. You only learn from failings so what you want to do is fail often and fail fast to try and do that. So we've all had some sort of process, this is a process we used to call just-in-time software. It was developed in the late 80s. At that point, just-in-time software, we just ripped it off from Toyota just-in-time and we applied the same techniques that were applied to Toyota in the mid and late 80s. That process later could be called lean and agile in the large and if I'd known that I could actually pimp it successfully like all these other processes around, I should have written it up in a big book and started putting it on a website but I actually just got tired of trying to do this. The problem is that basically a process-driven development, lean and agile works. You know, you can get 10% in almost any organization. The good thing about agile is basically before agile, they got nothing done, now they barely get something done and you do improve quality if you do testing. A lot of people, of course, just do scrum without any testing. Seldom doesn't really have the impact on agility that the executive is really looking for and it encourages a lot of micro improvements. Think small, you will do small things. It often discourages, I mean, for many years, the Agilista, there was no notion of documentation or design. Those words were not allowed, architecture. And as a result, where does technical debt come from? It comes from agile developers doing sprints. They start and they go very quickly, look at this, I've got one feature, I've got two features, I've got four features. Oh, wine, wine, I need technical debt cards because I didn't think. I see so many projects with this. It's biased to the tools and technology today and the biggest problem is it's very hard to sustain in an organization because although we wish it would emerge, leadership matters. To sustain a large organization of transformation takes two or three outstanding leaders to make this happen. Unfortunately, in agile, like in governments and businesses, we are short leaders with the values that are very important. These values are really, really important. And I want to quote a good friend of mine, Nigel Dalton, who was at the Lonely Planet at the time. And I was talking to Nigel at the Lonely Planet, if you've not heard the story, the people doing SAP, the people in finance, the lawyers were all doing cards on the wall. It was a most amazing place to go and see it happen. And the last victory that Nigel has, he actually got the UX people to put up their designs while they were working on them because usually what they would do is hold them back until they had it perfect and say, here's the new UX. I know UX designers aren't like that, but they were at Lonely Planet. So we had everyone participating and I was standing at the wall looking at the board and I said, Nigel, I said, what happens if you leave? And he quoted Dave, agile is fragile. Six months later, the BBC that owns Lonely Planet restructured a change and essentially agile at Lonely Planet changed from what it was instantly. In six months, it was unrecognizable to what it was before. Fortunately, Nigel went on to be CEO of realestate.com and is working miracles there. So you can fix your generic process. You can move from projects to products. Lean, okay, which is the mother of agile and the key thing of lean, of course, is think, value, waste, which is the inverse of value, and learning because then you can go faster and adopt. But lean is for a product culture. Products are from Venus. Projects are from Mars. And the classic problem in an IT culture is you have projects. So if you're going to do agile at all, you have to actually go to some sort of portfolio management. So it all starts at the top. And I'll mention this horrible word PMO but basically this is the portfolio management office. I got rid of that project word right away. And essentially, what you want to go is from doing 200 projects trying to get to 300 a year, which is where this client started to basically say, look, we've got seven products of which there's really only three that matter or seven, you know, big programs and those programs are kind of stories all the way down. When you organize things this way, you have much fewer things to do. So you're doing less. And you're putting all your efforts into that. And there are dedicated feature and component teams to be able to support that. So if you're really going to be effective, doing, you know, sort of class, you know, what what most people would call lean development. It's really a product culture you have to have. And if you don't have that, you will, you will won't be able to succeed to the same level that you can if you do. Another big issue, of course, is tangible requirements. I was way back working with Uncle Bob. And I basically, Uncle Bob, of course, then had cards that had white on the back and stories on the front. And I basically said, well, the problem I see is that people don't know how to, you know, how is this story tangible? So at that point, we coined the term acceptance criteria, which is basically how high do I need to jump for this story to pass. One of the things we found is that basically, an activity which we use heavily is called envisioning. Of course, we know the waterfall problem, you get a calculator, you need a backpack to carry. Or you get you get the agile pitfall problem, the sort of Bob Martin calculator, being unfair to Uncle Bob, since he he's so successful and so great that he can bear with my comments. And basically, people really focused, they go look at this, I've got this whole thing, and I made the plastics and everything like that. And I've got to calculate with plus. Really cool. Minimal viable feature, right? Minimal viable product. Not interesting. Not only you have to quote Steve blank, you have to have a customer. It's much more important you have the right customer. So what envisioning does it's a way to actually design quickly and try and get out the simplistic naive design mistakes so you don't end up immediately with a refactoring cost. Remember, refactoring came from a wonderful language not much used anymore called small talk where refactoring was really easy. It's nowhere as easy in a language like C++ or C sharp or Java. So this is an example from a very large product that products actually been bought by IBM. It looks at global websites tells it says whether accessible looks for security violations and so on. So it basically audits massive corporate websites. It was a wonderful product built in Canada where I'm from. You know, really solid, robust, great engineering with the most ugly user interface you've ever seen. So we were commissioned to try and redesign the product from the front end. So the first thing we did was build a low fidelity prototype. You can see little red dots there you can click on those things and run around it. Built a different alternatives. We accidentally put it on a publicly visible web page and sent a couple of links to our customers was a little bit of awful about that because the UX designers hadn't approved it and no one else had either but the customers had some really good input. We did a next iteration which basically went from eight, eight web pages to 48. Lot more detail. And then we did the final one went going to 148 screens. This is a very complicated product because you have to configure all the servers in the organization and a whole lot of stuff. I'm not not suggesting that this is great because of the number of the complexity of the UI. The investment was less than 2% of a two year product engineering effort to build this UI. How was it built? And it was built essentially part time over a period of three months. That's really because people weren't there for the meetings. You couldn't get the engineering team, the UI team together, you couldn't meet with the customers. So it's probably amounted to about six, eight weeks work. What's it built with? interactive PowerPoint. Very soft. Isn't that bad? Well, it is for lots of things. But the timings to go out and collect all the data from the server are real. So when you move from page to page and when you see all the servers filled in or all the web pages, those timings are actually measured against the real product. So it's authentic enough that people can play with it. Imagine doing this with story cards. Come on, folks. Story cards. Oh, well, it's always good to have training wheels. When are we going to take them off? All right, so enough about the problems of trying to deal with processes and changes. In the end, most organizations like to get two or three things that are important down a year. And if you want to make an impact in the business, and you want to build a reputation, you want to bring value to the business. Typically, what you want to do is find those two or three projects. And usually, they're they're they're projects that are responsible to someone who isn't enthusiastic about new things. Right, someone who has a track record for delivery because it's important to the business. So they're going to be very interested in you meeting your deadlines. And so in value driven development, which is just a term I coined, I'm not trademarking it, there won't be any more DDs from me, there's enough already. But what we really do is quickly envision a solution and experiment. We think out of the box, try and innovate, change the technology, change the people, change the hardware, change the software. We always try and use hardware because hardware saves most software. We're only alive today programming with these huge wads of stuff because the hardware is so fast. And we try and use appropriate practices. So this is kind of like a prototyping activity or whatever it is. The next part is the is really the important thing. We actually validated at scale. Because we all know it's easy to build prototypes, and then find out that they fall over. If you don't actually run them with billions of rows of data, or petabyte of data and pass it through, you know, that's what validation at scale is. And then we have to be able to complete the entire project in three or four months. So you're looking for a project that's going to save at least a million or more dollars. You're looking for, you know, ideally something like five million, you get too big, you won't be able to do it in the quarter. And then you have to put in a management buy in process that deals with the business issues, and deals with the technical issues. And they're listed there. So I won't go through them. You want to leverage as many innovations as possible. So you want to the easiest way to simplify a problem is to change the way the business works. If you're working with really senior business executives, you tell them, look, I can save this money. All we have to do is change the business the way the business works. Often they go, well, that makes sense. Otherwise you're in there fighting, trying to do some cruel on a natural act with the software when the whole way the business works doesn't make sense. So that's where the high value changes are. And that's where business agility, which I think is a very important topic. And more and more people are talking about that, not just the software. We can look at the hardware and we can look at the software and we'll do that. Now it turns out there's some ideal places to have this intervention. The intervention is really easiest to do and to insert at the point where there's data flowing. So these are the points where you can insert new technology, new practices in a very specific point and really impact the value. Because again, the optimization of values in many cases tied to the optimization of flow. So if you can take something that takes manual assembly and you can automate it or you can put a very fast pipe so that data moves quickly, then you can do it. There's a series of techniques here and I'll just leave them on the slide. The other thing is if you're going to do something in a very short period, you have to be very selective with the code you're looking at. So computational bottlenecks is a typical one. Highly structured rule calculations because they can be organized with a rule engine and optimize points of high variability because those can be taken from code driven to data driven and points with large number of defects because in the end, if you don't kill the module, it'll kill you. That one can be very different felt to do in three or four months. So fast software to me is about how fast hardware and simplicity. Simplicity in code, simplicity in architecture, the flexibility of data versus code. One of the great things and I think today simplicity is also enabled by what I'll call functional thinking. I won't use the F word too heavily. I'll use the little F word, which I'll call functional thinking or collection-oriented programming. And often with things like OO and frameworks, frameworks basically, you know, when the conversation starts with a word framework, what you just heard is, look, let's make this as complicated as possible. What we need is a framework. A framework is an unfinished component. Right. The framework was supposed to be a component with a value based API, but we didn't finish it. So we gave it to other people so we can inject their dependencies into the program. Because you have to know what's inside the framework to use it. So what you do is every time you buy into a framework, you buy into a bunch of someone else's dependencies, usually unfinished. Very, very high cost. So let's start by looking at today's hardware very quickly. $20,000 last May buys you a computer with a terabyte of RAM, 40 terabytes of disk and 64 cores. That's a lot more than even in some parts of the world a programmer costs. A lot less rather. Most people's active data, you know, I'm not talking about Google and so on. The keys that you need to join, the things you need, most people's active data fits in one to 10 terabytes. Next year we will have 10 terabytes of non-volatile memory on chips, on dims and so on. So your machines are going to be loaded with memory. 10 terabytes non-volatile memory on this laptop. 3D X point and to be these 3D memory chips. This is a very different world. When you have that much memory, you don't have to have, you know, the super-duper red black tree. You can just put it in a collection and sort it. You can just search it because the fastest way to go through memory with layers of caches is just to walk the pages linearly. Hardware is very cheap, especially compared to programmers. You can do very, very simple things with this kind of hardware. With 10 terabytes, what is a database? Three years to a petabyte in Iraq. Folks, the world is changing. Hardware makes software easier. What we've been doing is trying to figure out how to make it harder for the hardware by piling more and more junk and frameworks on top of it. Stacks and full stack developer, right? Fill it with as much software as possible and see if we can slow the machine down. Ship less code. I know people don't like the word K-Locks, but I'm old. In the end, the amount of code I ship is the amount of code I have to fix. So if we could, we have, we've always had great pride in trying to say that every release had to have 10% code than the previous release. It's hard to do, but it's a nice goal. Dependencies kill you. Now, of course, the solution to this is microservice architectures. However, I warn you that I've seen lots of dependency preserving microservice architectures. Because just because you put them in separate processes doesn't mean there aren't version signatures in the cyclic dependencies that have you the same problem of doing a build and deploy. So abstractions bloat. Most of the time, there aren't any objects. Sorry, yes. I pimped small talk and objects for many years and I want to confess and apologize for that. We actually thought that objects were components and message oriented, but they ended up being frameworks and tons and tons of code with huge methods and so on, which is not what the intent was at all. But objects really don't exist in most worlds. Apologies to Eric Evans, basically. Domain driven design is a great thinking idea. It's just a really bad way to implement software. Typically means you bring an ORM and you want to know the most inefficient point in typical Java architects or she's our architect. It's typically the OIM, the object relational mapping. It brings in zillions of classes, raises the garbage collector, causes all sorts of performance problems. You just hit delete. Data is rectangular. Hardware is rectangular. Objects around bloated and slow. Sorry. We need to realize that objects are great for capturing complexity from building that simulation model. But most businesses do not have any business objects. They have data and they have functional transformations that operate on that data, which you can implement in an object oriented programming language if the only thing you were given is a mid-tier server with Java or CCHAR. But most of the things that happen are actually very simple functional transformations. Now way back in times, I'm very old clearly, but you can tell by looking at me, the badge of honor was to learn table driven programming. This is what the systems programmers knew. And the idea was to basically express things very compactly in tables which were represented with data. For some reason this isn't taught anymore. It's really a very sad that it isn't because it's a secret to highly dynamic and variable code because data is much, much easier to change than programs. And there's all sorts of different kinds of tables and I'll give you some references in the talk. But basically tables are a very compact way to represent things. Much better than diagrams and much better than code. They can easily be done by normal people. The nice thing is if you fill in a decision table or a spreadsheet, you don't need to go and waste all your time taking this perfectly good spreadsheet and writing cucumber code, more code, to take a spreadsheet into something in tons of code, then writing all the Java code to make the cucumber code work with all the fragile stuff breaking over and over again. So we got two sets of code that's breaking when we could just run the spreadsheet and decision table directly. Why are we doing this to ourselves? Most business problems are really just tables. Tables and functions. These are very powerful abstractions they've been around for years. You can dip them, you can send them out on the fly, you can do hot deployment, all these things that people really talk about. They're really enabled if you let the data do the work instead of the code. I'll just give you an example. This is a large insurance system. It involves a whole bunch of legacy vendors, a bunch of enterprise application systems, things like SAP or PeopleSoft and so on, oracle financials. So there's a whole, it's a big mess. Large corporations are, they basically have a lot of legacy and they build it up and it's all working, but they want to put a new system in. They've already failed at this. They've already done the usual thing. We paid a censure $200 million and it didn't work, so we turned it off. Why do we keep repeating this? We just sort of rotate. Oh, a censure screwed it up. We'll have IBM this time, right? And next time we'll have CSC. Why does this nonsense go on just robbing us? This generation needs to grow up. We need to build the companies that can deliver value because we have them and communicate to our leaders in our government and our major corporations that we can build software with small teams that doesn't require wasting 10 million at a time as a minimum unit. So in this system, we went in there, we were supposed to do the usual, please come and spray agile dust all over everyone. Basically go, you're agile, you're agile, you're agile, right? And despite that I put the checkup to what they had to pass daily to an outrageous amount, they still took it, right? So we went in there and we said, okay, we'll have a crack team from object manner, they'll do the Java training. What people are there with Java skills? Oh, we've got some Java people. Oh, good. Well, turns out they didn't have any Java people. They actually went and found a few people down the road who knew C Sharp and they sprayed them green. So here we are going in advanced Java TDD session with people that didn't know how to program in Java. So at that point I just lost it, said we're out of here, called the team, we're out of here and Jeff who was upstairs came running downstairs said, no, no, no, he says, no, these people really know what's going on. I said, come on, we've been here, there's nobody home. Right? Yeah, we got IBM, we got other vendors, vendors don't know how to talk to each other, none of them do tests. You know, oh, that's the other big thing. In this world what they have is a notion of, you know, we have a great product for doing a ratings engine or whatever it is and you don't have to program it because it's configurable. There's the hype for you. Configurable, so you can use some crappy UI tool, you can't version the output, you can't get it in text form. And then it's sort of, oh, well you need a few changes, we'll customize it for you, specialize it. What that means is the entire testing problem is yours, which is what they count on. So we managed to convince the lawyers that acceptance tests mean executable acceptance tests and if the vendor actually provided the tests with the code, they would actually get the check right away, otherwise they would wait six months until we tested it or longer if needed. So we changed the process. So just as we were about to leave, Jeff called me upstairs and there was a room full of the most amazing BAs in the world. We need more business agilists. We need more people who understand the domain. They'd been sitting in this, you know, U.S. state in this small town for 20 years. They knew insurance inside out, commercial, personal, you know, home inside out and they'd worked everything out because they, of course, no one ever came to talk to them, but they'd actually put in spreadsheets everything about insurance. Now they were unstructured spreadsheets. They took, you know, several days to print out and you had to stick them out. So we divided those spreadsheets up. We did a small team interface to the enterprise service bus, you know, but you had to have one in those days because IBM was there. What else do you need one for? And we were able to generate all the code from the work that the BAs did. It would have been hundreds, I don't know how many story cards it would have been. Please think, innovate. This is another system and this is one that's very well known and I do it because I'm thinking of Kent Beck today. There's a large HR company, which will remain nameless for this talk. They had very complex calculations and they, what happens is when they do the union negotiations for who's going to be paid, they actually capture everything in an Excel, right, and build models and they say this is what's going to cost the business and we do the benefit plan this way and everybody gets their, you know, birthday off as a holiday and all the negotiations that happened. And then they would give it to a group of cobalt programmers and if they could give it to a group of Java programmers, but it wouldn't even be slower. I really believe that's true, by the way. And of course this was great because we were the, you know, in these days, this was small talk, we were the small talk superheroes, small talk was amazing, you know, you could do anything with it, you could refactor as both the floor wax and dessert topping, I mean, it was really cool and Kent, and I forget who else went in there, it wasn't Ward but and they went in there and they taught them all small talk and they managed to get 12 to 15 percent improvement and the executive said this is what, you know, IBM small talk brought me like nothing, you know, get out. And then suddenly as Kent was looking away, he thought suppose we could take these spreadsheets and we could write a rule checker because spreadsheets are very undisciplined typically. So we wrote a rule checker, so Bob, you write the rule checker and what I'll do is I'll write an implementation of a spreadsheet and I'll run it inside the main frame talking to all those cobalt programs that's still running today in Java and it basically allowed the 400 cobalt, the 200 cobalt programmers to redeploy because there's lots of work and it meant that two people could actually maintain the code base. This is an example of a successful DSL effectively. This is another one, data migration. Basically, big bank, you have to be able to mine, you've got a company that basically has a product. It runs on a legacy database, you know. If you don't have one yet, don't worry. Every NoSQL database is going to be a big legacy database, right. Guaranteed, what a dumb idea. I'll explain why if I have time but basically with NoSQL you get to be really easy to store your data so that every other poor person that wants to do a report or anything else whether it has to write a program to parse it, take it apart. So what you've done is take your simplicity as the person who restores the data and pass all the complexity to all the consumers of the data. What a brilliant idea that is. So the challenge here was basically how do we deal with customers that's screaming. I can't even get a report on this stupid system. So the first thing is there is an easy way to do this. You can buy an ODBC driver that's really basically there's two or three companies that sell them. It looks like ODBC on the front and at the back you can just hook it up to your database. You hook it up, go to the first customer meeting and all of a sudden you realize wow, look at this. All the customers say, hey, my money's in because they're already locked into this product, right? It's running their business. Now they can get, they can do reports, but they're still nervous because they know it's got that funky old database underneath. You know, whatever, it must be probably Cassandra or something like that. So what's the next thing? Well, basically the next thing is that you basically look at how we can migrate the entire database to another technology and to make sure that we do that migration correctly because that's a very tricky problem. Instead of writing thousands of unit tests or other wonderful things, we actually pay an independent team to write an independent program to basically check that the migration worked but not do the migration. And that allows us to move the vendor to a regular sort of style relational or maybe an old SQL database and do it. Again, all this is done in three months. You want to basically simplify integration, basically start using Adam and Rest APIs. Here's a simple one. We got a factory automation system. People got 200 million dollars have installed specialized PLCs and controllers monitoring the factory floor all built with highly proprietary patented interfaces. Does anyone know ABB or Rockwell? So you're locked into this stuff but you want to have a modern look. You want to have a nice UI. How am I going to do this? Oh, it's going to be so expensive. Well, the first thing is you put TCB IP in, change a little hardware, support a stack. So now all the devices can talk in a standard way, not over a proprietary bus. Second thing you do, you just basically turn everything into Adam Adam feeds coming out of the devices. Poof. Now your websters can go at it. You just made yourself a lot. Your entire control system into essentially a web server. Very simple. Again, easy to do. There's lots and lots of things. If you are using data, please stop using text. Text is aggressively serial to parse. It's very expensive to parse. It's a mess. Schemeless databases is just a hell that you're doing to people. Right. If you insist on using JSON, at least put a version number in it and try to add things at the end as opposed to the front or in the middle. Again, it's a thousand times faster to process binary data and at least 10 to 100 less data to do it. So if you really want performance, the first thing you do with text is get rid of it. Convert everything to binary. There's a bunch of techniques called lamburgation which I don't have time to go into. Clearly, what we want to do is enable loose coupling wherever we can. I mentioned that you can use REST, Adam. You can simply use simple pipes and filters. Have you ever looked at these new microservice architectures, you know what they say, a data flow pipeline? Any of you, maybe your dad or granddad, used to use these funny diagrams called data flow diagrams. Did you know that you could actually run data flow diagrams on a computer with lots of processes very easily? What went wrong? We opted to find a way to get rid of data flow. We'd shake the diagram and it'd fall into some tree and then we'd code it so we couldn't maintain it anymore because we lost the essential abstraction which was the data flow. With the exception of Morrison who has a wonderful book on flow where the Japanese banks actually directly implemented the data flow so they actually execute data flow in their organization. Lots and lots of ways to do this. In case you think that microservices are new, all these things existed for a long time so there's lots and lots of literature out there. It's just not labeled microservices and unfortunately there's 2,000 microservices talks now so you may not be able to find any of this if you Google it. But if you use the keywords here you'll find some very interesting stories about how to do that. And if you happen to really want to understand how to do loosely coupled things, look at Erlang. Erlang is the source of inspiration for the ideas. The OTP library in particular. Go maybe cool but Erlang is kind of the mother of service oriented distributed fault tolerant computing and the OTP patterns. Many people look at the language but the OTP pattern library really describes all sorts of fault tolerant architectures. Well testing is still important. Fred talked yesterday so I won't talk about his style of testing which is not to do anything you just deploy it. That's a really good way and again all friends really doing is saying we're replacing the acceptance test with an SLA that's executable so if it doesn't meet if it doesn't meet the SLA we're going to shut it down. So it isn't nearly as irresponsible as it may sound. Who in this room uses QuickCheck? Oh boy. You all must be an agile you're still doing TDD. Oh my God. What are we going to do? Well there's something you can take and add it to your TDD toolkit. It's called QuickCheck was developed in functional programming it exists for every programming language. What you do is assert the properties of the program it'll generate as many tests as you want randomly. Not only does it do that because random testing is not a new idea what QuickCheck does is it finds the smallest sequence of failing tests. This is a really critical weapon if you're not using QuickCheck the next time I'm back in town well we're going to have to talk to your boss I don't know. You really want to know this alone is worth the price of the conference trust me. It makes testing so much better it generates random values of all the types in your language and you can do it for stateful stuff it's not just functional. There are two examples React which is a large distributed database people heard of React or LevelDB which is the database inside the browser. Both of these have 90 percent test coverage written by really really smart people React basically two weeks into the testing of React with QuickCheck eight critical bugs were found. This code is 90 percent test coverage is written by some of the brightest minds in the world. LevelDB out of Google QuickCheck found a sequence of 24 operations where LevelDB failed because it just tries all these things. For a week or two I understand that Google didn't believe it was true they issued a fix two weeks later a sequence of 36 operations caused LevelDB to fail. This is really really high value stuff and again it does require a little functional thinking. The only way I know to test UIs I spent a good part of my life building IDEs which are really hard to test from a use because they're changing all the time and I understand there's Selenium and so on and God bless and Jasmine and we use all those things but the best thing we use is basically whenever a certain amount of code goes into the system we measure it we basically automatically say okay we've changed too much we do a code freeze everybody bangs on everybody else's stuff this also has the side effect of actually knowing what the person beside you is programming in case they might not be there one day. We get hundreds of bugs in a couple of days and a couple of days more all the bugs are fixed you get all sorts of bugs that are never going to come up again off by four pixels and this is the best way that I know to really test a product that's got a user interface on it just constantly exercising it over and over again. Automated tests is a wish for serious coverage of UIs and finally I want to talk about collection-oriented programming I know a lot of people get bothered people start talking about monads and maps and all this funky stuff because it sounds like math and you know it comes from there but the real thing that about functional program is important is that you actually program very simply you just work with collections most of the things you can do without adding any abstractions you just work with collections tables, dictionaries, lists some of them may be sorted or not and that's really the excitement about doing it and the nice thing is you can do it in many languages even C sharp and Java if you try and ignore all the other stuff that you have learned already this is really a way to make things very simple and very powerful it makes programming much more productive and once you get beyond this you'll start learning about things like immutability and type checking I'm not going to take you to Haskell school and punish you because your types are wrong but you will realize that having a powerful type system with inference really helps and this is not an endorsement for Scala or F sharp because they don't have sound type systems unfortunately Scala's basically the needles for addicts basically I already got Java so I'll put some Scala on top I'm sorry if you want but if you're going to do functional programming in F sharp or Scala or any other programming language closure that would probably be my favorite out of that list first thing to do is go learn scheme or Haskell go take a real functional programming course so you get the ideas and then go to it so everything is we used to say if you want to learn objects learn small talk then you can go find out how to mess it up in your other all language because when you're in a language like Haskell or Scheme you only got functions so you don't have this mix up of things and try and separate your functional code from your object code this is the dominant way that people are programming today and the good news is that JavaScript is ruling the world and so there's a lot of functional extensions going to ES6 and so on unfortunately all the old stuff is still there too so you know you can use prototypes you can use classes you can use functions it's both the floor wax and desert topping again but so I'll finish with one case study this is the most recent thing I built on a very large client came to me and said we've paid a large company a whole lot of money we've got 512 processors in a cluster we've got a high speed bus connecting all these things we've got this great stuff but we're not seeming to get the results we want so what we're going to give you is one billionth of the budget we gave them to try and come up with something different so what they wanted the requirements were very simple we'd like an order of magnitude improve productivity sure the good news is you can sign up for these because nobody knows how to measure them anyway you know so if it's good enough for the user in the end they would and we're going to work with lots of data so we want to be interactive in hundreds of terabytes interactive response hundred terabytes so we want to be working with sort of order trillions of rows if we're doing it in a or you know column store sort of thing and we want to be able to have normal people use this people who are not programmers people who don't know how to do queries and so on and these are the sorts of problems that I love to fail on and I often fail so I mean I'm giving you the successful ones because that's what I'm supposed to do but I can happy to share at length all the things I've done wrong as well so basically we had to get this done very quickly and the other interesting problem is this also has to be able to run in a large computer cluster like a cloud but it also has to run on devices out in the field so it has to be able to distribute so simple problem right you got three months to do a study write a report and tell us how to do this and then you get the contract to do it so we got lucky again and basically we came up with this three level architecture we needed something that was interactive and fast that led very quickly to a very small selection of technologies I work for the company the company that I currently work for bought this from me so I'm not going to pimp the technology here but basically we found a very very fast functional vector engine that would actually be interactive and be able to work with tons and tons of data and we have time in nanoseconds is the basic type so we can do sliding windows over time at any point in time which if you want to look at lots of packets you need to be able to do it's a common thing for streaming data we basically build an expert programming model so we have a functional language it's a little more friendly to use on top and we basically call C whatever you know we talk to R Python you name it we interact with it and then on top of that we have domain analysts who are specialized in different domains some of them know how to program some know how to program in Fortran some know how to program in R some can use spreadsheets so what we build is we built a set of visual and textual DSLs to support them plus they let them use regular office stuff in MATLAB so essentially you know we've got a spreadsheet that's like Google Docs except it'll work out you can put a trillion elements in any cell like a trillion rows of data built very quickly by a team of eight people the reason it was so many is the javascript stuff was huge pain it's a complete ID in javascript on the client does server side rendering because D3 won't scale if you want to look at large volumes of data and bin them and pack them and you want to be able to transmit the motor device you don't want to do that on the javascript side so we use high performance SVG rendering we bucket in parallel so we do heat maps and things like that so we can get performance then we just push the ping files with descriptors so you can zoom back in it just shows you that basically if you actually go and think out of the box you may have to pick a wild technology but we got such a mint productivity there and the thing is well we found out that from an engineering point of view this technology was great from a user point of view it was terrible so we had to go around and build on that stuff on top and there was much more work in the actual making it usable to end users not surprisingly than it was in the others but we were able to do this again small team of people with the focus and the key thing is driving the value and having to get something to the customer every quarter that works I want to thank you for the time you've allowed me to speak I hope I've given you a little bit motivation to try and pick some try and pick those few problems in your business that really matter this year and deliver value and really focus on points where there's data flowing where you can insert things because that's often when you can get big leverage and you can slide a new technology or a new approach in there without disrupting all the rest try and focus on building small teams that can do this and use that as the way to transform your company build small teams that can do this intervention as opposed to trying to spread all your money over everyone spraying them with whatever holy water is popular today don't teach everybody functional programming that's what the next thing would be right we did oh oh and now you did agile what's next well it's obviously functional programming I'll make you all cloudy think and apply to where the value is going to be in your business and look at the legacy systems all the smart people want to go build all the cool stuff but you can be very relevant in your company by taking the data in a legacy system and making it usable and consumable there is a fortune in gold in legacy systems and in making things work and you can do that with very clever technology I've had years of being able to use any programming language I want any technique and practices I want because I could help someone that really had a problem get it done and in the end they really didn't care what technology I use and today with running things as services the vendors usually control the platform now when you run things as service nobody cares what you write the service in Fred George's and our program lots of people are building services in closure you know I happen to build them in Q you can build them whatever language you want and as long as you interoperate with standard protocols you can do this you can build software very fast if you focus on it and if you build a team that has the right expertise when you've got to go work with a legacy system I'm working on a project right now that's looking at oracle financials and another banking system I don't know anything about those but I'm going to pay you know $5,000 a day for the best person I know to do that because they can tell me how to interface to do it they can answer all those questions versus me talking to someone that doesn't know anything how to worst reading the manuals so assembling the key people in the team the right expertise sometimes you do need specialists sometimes you need that deep knowledge to be able to interface to these systems so it's very important to do that I want to thank you for your time and I just really appreciate the opportunity to speak to you