 I'm very provocative but very insightful, always insightful speaker from HL Singapore a couple of years back and I'm very grateful today for bringing young conferences to Australia and out of South-East Asia as well so this is a great effort by Dave and the team or those and everyone else So you mentioned the next conference will be in September in Singapore and without further ado, welcome Dave Can you hear me alright or do I need a microphone? Did somebody say microphone? Okay alright, I'm used to one of these, not one of these Sorry Dave So thank you very much, Titan Soft and Agile and Engineering SG We really appreciate all the support and also our Platinum sponsor this year is Credit Swiss We're very appreciative that they decide to support the community and hopefully that means that they're going to be doing a lot of great things in Credit Swiss and I look forward to finding out about that We have a couple more Yow Knights, my apologies we had two Yow Knights planned one for February and one for March and both the speakers had problems with their schedules and their employers so we're a little late this year but our next Yow Knight will be on the 29th of May and Evan Laibor who you may know of because he spent some time in here working on transformations he formed something called the Business Agility Conference and we're doing the first Business Agility Conference outside of the US in Australia in September so if you're interested, Business Agility is basically about agility outside of IT so how do you change marketing, how do you change HR, how do you change the other business functions and finance and so on and some of you may know that that's actually required because a lot of the problems as an Agile development team is essentially completely confounded that the business doesn't drink the same Kool-Aid Our next Yow Knight is in June and Sebastian von Conrad there's a big software architecture conference in the United States called Saturn and last year he won the best presentation award for his work on event sourcing he's with a new company in Australia called Culture Amp and Sebastian will be here with you in June talking about event sourcing we have a conference coming up we've announced a good number of the speakers first one is someone who's just relocated to Singapore, Gregor Hoppe who's now back with Google, was the chief architect for Alliance and he wrote a small book that some people know called Enterprise Integration Patterns and another one that says 37 things that one architect knows and Gregor spoke for us here last year so some of you may have heard him next part of this is Goiko, specification by example you may have heard of that and he's got a few other books like Impact Mapping and so on so he wasn't able to join us in Hong Kong but he made it to Singapore last year and hopefully we will definitely be seeing him here the next person you may have know also is a book author called Dave Farley you may know the other author of that book it's called Jazz Humble Continuous Delivery so Dave will be here but he also talks a lot about high performance systems the next person is Jeff Patton, you may have heard of him as well known for his work on Agile products and he has a little book called Story Mapping which probably many people consider to be one of the Bibles in Agile they all be doing workshops as well two days before the conference and we have Heidi, I don't think many people know Heidi some of them may have heard of her, she was an Agile India she's an expert on dynamic teaming, putting teams together changing the teams and doing a very very important activity because many organizations are really bad at team formation and changing and so on and we have Michael Nygaard, he has a new version of his book called Release It he's a specialist in architecture and DevOps and very well known and then they have some fat old bald guy and then we've got a few more to be announced so tonight this is a talk which I get asked to give there's basically only a few people that talk about legacy because who wants to talk about legacy because obviously that's boring and no one wants to work on that terrible stuff you want to work on it, you want to program in Scala or Go or something exciting use Kubernetes even though it doesn't work yet but you can try it and see where the security holes are for your corporation and the two people to get asked a lot are Michael Feathers who's a very good friend who I work with at Object Mentor I was a managing director for Object Mentor when Dean Wompler and Bob Martin and a bunch of characters there and so I only do this talk, it sort of comes out of the box every so many years and this year I was forced to pull it out for Go to Chicago and unfortunately other people started hearing about it and so they said would you do it again so I do have some I'm sort of a bit opinionated, some of you may have heard me before I'm an old professor so I sort of believe in sort of poking the students so I will say many things and you'll have to figure out yourself which ones are true but I believe a very large percentage of them and I think one of the things that Yao can do is that the people we're bringing are people who have seen the movie that is they've actually built a lot of software made a lot of mistakes and what we'd like to do is share our experiences with you so I think the biggest thing that Yao does is bring people who actually have built a lot of software and made a lot of mistakes and hopefully avoid some of those things, I guarantee you you will make some of the same ones because there's no needle for experience right if you had a needle and you could just say bend over I'll give you all the experience I have that would be wonderful so this is just an outline for the talk I won't bore you taking it through the outline we'll just get going I think it's very important, I think everyone knows what a legacy is an application with substantial value most companies, I'll pick any large companies such as Oracle have not made money with their new products they're still making money from their legacy products the Oracle database, Oracle financials and so on many new companies find out that their wonderful Ruby on Rails tarball that they can't refactor anything with is now after 5 years a legacy state by definition I consider a legacy basically anything that's gone past 3 releases without a major rewrite so if you're on release 5 you're on that legacy train with the legacy pane but it's generating money and it's really scary to touch none of you will have that problem of course so it's classic, there's no documentation there's not sufficient documents lack of tests, there's lack of knowledge of the code base because the people have moved on or died and it's probably in a language that you don't want to program everyone likes working in Colbal, BL1, RPG just imagine those poor people who have to pick up that JavaScript code that they're working on so the reason we want to improve and I'll talk about evolve and I use that word for a reason and you'll understand what we want to do first of all for most people what they want to do with legacy is basically improve access to the data because typically the legacy system locks the path of access to the data and blocks people from getting easy access that's the biggest thing they want to do, the other thing is they actually have to change the functionality either because they want to be competitive for business reasons or because they have to comply with regulatory stuff it's a major problem in the finance industry I'm sure not in your bank but in other banks they often sort the backlog by which one will get us the smallest fine there were a few people from banks here and the other one is basically because you want to be able to improve the flow basically you want to be able to do more faster or get real time performance out of a system that was never designed for real time so there are usual solutions for this and I believe in Singapore you've all experienced probably all of these solutions the first one is to give it to someone else usually in another country if you're really bad at it you pick a country that's 12 hours apart from you and then you have both the people in India and the people in the United States, I'm Canadian by the way so you'll see I have a sense of humor so that way you can inflict the maximal pain on both the people in India by having them stay up at a regular time or get up at a really early hour in the morning and the same in the United States just the opposite time fortunately we're learning this is not the way to do things rewrite it in a modern language, ah yeah that's what I want to do we're going to redo this system in Go or Scala or whatever your favorite language is you know the one you don't know yet I'm sure you can do it with F sharp I guess that's it and then the other thing you can do is you can basically sort of take the classic TDD book and you've got Michael Feather's book and you can sit there and Michael Feather will show you how to work with a small number like 10 classes and spend 42 years to get it done the book is right, Michael's is right it just does not work because it's just too hard now there are examples of systems that have been rewritten I worked a company called Xerox you may have heard of Xerox park there's a couple of books one called fumbling the future they invented all sorts of things and none of which they made any money from because they got copied including the Macintosh and Xerox sold its computer business to Honeywell and they completely rewrote the operating system in a new programming language rewrote the whole operating system and made it functionally complete believe it or not they also replicated all the bugs then with the other previous system another one is companies doing microservices we know about microservices this is you want a whole lot of processes that are loosely coupled and one of the classic mistakes in taking a monolith and turning it into microservices you get what we call dependency preserving microservices so even though you've now got it into 20 microservices from one monolith the dependencies are all there so you still can't deploy it it's a very common failure mode for people doing microservices so there's successful rewrites but essentially defects preserved so I want to go this word technical debt because this is the big club that the agile guys bring up look you know there's this thing called technical debt and if we don't take care of that boss and put some money in there and get some technical debt cards we're going to be in big trouble and of course no one actually looks at the definition of technical debt which comes from Ward Cunningham and Ward was basically talking about he was actually working on a financial system at the time and actually the financial system that the first test driven development book that can't row the same financial system so the examples in that first testing book and basically the whole idea was that when you're building software you often don't know what you're doing that's for sure true and agile and so with no design and no architecture you're guaranteed to make mistakes with design art some design architecture you'll just reduce the number of mistakes but you'll still make them so the real question is how do I do this well in Ward's words what you do is you write a little code and then you realize no that's not quite right and then you refactor it and the technical debt is what builds up in between those refactorings the point is that Ward was talking about days not weeks or months and then a lot of people went off and made a big business about technical debt and so on but it's absolutely not what Ward was talking about you know you can't reduce technical debt that's accumulated over weeks or months most of the technical debt I see from agile projects starts because the agile team is so enthusiastic that they do almost no design prototyping up front they go for six to eight sprints and then they beg for technical debt cards right and the problem is as soon as you've got a lot of code you're in trouble refactoring this is another one of the golden touchstones refactoring works really well for really small programs there are no serious refactoring tools for a big legacy or any large amount of code move method you know move these that's not what you need if you're going to do serious refactoring you need that real metaprogramming environment and none exist yet today I think it would be a great product and a wonderful thing to work on but there's nothing but of course many of you go to your management and again we really looked at this and what we really need to do is we need to do a major refactoring and you're lying because what you're going to do is rewrite the entire bloody thing under the guise that it's just a refactoring tell me it's not true it's definitely not equivalence preserving and you don't have the test to make sure it works either oh but you use that word how many of you in the project report we're just going to do a refactoring it'll take 22 people in 72 months so now one of the favorite things to do for companies is to basically find something that they can make everybody transform everyone so this could be learn object oriented we're going to make everybody this year object oriented or we're going to make them all agile or we're going to teach them all to use functional programming or we're all going to use containers or whatever whatever thing it is it has to be done in a year and you know we have enough money so we can spray a little on all of you IBM did 300,000 people in one year and made them agile it did that by getting them in large groups and spraying them with agile holy water the problem is systematic transformation does very little for people it takes scarce resources which could be applied to do important things and sprays them around right people do agile and they put scrum in without any continuous delivery technical practices hey great how many the KPI for the MD is how many scrum masters they are in the year the MD got paid nothing happened so the problem is if you make systematic changes or you end up having to tackle a large co-base you have to put all these tests you don't have this is just a mission impossible some people succeed but most people don't and many of you have been on this project here's your opportunity we're going to do a rewrite of this major system you'll just be on it for a few months sure so what I want to talk about is a different approach and actually there's a pair talk with this keynote I did at agile a couple years ago and this is I call targeted value driven development and there's no book or anything else coming out on value driven development I just knew how I could pick a catchy title and the key thing here is we want to find simple things that we can do that will give large return on investment so what we're going to do is essentially do tactical interventions we're going to go in and blow something up replace it with something else that adds lots of value and it has to be targeted because otherwise it will never succeed targeted basically there's another way for saying we have to minimize the dependencies because the hell of software is dealing with dependencies so what we want to be able to do then is choose projects we can be very narrowly scoped and typically I'll claim that most of the interesting things and most of the way to make money doing this and I've been very fortunate to get away with doing a lot of crazy things with new technology working on legacy systems the challenge is to find a very very narrow scope and I'll claim that it's much easier to look at data than it is to look at code but let's talk with code and what we want to do is put a small team on it and we want to deliver it in a quarter nothing the other thing we want to do is we want to try and use some trick to make things go faster when one looks at a factory I mean if you're a student of lean and you should be there's only really one book you have to read you can read Mary's because it's very motivating but if you really want to understand lean principles read Don Reinerson's book called Flow it'll hurt your head because there's some queuing theory and some other things in it almost every person I know had to read it through or three times before they got it but it's a very very important book and hopefully we'll get Don here next year he had some family issues this year and wasn't able to make it so what we want to do is somehow come up with some clever idea think out of the box saying okay what could we do to change the game because we know if we approach it the conventional way we're not going to make a difference we'll just be there with a pick you know picking away at Code Mountain and it'll take us forever so the first innovation is very simple ask the business why do we have to do it this way no code approach the best code you can write is zero right never has any defects often there are simple changes in the business that you can make that will allow you to make a big improvement there's a story of the there's in the United States many years ago there's ice cream you get in a little cup they call this sort of Dixie cup right and you get the ice cream you take the top it's got a little wooden spoon in it and when the company that was manufacturing this when they first did it when they delivered the ice cream they found out that when you opened it it looked like it was half full because the vibration the ice cream settled they tried everything so finally they posted a big reward a very very large amount of money if someone can come up with an invention that would solve this problem and there were many engineers and you know many food producers and so on that I worked on this and one guy said I've got the solution and they said well you have to show us he said well you have to sign that I get the money he basically filled the cup and turned it upside down so the ice cream froze stuck to the top pretty important satisfied the company customers never knew there was air at the bottom it still settled but they shipped them that way that wasn't fun we didn't write any software so the next thing we can leverage is the one thing that's saving software year after year is hardware because hardware gets cheaper and cheaper and more capacity so that we can suck up all this you know we basically can burn up the cycles from all the bad software that gets written another thing we can do is we can look at using new algorithms database cloud and I put ML in here because if you don't have ML on a slide today it's not a good talk and then I can look at newer software practices so these are all things that I can actually do to try and make a change okay how do you get management buy in to do something radical because we're going to insert some wild and crazy technologies or some different things you know in doing this I worked in programming language you've never heard of database technologies you never heard of all of which were very new and very risky so the first thing is you had a clear measurable goal you need to make sure it's something that matters 2 million to 10 million more or 15% return if you're doing something that matters you'll find a business sponsor that actually cares about it and it'll stand behind you most people have no business value on their story cards right they get priority order by someone who hasn't even connected to the business that's why you need business agility so short implementation time the other thing is we want to not impact the day to day operations because it's so easy you know to install and have everyone change their production systems they're very flexible like we like to wreck the schema for you redeploying containers that's really easy to do in a legacy environment I know companies today that basically say look we can provision a new VM in a month that's a feature they've been working on it for 2 years sorry if I picked your favorite bank so the technical side is we need a team of good people good business people and good technical people that is people have experience and basically can work under pressure because this is always a pressure situation and not everyone can do this but it's short we want to have specialist technical skills working on a legacy system and it happens to be something like SAP BW Oracle financials SAP BW there's billions of tables in there you could get lost forever if you don't understand the technology so you need someone who understands that an expert in that technology not just someone that use SAP BW or whatever someone who knows SAP BW inside out it's going to be a little bit simpler for that particular thing going to be critical you can't do it just by reading the code yourself or doing something like that so you need specialist resources you want localized changes and minimal dependencies you want a clear service level agreement I want to be able to measure that what I'm delivering is actually delivered if I can't measure it I can't improve it so I want to be able to measure it so the first thing I do is what everybody who wants to use a new technology does they do a proof of concept well often it's called a pilot pilot means it's never going to work pilot means it doesn't matter proof of concept basically says okay so I'm going to implement this and I'm going to use Scala because Scala is cool you know so I'm going to try and do this and you build something pretty quick because you just took two guys at work with Martin Edersky building Scala and their experts so they figured out how to do all this themselves but the one thing they don't do is the next step and this is why most new technology fails most technology won't handle the pressure performance space storage all of a sudden you find out oh right I'm new on Scala but it's running on Java and Java oh yeah garbage collector oh yeah generating functions that turn into objects oh yeah oh continuations can't really do that functional programming likes continuations barf does not scale so scaling is really the thing and nothing goes ahead the other thing is you want to be able to deploy it right in the system ideally what we'd like to do is roll up a little box and insert it right in line in the system and just turn the box on connecting the inputs together and the outputs and do that not always that easy something I'll talk about if it's critical systems independent testing is still considered to be the best practice seldom practice except in things like aerospace where it's called IV and V so if it's selected code photos you know this is the stuff where Michael Feather stuff is really a good idea right small computational bottlenecks you can measure it I just was Martin Thompson in Chicago he may know Martin he's a performance expert he just got asked to go work at a large company and since it's being videoed I'll just cut off the rest and turns out they were using a different programming language which Martin has never used so they were very upset that the CEO was bringing in a performance expert for a technology that he didn't know the story goes that Martin offered to do it for free because he said look if I can't find any problems don't bother paying me so he spun up the standard performance tools you know the ones that you're all trained to use because every time you go to a company they show you how to measure the performance on a simple Linux or Windows machine you all can do that right what's the speed in space process overhead memory network and he went oh can I see the code and they said well you don't know Python but here you go they looked at the code moved something seven lines got a 15% improvement in performance and they said but you don't know Python he says no I don't but it's code the simple tools with a trick they gave away the fact that basically there was a whole lot of database I.L. going on and the database I.L. was inside a loop and didn't need to be most big performance wins are simple do your homework another thing that many systems have is they have a lot of things that change could be a rating engines for insurance could be the you know and they're classically these are hard coded with you know thousands of statements or K statements and they're very difficult to modify success well you know they're hard to test because there's a lot of possibilities so these are things that you want to change so the typical things you can do for an insertion point is that if you have something of high variability or some small little hotspot you can insert a small interpreter that's a small execution engine which works on data which we call a table this was best practice when I graduated from university I would take my nice piece of assembler code and I'd have all this great you know branching and everything in there I thought well organized but nice labels and so on and the people who did this table driven programming it was called in those days once it looked at me and they said well Dave you know it's not bad looks like it's pretty good but it's completely unmaintainable because we're going to change things in here over the time and you'll have to jump all over and they showed me how I could use one of a number of small table interpreters to do this plus the space would be a lot smaller and I just have to edit the table I don't have to redeploy a jar file and try to figure out how to get it running in a container I can just edit it live plus there's all sorts of techniques for testing these so these are examples and you could use DSL's today there's one very widely used DSL cucumber how depressing so I get to write the code twice for something that I could completely automate I'll come back to that in a second how much time can you waste doing spec how about running by example and set a spec by example I'll give you some examples shortly so in my book most people go after the code and attacking code mountain is really really hard and you have to be really smart to do that and I'm not smart enough to do that so I look for the lazy way and typically what I'm looking for is how can I improve the flow in this system and so the thing that's often most valuable is to actually look for the places where you can add lots of value and if you actually look the most important thing in most companies is actually their data that's why the legacy system survived no matter how many bad implementations they are they still got their data and they could just go forget it we'll just go back to the old system we've got the data that's before you get Mongo or something like that where you lose the data if you kick the power out and it's very stable it turns out that the physical page format for Oracle databases and IBM DB2 has not changed in 30 years unlike the code which has moved around all over the place so the data is very stable and the nice thing is most of the transformations that you do on the data basically you're taking data source from here over to there and you're applying just a set of functional transformations sorry I didn't need to use the word functional they're just data transformations that's why functional programming is useful forget parallel all the hype for functional program is a parallelism but the real value in functional programming is that you're taking you're doing data transformations you know map reduce so often these places are the easiest the other thing they're easy to test and monitor because you've got the data flowing you can observe it you can observe the input and you can reserve the output so this is really a sweet spot in my experience for impacting a legacy system so the places you can do this are any place where data is moving or data is at rest these are all many of the obvious ones sitting in memory or sitting on the disk but not many people go there because they think that's really scary you want to get the data out of Oracle financials, SAP BW and 12 other systems the fastest way to do it is to read the physical disk pages federate them in memory and you can query them in memory bit cutsy though right talking to the sand controllers read block write block it's a pretty simple thing to do compared to trying to use the APIs you know what APIs are right those are things that basically people give you when you whine and they don't do what you want here you go here's the API but I need these two more fields put in another card and another thing is of course sync replicate the way you do it because you get a copy of the data you can work on with an independent process and so you can do things in parallel without damage and for all you people who do do horrible things like use JSON please do not use text store it in binary I beg you it's 10x slower to parse it's 10x slower to store and if you put it on an HDFS it's 30x larger to store so if you wonder why you're dying with Hadoop or Spark processing CSV files you should know you're processing text you idiots get rid of the text as fast as you can oh but I want to be able to read it you can read 12 billion rows of CSV on the fly what you should do is have a small little printable program that actually prints it if you want to do that it's not too hard to write and please please God do not make it schemaless schemaless means I do not care about anyone working after schemaless is a great way to guarantee that no one can ever work on your legacy because the schema changes and you can't tell it's not a favorite for people doing UX we'll just throw a few attributes out there and now the whole data pipeline is broken because in December somebody threw a few more attributes and all the analytics we've been doing are broken for the last four months so let's look at some legacy pattern so this goes back to the first one I've got something that's changing a lot this one is a global HR provider you work for a global company and they give you a little card that says you know we love you if you have any problems with your health insurance or anything like that you just call this number 1-800 and someone answers saying hi we love you but they don't work for the company you work for at all they work for this company that's based in Chicago or Botswana or where you know where they are and this company has big mainframe systems that are written in maybe COBOL or PL1 the language you all want to be able to use and the problem they have is they have 100 developers whose job it is to take the latest negotiated benefits package from a national union and that's negotiated late at night and it's captured by analysts very bright people it's captured in very precise specifications called Excel and then the 100 COBOL programmers have to take whatever is in those Excel spreadsheets plus interview the analysts and get that into COBOL code and they're always late they don't really get it done at all so what do we bring to the party we bring small talk small talk is the easy language if you don't know by the way what a small talk programmer is that's all the people who wrote the XP books most agile manners are unemployed small talk programmers many of them good friends of mine big small talk a really fast much easier than any other language you can do as an old language and later we use Java much slower but in vogue so we bring Kent Beck and another bright developer whose name I can and those days it was called S unit and we train some COBOL programmers and how to do small talk and we do the project and we do the first one and we end up a whole 10% faster 10% now fortunately those two small talk guys were very bright so just before they left humbled by their experience since they believed the tool and the agile practices were the answer which they weren't what they needed was an aha an innovation that aha was the program has already been written it's in the Excel spreadsheets but those spreadsheets are a horrible mess so one programmer set off to write a spreadsheet checker that would basically throw up errors every time the analyst pushed the spreadsheet in and every time they did a push essentially it would basically come back and say this is wrong and the analyst would fix the spreadsheet the other programmer while that rule checker was being written wrote a spreadsheet interpreter which actually talked to the COBOL programs it's amazing you know there is something called and you can actually call a small talk program from a COBOL program and go the other way and this is a very dynamic piece that was changing and they were able to take those 100 COBOL programmers and redeploy them to other projects in the company this is programming by example today I see people with Excel spreadsheets they have something working or they have a MATLAB or a mathematical model and they go to the agile developers who basically say well what you need to do is you need to encode this and first of all you need to write a whole bunch of story cards for this and after you finish the story cards you have to write the cucumber tests because I don't really want to understand your domain now you wouldn't do that in Singapore you'd have programmers that understood the domain and train them in domain understanding and things like that assuming the programmers would let you do that so we've seen this trick the next one is a commercial insurance provider 200 million spent to a major consulting company we know it's got to be Accenture, IBM or company because they just take turns which one sort of works you over takes all your money you fire them and then you replace it with one of these other guys and three turns around they come back I pray that ThoughtWorks gets these jobs because I know they could do much better job than these other guys do but the problem here and so there was a new CIO coming in and said we've got the answer it's agile and of course the enterprise architectors came to help because they've been advised by IBM what they needed was an enterprise service bus we know exactly where to place that and we had an object database object database is the first thing to throw off the island and next is the ORM that's because you did naive DDD and put an ORM so we have this the technology's been specked outside rating engines developed with a different process with people trying to tell the company that's building the rating engine India with a waterfall process how they should be agile and screaming at each other over contracts you may have seen these projects I went on and on there were so many ugly pieces in this so we were charged as object mentor at the time to come in and basically build the agile team that would build the interfaces on the service bus and then help everybody in the company go agile we had 12 developers said okay great they have any experience oh yeah the other experience half of them were hard they were C sharp developers and took a Java course the week before supplied by a contractor don't worry you know spray them green they were blue last week the other people had programmed for most two months in Java we're supposed to teach them advanced TDD and how to do this so at this point I basically said look we're out of here you know I know that there's some customers you need to fire early because this is going to die I gave the team we're leaving I'm going up to see the CIO we're out of here and all of a sudden Ken said no no no Jeff and I have been working with all the BA's upstairs he says these people are amazing they've got all the insurance you know commercial insurance vehicle insurance personal insurance home insurance all this stuff they've got it all encoded in spreadsheets and I said so and I went up the spreadsheets scrolled and scrolled I've never seen that many rows and columns in a spreadsheet they couldn't be printed so they said no we have a solution we taught them how to cut the spreadsheets up modularize them you can generate all the code for the entire system from the sheets they're not all spreadsheets in the calculation form some of them are decision tables some of them are lookup tables and rating rules these people had been working for 20 30 years in insurance they knew the business inside out but we were going to take all these people and have them write story cards how retarded can we be please may story cards not be the end of the software this is a classic example make it look like the web you've got a bunch of ugly technology like old process control systems maybe you've got an IBM mainframe and it's running something called vSAM files or IMS or some other thing you don't turns out that all old systems whether from DEC or Univac or IBM now actually journal with atom feeds so instead of trying to figure out how to run a bunch of APIs and build a whole bunch of services like cobalt or PL1 or whatever C or whatever you're using use the atom feeds now all your websters can consume this stuff very simple trick APIs so if you have a system some people have this as they have an object oriented database very common in manufacturing to have a complicated Boiner connector database historically MRP ERP systems have very complicated data models and you can't get any reporting out of it the easiest thing to do is just put an OEM ODBC interface in front of it then you can just go at it with SQL oracle crystal reports or something make it look like a collection this is Eric Myers trick basically if you know Microsoft link or Rx essentially you turn everything you talk to you turn it into a collection and essentially you apply map operations across the collection how many people have a few APIs you know good web APIs how many hundred would you like here's my card I want the two more fields and of course you have to encode it all in ASCII so you can make it push through the wire and you give it a goofy name or some numbers and then you buy yourself an API gateway so you can try and figure out how to secure it and just pile this stuff on top right so the API field of dreams right oh you need another API it's professional life right it's so great about REST you need only one API select update delete the web should be just seen as a database schema now I've been saying this for years and most people consider me a raving lunatic so I'm now feeling much better since Facebook announced GraphQL essentially what you do is you turn your your entire web of data into a network schema you can also turn into relational schema and now you can enable that mobile app what does that mobile app does it does select of 20 things from 2 billion and puts them on the screen so you can pick three of them and send them back and for that I need Kotlin plus this plus React plus why stop building the API field of dreams just build select you do have to secure the select so there is some assembly required but you have a single model you can follow and now you can actually document what's out there because you can look at the semantics of the schema you might need to understand what an any relationship model is though because you really have a graph schema and there will be many to many relationships which many object oriented people don't know about because they don't like data that's why they use no sequel databases because they're too lazy they just want to stuff everything into a slot so other people can deal with it oh you mongo guys I want to kill each one of you so one of my favorite tricks is to use hardware all the time that if you go to Dell online you can order a server with a terabyte of RAM 64 cores 100 terabytes of storage for $10,000 US you can do nothing with a programmer for $10,000 US once you have a terabyte a terabyte of RAM is $3,000 today and there is something now called NVME memory which is persistent memory which comes in quantities of 10 to petabytes in two to three years you're going to be able to get a petabyte in a rack like this non-volatile byte addressable off the bus most people don't have more than a few terabytes of interesting data most countries don't have Facebook, Google, sure these people do that's right all you have to do is put it in JSON put an XML it will give you a larger so the problem here is that basically we can take advantage of the hardware and once you do that you can do all sorts of things we were challenged to build a cyber analytics environment having to deal with cyber problems in NATO countries pretty nasty things going on this project has happened a few years ago but it's still alive in the customer so our challenge was how do you build something that scales from an embedded system like a network controller up into running in a cloud and so what we did is we really looked and said look the only way we know how to do this is to use memory intensive technology and stuff that's going to really leverage memory because if we can get lots of things in memory we can query them very very fast and we picked a very strange technology for doing that and if you look at my credentials you'll find out what that technology is I don't plan to pimp it to you now I actually built this and sold it to the company and now it's called a product for the company I work for but it allowed us to do things that nobody else could do and it still runs 100x faster than Spark on a single box so once you use the power of the machine and if you look at any Martin Thompson's work on performance if you put hardware in you can build things that are very very quickly and you can deal with billions of packets another thing is maybe your dad or your granddad he had a book called Gain and Sarsen it was called there was a time called Dataflow Programming this was kind of in the 70s towards the 80s and they had all these little processes and these processes communicated to other little processes with messages this is Fred George and the first implementation of microservices are one of the first no one knows who the first in modern times was done in a company that sells power utility sells utilities power over a market called forward technology in the UK and if you look at Fred's work almost all the things he does use simple Dataflow one way this is the easy way to do microservices it's a very natural model if you decide you want microservices that are asynchronous and communicate back with each other and you have to do this then you need to go and become like Todd Montgomery and Martin Thompson and find out how to build asynchronous state machines and debug them so you can do simple microservices based on Dataflow easily put some kind of message bus in there and you can do whatever you want the other thing we want to do is leverage immutable data you've heard of Lambda architecture big data folks Nathan Mars you've heard of Topics Capca well the first Lambda architecture was the official airline guide done by Arthur Whitney in 1979 microservices, Topics, Pub, Sub Lambda architecture have been used for 30 years in the financial industry they are the essence of how people do high performance you know all those bad things swaps derivatives all those things that take the world's money and make it crash so the nice thing here is basically the world's very simple what you have is all the things that happened yesterday that's called the historical database or in Lambda it's called the batch database and it's immutable so all you do is a pin to it essentially and then you have all the things that happen since that's the real time database and if you want a consistent state guess what you do a query that queries two databases the stream and the batch and you're always guaranteed a consistent answer you don't know Paxus required no complicated algorithms and typically you take from the real time database and at the appropriate intervals typically in the financial system financial world is a few hours every night and you basically append the real time database to the batch database an amazingly scalable technology and you can make copies of these databases and remember now when you have 10 terabytes or 100 terabytes of memory on your PC because you're going to why are you even moving to the cloud you can give every quant in the business or every data science in the business a copy of all the data to the most expensive place to use storage in the world the cloud OS2 is cheap it's also inexpensive maybe you want to think where the technology is going maybe you don't need all that stuff maybe you just need a little decent hardware this is a solution that's been used in capital markets for many generations very successfully I mentioned that one earlier I do want to talk about testing how many people I tried this every year how many people are now doing property based testing thank god folks this is the best practices in testing it's been commercially in use for 5 to 7 years please learn about it please please learn about it one property based test is like writing 100 or 10,000 unit tests there you go there's some mode it's back practice there's something called quick check it's available for almost every language we had a nice talk last July on the subject it's all done with the examples are all done on the Scala but most people can read Scala because it's pretty straightforward it's not obfuscated by type signatures or anything like that it's straightforward we look up the video from last July property based testing is way way better than simple TDD come on Agile guys get out of the rut learn something new it's time right Agile is now 20 years old unit test we did S unit was in the 1980s and you can't move on scary but even more important is an independent implantation of validation that is a team which a separate team which writes some or all the code so you're actually writing the code twice way better than pair programming you're getting two implementations as opposed to one mixed up implementation with the pairs I'm being unfair pair programming is a very good technique this is best practice in things like aerospace and so on where things have to work because you're using independent brains on the problem if you look at people like Eric Meyer and Brian Beckman they do it their own way what they do is they write something that's algorithmic they write it in Mathematica and then they write it in C sharp that's how link was built so again two implementations of the same code either by one person or by separate teams and the example here in a previous life I was given the job of moving a large number of databases from many companies to a different database vendor this is called database restructuring or refactoring now some people might don't because it is in equivalence preserving now most people some of you know about this because you had to take log files out of Oracle and wait for the log to come out and it takes forever to try that's one of the great things once you put something in SAP BW Oracle is never going to give it back to you or did you know that Amazon charges you money to take your petabytes off you didn't know that I just had a customer have a quote of 10 million to move their data off so the problem here is how can we move all these databases but move them quickly if you're actually going to query all those things it's going to take you forever to do that and then you'd have to update so you'd have write all possible programs to do this now it turns out there's a very simple technique which not many people know and that is take the physical data and move the physical data first so just move all the physical pages page per page and produce the pages in a new database format this actually went from a network database to a relational database but it doesn't matter it works for any trick in the end you've got instances of records that have to become instances of records with the bytes swizzled around in some way and the ebkidic conversion to ASCII or whatever else you have to do and maybe you're going to split some records or something in there turns out you can do that at disk speed you can basically just copy and transform the pages there's no indexes and all of it's an unconnected mess well it turns out that if you understand anything like linking loaders you can actually in one pass with a sort go back and batch patch all those things because you can know the connection structure from one side and the index structure from the old system and then you can flip that then you can just apply it in a sort merge to update the database because it's this tricky because you're working at the physical page level which scares the daylight set of many people I love it because it's so simple we had an independent group write the validation and they independently wrote something to verify the data coming from the source match the data that got laid out in the target independent validation is still a really good best practice and for systems like this where you're making critical system changes look we have a guest performance too bad I missed all the bad jokes so I'll stop here I've gone on too long and it's time for a beer obviously thank you very much for your time I hope you got some ideas and clearly many of the things I said can't be true so if I stepped on your favorite technology which I hope I did just think about things try and keep things simple the key thing in all the things here is basically they involve the idea of being really lazy and trying to figure out if my job is to program 150 spreadsheets by writing cards and cucumber tests I'm going to go out of my freaking mind so maybe I could write a simple interpreter or I could provide a way for the user maybe I can take matlab program if the user can construct an example let that example not be a specification let it be a program that can run and validate that program first because it will have bugs in it check it and then move on thank you very much for your time and I look forward to talking if somebody has a burning question I'll take it but I'll go ahead you probably need to write a question so I'm going to use a story card actually I agree with what most of you say but the past years that we've seen we're all living in the real world and if you suggest to get a bunch of thought on the people I created and thought on the people I engineered and the people I've been ignored by all the media hyperbolic colleges in the back fancy and they are like on the sets they say but it's like it's a favorite day when you get all the standard people who use common sense instead of media it's a great question first of all most organizations don't have that many really talented people and so the question is do you spread those people around so they can get very little done in 20 places or do you actually say look I want to win in this case and I have to get a business value and deliver it and the answer is this works and you do two or three projects and you build more and more teams and so you get increased capability you don't need it's a small team that does this you're talking about five, six people that are doing these kind of interventions we're not talking about large teams there have been never enough programmers as we were going to the other planets to find them but you know I think the answer is there's enough to make a lot of big difference in organizations and if people apply these techniques they can do a lot more for their organizations and then many of them can do now by being spread over teams applying a whole lot of techniques that don't involve any innovation and so on so it works it can be done it's not a fairy tale but it takes leadership and it takes the ability to understand and absorb risk and be prepared to fail I mean you know I've got thrown out everything most companies have and they won't have a lot of them and the question is what's your choice? Am I going to target and deliver value for the things I can deliver or am I going to dilute myself and try and be under delusion that I can spread these people all over the place and get things done and my experience is that basically if you take your any business I know takes their key resources and they invest them and say we're going to put our talented resources here to solve this particular problem and they solve them by doing that's called tactics and that's how you win in business by getting things done if you do systematic change you spread these resources over and you believe in the tooth fairy leadership is scarce in everything there's no doubt about it the real issue is that you have to be able to work on things that are important enough and have enough value to the business so that they'll do this if you're picking on some problem that doesn't matter or just listening you have to be able to talk to the business and do that you know I understand that some people can't do that but lots of people do lots of talented developers don't take on the understanding of what the business problem is I wish a technology or do whatever it is which is fine I respect that they want to do that but if you want to make an impact in the business the business wants to see things get done every quarter so if you can't figure out how to make a difference in the business every quarter you're really not important to the business so make yourself relevant I do think it's often the case as well it's just not enough for people for like every year like it's a problem that needs to be changed that's always true there's never enough resources in the world so what you have to do is no so you have to figure out what to do with the resources you have businesses are successful because they manage the resources they have and use the people they have to do the job they can do the heavy lifting, other people make other parts work they try and take advantage of that when we do this we always have some young people in the team so every project that succeeds there's a few more people who learn more to do this I'm sure the same way that you do things if you go to a new technology you have some young people you take those you split the team you seed two teams it doesn't happen overnight you get an organization from anything to be able to produce decent software why do you see that often? there's good leadership with two stars to be able to be able to live it's not your experience in the end some people if you have to work with idiots you go someplace else basically my view is very simple if the people you're working with you're not learning from basically you're the best in the organization get out you should go someplace else whether you should learn because then you're the superhero but you're not learning and then you're going to be very frustrated obviously no one can fix the world no one can solve all the legacy problems in the end business is about triage it's about basically what can I do to make a difference in this and if you can get to the business leaders with a problem that you're going to save them tens or hundreds of millions of dollars or cost or improve the reliability of the system you'll get their attention but you'll have to stand behind it and be prepared to absorb the risk of doing it and guarantee it'll work anyway I won't be late this any longer I think that's probably a good time to end in there thank you very much Dave for your talk and sharing your insights