 Make sure you're in the same right room. This is a technical talk at an Agile conference. So it's not strictly about Agile per se, but it's something I do a lot of and talk about. So if you want to hear a really interesting Agile talk, I would say the mob programming talk by Woody down the hall is definitely worth hearing if you haven't had a chance to hear that before. Now I'm going to reverse from my talk, yes. But Woody has a great story to tell as well. You haven't heard Woody's story, it's worth hearing as well. This one you can hear by watching YouTube videos. You can see this one's elsewhere. So I'm going to talk about microservices. I've been playing with microservices since I toyed with this idea in my ThoughtWorks days back in 2005. So in 2005, I put a microservice project together. We delivered successfully. And ThoughtWorks kind of declared it to be a failure because it was a hard technique. It was a strange way to write code. Why should I write code this way? I can write these big, monolithic things instead. I ignored what they said, of course, kept doing this. And of course, now for various reasons I'll get into it, it's become a popular topic. If you don't have any, I'm not a good much introduction by myself, but I am, in fact, a disruptor in organizations. I tend to do a lot of strange things. A lot of strange things I do are probably not appropriate for your organization yet. In fact, let me ask you a question. Who's actually writing microservices as part of their establishment now? Good, yeah, about one or two, which is right number. That's about all that should be doing it. Because as a part of new technology, it works for certain types of problems, and I'll get into that. Interestingly enough, in Europe now, we're getting a lot of microservice attempts. We're beginning to get the same thing that happened with Agile. We're getting these early failures associated with it, by and large, because they're doing it very poorly. And I'm going to talk about some of the challenges we have associated with that that you should avoid. By the way, again, I'm a disruptor in organizations. That was my job description for one of my jobs. Again, if you can ever get this job description, you should take the job, because it's really powerful. So this has been trends in the industry. I talked about this yesterday in my talk, but microservice is certainly one of these trends, but there's a whole lot of other trends going on, with technologies, with various process stuff, with various roles in computer science, business thinking about that stuff, all around the concept of going faster. And microservice is a key part of the solution of our going faster. But it's for particularly a type of problem that's out there now that's new. This is a new type of problem. This is not the type of problem we're trying to solve back when I was writing code in the 70s and 80s. This is called a connection framework. It comes from a guy named Dave Snowden. Basically Dave says there's four types of problems in the world. These are simple problems where the cause effect's very straightforward. This is the world of best practices. There is a job for our manager in this role, because he's going to tell the people how to work better and get them faster and faster in doing it the right way, because there is a way to do it. For complicated problems, there's more than one way to sort of do these things. It's complicated. The cause effect relationship exists, but it's very convoluted. You need experts. But experts are expensive, so you basically put a team together to do what the expert tells you to do. And we've organized software this way for many years. We've had the architects and designers making these big decisions at the high level, and then teams of relatively junior programmers to implement what the guys tell you to do. And for solving complicated problems where the answer does exist, but it's complicated, this works really well. But Dave Stiller also says there's things like complex and chaotic problems, complex problems where the cause effect relationship does not exist. By not existing, I mean it doesn't really exist, financial markets are this way. They're absolutely unpredictable. Just because this happened because we did something yesterday doesn't mean when we do that thing today, we'll still have the same effect in the market. These are unpredictable things. This is why you can't really write algorithms to sort of do the trading and do it successfully. It's because it's a complex world. And it turns out a complex world, there is no concept of an expert. People think they're experts in this area. In fact, these guys over here who are making lots of money being the expert will claim they understand this world, but they don't. They're lying. And eventually, you figure out they're lying, you fire them and hire another expert. Because you don't realize there is no such thing as an expert. You hope there is. There's got to be one, right? You must be the right expert. Let's hire you and do this instead. This type of problem doesn't have experts. And this is the type of problem that's becoming very dominant in our industry. In fact, one of the things Dave Stodun says is also, most of the times, you don't really know what type of problem it is. You have a prejudice yourself to certain type of problems. You may think, I like to do things that are easy way. I mean, it's got to be easy. I've got to make it solve it. In my world now, this is Donald Trump saying he's going to build a wall. And he's going to fix all our problems. Of course, our problems are these sort of problems. And this sort of thinking does not fix that problem. If you're an organization that has a very structure and a hierarchy, I'm thinking about Indian outsourcing firms, for example, where you have this hierarchy associated with this, it's a matter of getting the experts and the business analysts and all these roles in place. Because that works really well for a complicated problem. But if that's how you organize, of course, every problem looks complicated to you, even though it's not. For myself, I'm a big fan of this stuff. This is where I like to play. So I've been known to take problems like this and try to turn them into this because it's more fun when it's sort of chaos and complexity there. But frankly, I've been working, frankly, in problems that are really are this type for about the last 10 years. And most of them talking about microservices fits really nicely for these sort of problems. I call them fuzzy problems because they don't have defined answers. Should I loan you money? I mean, where's the expert that decides I should loan you money? Who's going to tell me who's the right answer? It gets it right 100% of the time. You don't know. You're making guesses. And so it turns out, you get better at this if you're able to do things really quickly and try ideas out. So I talked about that a lot yesterday and I won't go much more into that. Now microservices have been driven basically by this change in hardware. And in fact, if you look at it, microservices kind of sprung up in parallel in so many different places over the last few years. And frankly, we didn't even know about each other. The stuff I was doing in London, I didn't know what they were doing in Netflix. The Netflix guys didn't know what the Goat guys were doing. And it all happened at the same time, primarily because of these trends. And it's how long does it take to get a piece of hardware? If you go back, and this is again a log chart here, if you go back in the 1990s, it took you three, four, five months to get a piece of hardware. And to figure out what you need to get it installed. Of course, you had to allocate the money for it the year before. So you had to make a lot of guesses about that. We got a little better over time when we first machines came along. We were stuck with this six month stuff. We could get a piece of hardware and carve it up later based on what our needs are now. And that was quite productive for us. And then the Amazon guys come along and says, oh, well, you don't have to buy a machine anymore. We can sell your virtual machines. And you can set these up in 15, 20 minutes. And so all this stuff about there is not necessary anymore. And then of course, Docker comes along and we're talking about seconds. How long it takes to bring things up? If you haven't played with Docker yet, make it part of the things you learned this year. The little contest now using Arduino. So how many Docker instances can you bring up on Arduino? At the conference last year in the US, or maybe this year in the US, they were bringing up over 500 containers on our Arduino. 500 on our Arduino. These are little things. This is a very powerful technology. There were only 500 web servers, by the way, for the entire conference on one Arduino. Now what happens when this happens, if you take advantage of this, what you find out for is, this concept of a whole capacity planning we have as organizations is dead. The need to have this really dedicated ops team to understand how to make these machines work and work effectively is dead. You're buying that service from these guys. And if you still have these parts of your organization, they're dead and holding you back. Now because of this sort of shift, all of a sudden now it becomes very easy to get a machine when you need it. And this is both a concept of microservices. And so you look to see what the impact of services because of this, then instead of having to worry about building this gigantic application that runs on this big machine you spent all this money on, one thing with thousands and hundreds of thousands of lines of code, we went to the service stuff when we got to the virtual machines because we could allocate a little better. But this is where the microservices are when you get down to the Amazon world and the container world now coming. That you can write these really tiny things. So instead of working on projects building these gigantic applications, which are fine actually for complicated problems, but don't have the flexibility to solve complex problems. Instead you wind up with guys like Workshop I Run where we write a few services like this. This is a company I worked with in London that did a lot of this same work. We had over 300 microservices, none of which were over 100 lines of code, running our business on it with zero documentation of 300 services. They need it. This is Netflix. 600 services, about 1,000 lines of Java each. So this is a new world associated with microservices driven by the fact that I can go out and buy commodity capacity of a hardware without having to plant it ahead of time. Planting ahead of time had plenty of time to build these things. In this reactive world, you don't have that. You don't need that. So this is where people are going and why they're going there in droves. So if you look at the principles of microservices and basically a lot of us kind of agree about these principles on some degree, they're really, really, really small. We're talking about 100 lines of code or in the Netflix case, which is really big service, 1,000. It's just 1,000 lines of code for the entire service. It takes one person to design and maintain it. If it takes more than that, it's not a microservice. This came from Adrian Cockroft, the guy who did the Netflix stuff. This is kind of his rule of thumb. If it took more than one programmer in Netflix to work on it, it wasn't a microservice. Let's tear it apart. You can make it smaller. They're very, very loosely coupled. The more loosely coupled they are, the faster you can deploy more of them into the system. If they're very tightly coupled with each other and you have dependency on each other, you have to deploy a bunch of them at the time. That slows you down. We're trying to go faster, not the other way. You actually want multiple versions being run at the same time. If you have an idea for an upgrade to a service, because some of the client needs some additional functionality, write a new service. Keep the old one running for the old clients. Again, makes you faster. So you're getting a lot of the ability to have multiple versions running at the same time. Don't have the idea, oh, I got the new service. Everybody needs to use my new service. No, they don't. Why would they bother? The old service is working perfectly fine. Keep it running. The service themselves have to be a little smarter than most services we think of. They have to diagnose themselves. If you come up as a service, you look around, you can't find your data, you need to yell. I can't find my data. It's probably not your fault. Some other service probably didn't put the data they needed to, but you need to stand up and yell so that somebody can take it. We'll see if something's going wrong. So fast failure is a very key part of these microservice architectures, and that comes in there as well. We borrowed this concept from Google, and that is you do something interesting, publish it to the world. Tell the world about it. Don't worry about who needs it. Don't wait for that guy to come to you and say, I need this information. That again slows you down. You do something interesting, you should just publish it. And finally, the whole concept of application, that's dead, because the boundaries around these things are so fuzzy that you have no idea where they really fit. So that's kind of a new world of microservices, and it's a very strange place to survive in. So let's get to the challenges. There's one common theme for all the challenges I'll go through, and that is, we're completely aided about what's going on now. This is new stuff. And when somebody says, even me standing up here and saying, this is how you shoot microservices, take it with a giant grain of salt, because we are really inexperienced in this stuff. I probably have more experience than most people have in this space, and I've been doing it off and on for 10 years, and I tell you, we're still learning things every day. So we're inexperienced. All right, so let's get through the more official list of challenges. The first thing you gotta figure out when you start to do microservices are gonna be asynchronous or synchronous. Asynchronous or synchronous? Now synchronous is kind of the more logical thing to think of. In fact, I have a colleague, Chad Fowler. Chad's got a lot of his common history with me. We worked in India at some point, but he's written books. He's much more smarter than I am about that. He's much younger than I am also. We actually interviewed for the same CTO job. He actually got it, which was good. I had other plans, but there was a good choice on their part. He's a good guy to do this with. So we were on a stage in Dublin one time talking about microservices, and we said all the same things. You know, small, and you build them really fast. You have parallelism. And Chad was still up there and said, you know, but if you have a choice, you should use synchronous. Because frankly, that's how your problems are described to you by your business guy. You're step one, you're step two, you're step three. You should implement those orders. You just wire them together that way. It's a restful call sort of thing. And that increases your productivity and understanding because it's not magic to them. It's very obvious what you're supposed to do. I said all those same things about great about microservices. I said, you should be asynchronous. Chad's wrong. You should be asynchronous because you want robustness and these asynchronous systems are much more robust. You want the decoupling you get with asynchronous. That the dependency of one service upon another is much less than asynchronous architecture. And yes, you actually have to teach your programs to think about problems differently. That is a barrier, and you have to overcome that barrier. Interestingly enough, Chad, of course, is at Six Wonder Kinder, which got bogged by Microsoft not too long ago, and he implemented asynchronous. So I point this out to him when I see him. So the asynchronous models kind of sit around this concept. This is a picture I drew with my new iPad back in the day when I was flying across the country. So it's all done with my finger, so no apologies required. The key thing here is we've been moving now from the concept of an operational relational database is gone. Operational databases, operational data stores are gone. They're being replaced by event buses. And so basically the whole idea is events are the interesting things you're actually looking at from an operational perspective. Things are going on, remember them as events. The fact this is your email address and you change your email address, these are both valid data points. You don't want to lose that old one just because they got a new one. And we do that with our operational SQL databases. So we want to build this concept of a rapid, which is basically every event that's going on is in one bus. Every event. And therefore you have access to everything that's going on in the world. This is your log messages. This is your user journeys. This is your traffic stats and your servers, as well as all the messages back and forth between the servers themselves. Everything's on that same rapid. That's a lot of information. So you want to tease a little bit about it into rivers, which are basically subsets of this information that people have. It's a slower moving body of water. And there is a still a need for reporting. There's still a need for a data warehouse and SQL is a good technology for that. And we call those pawns because they're relatively stagnant. There's nothing in mobile information, it's current state. So entities still need to be there, especially for reporting, but I call them pawns because they're stagnant. So sort of using this as our overall metaphor, we start building systems around this. So you need, first of all, a really fast rapid, something to handle this message traffic. And for that there is now several choices, my favorite is Kafka, but there's several choices of high capacity event buses. And by and large, these buses are really, really dumb. They're not the enterprise service buses of 10 years ago that you configure and they'll do transformations for you and all this other magic. There's none of that associated with it. They're really dumb, but their job is get a message, send it out to everybody who cares about it and put it into a store so it's always there. And Kafka, of course, is the bus that came out of LinkedIn. So these guys, LinkedIn guys have been using this over a decade and they've opened sources. It's part of the Apache project. The Apache documentation for this is truly excellent. If you want to read about why an event bus makes sense and what the protocols are, the Apache documentation is truly excellent. But the capacity of this bus, it handles about 250,000 messages a second, which for applications I tend to work in is way more than enough. Now, I'm not doing Google, I'm not doing Facebook, but in my world, that's a lot of capacity. The problem, of course, is it turns out they cheat when they count messages. It turns out if you put a message on the bus and it's read 10 times, that counts for 11 messages. You say, whoa, whoa, how do you count these things? Well, every time you read a message, it counts for one of your message count. So you don't want to hang 100 guys off of here because all of a sudden now it's not so big. So we try to hang that second tier river off of these guys. So we hang a few guys off of this that pull these things off, they subset them and pass them to somebody else and we call these the rivers. I like to use the zero MQ because I don't need a Q and it's already here. I just want to be able to broadcast this out to all my children. So ZMQ works really well, it's a nice library. You can use something else if you want to. You can actually chain these guys together if you want to filter on filters. But I try to minimize this. In my experience so far, I've never needed more than 10 rivers. So my capacity is still 25,000 messages a second. In my world, again, that's a lot of capacity. Now what you do is you wind up hooking your services. They're listening at a river level, but they always publish to the rapids. Don't publish just back to your own river. That says, I think this message is only good for these guys. You're not that smart. Do not try to anticipate the future. Push it to the rapids, it has the capacity. Don't worry about it. Now this sort of thing yields some new architectural patterns. So you get things like new asynchronous service architectures. This is called the need pattern. Basically the idea is you're sitting as a service and says, can you give me some information? Should I loan you money? And you hope somebody out here is actually listening for this, but you have to handle the case where he's not listening. Just nobody's gonna answer the question. So you gotta worry about that when you design these algorithms. But for a fuzzy algorithm, you can design that. So you may have some guys out here that are deciding, yes, should I loan you money or not? And they're listening to this. This guy may look at your banking history. This guy may look at your employment history. They don't know about each other. They're just making votes about this. And they vote. They say, yeah, I shouldn't lend them money. I shouldn't lend them money. And of course, this guy collects the answers and makes some decision based upon the answers. Call the need pattern. Very powerful for fuzzy algorithms. For these new things, you make money off of. Voting on, should I loan you a book? What book should you read next? Again, one of these algorithms. Now the nice thing about this, it's very easy as architecture to put variations in there. If I have a new idea of how to do blue, blue version two, I just put blue version two on the same bus. I don't turn blue version one off. It's still there. It competes. Maybe I like blue version two's answers better. I turn one off eventually. But I don't have to wait to turn this one off or to put the next one on there. If the green guy goes down, maybe he's not voting today, big deal. Yeah, I want to get him back up as fast as possible, but the system has not stopped. It's still a robust system. I'm still getting answers. So the variations are easy and the degradation is quite powerful in this sort of architecture. And that's what I'm really after if we're going faster. It's solving quickly complex, fuzzy problems. You can't be fuzzy when you're trying to put a bank account together, when you're trying to show your bank balance. And you'd be pretty accurate about that. So that sort of problem is not necessarily good for this sort of architecture. But if you're doing the fuzzy problems, microservices work really well. So an example of this in action. This is similar to something we did. The idea was we were working with a client that had a site. We're going to use rental car example in this case. And there's a place for advertising. You can actually try to sell him more things. He's coming here to rent a car, but how about give him a bigger car? Maybe he's some navigator. Try to get more money out of him. The McKinsey guy is called this share of wallet. You're trying to get a little more of his money just because he's already a customer in front of you. So we took the old legacy application, which basically put up the rigorous to the page. And we wanted to do something to fill in these little blanks. So we got ourselves an event bus. We put ourselves a little service in here that was going to basically interface to this guy and be able to put the information back. And we actually started building some services that actually would put offers out there. In the rental car business, the idea would be there's some brand level offers at the corporate level. You know, I want you to join my figure rental program or we have a discount on SUVs nationwide, some program like this. But each individual rental location has their own needs. They may have a surplus of vehicles. They may want to give you a convertible even though it's the middle of winter because they don't know what else wants to take them. And so basically the idea is this would be competing for the business, for the attention. And then you can add some more stuff in here, membership. It turns out that if you're a member of a frequent rental program, it turns out you're already an existing customer. Keeping you as a customer is way cheaper than trying to win you back. This is why all the frequent rental programs and frequent flyer programs exist is that if you're actually there, they want to know about you because they want to keep you as a customer. So we can put a little service there and just looks up and see if the person coming in is somebody we know about, in which case tell me if he's gold or platinum. We also do segmentation. Look at his history of usage. Turns out in the real estate industry if you rent the car between Monday and Thursday, chances are it's a corporate car. You're not paying for it. So I don't want to give you a discount offer. You don't care about discounts. But a bigger car, maybe so, or points. You may like some points. But if you rent your car on the weekend, it is your money. Discounts are very important to you. So just being able to dump that information back on the bus and says oh by the way, this person is a sort of weekend warrior. Or he's a road warrior. Goes into building business during the week. Tell me which one he is. And I'll make better offers. So you rent a little service and that's all they do. They just worry about that little piece of the problem. So if you bring up the webpage, what will happen is we'll put a message on the bus saying I have an opportunity for advertising. Don't know anything about this person, but have an opportunity for advertising. Because they don't know anything about this person, these guys don't care about that message. Don't even think about the user in there. Can't look anything up. So they don't care about the message. But these guys say here's an opportunity. Let's put some advertising up. This guy puts an ad up. This guy suggests this sort of ad. This guy collects those ads and decides which are the ones he wants to show. And he'll probably do a redirect and actually it pops up on the page. And notice this is not the content bus. This is not the data bus. It's the event bus. So these things may carry a little URL in, you're gonna redirect to, but it's not carrying the data. So now Sally comes in. So Sally says I'm here. Of course, that's more information. Everybody gets excited about the fact that Sally. So all happening in parallel. So again, we have offers. No, we know it's Sally. We know she must be an existing user. She logged in. So we have different offers knowing she's an existing customer. But we also get an opportunity to say things like, oh, Sally is also a silver in our program. And we're like, wow, silver in our program. Well, these guys care about that. So these guys picked that up and maybe a different offer. Because she's a silver level platinum. You remember in our program. And so you get different offers coming in based upon that information. And you'll see that we'll keep going. We'll sort of stop here. But of course segmentation will say, oh, by the way, she's a road warrior. She rents here in the week. So send that information back to the bus. So notice I'm gonna have lots and lots and lots of messages flowing around just for this little one transaction. I don't care. I got tons of capacity. I'm not, I'm trying to go fast. I'm trying to do flexibility. We built a system like this when I did a startup in California for one of the major hotels. We built the system for about, we charged them $6 million for the system. It's not that hard. They made $40 million additional revenue in the first six weeks, first six months. So they loved it. We loved them. We loved the money. They loved the results. Everybody was happy. It's not that hard to build with this sort of architecture. And it's a fuzzy problem. That's why you can make money off of it. All right. So you want it with sort of a service taxonomy in these sort of situations. You want it with lots of little services here interfacing to the outside world. So not only do we have web page interfaces to the system, we also have a call center interface to it. Because if you get to the call center and you got somebody in front of you, do you want to put another offer on the table? And we also send emails to all the frequent renters. Also another different channel interface. Another microservice. Individual microservices do one thing to do one thing on. It's a one for the web, especially one for IE6. There's one for Chrome. There's one for iOS and Android. There's one for the call center. There's no one for the emails. All those were different little channel interfaces. They all kind of pumped information into the event bus. The enrichment services, the little guys you saw at the bottom corner, they added more information to that request. Sally's Platinum, Sally's Road Warrior, and they republished it back to the bus. These guys are providing solutions. The ones I showed you are solution providers. But also I can keep track of what I showed Sally. Another service doing that for me. When I show a message to an ad to Sally five times and she never clicks on it, maybe I don't want to show it anymore. Let's degrade the likelihood of that occurring. So I can dynamically adjust the system. Another little microservice, very easy to write. And of course, if Sally already has a rental with us from Houston, and she's looking for another car in Houston, don't give her a discount in Houston. She already agreed to the contract. If you give her a discount, she's going to call you up. It's going to cost you money in every way you do it. So these are all solutions and solution blockers. The nice thing about these sort of systems is you can actually do interesting logging and monitoring, but more importantly, you can measure almost any of the key performance indicators. Anything that's going on in the world whether it's logins or quick traffic or page retention times, the events are on the bus. You can pull those events off and build KPIs to see how well you're doing. All right, challenge number two. Unfortunately, the database is a barrier here. You got to basically tear that operational database apart if you're going to be successful in microservices. So you really want not to get the one or two databases you have for operational store, but you want hundreds of databases. So for example, in this little system, the rental car example, I have a little piece of code that's going to publish an event on the bus. I have somebody collecting solutions, putting it into a Redis database. The Redis database has another service that's going to pick it up out of there and respond on the webpage. So that one service is actually running four containers, so there's no little database. One database, one microservice in our world. Somebody in the membership service is basically, again, has a little table, key value store, looking up, you know, Sally to see if she's a road warrior or not, or a partner with Silver. That process is filled in with an ETL process, runs in background, three little containers running. During a little database, it's one service. All right, so that's kind of the idea of the database. You got to tear it apart. Every microservice needs its own persistent store. There's an entity, what I call the entity microservice trap. There's a trap that most clients getting into that haven't literally looked hard at microservices. That is a service, a microservice represents an entity in my database. This comes from a little bit of something, Fred Brooks actually taught us back in the dark ages. Fred Brooks is one of the attorney award winners. He was the guy that sort of built the original IBM operating systems. And his theory was you get the data structures correct and the code will write itself. And I lived under this environment and I worked in IBM back in the 70s and it's true. So the idea was if you get this data structure correct, and then basically it was easy to write all the code around there and get it to work. Now you move, well, this is 1970s thinking. This is before we have relational databases. Well now we have relational databases. So you have a relational database and if you get that all right, it's easy to write lots of code around it. Now going back to Fred Brooks theory, we've turned out Fred Brooks theory was flawed. Because it turns out that with Fred Brooks, yeah it's true, once you get this data structure right, the code writes itself. But if you ever wind up touching that data structure, all the code is now suspect and we have to test it all. Hence was born a lot of the waterfall process and the QA departments and the concept of code freezes and all this rest of stuff. Born out of this misinterpreted idea that central database, central structures are essentially part of your system. We did tricks like we made sure we put fields at the end, we put fields at the beginning, we put fields all over the place and make sure we don't touch the base structure just to make sure we can actually add code safely. Now you move forward to today, this is basically what you call your database and if you have a centralized database you're gonna have this problem. We had a presentation at one of the conferences in Perth and the company stood up and said, yeah we're actually deploying several times a day with our system and you say wow, this is pretty cool. How are you doing this? We're doing this for anything that doesn't touch the database. You're like, oh, that kind of eliminates most of your interesting cases, doesn't it? As long as you don't change your data you can actually go fast. The database will hold you back. So don't do interesting microservices, it creates this sort of thing. All right, challenge number four is this is competing technologies in microservices. I actually think some of the functional programming language like closure are an alternative to microservices and how microservices tend to work. And this has been my experience so far some degree microservice are about like object-oriented programming. Every service is small, it has a single job, we have the concept of encapsulation because we have our own little database. It kind of has all the simple things associated with object-oriented programming. When you look at how you write closure code, closure code loves shared data. It's immutable data sort of concept. And you wanna put these functions together that are strung together which pass data back and forth to each other. And these data structures can be quite complex. In the case of the Daily Mail we were sending in an adjacent representation of the entire pages and turning it HTML at the other end. This turned out to be a marvelous way of doing it. But these interfaces between these services nobody else wants this. This is a hash of array of maps of a hash array of maps of hash arrays maps. No other service is interested in doing that. So why am I creating a microservice out of this little thing? Why am I just gluing these functions together? I've had experience now with three companies using closure and microservices. When I was working in London when we did over 300 microservices we wrote in lots of different languages including closure. We used to, we started out with sort of running cron jobs. We tried restful interfaces but we went to the Kafka bus, the architecture I talked about before. And that was our most robust structure. It allows us to go the fastest. When I went to the Daily Mail to work with those guys we actually, we had three services that were doing the main work in closure. And we had dozens of small services averaged of 18 lines of code each and no JS monitoring the behavior of those pages. And who was looking at what articles and what type of click rates were they getting? So we used closure in two different languages. We used revenue queue as our queue instructor. But these were things were not microservices. But together among the three of them were about 4,000 lines of closure code. You know, 4,000 lines of closure code you can move the earth and it's a lot of code. Then I did the startup in California. We were doing only closure. We're doing much more complex problems like I just described. And we basically had 25 microservices and the number kept going down because we realized there's no one else wants to use this and we just started glueing these functions together more likely. So I think there may be a closure in these functional programming languages or an alternative to microservices for solving some of these complex problems. I think there's a new hope in the rise that I'm looking at playing with a little bit. It's called Elixir, which is a functional programming language built on the Erlang virtual machine. If you're used to Erlang, you know there are thousands and thousands and thousands of threads and processes running in parallel to each other. So it's kind of a microservice with a functional programming twill. So I think this may give us some new insights. But watch out for functional programming versus microservices. I think there are two ways to solve the same problem. Fifth challenge, choosing architectures and frameworks. The problem here, of course, is we're new. We have no idea what the right architectures and frameworks should be. If somebody comes in there and says, oh, by the way, we have some tools to do microservices, you should use our tool set. I'm very suspicious. I get those solicitations periodically from various vendors saying, oh, you should look at our tool set. I'm like, you have a tool set for microservices? You already have lost me. You've already lost me. This is just probably a complete nonsense. I mean, web sphere of all things. Claims there have microservice support in web sphere. Why would I bring up an elephant to run a flea? I have no idea. But there is some interesting things out there. There's a lot of open source stuff out there that it's worth trying. And the biggest one is actually out there is Netflix. Netflix is microservice-based, very successful, obviously. They, by the way, are synchronous. They are synchronous microservice, but we know it's 100% one way or the other, but they're by and large a synchronous. They have 40 open source frameworks to help do synchronous architectures and clouds, over 40. And the nice thing is, as Adrian Cocker would say, even though it's 40 out there, it's almost more than any one program get their head around. This is too complicated stuff to sort of get your head around. You'd need a team to even be able to understand how to use these. But this is a framework that's being used a lot by the Department of Defense in the US. Basically, anywhere you were telling me I was going to start and try to build one of these big apps again, I would build it using this framework. This would be where I would start. There also, the whole Node.js community is extremely active. This is where we're going to see a lot of innovation come in. Because they're already thinking about small libraries and they're already working in small units, so it's been a natural fit for these guys. And so you see frameworks coming out of those guys, open source again. This is one I have a company in Ireland that I work with periodically, Seneca. I've been out there now a couple of years. I was just googling around, I found this one, which is a zero MQ, so I'm kind of feeling good about this one as well. It comes out of Italy. But you basically look at the open source community for ideas about how to do this. But don't have a vendor coming in here and tell you, I can do microservices for you. Buy my frameworks, buy my tool set. Walk away, or actually run away. Because it's going to cause you trouble. Sixth number challenge. We have not got a design patterns book. I mean, there's no gang of four for microservices. Yeah, there are a few books showing up, but none of them are really pattern-oriented books. A lot of them say, here's how you organize your teams, here's how you need to do continuous delivery, you need to worry about DevOps. These are the things you need to address. They're very good books for understanding microservices, but not a design pattern book. Not a book you can pour out and say, here's how I need to make my system. What you will find, though, is there's a lot of meetup groups and conferences that talk about microservices. And this is a very rich place to hear other people's experiences of how they did work. I mean, Chad Fowler talks about what he did at Sixth Wonderkinder. It's a wonderful story. The Guild Group talks about what they do with their huge loads, because they get a spike in traffic that's actually a terrible look at. Uber is using microservices. But listen to these guys talk about their experiences and learn from them. So make sure you either attend the conferences and meetups associated with that. There's actually one in Berlin every year. This is the first dedicated conference to microservices that existed. The third edition will be next year in February. You'll get 350 people showing up, all of which have been doing microservices. An amazing place to sort of get new advice about how to make this work, or hit the meetup groups. These are ones just from, I was looking at, San Francisco's had one, of course, forever. London has one. Of course, I love the Dublin guys. It's got a nice graphic thing. By the way, Dublin is a real hotbed for Node.js. And so they've been very, very advanced in this thinking as well. The seventh challenge, if you just do microservices by itself and say, I'm gonna do microservices, but the rest of my structure and stuff, I'm gonna leave the same. I'm not ready for the rest of it. You will fail. You will fail. Microservices is enabled by lots of other things in parallel. So of my big list here where I like to show lots of stuff, like microservices, if you look at this big list, you gotta pick at least a couple of these. If you're not into the cloud or into the Docker stuff, you're gonna miss it. Microservices without using Docker or the cloud, it's kind of just wrong. If you're not using some managerless process, because you're working in an area where managers are useless, you're solving complex problems. There is no expert. If you have a traditional architectural structure in your organization, it will not help you. If you do microservices, it won't slow you down. It's really gonna be kind of cool to write a microservice and then it's gonna take four months to get it deployed because of your processes. Why bother microservices? If you're not deploying the same day, it's not worth it. And look at, think about full stack developers in particular. You know, the guys who can basically solve the entire problem. All right, challenges eight through N, because I think there's gonna be more. By the way, you used to only be five here. Now we're up to eight. But eight through N, to seven degree, what's the microservice? That was the first question. What's the microservice? You're gonna hear everybody claim, like they do with Agile, that they're doing microservices. And you go in there, you wanna probe a little bit of what they mean when they say they're doing microservices. And I have kind of three questions I'm looking at. Three dimensions I ask, but when you say, oh, you're doing microservices, tell me about this. So before, especially before it's too late, are you doing synchronous or asynchronous? Are you talking about APIs, each of your services, you're talking about registrations for your service things which you need for synchronous, or you're on an event bus sort of thing. And by the way, you're never 100% one or the other, but what's your primary focus on how you do these things? Restful interfaces or you've got event bus or something else behind there. How many services do you have and how big are they? Give me the ratio between those. Oh yeah, I got some services there, 50,000 lines of code. 50,000, I mean you got three. Oh, you call those microservices? Oh yeah, we're microservices. No you're not. The other thing is talking about your ratio between your database, how many databases do you have and how many services do you have? Oh, we have three databases and 50 microservices. No you don't, you have three databases and three services, none of which are micro. So tell me about those sort of things. For more information, there's lots of stuff out there now that almost all the talks now get videotaped. The experiences of these various groups are worth seeing. And almost anything that goes on that comes around microservices, you want to take, again, with a grain of salt because this works for them, for their environment, for solving their type of problem. But don't try to project that to yourself. Your experiences will definitely be different. So for more information, obviously just go to Google. Ask for microservice for videos. The Micro Exchange Conference had the videos. The 2016 conference had the videos up the same day they were presented. So people were getting really good about this stuff. And you want to look at something like Program Anarchy or now Eric Meyer has something called One Hacker Way, which again is a sort of manager light process, which is associated with solving these complex problems. And that's the story of microservices. So I thank you for your attention. I think we have more than a few minutes for time for questions. We're just about to in five minutes. Question, let's see if they'll get the mic to you. We are a big room today. He's trapped. He can't get through. He's almost there. Hello, thank you. The question is about where you present the example of closure in a function programming where we have different functions and data moves from one function to another. But how do you solve that problem in the microservice where you actually want to move data from one kind of a service to another and not necessarily just the events? Well, first of all, I'm not a fan of moving data from service to service. I think any service that just publishes this data is not really a service. It's sort of equivalent to a data object back in the object thinking. I want a service to approach its conclusions to the base. I want my services to be behavior-based and publish their conclusions. I don't want them to dump their raw data. That becomes the word of the entity microservice. So I don't think it's really a role necessarily for using service architectures and things like closure and the mappings of that. I think those are solving different problems. Primarily because I'm thinking about once that interface. Did I miss your question completely? If the data doesn't move, then how does one service, I say one service has processed data or has done some service. How would the next service take the answer to do this? If you're passing the raw data back and forth between the various services, in my world you're not building services anymore. You're just basically replacing databases. And I don't think there's any value associated with that. What I want my services to do is publish a conclusion. I don't want to see all the data associated with how many rentals Sally had. I want you to tell me whether she's a platinum or not a platinum. I want you to tell me she's a road warrior and not a road warrior. That's all I want from you. I want you to draw conclusions from your data, not publish me the raw data. If I'm doing a medical instrument system where I have a hospital and I have an MRI, I want a bus that says, I want to publish to the bus, I did an MRI for you and your results are good, negative, very bad, whatever. That's all I want to publish. I want the raw MRI data floating on the bus because nobody's going to work on the raw MRI data other than the doctor that did it. But the nurse is going to react to this, the doctor is going to react to the conclusion. I want you to publish conclusions. And that's why I want to build behavior-oriented services that publish conclusions on the bus, not dump their data. It's the every way of thinking about it. I mean, the first thing you typically do with microservices that I've seen is wrong is you take your database, you take every table turned into a service. And then each of these services basically have to dump their data into some God service who's going to collect all the data and run basically a query on it. That's not a good design. That design fails. But still the conclusion that one service answers, it'll be an input to, something like an input to a next service. Perhaps an input to the next service. To some degree, in this asynchronous architecture, you publish it to the conclusion to the bus, you don't care who's listening. It's not your job to worry about who's listening. If you were designing in a synchronous world, you have a service you're going to pass this to with a restful interface. That architecture is called synchronous. And that's where you need some of these 40 frameworks like Netflix has in order to make this stuff work well. Because you have to have registration, recovery, database recoveries and all those sorts of things with it. Not my favorite implementation. Not for fuzzy problems. Yes. Is the implication when you have multiple instances of a service that you're going to have multiple of the same database as well or are you sharing a database between all those same instances? It varies based upon the problem you're trying to solve. If I need the extra things for capacity, in order to handle the load, but the database is able to handle the same capacity, then it may bring up lots of instances of version one of a service. So we have lots of these little blue things listening on the bus or responding to the bus, having one Redis database. But if it turns out the Redis database is not having enough capacity, I can either replicate that or I begin to segregate my traffic. A through Z, A through F goes over here and G through L goes over here. I start segregating those bases. You also sometimes just put load balance in front of these things. So by thinking of the bus it's going to be a load balance or an engine X. And he's going to have lots of children he can send the work off to. We build those configurations as well. So all those tricks still exist in the microservice world. Yes. In the effectiveness architecture, isn't the bus becoming the new database as in the archery base piece? So the question is with the event bus architectures the event bus becoming sort of the database. It's definitely replacing the persistent store of the database, yeah. It becomes your operational database at some level. And to some degree that's why you want them to be really dumb because you're not making a lot of use of it. And that allows you to switch technologies pretty easily on that. Well to some degree the only you needed to upgrade the bus would be well to some degree if you're using a Kafka you need more capacity. Kafka can be cloned and replicated quite easily. That's part of the Apache project. If you want to say go use the new bus that maybe Amazon has or Google has then you probably need to just take your messages and send it to that bus which means I have to change my rivers but almost everything else in my system is made the same. Because you think about it, service attaches to a river and it's independent of the core bus and it publishes, just publishes a message to rapids and you may have to use a slightly different protocol to publish but that's the only change you're making and that's probably a library itself. So we're not really depending upon the bus but not like we did with the back in the ESL days the enterprise service bus stuff. We're not anywhere near that dependency on our bus anymore. Other questions? Yes. You got a river, what's all things and what types of messages and do you run the library? Well in my world my rapids when I set these up they're actually the place I'm gonna put every one of my log messages goes there. When you're logging things you're logging into that bus. Every service heartbeats on that bus. So I'm always putting heartbeats out there to make sure I'm healthy and I got another service that's actually looking for heartbeats. They stop hearing your heartbeats they're gonna start raising up flags and say I think he's broken. So we do everything on the rapids. Every one of the messages goes on the rapids and we hang things off when they're looking for that. Now we tend to segregate things off of that for the purpose of services. For example if I'm looking for an advertisement put on a page. I will segregate people that are looking for ads into a river and anybody attached looking for that wanna satisfy that need all those guys satisfying the need are gonna attach to that river. I have another one that says I have some information about Sally. So if you have information about Sally there's a river associated with that. Says I have information about which user I have. If you wanna know if you're gonna react to that situation you hang off that river. And again I've never needed more than like 10 rivers. One place I don't use a river is I actually we usually take a service that actually listens to the bus. These are persistent buses anyway but we'll take that information to the bus and we'll still stream it into an Amazon big store somewhere. So we'll save this data for years because data is cheap. By the way debugging a system like this when you have an event bus imagine that the fact that you gotta know a time sequence of events where every service and every part of every service all your logs are in there as well as all the traffic stats. You wanna try to debug a system like this 90% of your work is already done for you. You just gotta find out who put a message on the bus that's wrong and why did he put it on the bus. And you can figure out what service that is very quickly. So debugging this stuff it took about 90% less time than it used to. The guy says it's in although it's backwards so I'm not sure about upside down admins. Okay, I guess it's right side up now. Well, thank you very much. Again, especially for a technical talk at a casual conference. Thank you.