 Okay, let's get the show on the road all right. Good morning everybody. How are you? That was terrible just just awful really really crap try again. Good morning everybody. How are you? What is going on? Um, well, all right. Well, thanks very much for coming. We've got what do we have three hours something like that three hours We've got a lunch. Is that still leaven ish? We have a break after about an hour. So around 11. Yeah We've got a break around 11. So we'll uh, we'll take a quick break there But we have two sections to this discussion today. So we're gonna keep in mind that around 11 o'clock. We're gonna take a break. So Thank you very much for joining me today. Thank you for joining us today before we get going before we get underway We have a matter of very pressing important that we need to address So if you'll indulge me my friends, I need to I need to get a photo now When I say open source You say open source. Okay ready? Open source So money good stuff. So moving on Thanks very much for joining us. Like I said, I've got a some code for you my friends I've got some code. I'm a big believer that if you're among the four or so billion people on the planet Earth were connected to the internet Then code software is the single most powerful thing you can do to affect change and because I believe that I start every single one of my presentations with a pointer To a github repository, I encourage you however much you don't believe me now I encourage you that to note that code and then record it for your own reference later You're gonna want it later. Think of it as a Think of it as a life vest Right, you don't believe me, but you're gonna be swimming soon. So so note that we're note that repository. Okay now I'm also on the internet. I'm happy to answer questions. I'm happy to Correspond if you have any questions comments feedback, whatever, I'm happy to hear from you So by show of hands, how many of you are on Twitter? Twitter Okay, that's that's not nearly enough The rest of you get on it Twitter is great There's a all the all the stakeholders and the the drivers and the open source that powers your business today They're on Twitter. They're happy to answer questions. It's the new IRC. It's a great place to be I love Twitter and you should too. So get on Twitter if you're not already there Otherwise, what about email? How many of you have email e? Email anybody nobody okay that one's You know if you I'm happy to answer questions there as well But remember if I if I help you with something there nobody else can learn from it nobody else can benefit from it So it's a really greedy way of communicating, you know But either way, I'm happy to talk to you in either either approach. I just prefer Twitter or Stack Overflow or whatever A little bit about me. My name is Josh Long I'm the spring developer advocate on the spring team at Pivotal I've been an open source contributor and engineer and several different open source projects Not the least of which are Spring boot spring cloud spring integration spring batch modern timely of activity. I am the the number one number one top ranked for Seven years in a row seven years consecutive and successive number one top ranked highest ranked contributor Bugs but still number one number one know more more bugs per commit than any other engineer for seven years So, you know something I'm also a Java champion, which is a rare honor It's bestowed upon those of us who do our level-headed best to engage the community and help them You know level up with the DVM Which I've done and that double tells very nicely with what I do in my day job It in my capacity as a spring developer advocate where I do my level-headed best to help the organizations communities customers, etc Work and build interesting systems usually in terms of spring And as part of that I've written blogs magazine articles books, etc. Training videos the latest and greatest at which are of course Building microservices with the spring boot live lessons, which I filmed with my friend the one the only the inimitable spring boot co-founder and the amazing Phil Webb and In my latest latest book, which is called cloud native Java now For those of you are wondering and I know that you're all wondering because everybody wonders This is a natural consequence of not having an idea That that bird That bird is it a blue-eared kingfisher It's a blue-eared kingfisher From Java in Indonesia So you see it's a bird That flies in the clouds Birds fly in clouds and it's a native Java bird. So it's a it's a cloud Native Java. No, okay. It's fine. Don't don't worry about it. It'll come Anyway, there's that and I've also, you know as part of what I'm doing I I work at pivotal. I'm very very happy about that part, right? And I love what we're doing at pivotal and we have lots of great open source technologies And that's certainly interesting to us as well, and I love that part as well obviously, but Let's be very clear Let's be very clear the open source technologies at pivotal how however interesting are not the reason we're here They're not the reason that I'm here It's not the reason anybody at pivotal wakes up excited to go to work and to do what they're doing You see for as much as we love the open source bits at at at at pivotal and as much as we champion them as As as as much as we invest in them for example in Apache Tomcat. How many of you know Apache Tomcat? So we are the lead sponsor spring source and then VMware now pivotal have been the lead Contributors to Apache Tomcat for the last decade. So if you've used Tomcat, you're welcome Also, the same is true for patchy, you know Apache the web server the most widely used web server in the world. Yeah, you're welcome That's us, right? Clive friendly Rabbit MQ and Redis for five years, right? These are all technologies that we work on at at pivotal and of course spring So this underscores our commitment to open source into building amazing stuff. But again, it's not why we're here We care at pivotal about helping customers community members and organizations quickly and safely move ideas thoughts Value through the value chain as quickly as possible from product management all the way through to production Product management user experience developers testers administrators all the way off into production And we see that a lot of organizations struggle with this movement this progression They struggle with how to move something as quickly as possible through that loop They want to go faster, but they have trouble doing that They don't really see how the they can capture that agility and they know that agility is king speed is king They know that the faster they can deliver value into the production environment where the customers can use it the faster They can capture that value and Turn it into profit and we don't need to look too far for supporting evidence of this What is the largest taxi company in the world today? What is the largest hotel brand in the world today? What is the largest video streaming? Video rental service today These companies have one thing and one thing in common. You see they didn't have More money more resources more brain trust more intelligence more anything or even more time compared to the incumbent competition They have the exclusive advantage of agility They had the ability to have an idea put it to work Deliver it see it see it see it in production get the feedback and then turn that feedback into change Faster than the incumbent competition And they did so very well very capably you see these companies realize that software is king. They have to treat themselves as software businesses They didn't just compete with the existing incumbent competition in the Bay area where I come from They have a patronizing. I think it's a pretty patronizing You know kind of word for this they call it disruption right These companies didn't just compete the tax your local taxi company didn't doesn't compete with Uber Or if it does it competes in the same way that an ant competes with her with a rhinoceros, right? It's not the same thing So these companies have captured agility they know how to go faster they treat themselves as software companies There's an amazing book. Have you ever read the inmates are running the running the insane asylum? This book has a lot of great ideas one of which one of my favorites of which is this idea that if you Take any business any business at all and you mix it with software what you have is a software company If you take taxis plus software you have a software company if you take hotels plus software You have a software company the differentiator there is software You're not going to get the same leagues of return the same orders of magnitude of return by investing in the taxis You're going to invest in better software to automate the taxis to enable taxis That's the differentiator if you take any business at all you get a soft and put and mix it with software You get a software business and these companies have understood that so they run themselves as software businesses They deliver on software time frames. They innovate on software time frames like a software company like a silicon valley company So a lot of organizations look around and they see this landscape of really really fast agile companies And they understand that they need to go faster as well And they struggle with this because they've got large existing applications. These these organizations are lucky enough to have been around Before the era of cloud computing they've got software that predates the era of cloud computing and the economics of cloud computing They've got large existing applications upon which a large group of people of toil Large existing applications that are very difficult to change to evolve to to Update and the reason is because it takes a lot of people to make any kind of small change to affect any kind of change And these large unbroken code bases these monolithic applications So these organizations understand the necessity they understand that they have a need for speed They understand then in the existential threat implied by those who have agility, but they can't quite get there They're looking for ways to do it and they struggle they know that these companies when they started were very small So they see that that's part of the the formula. They have to somehow Think about small batches work in terms of small batches the smaller the batch of work that they have to push to the value chain The faster they can go But they don't know exactly how to take this large application and decompose it to break it apart into a smaller batch We can turn to Dr. Eric Evans. Dr. Eric Evans wrote an amazing book called domain driven design Domain driven design has this idea of a bounded context A bounded context is a part of the domain model that when extracted from the larger hole stands unto itself internally consistent and reusable A bounded context is A crispening if you will of parts of the of the domain of an application And i'll give you an example. Suppose you have two contexts that of sales and that of customer service Nominally in both contexts, you've got this idea of a customer But is it really the same thing? When you have when you have somebody who's upset and they're trying to get money back on customer service line You have somebody who's already paid money and you've already got full account information Compare that to somebody who hasn't yet invested in you're trying to incentivize to buy You're trying to sell right these are not these are not the same thing. They have different states different life cycles They have different known data sets If you muddy the concept of a customer By trying to share the same entity across both contexts, you'll only think only make things more confusing Because that state will that object that entity will always be in different states instead tease them apart Treat them as two different things make clear that this is the this is somebody who's got a complaint And this is somebody who's a lead Extract them out if you can do this if you can identify these bounded contexts You have a boundary a natural boundary along which to cut out parts of the domain A smaller batch of work A smaller batch of work upon which a small group of people can work The small group of people thing is very important You see the whole point of reducing the size of the of the context They're of of identifying these small batches of work is to reduce the size of the team working on it That has extraordinary benefits the most obvious of which is that it takes less time to to communicate with everybody in the Team about what we're about to do another benefit of course is Now you can collocate all the people involved in delivering that particular feature on that team Which further reduces the time to deliver value because now instead of me as a developer throwing my work over an imaginary wall And waiting for somebody downstream to test it when they have time on their schedule Now that person working in the testing group is on my team He or she has no other priority but to test the code that we're working on in this in this feature Right you've reduced the waste the inventory the queue time in between different stations in the in the value chain By doing that We go faster now The question of course is how small should that team be what does that mean to be small Amazon.com founder Jeff Bezos talks about this. He calls this a A two pizza box team It's a team that's small enough that you can feed everybody on the team with two boxes of pizza now I know I can see her. I can see some eyebrows for unfurling I know what you're thinking Two pizzas two boxes of pizza feeds a lot more people here than it does in in california where I come from But but in the states when we talk about two pizza box teams We're referring to about five six seven people right very very short amount Very short small group of people the the important part here is that you can keep in your head what everybody else is doing Right you can keep the cost of communication down And that is after all what we're trying to do here. We're trying to identify small Batches of work that a small group of people can toil on and work on and push through the value chain as quickly as possible Independent of the rest of the organization independent of other teams We're trying to reduce the cost of communication and synchronization in an organization That's the that's what we're trying to do here. We're we're trying to build microservices A microservice is a hack on conway's law. It's an optimization Conway's law, which was which originated in the 70s says that software is a mirror image of the organizational structure that it serves So if you have different teams in an organization that do a crap job communicating Then by definition the modules upon these different upon which these different teams work will probably have poor integration They've done numerous studies successive studies to confirm or at least understand this this effect And they've done an example where they compared open source software to proprietary software So two code bases of analogous function One open source one proprietary and they they saw that time and time again the software developed developed by the People in the open source world Tends to have better modularity Whereas the proprietary software tends to have far less rigid modularity in the components in the code Can anybody can anybody uh hazard a guess as to why? yeah, well the uh The open source software is developed by people spread about the world spread around the world in different time zones working at different Paces different availabilities. So in the evenings on the weekends Uh, uh different days of the week, etc. Whatever they can right? Maybe they're on vacation as opposed to the people who are working in the proprietary software in the same office in the same time zone At the same company in the same geography, right? these people these people are Easy they have an easier time of talking to each other They can just turn their chair and ask questions of their neighbor and say what do you think of doing this? Whereas with the open source people they have less of an easy chance to do that It's harder for them to all jump on a bridge and ask questions and to have a long discussion It's harder for them to be in the same room to have a brain set brainstorming session So the componentization is important because that becomes the communication channel The api itself becomes a formalized boundary for communication Microservices aren't a hack on this effect We are making it easier for teams to go faster by formalizing the modulators module boundaries between different components in the system You get a lot of benefits by doing this we get autonomy we get the ability to go faster But we run headlong into two big problems when we make this when we make this move Two big concerns that we have to address before we can enjoy some of the benefits of this architecture approach The first is The first question the first concern that we're going to have is how quickly can we stand up a production worthy service? And all of that implies How quickly can we stand up a service that is destined to be in production that has the The appropriate infrastructure and middleware and you know load balancing and in an environment and dns and and uh And how quickly for that can we for that service? Also address non functional requirements things like security and observability things that you need to do to ensure the proper Function of a of a system in production, but that doesn't better differentiate you in the marketplace It doesn't make you or your business a better business right it's stuff that you have to do in service of Of operations, but not necessarily for the business. How do you do that? That's that's the first Question how quickly can you do that a lot of organizations struggle with that because most organizations that I've been doing again? Not yours of course surely not yours surely not but But most organizations that I have been to have a terrifying nightmarish wiki page The wiki page with 500 easy steps to production That wiki page is the enemy of velocity that wiki page is all the stuff You need to do before you can stand up a production worthy service and it defeats our ability to go faster It creates friction And think about it How hard is it for you to get a new environment right now to get a new server to get new databases? If it's if it's a Very hard to do it you won't do it I'm a I'm a big believer that there's no such thing as a a good person or a bad person There are good systems and bad systems and you are a product of the system in which you exist If the system in which you find yourself makes doing the wrong thing the easy thing then naturally you're going to do the wrong thing That's just human nature It's the path of least resistance Case in point if it takes Weeks to get a new environment stood up and new new permissions and credentials and all that setup So I can log into a machine and then deploy software I'm not going to do that Right, I'll just I'll probably just take this existing application that I've gotten add more rest end points to it Or maybe I'll shoehorn my data into the database that I've already got permissions for and credentials for The right thing to do in this case is to factor out the code into a separate application a separate service And to use a database that's appropriate for the task at hand But it's so much easier to just shoehorn it into this existing code base and existing a database And we see this time and time again If the right if what you're trying to do is to model full text Then the right thing to do is to use a full text search engine something like elastic search or solar But we won't do it. It's so much easier just to make oracle, uh, you know do some full text searches on their indexes If the right if if the right thing to do is to use a geospatial aware Engine something that supports a geographic queries like couch base You know sometimes people know into that the right that's the right thing to do but they'll put it Instead they'll pay 90 000 for oracle spatial because they've already got a license for that Maybe you're trying to to store data binary data Right, the right thing to do is to use something like MongoDB's grid fs or to use s3 or a service that's optimized for storing bytes and scaling them out But most people don't do that just they just shoehorn it into oracle blobs right If you're trying to model interconnected inter-relational data in which the relationships themselves have semantics The right thing to do there is to to use a graph database like neo4j But so often people try and shoehorn it into uh rdb ms's relational databases, which don't actually have They have foreign keys to to disk offsets and sequential disk based systems Which is the how SQL and foreign keys are sourced from the 60s in 70s If the right thing to do is to randomly lose your data at random times Then you should use MongoDB right But many people don't they just try and use something else for that again You have to you have to care about the use case at hand But it's so hard often to do that So that's the first concern you're going to run into the second concern That you're going to run headlong into is how quickly can we address the complexity that we've invited into our system The complexity of building a distributed system because now we've got lots of small interconnected services deployed across the network Across network partitions talking to each other and this complexity is significant This complexity is very very significant And if you fail to address these concerns, you'll have a lot of pain For if there's one thing upon which i'm sure we can all agree It's that building distributed systems is hard And so we need to address that complexity So to address the first concern today, we're going to look at spring boot and spring cloud How quickly can we stand up production worthy services individual services and and the Corresponding infrastructure and middleware will use spring boot for the the application concerns And we'll look at cloud foundry as a way of managing environments and and so on Then to address the second concern the the complexity implied by moving to a distributed systems world. We're going to look at Sorry, we're going to get spring boot and cloud finder. And then for the second concern, we're going to look at spring cloud Okay now I don't I don't even know why this is go away shoo All right, so we're going to begin our journey here to production on my second favorite place on the internet Anybody know what my first favorite place is? Sorry production I love production You should love production. You should go as often and as early as you can It's great this time of year especially the weather is amazing Bring the kids bring the family go as often as you can. I love production. It's the happiest place on earth. It's better than disneyland If you can go to production, you should but if you haven't been Then you can begin your journey here at start that spring If You want for inspiration in the early morning before a cup of tea or coffee Start that spring That I owe if your children are restless and can't sleep start that spring that I owe And if you suffer from indigestion and seek relief Perhaps after a long night of laxah start that spring That I owe Bookmark it keep it close to your heart. Keep it under your pillow. Keep it near you at all times and what we're going to do is We're going to build a very very simple service I'm going to call it the reservation service and we're here We're going to take advantage of different technologies that spring supports different types of workloads that spring supports We're going to use spring support for building web applications We're going to use spring support for centralized configuration We'll use spring support for service registration discovery We're going to use a rabbi mq for stream processing zipkin for distributed tracing We're going to use the rest repository support We're going to use jpa the the java persistence api because I make poor life decisions. So jpa and then Then we use h2, which is an in-memory embedded sql database And because it's an in-memory embedded sql database, it's going to randomly lose its data after every restart in this way It's very similar. I think you'll agree to to mongodb We're going to also add actuator for observability and operational concerns And that'll do now I could of course elect to switch to the full version If I switch to the full version I'll be given a variable ocean of checkboxes Options things I can elect to include my application if I wanted to and I I'd certainly encourage you to Peruse this list at your own discretion later on But for now it suffices to leave the checkboxes as we have them because that'll do now You should see here all manner of different technologies Some of which I'm sure will appeal to you even as we score by them So keep this page in mind like I say lots of good stuff. Okay Now You have up here three drop-downs This last one is kind of interesting. This last one Is the choice of language What language would you like to you would you like to use to build your application any language on the jvm that supports Annotations and objects will work just fine. So java groovy scala kotlin even saline for the two people using that It's just fine right use them if as you like anything will work And Here we have two more drop-downs and this is where people get very confused These are you see these are drop-downs and they look like choices, but they're not actually choices They're what I like to think of as non choices They're choices in the same way that stripping naked and running in traffic is a choice You could But but please don't Please So for example In 2016 Which version of the jvm would you like to use? As both 1.6 and 1.7 are end of life expired gone No longer available past their prime Not supported Not available for updates extinct deprecated As both of these for more than a year that's 12 months in human time If as both of these are more than a year past being end of life To continue to use either one is irresponsible and an active source of technical debt in your organization To start new projects in 2016 on either one is insane And web sphere by the way is no excuse So don't ever use these ever We also have the choice of packaging And people get confused by this they don't know when and where to choose which so i'm going to do my level headed best Here now to explain and when and where to use which if by some freak accident of physics Some terrible terrible accident of physics You find yourself stuck in the very very very very distant past Far far far far far far beyond modern help Then then choose dot war But if you're here With me in 2016 Which isn't even the future really it's just now so it's not even that impressive Then Then choose dot jar This is a big part of my overarching guiding personal philosophy of make jar not war and again You have options You have you have choices you should do what works for you Now i'm going to leave the options as they are because they sue us Uh and they're the right choices. I'm going to go ahead and generate a new service like so And i'm going to open this up in my ide and this is just a typical maven spring Application nothing all that fancy or involved about it. What we're going to do is we're going to Develop an application a simple service We're going to develop a simple a simple service And we don't really care all that much about the domain of the service or even the service itself We just need something with which we can play an experiment Can you all see that in the back should I make that larger? Let's see Can you read that okay good very good so So now I have a a typical spring boot application. I've opened it up in the intelligent But let's make sure make make no mistake. It doesn't matter which ide you use How many of you using intelligent? Very good good stuff hot sauce. What about eclipse some sort of flavor of eclipse good stuff right well done hot sauce as well What about net beans net beans? Okay, well that's great stuff as well works great What about emacs are you are you here sir? Are you here there you are it's that guy it's the one guy He's in every talk I do I go to hundreds of cities every year and it's always him the same guy I don't know how you do it man, but you follow me everywhere and it's getting a little weird Every time I ask who uses emacs he raises his hand. I use it and then he leaves He doesn't even say for the whole talk. He gets on the plane to go to the next place It's just terrible troll Anyway I've got now My spring dependencies now these are opinionated starter dependencies these dependencies Give me everything I need to be productive out of the out of the gate or out of the box with the For example jpa This gives me hibernate 5.x jpa 2.x transform transaction support or I'm support etc etc etc Now I don't have to work to make this to line up the different dependencies I don't have to specify how to you know, which libraries to exclude and which ones to include It all just works and if any library Shares a conflicting dependency with another library and there are different versions We've made sure to line them up so that there's at most one version of any library on the class path Avoiding all the painful conflicts that make using different components in the java ecosystem so so painful That leaves me with an empty java class a public static void main class Now this is a spring boot application. How many of you have used spring before? Okay, some of you good This is a spring boot application. The spring boot application is in turn actually just three annotations It's semantically the same as saying At configuration at enable auto configuration and at component scan These are the same exact thing now These annotations are standard spring. This is stuff you've probably used used before in spring This first annotation Tell spring that the class on which that annotation is set is a java configuration class So suppose I have class bar and class foo And suppose that foo has a dependency on or sort rather bar has a dependency on foo I can I can tell spring about the arrangement of these objects I can tell them about the wiring of these objects by saying return new foo and return new bar And the collaborating object the dependency here that bar has on foo I can express as a as a parameter into this being provider method. So there it is right there I'm telling spring that I want the configured Initialized instance that comes back from this method here. This is a being provider method. And so I can inject this foo as many times into as many different collaborator objects And I only I'll always by default get the singleton instance. I'll get one instance scope is still preserved here, right Now this is one way of telling spring about the wiring of the objects And this is based on the java configuration annotation. That's this one right here Another way to tell spring about the wiring is to use annotations on the components themselves and this this tells Spring to imply the structure based on these clues that we're giving at these annotations these conventions Now you can use either or or indeed mix and match both approaches. So you can say being foo return New foo Okay, and we can say that This should be just a regular class without the component annotation and that will still work We've defined the being here and we've defined it here, right? We've defined we've injected it into the collaborating object into the constructor So the wiring still works another way to get spring to see the arrangement of your objects Of course is to use the the classic xml configuration format So if you have an xml application context configuration file, you can import that here and that'll still work as well But I wouldn't use that right? I prefer the java configuration And this finally is the thing that activates spring boot. It's what makes spring boot do what spring boot does and we'll come back to it Later on but so for now it suffices to know that this annotation Is a syntax shortcut for all those things that i've just shown you okay now With that done we can build a simple entity because we've got spring data On the class path and we've got spring data jpa and i'm going to build a jpa entity here And i'll use a Primary key that i'll signal is a primary key here by saying at generate value At id i'll give it a field here. I'll say private string reservation name I'll create some getters and i'll create a constructor and i'll create another constructor because of jpa right There's this and i'll create a two string method and there we go now i've got a jpa entity I've got a few fields that i want to persist into the database. I'm not really interested in the jpa entity I just wanted something with which we can play Really i've only got one field now based on if you've ever used jpa Then you know this will get mapped to a column in the database called reservation under underscore name right for reservation underscore name And this will get mapped to a table called reservations And i want to be able to read and write and manipulate instances of this entity So i'm going to create a repository now again We can turn to dr. Eric evansen and we understand that a repository is a is an object that's meant to handle the boring tedious Sol annihilatingly stupid creation reading updating and deleting of entities It's meant to interface with the underlying persistence here, but it is by no means business differentiating functionality It doesn't further your lot in life. It doesn't make you a better partner to your significant other It doesn't make you a better person. It doesn't make you a better mother daughter father or son right It's just stuff that you have to do to talk to the database So if you can reduce the amount of time you spend toiling on that to as little as possible Then you should absolutely do so and that's what we're doing here instead of writing a whole repository We're letting spring data do the heavy lifting for us. We're declaratively defining the repository Based on convention and we can see that um spring data's jpea repository for example has methods like find all find all uh save flush delete find by id etc All of those methods will be implemented for us automatically. We don't have to implement this interface We'll get a bean that implements this interface for us and we can as I say create methods by convention I can say find by reservation name string rn And this will turn into a query at runtime something like select all from reservations Where reservation underscore name equals rn naturally I could override it I could say at query and then pass in a custom query and that would uh that query would um You know override the default and I and and you should if you want to right We're not trying to to position spring data as the lowest common denominator works on every kind of persistence to your technology Uh abstraction instead We want you to take full advantage of the lower persistence technology if you need to it just suffices in this case that we Can leave it as is we don't need to provide the custom query But if you're using jpa, then absolutely use jpeql if you're using neo4j Then use cipher if using mango db then use b sign if you're using couch based in or set rather casander then use cql Right you should absolutely leverage the language uh that powers your technology You didn't invest time and money and resources just to treat your underlying uh persistence here as a as a as a Crud machine right create read update and delete you want to do queries and analytics and take full advantage of the power That that thing has so you can do that in these custom queries We even support, you know specialized kind of parameters suppose that we were using mango db right now If we're using mango db or couch base, they both support geo spatial queries I can define indexes that are based on geographic indexes right so I can say find my reservation name and point And I can pass in for my second parameter a spring data geo point right So this would actually do a query in mango db or couch base that looks for the reservation whose reservation name equals this And whose geo spatial index for point matches that For example, so you can fully and natively exploit the underlying Technology if it makes sense to do so Okay Now let's go ahead and save some sample data into the into the database so that we have something with which we can We can play for today So what I'm going to do is I'm going to build a command line runner a command line runner is an Callback interface in spring boot when spring boot starts up. It's going to call the The run method here. It's going to call the run method On this object and it's going to give us a chance to do any kind of application initialization This is an ideal place to create or to put in application initial initialization to put in Any kind of batch or etl or messaging or integration logic that has to happen outside of the traditional request response flow of the application Right in our case, we're just going to create some sample data So I'm going to say that we're going to have a collection or stream rather of names. My name is josh It's lovely to meet you. Uh, what about you buddy? What's your name? How do you spell it? Sorry Nice to meet you. What about you buddy? Are you in? Very good. Nice to meet you. Thank you. What about you buddy? L you in? Okay, nice to meet you as well What about you buddy? You yeah Oh, I knew that That one is that about right? Okay, very good. I forget names We've met before um Oh, let's see Who wants to go next? Oh, I need another name Wow, we have One one woman How many women are there's not I need a woman. Is there a woman name I can use? Throw it out. Yep. How do you spell it? A d i t i very good. Thank you so much. Nice to meet you another one Don't be shy I'm just gonna call in you then Uh Miss over there in the far far back. Yep. Hi B E R O Like this very good. Lovely to meet you as well any more Okay, well, thanks for coming You know, okay. I need more names. We can't just leave it at six. I need at least eight Because it's a nice even symmetric number and I'll sleep better tight. Six is okay, but eight is great now anybody What about you buddy in the red shirt? What is it Very good. Love it to meet you and uh, one more. What about you sarah with a the The skin that looks kind of like mine What? No, right behind you. Although you too. Yeah Andre and d re Love it to meet you and you buddy Sha un or sha wn Very good. Cheers. Lovely to meet you all Thank you very much for coming. That's that's enough names I wish we could add everybody but we don't have nearly enough time So what we're going to do is I'm going to visit every record in this report in this uh stream And we're going to save a record into the database then we're going to Confirm the existence of those records by calling the print line method in the repository and then Iterating or visiting every record that comes back and then printing it out now here I've used a a nice feature in java 8 called lambdas lambdas make it very simple to express Uh Functions as a first-class citizen, right? We can treat them like like objects which sadly This actually does get turned into at runtime. So there's that anyway, um, we now have Data on the uh on the console in the database, right? We've confirmed that everything is working It worked as we expected. Of course it worked It was a demo What were you expecting? It was always going to work Instead what I really want to talk to you about this now This is the ascii artwork in spring boot This ascii artwork took a long time to get right. You see we have people on the spring team that are doctors PhDs They work in nuclear physics in their previous lives Very very very very intelligent people If there's somebody who has a heart attack and somebody says is there a doctor in the house There's several people on the spring team that would raise their hand Just wouldn't be the right kind The point is it makes me very happy to imagine that someday somewhere Somebody got a github issue that said damn it. We need ascii artwork And I think you'll agree they delivered It's absolutely gorgeous very well done very well thought out now It's at this point that I'd like to take a brief moment to talk about what I consider to be a very Glaring deficiency in the jet brains intelligent product for while I am a fan I think this particular feature was poorly conceived What the hell Why is that there? That's a dumb checkbox And so I did what all people do when confronted with adversity and challenge I went on the internet and I cried loudly And I was given a message of hope which I share with you here now This is a message from my friend Jan Sabrone who's a software developer by passion at intelligent idea jet brains Here's his response Don't worry my friends. We're going to make intelligent great again. Now a lot of times people ask me How can I change the ascii artwork? How can I override it? And even now even as I pronounce those words I resist the urge to start flipping tables and leaving the room because that's a stupid question Know your station life. Know your role. You're not going to do a better job than what we've already done That's a masterpiece But that said I'm willing to concede that there are some very talented people out there and Anything is possible. So I'm going to show you how to override the ascii artwork All you need to do is to create your own banner.txt And put it in the source main resources Directory of your application. So here I'm going to curl a banner.txt into the source main resources directory of my application And then I'll restart having done all that hard work. Now. I don't think you'll agree that this is this is better now There's a few things we should focus on In this uh in this artwork First of all Meow Second of all and this is basically why we're here and why I'm so happy to be alive anyway Now we've got data being written to the database And we're confronted with an existential question something with which I'm sure we all wrestled in university If we write data to the database, but we can't read it from a rest api Did we actually in point of fact write it to the database and the answer is no You didn't so you could have skipped that stupid philosophy class Instead we need to build a rest api and we could do this a few different ways We could take the long way around by bringing in spring boots start a web like this and then go up here and I need to make my font a little I need to make my screen a little larger in my font a little larger Okay Close that font 24 Can you read that? No What did I do? Buddy Was it something I said? Nope, I didn't work. I'm all out ideas We're in for president By the way, he uh He sat through my terrible jokes before And got this talk a version of this talk online in less than like something like I don't know 12 hours or something like that after I gave it at the Singapore spring user group. So I'm a big fan. I'm a big way ran fan. Thanks, man. You're awesome What did I do? It must hit my jokes too huh My computer just re refreshed Let me see displays Uh-uh scaled Could you change it to p instead? Sure Okay scaled p There we are Now we're cooking. Thank you. We ran Okay So now We could take the long way around we could go back to our build and bring in spring boots start a web And this allows us to this gives us everything we need to build the rest applications and and that works, right? By the way, can you see the font? That's what I meant to ask Everybody anybody can't see it Okay, so we can take the long way around we can build a rest controller. We can say a reservation Rest controller and we can inject the rest a reservation repository Right, and we can create a constructor and then inject that value into the constructor create a new endpoint here And then all we're going to do is we're going to return all the records in the database In this end point and we're going to map that endpoint to htb get value four slash reservations Okay, so that is certainly one approach to building a rest api But that's an awful lot of code for what i'm trying to do I'm trying to map the business state transitions that the very same state transitions about which my Repository is already aware to the appropriate and corresponding htb verbs So instead of me doing all that hard work Why not just let the repository which already knows how to do that? Let it do the heavy lifting right it already knows about the the business state transitions We're going to bring in spring data rest via the spring boot starter data rest dependency And in so doing we can we only need to add an annotation here repository rest resource And then we're going to annotate the finder method here. We're going to say path by Name and we're going to annotate the parameter. We'll say at param rn and then where we start an i will take some t Because I've earned it now We'll go to local host 8080 forward slash reservations And we can see here a hypermedia api. Let me see if I can make that out. There we go We can see a hypermedia api. We see the payloads as one would expect But then we have a collection of links and these links are metadata their Information about the response about the rest api. They promote self-describing services. This is a a design pattern called hot os Hyper media as the engine of application state It's the idea that every rest resource should have information enough In the response for the client to be able to further manipulate that resource without any apiary knowledge This is very very useful in a distributed systems world where everything is an api Remember and remember very very very few developers write good documentation and none read it none zero nobody And so the only humane thing to do is to make it as accessible as possible to understand how to work With and interact with a service hypermedia gives you that effect You can see that this achieves two things first of all We've decoupled the client From the urls themselves right the client doesn't care about reservations four dash one. It cares about the link called self That's the contract so long as that link id stays the same Then it doesn't matter what the url is and you're you're probably used to doing this all the already Nobody even realized it so link href food at css type equals text css Etc that it's a link element in the html markup It's a link element the contract is the text css The html browser the the browser doesn't have an expectation or an assumption that that resource is always going to be available At styles.css it just looks for the link and then it follows or traverses that link The link can change tomorrow the url the resource can live at foo one dot css But the client doesn't break The same is true here with the the rest api This link the url can change tomorrow, but the client's not going to break because They're only looking for this the link id So we've now given ourselves the ability to change our api topology Over time as our businesses evolve so well our domain and that's fair. That's that's very reasonable This helps you with a versioning problem, you know making sure that as you evolve your system over time you have flexibility Another benefit of these links is that they give us an idea of state Imagine you're on amazon.com and you've added something to the shopping cart something for which you haven't yet paid Do you imagine for an instant that amazon's going to offer you a refund? A button you can click to get a refund for products for which you haven't yet paid Of course not right? That's bad business. That's no way to run a railroad. We need to do better They're not going to give you a button for a refund until you've paid until you've checked out Similarly once you've checked out there's no button to pay For products right you can't like check out again This is state. This is contextual in the same way these links are contextual You can dynamically contribute or remove links based on the state of the resource If you have a product in a shopping cart, you might have a a checkout entity or you know a cart entity and you can Change it you can change the status by posting to it, but that entity won't be there if the product's already paid for Now the client doesn't have to remember a prayer You know beforehand or our priority which end points to call in which order It's all explained the navigation the navigable state transitions are given to you by the api This is how you impose state in another way stateless architecture This gives you a lot of flexibility in a distributed systems world now This also gives us some other niceties right I can scroll down and I can see that there's a A search link and the search link has an endpoint here For search and I can go to the search endpoint and it says that I've got one particular finder method called by name And if I click on reservation search by name Rn equals and I'm going to find one of my friends here. We're in the the champion And I then do a search for we're in and we find the result right so there we go That search endpoint gives us what we want out of the box Um, I can even do paging I can say page equals one and size equals Two for example, and if I do paging I get that for free right? So there's those two records and contextual or stateful Metadata links that tell us the navigable state transitions from here to there I can go to the first page results the previous one the current one the next one the last one etc And we can do get put post delete etc. So this is all being powered by spring data rest Which is in turn using spring hot to us to describe the the resources the envelope objects that are Embodied by this j-sign here. Let me expand that. Can you see that box boundary there? There's a box that box The contents of that box is mapped more or less back to an all an object called a resource of tea so a resource object in a a resource object in a in In spring hot to us is an envelope object that has a payload and a collection of links We'll see that later. So keep that in the back of your head, but spring hot us in turn builds on spring mvc All right, so now we've got a nice simple api. Are we done yet? Can we go to production yet? Are we done? Are we are we finished? Can we ship it? Not yet, right? There are things that we need to do now that we've built a very simple service We need to do in the service of operations in the service of these observability requirements and maybe security These are things that we can't ignore as much as we'd like to even though they're not glamorous They still need to be done Observability in particular is something I want to touch upon We need to make sure that this application can express itself You see your application when it's deployed into production doesn't give off a telltale smell It doesn't give off a telltale sound that that tells you that the application isn't working It's not like a car engine where you can hear the the the whir of the engine and you can hear Something that alerts you as a problem in the engine It's not like a food where the smell tells you something is wrong. It's completely silent You need to make your application explain itself And who better to to identify problems in an application than the application itself So we're going to stand up Endpoints that surface information about the application by using the spring boots actuator sub framework or the framework inside of spring boot called actuator actuator is inspired by google's borg monitoring approach to infrastructure and monitoring The the idea that the thing that google does is that for every workload for every service for every process They have in that process htdp endpoints that surface information about that process Even if the workload itself isn't htdp They'll have htdp endpoints So it might be machine learning or a batch or integration or something that has nothing to do with htdp But they'll still have a web server that stands up and exposes this information That's what We're doing here. So by adding actuator. I can now Drive some traffic. Let's do some traffic here. One two three, etc Do a search if I want right And now I can go to metrics or m e m e t r i c s If I go to metrics, I can see an enumeration of Quantifications of the different status states of the application how much memory I've got how much is free the processors The uptime the the uh the heap the non-heap the threads the class is loaded the data sources the Etc I can even see down here counters this counter says that I've made one request to four such reservations And I had a status code of 200 I can see here that I made one request or three requests to reservations four such one two three and had a status code of 200 Now you may be wondering about the distinction between a gauge and a counter. You see spring boot knows about two special kinds of metrics These other metrics are uh ad hoc public metrics, but Spring boot has two very very interesting specific types of metrics that we can use the first kind of metric is a gauge A gauge is a metric that you can calculate on demand You don't need to tabulate the information over time to arrive at the value that you're going to then report in the metric How much ram do I have right now? I don't need to keep a counter to ask for each megabyte. I can just ask the operating system. How much ram do I have? That's like that's a gauge Right. How many how many users are in the chat room right now? I'm not going to do an increment or decrement. I'm going to just say select count from chat room Right on the other hand a counter is something where you're tabulating the number over time How many users have checked out For each new checkout I say plus one Right, there's one more htp request plus one There's one less job in the task q minus one counters are incrementing and decrementing metrics How they're how they're arrived at is different, but the result ultimately is the same you have a key and a set of numbers These metrics are interesting because they give us information about the application You can of course inject the counter service and the gauge service in your own code And use that to capture or emit custom metrics if you do that If you do that, uh, you can add business metrics or pkis product, uh, sorry, um, yeah Key performance key indicators, right? You can add these things into your own metrics and capture them and use that to drive Insight, but you can't really do a good job of that without one very important thing you see We don't have any context for these metrics We don't have any idea of whether these metrics are an improvement or deterioration We don't know that because we don't know what these values were before These metrics lack the ever so critical dimension of time Without time we have no basis for for understanding whether these are are better or worse good or bad Without time we can't see patterns and if we can't see patterns then we cannot make predictions These numbers are at best a latest in greatest reflection of the point in time value that they most recently had That's not good enough and we can capture time by using a great library by a guy named kota hail How many of you know kota hail? Kota hail is one of my friends and spirit animals This is kota hail I share him with you now Gnoming pretty hard right now dum dum dum dum That's kota hail also This is kota hail and I want to take a moment to really focus on on this photo now There's a lot going on here There's a lot going on. I get that you're overwhelmed by the awesomeness of it. First of all First of all notice how both master and dog Are intently focused on the code notice How the dog's ears are up in tension His priorities are clear. He's he's listening for feedback. Do you see their their pair programming? Notice how both are dressed for the occasion. They both understand the task at hand and they both came in pink And notice that amazing hat The hat wins So this is kota hail and I show you these photos so that you'll remember them because you're still going to use his library because it's that good And then you're going to know that kota hails code is all up in your production systems operating your banks operating your governments That's this guy And you're going to feel better about the world So he created this library called the droppers of metrics library And if you have it on the class path that it integrates seamlessly and automatically with spring boots actuator framework If you use it you can capture a third type of metric beyond a counter and a gauge a third kind of metric called a histogram A histogram is a statistical structure that captures things like the mean the median the max the average The minimum the the 95th percentile the 99th percentile the 99.9 percentile the 99.9 Point, you know 95th percent 99.99 percentile, etc. It captures all sorts of calculations Statistics about each of the metrics that it sees but it does this by using something called reservoir statistical sampling that is to say It doesn't keep every metric that it sees over time in memory It recalculates the value as it moves forward in time This means that the values that you see captured are necessarily a little bit lossy They cannot be accurate right you can't get an accurate average unless you have all of the data points in the in the set But you can get a good enough average and they do some pretty sophisticated math to do that work for you The benefit of course is that you don't have every single metric in your system inside the memory pool of your each of each application Which means they don't you don't run the risk of overwhelming the ram buffers in your machine Which is very very useful in a production system That's very very powerful because you have an at-a-glance idea of whether the number you're looking at the metric You're looking at is About the status quo or if it's a complete deviation from the normal right Now sometimes it can be very useful to look at all the values Sometimes it can be very useful to have all those values So you can do querying or an analysis on those values and for this we can use the drop wizard metrics libraries various various reporter objects These reporter objects let you talk to downstream time series databases Tools that want to help you understand and graph and work with this data Most of these time series databases speak a particular protocol called stats d Stats d refers to both a daemon and a protocol and while while very few people use the daemon the protocol itself is becoming ubiquitous It was developed by etsy It's a way of shipping or it's become a de facto standard for shipping metrics And so you can use the drop wizard metrics library for shipping stats d metrics to time series databases A time series database is another type of database that isn't oracle It's optimized for storing keys and values over time in the continuum of time And once you've got it You can then use tools like Once you've once you've got your drop wizard metrics reporter rather you can then use tools or time series databases like influx db Or prometheus from soundcloud.com or open tsdb from stumble upon dot com or graphite or ganglia Or the newly announced riac ts or time series from the people that make riac at basho These tools are all optimized for storing large amounts of data over time And then you can use tools like grafana to graph that data This is grafana It gives the ability to drive that ever important single pane of glass dashboard experience that is critical to a production system you see a cloud native system is four things and we're going to touch upon these points over and over and over again for the next you know Next little bit of time that we have together today A cloud native system is one that is That lends itself to easy and agile evolution and iteration A cloud native system is one that benefits from the elasticity and dynamicism of a cloud environment scaling out A cloud native system is one that does the right thing in the face of service outages and topology changes And a cloud native system is one that is observable that is to say it's Monitorable by its outputs. So the actuator Speaks to and helps us with that last point It specifically helps us understand The state of each individual node in a system Right the metrics endpoint is only one of the many endpoints that you can use in the actuator. There are others There's env for example, which uh, doesn't work because my application seems to have been Oh wrong port, haha There's env Env is in is the environment. It's uh, the system properties and the environment variables for this application You can see however that sensitive information like my underscore passwords and my underscore key and my underscore secret Those are all Obscured their hat. They're they're masked I've also got uh mappings Which shows me all of the endpoints available to htp that are that are stood up by spring as well as any metadata if available So I've got an endpoint at four slash heap dump that responds to htp get that produces the application application octet stream I've also got trace trace shows me the last 100 requests that have been made into the application by default And their headers and their time stamps and so on Right. This is great for debugging interactions between uh, job job script and services. For example, it's great for security Handshake exchange information, you know, just great for a lot of stuff. Okay There's also info now info is empty by default. It's up to you to customize it There's a lot of useful things you can put here namely Sir identifying information things you can use to identify which service is running in production When you move to a continuous delivery pipeline, there's the possibility that the the wonderful possibility that every get push Could trigger a build that will ultimately result in a build that will go through an exhaustive Test suite harness that will then deliver something into production That's a very very powerful thing, but it means that every get push could be something in production in 10 20 an hour whatever If that happens, it becomes very important to be able to identify the version of the code that's in production Because you might have 10 new versions in a single day or 100 So you might put for example the the git commit id Using maven resource filtering you can you know At compile time you can copy a value into a property file and then have that included into the application There's actually a documentation on how to do that with spring boot in particular Now you have an identifier so you can use to understand which version of the the code is in there And of course there's my favorite endpoint health Now health Tells me it gives me an enumeration of the different health indicators in the application It tells me information about different subsystems in the application I've got for example data source and the data source is Got a connection pool and the connection pool has a validation query And the validation query is something like select one from dual or whatever. I can see the results here So very good I may want to customize these management endpoints I may want to for example contextualize and move everything under 4 slash admin And I may want to also create my own health indicator So i'm going to go back to spring boot and i'm going to show you a few ways by which we can extend or override the default behavior The first is to go to source main resources and uh Use any of the well-known properties to which spring boot respond to change or Override the default behavior. I can see for example that management.context path is useful for prefix or contextualizing the management endpoints If I go to my code I can also create a custom health indicator. So I can say class custom health indicator implements health indicator. Okay So this is just a simple health indicator It's an object of a well-known type that will get plugged into the machine As a cog in the machine so to speak. So I'm going to say return health dot What health dot out of service surely not singapore never Down I don't even know what that means So we're going to say we're either we have a status or we're up always right like Chuck Norris So I'm going to say I hard Laksa, okay Uh, and then we'll say that build Or actually I'll just say that of singapore and it's food okay So now I've got a custom health indicator. Let's go ahead and restart and we'll see here That this no longer works, right? I've changed the management endpoints to be forward slash admin So there's that And if I go here, I can see that there's a custom health indicator there as well status i hard singapore and it's food And that works this health indicator also returns the right status code So if any of these components says down then this will say down if that says down the status code will be 500 But it's it's 200 otherwise now This isn't the only way by which I can access this information. You can also use jmx for example jmx is uh super is super convenient if you um Already have it in your environment. So we already exposed that information for you via the uh jmx endpoints here So there's my health endpoint operations Get data And there's my custom health indicator i heart singapore and it's food That that that should reinforce the idea that these actuator endpoints are They have an endpoint object and then they have different views for that data. So jmx rest etc In fact, one of my favorite views or representations of that data is the spring boot remote shell So i'm going to add this here for you now. So spring boot starter remote shell I'm going to restart and now what it's going to do Is it's going to start up an absent any specific configuration with spring security It's going to print out a user on the console That we can use now. I haven't used spring security. I haven't told it about my identity provider So it's going to give us a username and password the username is called user and the password is called uh this right here And once I've got that I can ssh into it so ssh minus p 2000 user at 127.0.0.1 paste Paste the password enter good ascii artwork. That's important Very well done And now I can say show me some help And I can say oh system prop ls Or show me the metrics and my my hands aren't on the keyboard But you can see these numbers are animating they're changing without me touching them or you know manipulating them I can use the endpoints. I can show endpoint list endpoint invoke health endpoint, right? And if I do that I can see my heart singapore and its food right My favorite spring boot remote shell Thing is the dashboard The dashboard is basically java top It shows you the running processes in the application As well as the metadata, you know the memory pools in the application itself This is the meta space for for java 8, but if you're using java 6 or 7 which you're not supposed to do ever Then it'll show you perm gen right Okay, so hopefully you see that the actuator is really really powerful There's a lot of extensive things you can do there and you you should see it by now that I've been able to change little things here and there and get useful results I've been able to override the default so that having to tear down the whole object graph This is what spring does best spring at the end of the day is a framework The ifo in the ifo sense of the word it is open for extension, but closed for modification Although since it's open source, it's not actually closed for modification It's just that you don't need to recompile spring to observe different behavior in spring You can plug in components at well-known places and see that in action this mechanism in spring boot Is powered by something called by something called auto configuration When spring starts up it looks on the class path for a text file in various meta inf jars And it looks in the class path for a text file called spring that factories For example, here's the spring that factories for the auto configure jar in my application And you can see that it says or spring framework boot auto configure enable auto configuration equals and then it has a Long long list of java configuration classes these java configuration classes would seem to do just about everything There's aop rebed mq batch processing cache support Cassandra support couch-based, you know liquid-based elastic search mango db solar neo4j redis Hazel cast integration jdbc jms jmx jta jndi database migrations with flyway and liquid-based joq java mail mobile mustache security different types of templates Social network integrations etc etc etc everything And at first blush that might be a little bit terrifying It certainly was for me when I saw that because I know that spring is going to try and run every single one of these classes when the application starts up And I certainly hope I don't have all of these libraries on the class path Right that wouldn't be very good. So we can take a look at one of the examples to kind of understand what's happening This is the rabbit mq auto configuration It's just a configuration class. Remember we talked about configuration classes before it just defines beans But these beans are conditional. They're based on certain conditions So here we're saying I want to create this bean this configuration class But only so long as the type rabbit template dot class or channel dot class are on the class path If they're not there, then don't bother evaluating all of this configuration Short circuit the evaluation there Similarly, I've got a bean here. I've got a configuration class Which defines a bean of type connection factory a rabbit mq client connection factory But it does so only if the bean isn't already defined if the bean of that type isn't already defined We don't want to define two of them If you if you feel like you can do a better job and you've got some particular Thing that you want done in your connection factory, then we'll defer to you. You know best Right, but if you want spring boot to do it, we'll do our best to do the that give you the default version, right? And we can do a lot of things with just properties. You see earlier. I showed you that when I use Uh application dot properties. I can change everything. I can change my port. I can change rabbit mq, uh, you know This is the rabbit mq addresses Well, rabbit mq addresses if I click on this property You can see that it actually goes it corresponds to this property on rabbit properties Which is the object that I was just looking at in my rabbit auto-configure rabbit auto-configuration. That's this right here Spring boot when it starts up it looks at your properties and it maps them to fields on these pojos These configuration property style objects here It's saying anything that starts with spring that rabbit mq dot host will be mapped to this field spring that rabbit mq dot port spring that rabbit mq dot username spring that rabbit mq dot password etc The effect is that you now have the ability to build uh applications and get useful defaults that you can easily override You've seen this several times today. I added spring boot start a web and I have an embedded web server and spring mvc And the servlet infrastructure. I had to do nothing. There's not even a web.xml in there I added spring boot started that at jpa and I got transaction support I got a local container entity manager factory being uh for my entity manager from jpa. I've got the transaction support I've got all the sequel and you know all that stuff set up for me When I when I used actuator, I got observable operational endpoints in the application I did nothing. I just added libraries to the class path and that activated these tests And you can do the same thing If in your organization you have things that need to be the same from one service to another Maybe you've got security or maybe you're doing something with a framework that we don't already have an auto configuration for Describe it once get it right just once Put it in an auto configuration and remember you can auto configure Servlets and filters too just as regular beans you just say at bean and then define javax servlet filter or javax servlet servlet right Um, you define that bean just once and spring will automatically install it for you when it's on the class path If those tests are true and you can say okay, I'll do I'll provide the default bean But maybe the the user of my framework has some particular thing that they want to override and they can either use your custom properties Your custom configuration properties or they can just define the bean wholesale and you can have a test for that bean's presence This makes it very easy to build systems on this dynamic sort of uh adaptive approach to configuration Indeed you can build your own spring initializer. Remember the spring initializer my second favorite place on the internet Start that spring that i o that source code is an open source The application itself is open source so you can fork it on spring hyphen i o initializer Now you can have a start that your organization that i o add custom check boxes Maybe you're doing siteminder or something like that and you want to configure that to work a certain way Describe that auto configuration once package it up and make it a standard for all new microservices that when you go to start That spring that i start that your organization that i o that that checkbox is automatically added to the build Now you have the ability to get past those non functional requirements Now you can remove the cost of getting past those 500 easy steps to production Right now you can focus on writing the business differentiating functionality that you set out to write in the first place now We have an application and this application is Ready for production. It's production worthy and and one question we have now of course is What does it mean to make this a production application? Well, first of all, you should know that this is a a so-called fat jar So downloads reservation service maven minus d skip tests clean install Crack the nick I now want to go to the target directory d minus hs Jar there's my 39 megabyte jar That jar is a so-called fat jar or you know american jar That jar has everything that we need to be able to run this application It's self-contained. I can add this jar as an attachment to an email I can send that email to my dear sainted grandmother And my grandmother is really really smart, but she's just not really conversant with computers. However, she can still run this She's got applets on a machine So she can get to production very fast if you have a Web sphere addled operations team that insists on using that tell them to call my grandmother She'll help them get to production faster So java minus jar reservation service dot jar There is the the application really good quality animated or sorry ascii artwork and the applications up and running Getting the application up and running isn't the hard part, of course Right. We have other concerns related to for example, how do I do load balancing? How do I provision an environment? How do I handle dns and all that and for this a cloud computing technology? Like cloud foundry isn't is a invaluable So i'm going to do cf login it's going to log into my part the particular cloud foundry's Installation that i'm using but you can log into your own of course. I'm going to log in here I'm not going to tell you my password even though i'm all about live demos and once i've authenticated It's going to ask me which organization now you'll notice that I have two organizations This is the rnd group in which I sit. That's the group You know that's the organization in which I sit and this is my private organization the one that I josh pay for Yes, yes, if you're wondering pivotal doesn't give me a free pivotal cloud foundry account Makes me sad Anyway, that's okay. It's still worth that. I pay for it So I have my own account that I pay for in a separate org So i'm doing a demo and I may forget to turn it off So i'm going to put it in the platform engineering organization and let them pay for it Okay, so just like with github you can be part of multiple organizations Same here for with cloud foundry you can target different orgs in the same account same installation and you can have different installations That done I can interrogate my installed apps I can say cf apps and I can see that I've got two applications right now And what I want to do is I'm going to push this application that I just created Up to the cloud. So I'll say sg reservations Uh, that's probably already been done just in case well, we'll try So I'm going to take that jar. I'm going to push it into the cloud environment And it's going to say I'm just going to take that incoming jar and it's going to try and detect what kind of artifact it is What kind of application it is it's going to Detect this using something called a build pack a build pack is a set of well known scripts It's a directory full of scripts that are in well known places that do certain things in different You know phases of the initialization it lays down the file system That is needed to be able to run this application in the case of java And the java build pack it's going to lay down a java virtual machine And it's going to set the ram and so on so that you can run the application It's going to do all this for you So that you don't have to worry about that stuff You can override the default build pack, right? I'm reusing There's a set of well known default build packs that are provided by cloud foundry But as it is just a git repository containing scripts You can provide any build pack you want you can fork our open source build packs and tweak it if you want as well It doesn't matter and you can specify that I want to always use this version of the build pack So now if you have the version of your source code in the version of the build pack As long as those two are locked up you can guarantee the same runtime environment five years from now Right or ten years or whatever. It doesn't matter the point is you can you can get a reproducible build Without having to do the the heavy lifting of Provisioning configuration so once that's done once it's created that file system It's going to turn that into a container and then deploy that container across the cluster Right now. I've only got one instance and we may indeed want to scale it up and and we can so the application is now started I've got the the command line that's been run to Confirm it's working and there's the url. So sgreservations.cfapps.io. Our application is live Okay, I'm in production There's that now I've only got one instance and that's not going to be great if I have a You know denial of service or something like that. I can see that I've got one instance by going here to instance index So vcap application instance index And I can see cf scale sgreservations I can see that when I do that There's one instance with one gig and one gig of space. I'm going to scale that up I'm going to say cf scale minus i to sgreservations and We'll up the instance count. Oops, or I'll or it'll fail because I didn't put a space So now I've scaled the application up. I scaled the application up And it's behind a load balancer. So now if I refresh this giving you know, assuming I have a 20 seconds or whatever it was to get the application started We'll see it to slowly start to to load bounce from the first application the first container To the second one Come on There we go one zero one Etc. I'm hoping you can all see that. Okay, so now we have an application. It's in production. It's production worthy We have a dressed observability We've looked at how to expose information about the application and make short work of Of getting past the configurations that we can deliver our application This is a quick look at both spring boot and spring cloud. I hope you saw something that you like in all this This is old stuff Everything we've just talked about You know is at least 2013 some of it much older Right, so this is not even the reason we're here like I'm I'm happy to talk about this, but this is like review What I really want to talk about is what we're going to talk about the next section Which is spring cloud spring cloud is the thing that allows us to build a distributed system And to do so effectively so that we can actually get the value out of this approach Now I hope you enjoyed some of what we talked about here. I'm uh, uh, I should stress that this is Very very ubiquitous spring boot for example is used by there's 4.5 million unique downloads every month For comparison the entire node.js community npm every month the entire thing 10 million So the fact that one technology inside of the jvm community has almost as much as all of node.js Including all the front-end developers all around the world. It's not bad. It's not bad at all. We're doing good. Um spring boot itself Is far and away the number one most widely used technology for building microservices. There's a great developer Uh, productivity report by zero churn around that was just published that showed that both spring boot and spring mvc are both Like 30 or 40 percent or whatever of the uh, the the the usage for microservices and the distant and the number two The distant number two wasn't even close, right? Um, or distant number three actually So spring boot or spring mvc spring boot and then distant number three Uh I think that'll do it for now. Now's a good time for lunch. I'm happy to answer questions. Uh, if you have them And uh, otherwise I look forward to seeing you the after the break where we can actually cover the interesting stuff Thank you so much. See you guys soon We have how long is the break by the way? Does anybody know how long the break is supposed to be? Okay, everybody. We're back so Before we left for uh, before we left for the break We had built a single service a single application that was running on a single, uh host important and and uh We took care of operational concerns things like observability We we looked at how spring boots approach to convention over configuration And ease of configuration makes it easy to override the default behavior if you need to in a pinch This uh, this mechanism Uh, I demonstrated to you in terms of the application that properties property file inside of the source main resources directory but uh One one thing I want to underscore is that you can provide configuration in other ways For example, you could have an application of properties Or rather an application dot yaml a you know a dot yml file As well and that'll work as well. That's a hierarchical way of representing Properties, you know dotted dotted trees syntax, right? that's certainly one approach but That's not going to be very useful if we have to change the application Or we have to promote the application from one environment to another if we're taking the application from development to to q&a To staging etc. I don't want to have to recompile my code To affect changes I want to see those changes those property configurations Without having to recompile the build and as we have it right now I have no way to do that so We can look to a design pattern called called a 12 factor style configuration The 12 factor style configuration originates from the 12 factor manifesto the 12 factor manifesto was written by haroku in 2008 It's a set of good clean cloud hygiene things that you need to do to build applications that Do the right thing in the cloud They live and breathe natively in the cloud The 12 factor manifesto describes 12 principles One of which is that configuration that is unique to an environment should be kept external from the application In the environment for that application And we can see that in action here. So I've got a a jar, right Suppose I wanted to change the port Right. I can change almost anything In spring boot. I can change almost anything even things that you wouldn't otherwise expect to be to be something you could change Uh In the code for the application itself. I can change things like the port the ssl I can change change the the gzip connectors I can do all that Instead of spring boot itself because spring boot is running my embedded web server not the other way around Spring is the container and it's running tomcat or embedded jetty or embedded undertow Which is the server that container that powers a wildfly It's running all of those as a library as a as a as an api with which it's interacting to respond to htp requests on a certain port So you can actually do some pretty amazing things that way you can actually Imagine doing aspect oriented programming on tomcat, right? Because tomcat isn't running your application your application is running tomcat. So the the inversion Leads to some very very powerful options including for example changing the port I could do that here But I want to do it when I run the application because that's something that's going to change from one environment to another Or even from one desk to another. So here I can say java minus d server dot port equals 8020 minus jar Right, so now now by overriding the default configuration by overriding the default value I uh, I get the result that I want without having to recompile the bill. There it is right there 8020 I can also use environment variables. I can say I can say export server underscore port equals 8030 java minus jar reservation service dot jar and uh here I can see 8030 Right, so my application is now uh on a different port and I've My configuration is converged. It's been fuzzily transformed into server dot port It's been lower cased the underscore in between the uh the tokens and the environment variable has been Canonicalized and then We can see the result there that that works So this gives us some flexibility I can now move my application from one environment to another without having to do too much work But this falls for falls short of four critical or key areas The first is What if I have configuration if I have more than one service or more than one instance of one service I would still have to tously duplicate and copy and paste my configuration from one instance to another That would be error prone and and slow Another another use case that this doesn't quite address is how do I how do I handle auditing and journaling? How do I See who changed the configuration and if necessary to roll that configuration back for that I don't quite have an answer here. What about um What about sensitive information passwords credentials locators things like that that shouldn't be stored under any circumstances In plain text on the file system at rest How do I how do I address that use case? And then finally, how do I change the configuration whilst the service is running whilst the process is up Without having to restart it While what I've shown you is a good Start it's it's not nearly enough for those use cases and I'm sure you can imagine a few ways By which we might address some of those challenges some of those questions We could for example, how a directory full of configuration a directory with with configuration property files We can share the directory That might solve the centrality issue you could have everything one place I could even make that directory based on git or subversion And then by virtue of the fact that it's using those technologies I would I would have auditing and journaling I'd have a log of who changed what and when And I'd be able to see it But that doesn't address the security concern Right, and it doesn't address the fact that we need to have a way of Configuring or reconfiguring our spring applications or clients or configuration Applications or the things that use that configuration without restarting So for all of this we need a little bit something a little bit more sophisticated for this we need something a a bit more powerful And so we're going to use the spring cloud config server The config server is just that it's a configuration service that manages different configuration So we're going to say a config server Config server and I'm going to hit generate And this config server is going to babysit a directory full of configuration A directory full of configuration that we're going to store On github or git, right? I've got a git repository here that you can clone if you want for your own reference and edification later on Under josh long beautiful micro services config with dashes in between the words I'm going to clone this into my local machine So there it is Git clone That to the desktop config directory Wi-Fi permitting of course Okay, and in my code here I'm going to configure The application I'm going to say server dot port. This is the config server that we've just stood up We're going to configure to run import 888 And we're going to tell it where to find the directory full of configuration I'm going to say spring cloud config server dot git dot ui Equals and we're going to point it to the home directory For such can desktop for such config And then we're going to turn on the the config server. We're going to activate the config server like this And now This is going to act as an intermediary. It's going to act as a a broker for our configuration clients are going to connect to the config service to draw their configuration But the configuration will be managed by that directory or by the config server, which will then talk to the directory So now we solve the centrality and we solve the auditing and journaling requirement And because we've interposed this extra server this little bit of indirection between the configuration clients Which rely upon that configuration and the downstream services that are the downstream directory that provides a configuration We now have an enviable place to Address things like security Right, I can require authentication to talk to the service to get the configuration And I can have the config server, for example, symmetrically decrypt Encrypted properties in the property system in the property files on the property in the file system So if I have an encrypted value in the property file, I can tell the config server Here's the cipher. Here's the key when the client and the server authenticate I want you to decrypt that value But only in the client right not in transit so This will start up in port 8888 And if I have a microservice that connects to the config server on port 8888 That goes by the name reservation hyphen service. It will see the configuration here And uh This configuration, uh, this configuration shows that we have two property sources a property source is the properties that Our spring client will will see it'll be able to use to be able to configure itself We'll see a properties. We'll see a set of properties from the reservation hyphen service dot properties. These are uh keys and values from that property file That we can use to to um to configure that one service and we have a separate separate second Separate or second set of properties called application of properties These are properties that all microservices no matter what their name will see Think of this as a fallback set of properties So all microservices will see these values, but only the microservice identifying itself as the reservation hyphen service will see these values These values get squashed together Merged together to form one basic set of keys and values And if there is a conflict as there is between server dot port And server dot port then the more specific property file Reservation hyphen service dot properties overrides the default fall through property file So you get a cascade sort of you get global properties and you get service specific properties So we've got two properties here. We've got server dot port Which says I want to I want to be equal to the value of the environment variable called port Or the literal called 8 000 the the string literal And we've got a message called that says hello world. So let's go ahead and connect our reservation service to this newly stood up config service We're going to act as a client to the config service and have our reservation service Talk to that config service to draw its configuration The first thing we're going to need is the client the config service client And we'll get that in by bringing in spring boot. I'm sorry spring cloud starter config spring cloud starter config Needs two things to do its work It needs to know the name of the application, which in this case is the reservation hyphen service naturally And it needs to know where to find the config server. So we say spring dot cloud Dot config dot u r i equals htp localhost 8888 Now this information isn't necessarily I think uh and understandably Used earlier on in the initialization of the application think about it The application starts up and then looks for the config service where it then finds a set of property sources keys and values And then it then applies those keys and values to the application It does that at the same time as it would the properties in application at properties So necessarily these two properties have to be read and and loaded before the rest of the configuration In the bootstrap of the application because you know, we we cannot resolve the server at the same time as we're applying the configuration from the server So by convention spring cloud expects this information to be in a property file called bootstrap the properties And indeed for most microservices, these are the only two things you're going to specify Everything else will live in the config service right So now I've got just these two keys and values and these are the only things I need to really specify per app And in fact this as I say could have been an environment variable You could have said export spring underscore cloud underscore config underscore uri equals Right and that could also be mapped to dns. It could be a load balanced dns application on on cloud finding Uh and in fact, that's the default So I just show you I just did that so you would see what's happening But one is most of the time, you know, uh, you'll either specify it some other way, okay Okay, so now having done that we should see the application spin up on port 8 000 if we started it right now I want to take advantage of that message that hello world message So I'm going to create a a rest controller here called message rest controller or controller Okay, and I'm going to inject the uh the value. I'll say private final string value private constructor And uh in the constructor, I'm going to say at on a wired And I'll say at value dollar sign dollar sign or sorry dollar sign curly bracket curly bracket message So here I'm using spring to inject that value from the config service Just like I would any value from the uh the property files or from any other part of the spring environment And all I'm going to do is I'm going to return When asked I'm going to return the values For consumption I'm going to parrot the value out now I may Want to change this value later on in the future So I don't want to restart and in order to support that I'm going to make this bean refresh sculpt So I'll restart the bean like so Okay localhost 8 000 reservations There we go seems that's working right we've got the config server Uh correctly configuring the applications uh port. What about the message? And that seems to have worked as well. Good. So we're making progress now This message however useful isn't exactly correct. Is it we can go to the config directory here desktop config And we can see We've got a property here adam reservation service dot properties Say hello world and instead of hello world. That's uh, let's be a little bit more specific Let's say hello singapore. Okay extra exclamation marks so as to reinforce my Authenticity credentials and authority on reddit And uh, then we'll say get commit minus a minus m yolo. Okay now having done that I can visit the config server Uh, which I seems to have closed I can visit the config server and I can see that the value is immediately visible and reflected in the config server But our downstream microservice has no idea. What's just happened? It doesn't know that there's a new value. This is by design The last thing we want is for the client to constantly We pull the the downstream config service for updated configuration Instead we need to tell the the config client to redraw or refresh its configuration by reconnecting to the config server And recreating that one bean in situ now We've seen that I've already annotated that bean to be refreshed scope So that gives us the mechanism we need to do that we can trigger the refresh action One of two ways we can Connect all of our microservices to an event bus powered by something called spring cloud stream Which we'll look at in just a bit And also connected microservices would automatically refresh their configuration when a message arrives telling them to The second option As we could for each individual node explicitly trigger an actuator refresh endpoint. So curl minus d curly bracket curly bracket htp localhost 8 000 forward slash refresh now. This is an empty htp post Okay, so let's line things up. What's going to happen is i'm going to hit enter And then hit command tab and then hit command r as soon as and as fast as my little finger is going to let me what? Oh, thank you Go team. See that's what I like Some people Some people just want to see me suffer on stage so they see the errors and they say nothing It's not fair mob programming is awesome Okay, ready Go Okay, so as soon as I hit enter I fat fingered it. I fixed it of course corrected hit command tab again Went to the browser hit command r and there we go. I was able to observe the updated value immediately I didn't have to restart the process. That's because spring has recreated this one rest controller it discarded the internal representation of that rest controller and We created it anew and we can then see that the injected value gets refreshed automatically from the config service This supports feature flags. I can now Change things while the service is running live without having to restart the service This gives me the ability to decouple the release of software from the deployment of that software I can have functionality that is latent or inert in the production software that isn't available. It's not active, right? This gives me the ability to do a b testing. I can now loose Some functionality on a subset of the population and have them Use it in feedback and then I can use that feedback to decide whether to promote the functionality to the larger population or not I've got a lot of potential here a lot of really interesting things we can do with feature flags now naturally There are other things we could talk about here including the fact that You should and could secure Every end of the communication chain, right? I can make sure that the client does a mutual authentication x519 certificate based mutual authentication with the config server or that the client does htb basic authentication With the config server, uh, and I should I should I would be remiss if I didn't also say that you should use SSL to make sure that all communication from the config service and the config client are secure But for now it suffices to leave it as is and instead to to move on Let's talk about How services discover each other in a distributed system in a cloud environment specifically a cloud environment Services come and go as they need to as demand and capacity dictate. They're dynamic. They're ephemeral. They're fleeting Things are going to change the location of these different services are going to change That's the nature of a dynamic cloud environment But we cannot be too sensitive to those topology changes our our code needs to be decoupled from that My client should be able to discover another service without worrying about having to manually restart the service and teach it about a new IP At first blush this might seem like a use case for dns But dns as it turns out is actually a really poor fit for dynamic cloud environments for several reasons There's a few obvious reasons some of which are you know, they can't be helped But they're worth underscoring a worth restating the first of which is that dns requires resolution It's not a lot of cost, but it is still latency right. It's still something you have to pay You can get rid of that resolution that costs that latency by caching the result the resolution The resolved rather dns the the ip's and ports If you cash that Then you run the risk of having stale entries in your in your in your view of the world If that service is no longer there and you and your client has cashed that dns entry You're going to call a service that doesn't exist right Which is not a good idea in a dynamic cloud environment So you either have to constantly incur the resolution cost or you have to uh Be prepared to have stale entries, which is true for almost anything naturally, but just keep that in mind That's a compromise another problem with dns is that it requires dns server naturally But dns servers are typically Managed by other people not the application developers. So it becomes another ticket. It becomes something that somebody else controls Another problem with dns isn't so much dns as load balancers and dns load balancers as well and specific You see load balancers are pretty dumb instruments. They don't really have the ability to to answer Interesting questions nor does dns for that matter. They don't have the ability to answer the question Is that service there? If I call that service, am I going to get a response? We don't know if i'm going to get a response So all I can do is make the call and then hope that there's something on the other side that'll respond to me If there's nothing there that my client is going to block Which is a non starter. That's unacceptable in a dynamic environment in a cloud environment in a high performance system If i'm stuck there waiting for a response, hopefully i've done the right thing Hopefully i've specified at every level at every place in my code an aggressive client-side timeout I've made sure that whenever a client calls another service that there's a timeout that Stops it from hanging on the on the on that socket waiting for a response that'll never come Have you all done that? Everybody and all of your code everywhere Pop quiz. What's the default timeout for the java net url connection anybody? You don't know. Yeah, so then you haven't done it, right? That that is probably used a thousand times and even the basic even the basic application in your program, right? If you don't know what that timeout looks like you're in trouble And by the way, it depends on which part of the java.net stack But some of those parts are configured for perpetual timeout by default That is to say it never times out. It just blocks forever and that kind of behavior you know that The underpinnings of the java.net Package in the jdk from java 1.0 more than 20 years ago A lot of the behavior there makes perfect sense in the context of 20 years ago Where services were very static and they didn't change a lot and Client applications were also fairly static as well But those underpinnings They disservice now One such example is that java clients by default cache resolved dns entries Right, so if I have a java client that calls a host and a you know a host And I use dns to cache that to resolve that ip The jvm by default keeps that ip So if I use it if I'm using a load balancer And the load balancer gives me a back in ip a dns load balancer and it gives me back an ip The jvm is going to keep the resolved ip and then use that for all subsequent connections It's going to defeat the load balancer The whole point of the load balancer is to move the work across different nodes so as to To load balance But the jvm is going to pin it to a specific node Right, that's a useful behavior for 20 years ago But it's a bit of a problem now in a cloud environment where these things will change Load balancers in general Are pretty dumb instruments as I say they don't really know about the nature of their workloads They don't understand that not all requests are created equal as far as they're concerned There's 10 requests coming to the door They don't know that some of those requests are going to take two minutes and some will take two milliseconds So they don't know how to evenly distribute based on the weight of the of the response, right? Not most of them Your average load balancer also doesn't know about the nature of your business about the kinds of routing decisions You need to need to make for your business. Suppose you want to do something stateful Maybe you have a service that has something stateful and you want to pin Requests from a specific client to a specific node. Sure. Some load balancers will have a uh, you know, uh, they'll have built-in support for Well-known types of stateful, right? They'll have support for j session ids or cookies or whatever But what about oaf Let's imagine your netflix and your streaming video And that streaming video is on a certain node I've got an oaf client an oaf token coming in from an oaf client And I want to pin the request from that client to that specific node. How do I do that? There's no there's no checkbox for oaf tokens or any arbitrary token, right or x519 certificates, right? How do we say that this identity corresponds to somebody who's doing something stateful on this process? What about more interesting kinds of load bouncing like data center aware load bouncing? How do I load bounce to nodes in the same rack and then alternatively fall back to another data center if I need to? What about data sharding and locality? How do I say that this data lives on the certain node so the request to handle it should go to that node for example There's not an easy way to do this if you have a centralized load balancer So dns and dns load balancers or just load balancers in general have a fair amount of limitations Mostly due to the fact that we can't control it It doesn't it's not our application logic And it's an extra piece of infrastructure that we have to manage that this that doesn't serve as well In the cases where we need it the most so we can get around this by using something called a service registry A service registry like a dns server is another piece of infrastructure But as you'll see it's much easier for the the developers to have control over that Just in the same way that they would have control over for example the database, right? They'd have control over what how that gets deployed managed Uh a service registry is like a phone book for the cloud. It gives us the same effect as dns It's a logical mapping from service id to hosts and ports And then we can use that to make decisions about how to route the request now the uh the logic for the routing itself Lives on the client now in this world not in the load balancer Because it lives on the client you can do anything you want You just interrogate the look you insert interrogate the registry and use the results that come back to to decide upon which node to send the request Spring cloud has a discovery client abstraction The discovery client abstraction lets it easily talk to all manner of different uh service registries including apache zookeeper Hashicarp console at cd. Although that one is not ga or production worthy yet Cloud foundry itself can act as a service registry because by definition Given a service id cloud foundry knows the hosts and ports on which that service is available It put them there, right? Uh my favorite service registry is netflix eureka Netflix eureka has two big benefits The first of course is that it's been used at scale by one of the largest websites on the planet for many many many years How many how many of you have netflix? Okay The rest of you don't have it yet But I say yet because you will like um netflix is a uh Pretty popular they stream something like 200 percent more video than youtube every day And i'm sure most of us thought that youtube is the largest video portal on the on the internet Right 200 percent 2x youtube That's insane So we know that it works. It's been used by them for many years at scale So uh i'm gonna that's the first benefit the second benefit the second reason i love netflix eureka Is because it's really really really really really easy to set up and i am i am nothing nothing If not lazy So i'm going to go to start.spring.io and i'm going to configure myself a eureka service eureka server config client generate Okay, open this up. I'll take some tea Oh, that's good Okay spring dot cloud dot config dot uri equals htp local host 8888 I'm going to say spring dot application name equals eureka hyphen service and then we're going to do the following we're going to say at navel eureka server Now i'm going to rename this property file It's tricky you have to remember to rename that rename that property file The worst part is it would still work if you don't because the default is this url this value. Anyway, uh I've got my eureka server. I've said abracadabra. You're eureka And i'm going to start it up and we should see it's been up on port 87 61 now What we need to do now is to make it to that other microservice our reservation service principally We'll raise its hand. We're going to have the microservice raise its hand and say listen if anybody needs me I'm here. This is my host. This is my port find me here. So right now it doesn't do it Right. So here's our registry a few things worth pointing out about our registry. First of all very well done animated give Okay And then also No applications registered yet nothing here This is the one drawback to using service registration is in discovery. It is invasive Your code has to be made aware of it for it to work Whereas the the handling for dns is typically handled either by the jvm fundamental libraries or by the platform itself Now granted If they worked then we wouldn't have wouldn't be having this discussion in the first place So it's not necessarily bad that we're doing something that was otherwise being done by the the platform But it's still something that you have to care about right? Thankfully with spring cloud Because of the abstraction it is a minimal You know intrusion or a little bit minimal minimal, uh, invasion So i'm going to say spring cloud starter eureka. That's going to bring in the discovery client abstraction implementation for netflix eureka And i'll say at navel discovery client And then having done that hard work i'm going to go ahead and restart the service Oh ice tea You know what i think would be a good business in this beautiful country More uh vending machines with tea ice tea Or just ice Okay Good so having restarted We can see that the service is now registered in the registry There it is reservation hyphen service and we consider that we can see that it's available on this ip this service id and this port It's now available for discovery. I've only got one instance of the service But i may as well have 10 or 100 a thousand in which case it would say up parentheses 1000 right instead of one Good so now we're ready to create a client something that we can use to talk to our service and to to interoperate that Interoperate with it by way of the registry and that's what we're going to do We're going to create our first client the reservation clients and to build the reservation client I'm going to use spring's web support. I'll use the config client. I'll use eureka for service version discovery I'll use drabdomq for zip for stream processing and zipkin for distributed tracing I'll use his tricks for circuit breaker zool for micro proxy and the rest repository support And because we have the time we're going to use oauth for security Cuz yolo Okay, now that'll do For now That'll be okay We're going to go ahead and hit generate And uh, we're going to open up the uh the client now this we'd have to do the normal pro forma kind of stuff First of all, we have to say uh at enable discovery client and we have to open up the property file We say spring dot cloud dot config dot url equals htp local host 8888 spring dot application dot name equals reservation hyphen client And uh, then we rename the property file to be bootstrap dot properties as we've done Countless times before so bootstrap dot properties Now the question is what do we want to do here? This is no mere service This is no regular service. What we're trying to do is to to call the downstream data and then return it back to a client This is called an edge service And when I say client I'm referring to iphone's androids html 5 experiences roku's playstations xbox's your smart tv's In the internet of things darn near everything has an ip address these days everything Your streets here in beautiful singapore have sensors What do you think that sensor data goes? It goes back to a computer. It has an ip address your streets have said ip's right? There are human beings walking around on this planet with organs That have ip's on them Seriously, so you cannot afford to ignore the myriad possibilities for different clients even the most conservative and boring of organizations today Has to support at the very least android ios and html Experiences right for your for your services for the for the experiences by used by your customers You have to and these things are all going to talk hopefully to some of the same set of services So you can't afford to make them siloed to have duplicated investment there so instead of Instead of connecting each one of these different clients to each one of our microservices Insert you we're going to insert an intermediary an intermediary called an edge service And the reason we're going to do this is because the clients have often different payload protocol and security restrictions Sometimes the the data that they need is different as well So the the pattern here is to build an edge service that is specific to the client We're going to say this is the html 5 edge service. This is the ios edge service. This is the android edge service, etc Right we can handle client specific concerns in that edge service so Let's let's talk about html 5 html 5 browsers today are really really really powerful. How many of you have ever seen js linux? This is My favorite example of the possibility of the web So what I just did was I went to bedal.org forward slash js linux as in javascript linux and uh, it is a x86 machine code emulation Or emulator written entirely in javascript When I loaded that page it downloaded a linux kernel iso And then it booted the iso Entirely in javascript. There's no server-side state at all to demonstrate lsla vi hello.c Let me open this up. I'm going to hit escape I I'm going to say Hello Singapore okay extra exclamation marks Remember I'm I'm using my browser to command up and down Because it's just a div element the ttys device is a dom element in my browser Okay, so now I'm going to say escape wq I'm going to say tcc minus o Hello hello.c. That's the tiny c compiler I'm going to there we go So I compiled c code in a linux distribution running inside of a div element In the browser I booted linux In an html browser an htp browser So when people say that you need to build a native desktop application and you need to use windows forums or gtk Tell them that the browser boots linux now and that's no longer necessary right The point is you can do really really amazing things on the browser html 5 experiences today are insanely powerful You can do 3d rendering. You can do all sorts of cool stuff right that said Browsers htp browsers live in a secure sandbox necessarily They cannot make requests to different hosts and ports outside that sandbox You can get around that by adding a policy to every single microservice a policy that exempts cross origin requests Right that would be one idea But that would require retrofitting every single microservice just to accommodate one new client If you had to do this at scale if you had more than a handful of services You'd be spending a lot of time having other teams to redeploy their services Just to accommodate your client That would defeat the autonomy that you're trying to obtain by moving to this approach in the first place So an alternative and much more scalable approach is to instead just proxy the data back and forth From the outside clients to the downstream array of services hidden behind this Edge service and that's what we're going to do. We're going to use something called a micro proxy Now we have on our class path here in the reservation client the netflix zool micro proxy. That's this bit right here Now i'm going to go ahead and comment out a few things that I don't need just yet And we're going to see that we've got spring cloud starter zool there on the class path Zool Is a reverse proxy and you can set up arbitrary routes You can say I want to I want to have a route on my edge service that goes from local host 99 99 foo to go to google.com or google.sg if I want In this case, however, it suffices just to just uh, let spring cloud do its work because spring cloud is already aware Of my service registry. It knows about the registered services because of the discovery client abstraction And so as a result, it's been nice enough to set up convenient, uh routes for us on the edge service that map to the, uh service IDs in the registry so Reservation hyphen service forward slash reservations. There's my edge service right edge service actual service Edge actual edge actual Edge actual There's a couple questions you may have at this point First of all How does it know to which instance it should route the request? We've given a service ID and it has to pick right now There's only one instance in the registry so naturally it's going to use that one, but suppose I had 10 or 1000 How does it pick that instance? That routing decision is as I say done on the client It's done using something called Netflix ribbon. Netflix ribbon is a client-side load balancer It is the enterprise distributed systems microservices equivalent of any mini money mo It's just going to pick one Based on some strategy that you provide by default that strategy is to do round robin load balancing right It's going to do round robin load balancing for you But it's an object the strategy The strategy is something that you can override you can do weighted response based load balancing You can do data locality based load balancing you can write your own strategy that does something with all of tokens You can do whatever you want Now you have the right you have the ability to write a load balancing strategy and then unit test it and then version control it Control over how nodes get routed to is in your hands now And you can plug it in centrally and have that work for all the services And now you have one less thing to manage in the system So that's the first thing that we need to understand the second thing is that when this proxy makes the request to the downstream service it passes in the origin url as part of the headers And so you can see that the downstream service has adopted its urls It's rewritten the urls to say localhost 99 99 when in point of fact they're actually coming from localhost 80 80 From the perspective of the client Everything looks like it should everything looks like it was generated on that port on that service on that instance The client has no idea that that json originated in some other process some other node some other place, etc And it doesn't need to know Now we have everything for we need for example to build an interesting html 5 based application our html 5 application can talk to locals 99 99 and Talk to the to the to the different services and they're often right off to the races They've got everything they need to build an interesting experience. We should absolutely absolutely use htps Remember it's a requirement for htb 2. So you're going to have to get there anyway We should probably use s s sorry o author htb basic for some sort of authentication to protect this service But suffice it to say we've done more than we need to at this point right the html 5 developers have access to all the back end services And assuming you're using using ubiquitous htp and json you may be done Now sometimes it's required that you send different types of data back that you transform the data that you Sometimes synthesize data based on two different services Maybe you've got service a and service b and they want to compose the data and then send the results back to the client For for consumption there In this case we need to do something more than just blindly proxying the data back and forth In this case we need to do some sort of transformation or translation on the downstream services This kind of edge service is slightly different than a micro proxy This kind of edge service is called an api gateway And so we're going to do that We'll go back to our build Or rather our application And we're going to stand up another rest controller on the edge service called the reservation api gateway rest controller Okay, and we're going to map this to request mapping Uh forward slash reservations We're going to make this a rest controller And we're going to create an endpoint in our case. We're just going to create a simple endpoint that just streams the names Back to the client, right? It streams the names. So what I'm going to do is I'm going to I'm going to call the downstream service I'm going to call the downstream jason and strip away all the surrounding strata from the jason I'm going to keep only the names so josh and neos and aran and Lune and we ran and and so on right. I'm going to just keep the names nobody else nothing else Right just those So I'm going to say whenever somebody goes to get value equals names we'll uh We'll serve back a collection of strings in order to do this I'm going to use the spring framework rest template The rest template is an object that we can use to make hdb calls in a convenient way It makes short work of of common hdb exchange patterns get put post delete etc The rest template however has no idea about our service registry and it doesn't know about ribbon by default We need to teach it to do so. So we're going to configure a bean of type rest template Here and we're going to configure an interceptor on that rest template that will pre-process any request And extract out from their request you or I the host It's going to treat the host as a service id It's going to pass that service id to our registry in this case ribbon. I'm sorry, uh, eureka It's going to get all the service instances back the collection of service instances And it's going to pass that collection to ribbon Which is then going to pick from among them And then use that and then we're going to use that final resolved host import to make the request Okay, so Let's see what that looks like this dot rest template dot exchange hdb colon forward slash forward slash reservation hyphen service forward slash reservations And we're going to make an hdb get call so get we're not going to send any data It's going to be a git not a not a not a Put or post for example, so we have no body in the request So we're going to send null and then for this final parameter We need to tell the rest template what kind of data we want back Remember we're going to call a downstream service a rest api that has jason I can tell the rest api to give us a string payload that would work certainly I could say give us a collection of bytes although that's a little I don't I don't think that's particularly useful in this case What I want to say is, uh, you know, I want to get I want the data back I want the jason back and I could say give me a map of of of the jason where each Attribute in the jason becomes a key in the map Or I could say give me a jackson jason node. So jackson is a the java api for marshaling jason Right and the jason node object is like a document object model like structure that you can use to traverse the jason tree I could do that or I could ask it to give me a collection of Resources whose payload is of type reservation recall that in spring hot os spring hot os resources Each one of them has a payload and a collection of links So I want to take the jason and turn it back into an envelope object I want to turn it back into a resource with whose payload is of type reservation who in turn has a collection of links So There's a few reasons this doesn't work The first and most obvious is that we don't have the type reservation on the class path Right I can add that here. I'm going to add a representation of that type on my client I'll say reservation name and I'll get a getter and there we go And it still doesn't work right The end to understand this we need to understand that Java has trouble with instance variables and generic parameters If I look at this list of x new list of a ray list What is that generic parameter? What is t here? Sure at compile time now. What about at runtime? Yep, it's it's basically the same as it's almost the same as object It actually there's some there's some nuance, but it's as far as java is concerned What you wrote was list x equals new realist. There's no parameter there at all. It's untyped At runtime java doesn't see the instance type That's because of a compromise that the java language designer designers had to make in 2005 more than 10 years ago when they grafted on Generic parameters onto the java language. This is a process called type erasure It means that at at runtime using reflection you have no ability to capture that generic information You can cheat a little bit. You can say class x extends array list of string And then you can say list of string x equals new x and at runtime t will actually equal string as you expect And if you understand what we're doing here that we're creating a subclass Then you know that we can also use an anonymous subclass, right? Which is a java 1.1 feature So here i'm just going to create a subclass in situ. I'm overwriting the definition of that object In the same place as i'm instantiating it, right? Not a big deal But it works now. I now I have list of string t If you understand this Then you know what we need to do we need to use sub typing to capture generic parameters This is called the type token hack. Uh, sorry design pattern It's called the type token design pattern And the way it works is fairly similar. You'll see different implementations of You know all across the spring and java ecosystem There's you know also all across the java ecosystem You'll see different implementations spring has one because it's not a novel idea, for example It's just something you have to do so we're going to say parameterized type reference equals new parameterized type reference And the parameterized type reference, uh The parameterized type reference Gives us back a java lang reflect type so get java lang reflect type And if I go to the type I can see that it says that that is a parent of java lang class Right, so the java lang class is what I want ultimately So I can use that now I can pass that last object there into the rest template and the rest template understands that Okay, you want me to take the json And turn it into whatever type is embodied by that parameterized type reference And then we can use that response entity. We can say, okay I've got the body Do I you know, I've got the status code. Do I want that? No. Do I want the value? No Do I want the headers for the htb request to know what I really want is the body The body has the collection of resources whose content is a collection of reservations And I'm going to stream over each one of those reservations and I'm going to transform them or map them Using a lambda. I'm going to say for each lambda for each reservation reservation I want to keep on a return only the reservation name and then I'm going to store all the reservations in a list So collectors dot to list et cetera And I can replace that as so Okay So now I've got a three liner I'm calling the downstream service. I'm doing some basic transformation. We should be able to restart it I'll take some t. It'll be great Who ah That's not t It's lemonade Okay local host 99 99 reservation name. So there we go My edge service Is calling the registry. It's getting the service ID. It's calling the downstream service. I'm getting the data back I'm transforming it from a collection of reservations to a collection of string names All in the blink of an eye I'm I'm ignoring For the moment for the instance I'm ignoring for the for the instant the fact that right now I've only got what eight records 12 records. I don't know not not many right? I'm ignoring that. There's very little there's very little data here I'm also Ignoring that I'm just calling one service if I were to call more than one service Then I would want to do this concurrently I'd want to call two different services at the same time and then join the results back up If I were calling a service with more than say, you know a thousand records I'd probably want to stream that data back Right, I wouldn't want to I wouldn't want to do a select all so to speak And so for both streaming use cases and for easy Declarative kind of concurrency I invest I recommend that you take a look at and investigate the realm of reactive programming For which pivotal's reactor project is an ideal candidate There's a lot to be said on that and I'm sure we could in a whole Whole other talk on that but for now we've only got one service. It's got a very small amount of data It's fine. Okay Now we're making we're taking some liberties here. We're making some assumptions We're saying we don't care You know, we don't we're happy to content ourselves with the fact that this is going to load bounce across the available Instances of the service if there are available instances of the service If there are one or more registered instances of that service, this is going to work just fine But what happens if there are zero instances of that service? What's going to happen? And here we uh, here we run into A A bit of a roadblock right we run into a bit of a problem. You see if there are zero instances of that service registered It's going to throw an exception It's going to try and divide by zero basically it's going to throw an exception And we're going to have a big fat job as stack trace in our iphone clients, which isn't good. Can we what? You're going to turn the air on what yeah, it's really are you guys warm? Me too Not not as cold as before but not as hot as right now Just right the gold deluxe principle Okay, so we we're confronted with a problem if if we try and load bounce across zero instances We're going to get a big fat java stack trace and that's unacceptable We have to understand that in a sufficiently distributed system failure is a statistical Inevitability it's going to happen. It's not a matter of if it's a matter of when we need to optimize Four times remediation. You see we cannot build a system on the predicate on the lie that services will always be available As you add more capacity and more instances that That uptime will diminish For a few nodes. It's already pretty significant. You're not going to have nine nines. That's for sure Or five nines even For tens of nodes or hundreds of nodes this can be very quickly hours per month of downtime Hi Didn't we already do oh, yeah, sure Oh, yeah This is I thought we did. Okay Ish Okay, so we need to understand that uh failure will happen It's a guarantee right failure in a sufficiently distributed system is a guarantee It's going to happen eventually and we need to understand that we need to address that in our code High-performing websites and organizations understand this fact intrinsically natively. They live and breathe this reality There are people at google that wake up every day And they put on their google socks and their google underwear and their google t-shirts And they jump jump on the google bus heading to the google campuses Whilst talking to their google colleagues and checking their google mail and their google phones And when they get to the google campuses, they no doubt proceed directly to the google cafeteria or they have their google gourmet breakfast And whilst chugging voluminous amounts of google gourmet kool aid They may accidentally have a google accident and spill some of the google kool aid on their google shirts This necessitates a trip to the google laundromat Where they will do their google laundry at the google laundromat while strolling about in a google gold sequined google bathrobe When they're done with that they may decide that they want to get back on that horse There are no quitters after all so they go back to the google gourmet cafeteria and have more google breakfast And then having done that they may then decide to get on with the business of the day It is after all getting on past three o'clock in the afternoon And for some of them they may get on a google bicycle or a google shuttle or a google bus and go to the google data centers Where they will walk the google data center aisles And find completely and utterly unsurprisingly to them that some of their google servers have google died It's not surprising to them because they've got more than two million enterprise grade google servers deployed in google data centers all around the google world And they don't care They don't care that some of their google servers google died Because they didn't build their systems to be sensitive to the loss of a few nodes That's why they're google And we need to do the same thing We need to build our systems in the same style we need to understand that failure will happen They didn't care Even when they got the alert at eight in the morning telling them that the services had died They had far more pressing matters like the google kool-aid and their google t-shirt Their systems aren't in fire nothing's wrong everything's going to be fine They built their systems to be robust and resilient to that failure netflix also understands this this fact Netflix has a suite of software components called the simian army The simian army are basically little agents of agents of chaos Little terrorists that run around in production causing all sorts of havoc and craziness They purposely kill minus nine processes They are m rf database and disk partitions. They they block ports They even have one called chaos kong Which purposely kills a whole data center availability zone In production They do this to themselves during office hours when people are on hand and available to respond to change to respond to the crisis So that they can be sure that if there's an actual crisis when people are four in the morning People went at four in the morning when people are asleep Then everything's going to work as expected They'd much rather find out During two in the afternoon at two in the afternoon when people are at the office on hand to respond Then at four in the morning when people are asleep Netflix knows that Eventually failure is going to happen These organizations and many others besides say You build it you run it This is a very adult mindset. They say you're an adult. You can use whatever technology you want to solve the problem at hand But be prepared to get the phone call at four in the morning when something goes wrong There's no separate operations team. That's going to that's going to handle it for you. You wear the pager Right you build it you run it this by the way, I think speaks volumes as to why organizations like Netflix Use spring boot and spring cloud to build production worthy services at extraordinary scale Because they want to get production worthy services up and running first and fast You see we live in a different era now. There's no longer this artificial divide between developers and operations There's no longer. It is no longer the case that Developers are the only ones Charged with delivering business Different trading functionality and it is no longer the case that operations Are the only ones who are charged with ensuring the stable Evolution and management of the software and production operations job isn't to say no no no no And to stop change and it's not developers job to to write code that doesn't run say safely Right developers have to write production worthy code They have to care for it And there's no better way to make developers care about production worthy code than by making them wear the pager They'll care a lot more when they're worried about being woke up at four in the morning These organizations understand that This is what this is the very spirit of DevOps DevOps is an ancient malaysian word A very old malaysian word It means empathy Empathy between operations and developers empathy for the situation in which we both find ourselves It means that these two are the same page They're both charged with delivering new functionality for the business as fast as possible And they're both charged with delivering stable production systems This is not either or No longer our operations the sin eaters It used to be that we would take our terrible code and throw it over the wall And operations would take it and if they ran it then it's their problem The fools Right that mindset makes no sense anymore. It's gone. It has to go So we need to care now about making this service production worthy We need to care about resilience a big part of that is Systemic observability a big part of that is Some of the patterns that we're going to look at when we come back Now we're in a good place to break So There you go. I'm happy to answer questions on what we've just talked about so far When we come back, we're going to look at Oh boy, we're going to look at resilience and reliability patterns How to make the system robust in the case of service outages for both reads and writes And we're going to look at how to make different systems agree upon distributed state We're going to look at observability and how to get systemic Observability across the different nodes in a system We're going to look at security and then we're going to look at data orchestration or messaging based microservices It'll be quick Okay. Thank you very much. I'll see you guys and gals as soon as possible. I don't know when that is I guess it's an hour from now. So, you know Cheers Do not step on that white cable Lunches is a real experience here So, all right, welcome back. Where are we left off when we left off before the the break I had brought the idea that we have to build services that are going to do the right thing in the face of service outages and topology failures This is critical and I and I hope I underscored how important this is As you move to this cloud native architecture a cloud native system is as I say four things It's one that lends itself to easy agile iteration it's one that Does the right thing in the face of elastic scale benefits from the elasticity of the of a cloud, for example It's one that does the right thing in the face of service outages or topology changes And it is one that is observable. So let's talk about that third one that third point That third tenant we introduced in the last example a rest api that Calls our downstream service And it does the right thing should we have one or more instances of that service registered in the registry But if we have zero instances then it's going to it's going to blow chunks We're going to see a big fat java stack tracing and that's unacceptable So we need to think about how to make this a little bit more resilient how to optimize You know for the fact that failure will happen high performing organizations Optimize for time to remediation Instead of trying to build a system where everything is highly available instead build a system where the time to fix that error an error that will come Is as minimal as possible if time to remediation that is to say How long it takes for you to get the system in a state that the client can continue using it if time to remediation is zero seconds For any given node or service then you are effectively 100 highly available right, but While the results may appear the same do you have a Okay, i'm screaming for now. It's fine, but when you can please Um while the result from the perspective of the user may seem the same It has profound implications on the way we need to build our systems The result the approach that we have to to embrace to build our systems is very different So What can we do here? I want to make sure that if somebody calls this endpoint and we try and load balance and it throws an exception That we handle that gracefully and I want to make sure that We give the downstream service the service that we're calling that's sick time to recover After all the last thing we want to do is deluge it if it's trying to come back online So i'm going to introduce a circuit breaker a circuit breaker Very much like the component in a building in a modern building Is a component that when there's a risk of an overwhelming amount of electricity or traffic Opens and it prevents successive Calls from going through this in a building Stops the risk or prevents the risk of a of a fire, which is good. Which is what we want I would much rather lose the electricity and lose the lights Than to lose the whole building for a fire Um, so How can we do that here? I'm going to go back to my build and I'll use spring cloud starter hystrix Hystrix is a circuit breaker from netflix People I'm not sure what's happening people are stirring Anyway, uh, we're going to use a circuit breaker from netflix. We're going to enable that circuit breaker. Hi No problem Cheers, thank you very much We're going to use a circuit breaker from netflix. We're going to say at enable circuit breaker And with that we can now Cordon off this particularly risky service to service call the shaky service to service call By using the hystrix command annotation. So this is a a third party api called javanica Which is itself just a uh an annotation based approach for consuming the hystrix circuit breaker library And one of the conveniences is that you can now Dain that a method should be a fallback method. So here i'm going to create another method with the same Prototype if you all the same signature. Thank you Ah wonderful. Yep good. Let's see Money can you all hear me now? Better? Oh, that's so good That's what I've always wanted. Uh, anyway, uh I'm going to build a a fallback method that just returns an empty array list And this is not a particularly useful fallback But uh, it does give us something instead of a big fat java stack trace High-performing websites will do this sort of thing all the time They'll say oh, well you you went to the search engine service, uh, but it's not available So here's some machine learned recommendations from across the web or even better instead of showing you the option to do a search You just get the recommendations right parts of the page can appear and disappear based on the availability of that service You can achieve this effect in part by using the discovery client and talking to the registry to see if the service is there And then also in part by protecting against uh pathways that have errors in them here We've done that uh for for our service here. Now, of course, we have Information about both the service and the client So we could use the registry to ask the question Is that service there and then to not bother calling the service in the first place But we may not always have the ability to interrogate some other system's uh registry We don't have the ability to ask third-party apis questions about whether their services are online So the circuit breaker is the the sort of band-aid. It's a reaction instead of the pre pre action, right? So here i've got enough a fallback method. Let's go ahead and see it in action I'm going to go ahead and call the the happy path the 80% case Uh, do I have my service running? 8 000 I'm not sure if I killed the service or not If you change, uh, so when I when I went away for lunch, I My network changed so I've got a different ip so Things are nice and different Okay localhost 8 000 reservations good. There's that Eureka There's this Client Can you hear that? I wish I could share that I think there would be far less war if we could all just crack our knuckles Anyway, let's see There we go. So there's the happy path Right, everything's working just fine. I'm making requests. It's going through the edge service to the downstream service Now let's kill the downstream service here And as we do that the service becomes no longer available We call the service it throws an exception after a timeout and then finally we get the the fallback method now The circuit breaker would be kind of interesting. Uh, if it were just a mere try catch block, but it's more than that You see it's stateful It's going to look at this this pathway and it's going to see that enough successive attempts to call that pathway To traverse that pathway have failed and so it's going to instead of deluging the downstream service It's going to route directly to the fallback sort of switching the train tracks Right, so now let's see that in action here As I make requests you can see it stuttering. It's hesitating right there. It's trying to call the service, but It's timing out and then we get the result if I Lay down on the refresh button And drive traffic will see it eventually stop doing that eventually it'll go directly to the fallback So we can see that here It's just going as fast as I can refresh Right, it's not timing out anymore It's smart enough to see that enough attempts have failed and to then divert traffic It's also smart enough to heal itself, right? So if I restart the reservation service, it'll eventually percolate back to life The service will re-register with the registry the registry will will propagate its Its understanding of the world back to the clients the clients will then re Reallow traffic to go through it will attempt to allow traffic to go through again So we build a self healing system so to speak as long as parts of the system come back into place They can self describe self configure themselves through the registry This is very very useful, right? We all know that If if a website isn't working If a website's going very slow, the best thing that you can do is to refresh the browser a lot, right? Is that is that true? Of course not, right? Like you're just going to overwhelm the service. It doesn't help anybody So this defeats that effect this protects the downstream service from the torrent of activity Hammering it as it's trying to come back to life. It gives it time to warm up if you need as well We're protecting our system if you're using something like cloud foundry cloud foundry will move heaven and earth It'll do it'll work all day and all night to make sure that if you say I want 10 instances of this service available at all times that darn it There are 10 instances of this service available at all times It'll wear the proverbial pager Right, but it's our job as developers and architects to build our system in such a way that we're resilient to topology changes as as happened in this case now In this case, I'm just reading data from the downstream service, but what about a right? What about writing data? I've got an edge service. I've made a request. I'm posting I'm putting something to the edge service and that's going to be transmitted to the downstream service What happens then? Well, how do you what does that mean like if I have a downstream service that uh, isn't there How do I you know it's I can't send a request to a service that's not there Sorry Something like that right you want to buffer it so What we're trying to do here in this case is to to get two systems degree upon state one of which may not be available So let's let's modify our edge service here our api gateway and add another endpoint. This is our risk controller. This is the There we go. We're going to add another endpoint that will take the data and then write it So we'll say public void write I'm going to say request body And I'm going to say that whenever somebody posts Jason to my edge service on port 99 99 That the json should be converted from the json to this reservation entity and then my job here is to somehow get the data To the downstream uh reservation service, right? That's what I'm trying to do and I could use the rest template here as well But for the reasons we've just described that may fail catastrophically This is an age old problem right that getting one service separated by a network partition to work with another Is a hard problem But there are many different ways we can get around this many different easy to use patterns or uh fairly easy to use anyway The one way that I actively discourage and that I do not recommend Is to use distributed transactions Distributed transactions are provably the worst way to solve this problem If your goal is to slow down the system And not actually guarantee consistency Then there's still The possibility that they're the worst solution for that problem, although they usually are the best for that one particular's goal Don't use distributed transactions Distributed transactions on the jvm are modeled by the jta api. How many of you have ever used jta? Okay, so I give away hugs I give away hugs for free for the people that have been using that Uh jta is middleware. It's a client side binding if you will for the x open protocol Just like the servlet api is a client side binding or middleware for the htdp protocol Right the x open protocol is a very old protocol It describes a single point of failure called a transaction manager Which is supposed to keep a transactional log of different transactional resources xa resources right Each resource is enlisted in a transaction They all hold hands on them whenever the transaction transaction manager says go and then they jump they commit And if something goes wrong then the transaction manager has each one of them roll back Now This is a bad idea Lots for lots of reasons first of which is that it's a single point of failure The the second reason is that it's completely irrelevant in modern distributed computing most resources aren't xa resources The arrest api doesn't implement xa at all Right, so it's not a good solution. We need uh other solutions different solutions And so we can use a uh a markedly better and newer idea A more contemporary a more modern approach that came from the 1980s It's called the saga pattern The saga pattern has two key constraints you design your systems as a set of interleavable That is to say reorderable transactions as long as they can be reordered It doesn't matter which order you run the transactions in that's the first constraint The second constraint is that every transaction that you have has to have a semantic That is to say not general purpose, but semantic business logic specific compensatory transaction a transaction that rolls back the state to a semantic A well-known semantic state rolls back the system to a well-known well-known semantic state So a good example of this is uh like kayak calm or orbits calm these websites let you book for example a hotel a car and a flight Well when when you go to that website it tries to call the hotel website It tries to call the airline website and it calls the car rental website It uses web services to get all of these uh, uh transactions booked If any one of them should fail It then cancels the booking it cancels the hotel reservation. It cancels the flight It's a semantic rollback. It undoes somehow via the api by the by the thing what it had done And the only other constraint there is that the the compensatory transaction has to be retryable It has to be edempotent I should be able to retry the compensatory transaction as many times as I need to to get the result If I and I should be able to do so without any extra side effects If I can build my system and comply with these two constraints Then I can guarantee consistency across the distributed system now. It's worth noting that the saga pattern originated in the context of a single node A single long running process But the result the constraints the dilemma the dimensions of the problem are very much the same. What is A network partition, but time It is a delay They are the same thing in terms of how we have to solve the problem If you're solving a long running process and you want to have a transactional resource that uh, you want to You can't hold open for a long time It's the same Dilemma as you have when you have a network partition So you if you use the saga pattern you can you can solve very very interesting sort of distributed You know distributed consensus problems in our case and our for our purposes. However, that's a bit overkill We don't even need to do the the saga pattern as novel and innovative as it is for the for something that came out 30 years ago Instead we can use something even older from 40 or 50 years ago called messaging Right, we're going to use eventual consistency and that's just a five dollar word for messaging It's the idea that we're going to store and forward the right Through a broker a message queue I have on my machine rabbit mq It's one of many choices and I could use spring integration Spring integration is a framework for building event driven or messaging based systems It solves a lot of the same use cases as you might otherwise solve with axway integrator or tip tipco or web methods or Mule or whatever Except instead of instead of having a centralized broker instead of having a centralized message bus It is designed as a framework is designed as a set of components that you can hang off the side of any spring application So the integration logic lives where you need it to be spring integration Has at its heart at the core of it this concept of a message channel A message channel is a java util queue. Basically. It's a pipe through which messages pass Spring framework has message objects. These are envelope objects that have headers and payloads So message objects transit through these channels and on the terminuses of these channels. There are components And there is where you put your business logic a lot of the business logic a lot of the logic in those components isn't really business specific It's just uh integration work For example, you might use an inbound adapter on the on the origin terminus on the genesis terminus You might have an inbound adapter that says whenever a message comes in from a third party external system I want to adapt it into a spring framework message Maybe it's a x mpp message or an mqtt message or a mqp message or a jms message or a kafka message Or a tweet on twitter Or an email or a new file appearing in an fdp server an sfdp server Right all sorts of classic integration style stuff and I can do the same thing in reverse I can say i've got a message and i'm going to write it out And I can even interpose other components that do things like splitting and uh and joining or you know aggregation or splitting and aggregating Uh messages across different Qs or I can transform them enrich them etc This is all classic enterprise application integration stuff In fact the patterns the api elements in spring integration map one to one to the patterns Set forth in the canonical tone by bobby wolf and gregor hope called patterns of enterprise application integration So there's that I could use spring integration It's certainly a good fit for what we're trying to do but it's a little overkill because we're not going to use twitter To connect our microservices Right i'm not going to tweet to my reservation service that i've got a new i've got a new post I'm not going to send an email to that microservice either I I can take for granted that i'm going to use a message queue A highly efficient highly concurrent highly transactional highly scalable robust message queue something like apache kafka or redis or rabidim queue These are all very very very easy to use very easy to deploy commoditized technologies You know that we all have i'm sure access to So let's do that Let's do that instead and if we if we're willing to take for granted that we're going to use a message queue Then we can move up the abstraction stack a little bit We can move up to something called spring cloud stream spring cloud stream Builds upon spring integration. It has the same idea as a message channel and it has the same idea of messages But it makes the work of composing these different solutions A matter of convention and configuration Our business logic our java code interfaces and works with message channels With no regard to how those message channels are wired to the broker. That's the left as a matter of convention and configuration handled outside the code Right, which means that you can arbitrarily reorder things later on and that'll prove valuable as we'll see in a little bit now Let's take a look at that right. We're going to use spring cloud stream I'm going to say bringing i'm going to say bring in spring cloud starter stream rabbit mq or rabbit This is but one of many binders right spring spring cloud stream has binder implementations. This binder is for a rabbit mq There are binders for jms for Redis for kafka apache kafka, etc so When I bring that in I now have to tell my client my reservation client the edge service I have to give it something to work with I have to give it a message channel and that work is declarative I can define the channels here in an interface I'm going to call this interface reservation client channels because it doesn't really matter at all The name is completely arbitrary and I'm going to create in this channel an output channel now This is an arbitrary name There's no reason you couldn't have multiple channels in the in the interface definition Each one of which is named for some downstream messaging based microservice You might have one for products for orders for customers for whatever right In the same way that the service registry uica or a console or zookeeper, etc Act as a phone book for our rest-based apis So to the spring cloud streams interfaces acts as a phone book for our messaging based apis We don't have to worry about how or where these things live We just know that if we send a message into this Output channel that it'll get delivered to the downstream service appropriately Right and we'll talk about how that actually wire gets wired up and lines up later on but for now We've created an interface and I can say reservation client channels class There we go. So I'm saying enable binding. That's what activates the spring cloud stream Once that's done. I can inject either this object or an object of type message channel whose id is type output And I can dereference that channel. So let's go ahead and do that. I'm going to inject here private final message channel out and then I'll say That I want to use the message channel. I'll say message channel out this dot out Uh, actually I want the uh reservation client channels Okay, so client channels Dot output, okay, there. Voila So those are output channel Now I can use that channel my code So I need to create a message a spring framework message which has a payload of type string I'm going to use the spring framework message builder And I'm going to say that I have a payload reservation dot get reservation name That's the uh string that I'm going to send to my downstream service I don't have any headers. I'm going to go ahead and just omit those and then I can send The uh the message on the channel like that right If I wrote that a little more cleanly, I'd say that So I'm going to say I've got a payload the string name I'm going to create a message around that and I'm going to send it to the downstream service now On the other side of the code on the service itself. I need to do the same thing in reverse I need to say that data is going to come in On a channel and for this I need to first of all reinstate spring cloud start a stream rabbit And I'll just bring in this other dependency so that I don't have to worry about it later on Uh, and I'm going to bring it. I'm going to say at navel binding And I'm going to define an interface. I'll say reservation reservation service channels And I'll say input Subscribable channel input okay Reservation service channels dot class And there we go. So now I'm describing a channel that'll take data in and again the name is arbitrary I'm just using what I what I want now here. We can use a spring framework Rather a spring integration component called a messaging endpoint. It's just a declarative, you know, uh processor Data will arrive and I'll process it as it arrives It's a you know, the hollywood principle applied to distribute systems. Don't call me. I'll call you right So on new reservations Whenever a new reservation arrives, I'm going to coerce the payload I'm going to have it injected or passed to my method As a message whose payload is a type string I'm going to signal to spring integration that data coming in from the input channel should be Passed to this method, right? So this is a listener if you will a message listener container And what am I going to do with a message that arrives that ultimately arrives? I'm going to write it to database That's what we're trying to do here ultimately, right? So There we are this dot reservation repository at save new reservation message dot get Or get payload and there we are Okay There's my message my message a processor my reservation processor So I'm going to restart that now Let's take a quick look at the configuration that makes this possible so we can understand the dynamic of what's happening here If we go to the producer the producer is the edge service I'm going to send data to the downstream service from my edge service The producer has in its configuration here spring cloud stream bindings dot output dot destination Equals reservations output my friend is the name of the channel in the interface on the edge service the client This is arbitrary if you have multiple channel definitions, you'll have multiple lines Look like this Reservations is the agreed upon rendezvous point in the broker It's where the the producer and the consumer are going to agree to meet in rabid mq in apache kovka in redis in jms Whatever in a jms broker Yes Well, we'll give it let me get to that the question is is that the topic and um And as it happens yes, but Hold on so Let's look at the other side The consumer now the thing that accepts the data you see We can expect to see the the same kind of thing in reverse there We see this right here. It says spring cloud stream bindings input that destination equals reservations That's natural But one thing that we need to understand is that by definitions, you know by default spring cloud stream bindings are publish subscribe They're broadcast. They're one to many their topics Right, they're not point-to-point So if I have 10 consumers if I have 10 reservation services Listening on the other end of my rabid mq Then all 10 of them will get the same one message if I send one message That's not what I want here I don't want to duplicate You know the same message 10 times or end times instead I want to do load balancing I want to divide the work by as many consumers as I have So if I have 100 messages that I send and I have 10 consumers then each 10 You know every one of those consumers should handle 10 different messages So I need to make sure that The consumers divide themselves up so that only one consumer gets any one message I can do this by using what's called an exclusive consumer group here I'm saying spring cloud stream bindings input, which is the channel still dot group equals reservations group Now all so configured consumers in the same group Will arrange as or arrange themselves as an exclusive consumer group only one consumer will get the message at any given time The final line of interest here Is this this is this specifies that the subscription to the configuration of rather to the To the exchange and to the to the queue should be durable That is to say If the broker has messages That haven't been delivered because the services are all down Then as soon as they start up, it's going to retroactively deliver all the messages that are in queue That means that I can restart my service or I can have None of my services available and I still eventually will handle the data They're not just island or stranded on the in the broker Okay, so these three things together give us the effect that we want That's one to one point to point as opposed to publish subscribe It's durable. So I'll replay I'll retry the transactions as soon as the service becomes available And of course it's connecting to the right exchange now Uh, I restarted the service. We should restart the client. I realize now that I forgot that Okay, we can actually pop into rabbit in queue here on my local machine 127.0 .0.0.1 And it's called guest guest Nope, and I've got lots of different Queues in here exchanges and queues Where's my there we go? This is the queues. So in rabbit in queue parlance. How many of you have used rabbit in queue? How many of you have used jms? Okay, so imagine In the jms world you have a javex.jms.destination A javex.jms.destination is a Super type for two specific types of things a javex.jms.queue and a javex.jms.topic A topic is meant to be a pipe that has multiple consumers a queue is a pipe that has one consumer But otherwise, they're the same thing. They're both a type of destination This is this means that The client or the consumer and the producer have to agree to meet at the same destination In rabbit in queue there is no such limitation There's indirection even in the broker in in rabbit in queue you publish a message to an exchange The exchange is sort of the revolving door into the broker and the exchange can do all sorts of cool things It can do a replication to other brokers. It can translate the message from amqp to another protocol It can do all sorts of interesting stuff that has nothing to do with delivering it to a consumer It can also act as a topic for example It can do point to point it can do hierarchical topics as well, which you can't do natively in jms and the point is the Matching of the binding of the of the exchange to a queue is Random you can even change it at runtime So I can have I can flip the train tracks so to speak in the queue. It's in the broker itself So I have here the queues And I can see here that the consumers are going to connect to the queues. This is the reservations group And then I have the exchanges here And uh here I can see That I've got the reservations exchanged So the messages are going to arrive at the Topic exchange here, then they're going to be sent to the exclusive consumer group and only one You know one part it'll be a partitioned consumer group basically Okay, so let's see if everything's still working There's the read i'm reading data Okay Let's send some data now. I'm going to send an empty. I'm going to send a post with some data in the payload. So, uh, let's see We're going to line this up application Jason not Jason Jason HTTP local host 99 99 reservations Okay, and I'll say reservation name And I'm going to send a few of my favorite doctors because if if we've established anything here today It said I love good doctors now Uh, I'm going to say doctor who okay, and after I refresh I can see doctor who immediately reflected in the output on the edge service So I did a post to the edge service it delivered it to the rabid mq broker Which then was the message was then consumed by the downstream service the reservation service Which then voted to the database which then makes it available for me to refresh the browser and read it Okay, we can send some more doctors because we we all love a good doctor Okay, so doctor Seuss And doctor strange Okay There we are so there's all three of the good doctors now Let's go back and kill the poor poor poor reservation service Which for the bounce over to talk is going to be killed a lot. So poor went out for the reservation service, okay If I read that if I refresh that it's going to hesitate it falls the circuit breaker falls back Now we're going to write some data. I'm going to write some data and I'm going to say doctor Subramaniam doctor subramaniam. Dr. Venkat subramaniam is one of the coolest human beings I'll ever meet He's a an amazing terrific speaker really really nice guy And just really really genuinely cool. So you get a chance to watch any of his talks. You should so doctor subramaniam I'm going to send doctor Pollock dr. Pollock is the co-founder of spring aim qp spring cloud data flow in spring xd He is the founder of spring net and he's a spring framework committer So there's that also super duper terrific guy and then we're going to send doctor sire now doctor sire is the Founder of spring batch. He's the co-founder of spring aim qp spring innovation spring boot spring cloud And also a spring framework committer So those three doctors have just been sent Recall that we have a command line winner and the command line winner is going to be rerun when the application starts And that's in the reservation service. So you see that here. Here's all the names This is an in-memory embedded database. So as the application starts up, it's going to recreate these 10 records That's why we don't have a thousand records by now because I've been restarting the service But it's discarding all the names and they're recreating them on startup So what we're going to see is as the application starts up It's going to take delivery of those cued messages those three messages that are waiting to be delivered And then it's going to write the data the the command line winner Or it'll probably do some mix of that. Maybe it's at the same time because concurrency, right? So Okay Goody So there we are. There's the good doctors. There are the good doctors doctors supermonium polykin sire They've been delivered and then we saw the the other command line winner written Records being written to the database. We've now built a system my friends that does the right thing in the face of topology And service outages for both reads and writes And it wasn't too much too much code But I would submit that it is still more code than I want to write you see The read use case had three lines. I was using the rest template. I created a parameterized type reference And then I made the call with the rest template The the write use case had three lines as well I was using spring integration and the low level message channel and so on and I was creating a message We're not so bad But consider that I'm only writing and reading a string here Right or a collection of strings, but really That's a lot of work And it's going to become very tedious if every client that has to use the service Has to write all of this code for both for both the messaging and for the rest logic One solution and I say that with air quotes Is for the serve for the team that builds the service to write the client for that service This runs the risk, however of the client leaning upon the fact that they're also Implementing the client and baking too much magic quote-unquote into the client The team that builds the service might use it as a crutch So a lot of organizations frown upon this they say okay well, uh, you We should be able to automatically generate The client or the client should be built by somebody else So amazon.com for example says if you're building an api somebody else builds the client that way the service stands on its own There's no magic in the client itself And that way anybody can build a client for different language as long as they have commoditized tools Next Netflix, you know, they want the same they want to make sure that that magic doesn't happen as well But they're perfectly fine if uh, the client is automatic that way. There's no cognitive You know overhead. There's nothing that you have to think about it. It's just automatically done for you And and and there's no problem if the service team ships that so long as there's no like chance for inconsistencies and Special pathways. So let's take a look at how we can do that using this How look at let's take a look at building a declarative rest client using Netflix's fame now in english Thane Means to pretend Means to act as So suppose you saw an animal in the forest. Do you have forest here? Is it is it in the mall? No Oh, it's on the top of the mold. Yeah, right makes sense. Okay, uh, suppose you saw an animal in the forest here You know laying supine with its tongue out playing dead It's trying to avoid You know entrapment You'd say that that animal is feigning dead Right, it's pretending to be dead in the same way that websphere feigns utility. It's not actually useful. It just pretends to be so We can we can use feign here. We can use feign to build a declarative rest client. I can say up here At enable feign clients and then I can build a interface a declarative client. I can say reservation reader And then I say feign client and the client is going to uh call For me My service the same service that we've registered in the registry. So again here too Here again spring cloud is automatically wired wired up this this client to know about ribbon and to know about a registry for us And I can now Create a client side method I'm going to say that whenever somebody makes a call to the rest service called the reservation service and they call this method We're going to create an htb get call that goes to the reservation's endpoint and the return value Will automatically be converted into a spring into the spring hideous collection of resources Whose payload is this type reservation now with that done? I've got a client that I can use to talk to that service and I'm using spring mvc server side mapping annotations on the client logic I can now rewrite my api gateway. I can inject here In my code I can re inject the reader instead of the rest template Reader and I can rework this code. I don't need that anymore, right? So I say a reservation reader reader this dot Reservation reader equals reservation reader Good much much less code. Uh, we can see that here. I can change this logic That all goes away And instead of saying anything else I just say read and get rid of that So now it's become a one-liner and it's and even though it's not much less code The real value here is that we didn't have to think about it. The client just works We're not parsing the lines of code to understand what's happening. We know. Oh, it's a feign client It's just going to work, right? We can we can write tests for that, but that's not we're not actually testing whether we've correctly manipulated the htp Programming correctly, right? It's just about whether we've got the right payloads and so on So that's the rest stuff. What about the right? What about the right use case the messaging code? Well feign doesn't quite work. It doesn't work for messaging. It doesn't know about rabid mq and spring negation It's for rest, but we can use spring integration messaging gateways Messaging gateways is a pattern first and foremost and it's the idea that you're going to have an object that hides the the messaging The interaction between the messaging systems behind what it looks like a synchronous method It's a facade right for the messaging code. So I'm going to say interface reservation writer and I'll say avoid write String rn And I'm going to signal to the signal to spring integration that this is a messaging gateway and that this method should be sent On the request channel called output, which of course is the same channel that we have Defined by spring cloud stream up here So now having done that I can rework this code as well I can get rid of the channel because I don't need anymore and I can just use the writer reservation writer reservation writer and I'll say reservation writer Okay, this dot reservation writer Equals reservation writer good Now we can get rid of this logic becomes markedly simpler, right? So this dot reservation writer dot write reservation dot get reservation name or rather reservation Dot get reservation name. Okay So now it's the same business logic. I I gain all the same benefits. I get client side load balancing I get I get messaging. I'm not using rpc I'm still using rest. I'm still using ubiquitous transport. I'm still using highly decoupled interfaces I still get hyper media so I can traverse the links if I want I get all those benefits, but I haven't had to write so much business logic to get it This is one of the pain points that people perceive when they move to a distributed systems world Does they think that they're going to spend all their time twiddling htb packets and twiddling? You know with the message brokers Which is just not true. So there's the read. I'm correctly correctly reading, right? That's working Let's go ahead and Send some more data. So I'm going to go ahead and send the good doctor sire again a second time And when I go here, you see that the doctor sire is there now All right Good stuff, right? So now we've reduced the code. That was just a matter of style It's not fundamentally anything new or a concept, but it's just a matter of making it easier now Now that we've got this up and running that circuit breaker Strikes me as an interesting opportunity In this next section, we're going to talk about Observability and specifically when to start with the circuit breaker You see that circuit breaker is a connective tissue between our client and the downstream services It represents the connection from our api from our code to another api not necessarily our own The circuit breaker is very useful for services, you know for for guarding against failures in any system, of course But it really comes into its own when you're using it to call third party services Over which you have no control You have no ability to inform or change Other people's operational characteristics for their services You have no ability to to make them run in cloud find you to guarantee that the system is up all the time You have no ability to influence influence all of that So the best that you can do with a circuit breaker is to protect your system against their failure That's what the circuit breaker does and because it is basically It is it's a stand-in if you will for their system It represents the link to that system and so monitoring the circuit breaker becomes useful because that is basically A proxy if you will for their system If the circuit breaker says something is wrong, then it is as it is as good as saying that we have monitoring on their system And something is wrong Right, so let's let's build a dashboard by which we can monitor the flow the traffic through that circuit breaker We're going to build a hystrix dashboard like so we need to configure client and uica service registration discovery and we'll hit generate We'll open this up Oh, did you hear that? Yeah, okay application that properties spring dot application name Name equals hystrix hyphen dashboard spring dot cloud dot config config dot uri equals htp local host 8888 And i'm going to rename the property file to be bootstrap dot properties if I can type the right key And I'll open up this code and I'll say first of all that I want to participate in service registration and discovery There's that and I'll say at navel hystrix dashboard and then I'll restart now. What this is going to need Is a service end event heartbeat stream It's a heartbeat stream that every circuit breaker and all of the services and all of my system Will automatically emit that service end event heartbeat stream Tells me the status of traffic through that circuit breaker So let's visit the edge service and we'll go to the circuit breaker stream for that edge service You can see that this circuit breaker stream is as I say it's service sent event. It's push There's always new data It's endless. It's infinite It goes on and on and on It has no end It has no finish It is boundless without finish It goes on like the stars And the seas And the oceans And the bugs in your code just infinite just on and on and on So whatever you do my friends Whatever you do Do not Do not and I cannot underscore this enough Do not curl this endpoint now We're going to take that endpoint and paste it into our hystrix dashboard And this dashboard is what we've just stood up. I'm going to paste it in there. I'm going to hit monitor And now as we draw drive traffic on the left here, we should see reflected on the right the moving average You can see that as I drive traffic on the left. It says 12 19 23 30, etc That's the ever arcing ever upward trend of the traffic through that that circuit breaker Everything is healthy. Everything is happy traffic is flowing just fine and the circuit is closed I can also see that there are zero percent error errors. Nothing is wrong. Everything is healthy. Okay. Now if I kill my downstream service here We can see that the circuit breaker is going to eventually fail It's going to open up and you can see that 100 of the rights are Read rather are failing and that we've got now this ominous looking red glowing orb And now the circuit has eventually forced itself open so it's going directly to the fallback Right, so now you can see that that's happening This gives us visibility into the state of the communication between these different services and that's very important earlier on and we talked about The uh, the actuator the actuator is a good way to understand what's happening in a specific node on a specific instance of a specific node in the system But we haven't really looked at capturing systemic behavior You have to remember that in a distributed system the map is not the terrain Is if you are walking here in beautiful singapore, is that exactly the same thing as Looking at a map a google map of singapore Of course not right When you walk in singapore, first of all, it's markedly warmer Second of all the food smells much better than google maps does It's much more beautiful cleaner. It's amazing. You know the the people are amazing There's there's so much more happening in singapore than you can actually understand or capture by looking at the map of singapore That's pretty obvious What this this is also true for distributed systems though The be the emergent behavior of your system in production in the wild is very different from the architecture diagram of your system The map is not the terrain And you need to capture that emergent behavior To be able to truly understand what's happening the circuit breaker dashboard is one way to do that one thing you may be wondering of course is How do we get multiple circuit breaker dashboards? You know multiple circuits in a single screen and of course you can use spring cloud turbine to do that If you do if you use spring cloud turbine Then it'll multiplex all of the circuit breaker streams into one unified stream that you can then plug into the dashboard And you can see that updated here Right So i'm not going to do that now, but it's a it's a fairly trivial exercise to then use spring cloud stream to do that okay Okay, so the next Another a good way to capture the emergent behavior in a production system is to use distributed tracing I like distributed tracing because uh, it's appeal is obvious And it's such a big win and when you do it Distributed tracing is in theory very simple What you want to do is to for every message that flows through a system From one node to another node to another node to another node You want to make sure that you affix a unique id a unique identifier a unique correlation id An id by which you can trace the flow of that message And so if a message enters in the spring mbc rest api And then it gets pushed to uh, rabbit mq or you send another message via the rest template or a message arrives in the Zool micro proxy, and then it goes to, uh, you know the the rabbit mq or apachi kafka or or whatever Any ingress and egress points in the architecture any place where messages arrive or leave You have to instrument that code to make sure that you check for this unique identifier and if it's not there at it And if it is there then make sure to perpetuate it so that as the message moves along it always carries in band This one header this one unique identifier As long as you've got that and as long as you've registered where this id has been observed Then you can see the flow of the message through the system And we already have that basic functionality working for us right now. I uh sneakily Added spring cloud starter zipkin to our application for both the the client and the service earlier They've been on the class path. They're dormant right now They're on the class path spring cloud starter zipkin is an implementation of an abstraction called spring cloud sleuth sleuth Right sleuth is our distributed tracing Uh abstraction in spring cloud spring cloud sleuth okay now We've already got it on the class path and it's already doing good work for us We can see that here on the console for the client. Where's the client? Here's the client Or is it doing well good work for us? No, it's not it's just sad So I'm going to add it to the class path there and we're going to sit here awkwardly waiting because I failed spring cloud sleuth Has at heart the idea of a trace A trace is the aggregate journey. It's the whole journey from a to z of a request through the system I didn't do it wasn't me We ran it wasn't me man It wasn't me Poor guy um A trace is the aggregate journey represents the aggregate journey of a of a message through the system from a to z If you have five services, then this the trace id will be the same across all five calls A span on the other hand is each individual leg in that journey each hop in the journey from a to b From b to c from c to d from d to e etc. You'll have a different span id for each one what It's static. I I didn't even know that was nothing anymore I've made this guy's life so miserable So weird. Thanks, brother Oh Anyway, the point is That's already very useful right that that abstraction is already useful and here I've restarted the application I'm driving some traffic now And spring cloud sleuth is already in play and you can see it here on the console You can see that it's it's got as its output the service id That's ridiculous My code is cursed this guy's my hero We were in for president. I'd work for you, but I would Sorry Now's a good time to to go pee And if you have questions on such on some of the stuff that we talked about just up until now Uh Yeah, you can do request reply based interactions with messaging as well so that if you if you have a If you're using revm q or kofka, it's very easy to say that there's a return address. What's wrong, buddy? Your computer is eating up Too much power It's eating up too much power. Yeah, how can we use the vg? You're running too much application But I've done this demo a million times nothing has changed I can if you want to kill kill minus sign everything and start again Let me see top Java updater z shell java so the java processes are down here Google chrome is eating a lot of power. Goodbye. Let's kill some stuff p kill of java Kill all that Top I just killed all the java Everything is sleeping. The only thing that's running right now is top itself Don't try the other one No, I'm sorry, um Yes, sir um, so the the The history. Okay, so let's talk about the circuit breaker first of all If you're using cloud finder, you can do individually addressed addressable Instances And for services service communication in the cloud you would still use eureka Right, you're not going to use you're not going to route back and forth between the the different services Through the ha pro through the go-rounder for example, right? So you'd still have individual ip's on individual instances and you'd still do load bouncing there Uh for iPhones and so on For html 5 devices, they're not going to use eureka. Can I go again or? Should I try Okay, so uh, you're not going to interact with um You know you're not going to your individual your clients I'd load balancing work just fine there and uh history is you know work just fine as well because it'll it'll you know There's nothing different as far as you're concerned It's just two different services that are running in the cloud as opposed to my local machine But the the the dynamics are still the same Uh, what was the other was there another part to the question actuator? Ah, so the actuator again Those are host by host and node by node specific information. You can individually address some of them where where You want or where you have a need for global visibility. I've talked about some ways to do that I've talked about how you can capture some of that information Uh using for example the the drop wizard metrics reporter objects that then Publish those things to a shared a store, but a lot of that information doesn't need to be Shared in a single place, right? That's it's by definition a node by node Kind of configuration And and we're looking right now at some information some some of the things that you can deploy uh to to Centralize this information, right the history dashboard and and so on and this all runs on cloud foundation It's all just it works just fine. Indeed We've got work underway right now so that the metrics that are in In the spring boot actuator can be used as a way of of uh as one of the ways by which we can do automatic um Uh Auto scale right and we've also got Uh work underway right now so that when you see when your spring boot application says it's sick or it's down Then the ops manager will actually reflect that as well in cloud finder, right? So this is a deep integration between cloud finder and spring Okay, let me go ahead and restart the world Config service Okay, and we need the uika registry so start that up Okay, and then we need the Reservation service Start that up local host 80 81 80 30 What? Oh son of a gun see Remember that earlier demo much earlier So I just opened up all my intelligents in the terminal on which that variable was set And so all of them are seeing the overriding server underscore port equals 80 30 So all of them are that that has priority over the built-in property file So they're all trying to start in the same port So now I have to kill everything Echo server underscore port Not there array Okay, okay Computers are terrible Okay So config service There we go up up and away and uika service Once that's up Okay, and then we need the reservation service Here and that should spin up now any moment local host 87 61 There we are. There's my service registry Close this and we should now see the reservation service itself eventually register there Okay, now we need their client local host 99 99 reservation names Okay, there's that now the dashboard district dashboard There's that Now as I was saying before the screen gave up the ghost I have on the console here Reflected in the output logging that spring cloud stream is automatically doing force rather spring cloud sleuth So here we see the service id The trace id and the span id and already this is pretty useful I can drain my logs to a centralized log analytics platform using for example Cloud Foundry's logigator cloud foundry will centralize It'll take all the logs multiplex them into one stream and then drain the logs to any sys log d compatible spout anything like Paper trail or splunk or elastic search and then I can do log archaeology Right, I can sift through the logs and see the I can trace and find the The flow of messages through the system and that's okay It's a start But i'm a big believer that a picture is worth a thousand spans. So instead we're going to go ahead and use Zipkin Zipkin is an open source distributed tracing platform that originated at twitter. I'm going to build a zipkin service here zipkin server config client discovery client And generate it originated in 2010. It's a it's a way of capturing and modeling and analyzing Trace data spring cloud.config.uri. We are lucky We are very fortunate on the spring cloud team to have committers from both Netflix and from twitter among others who contribute to the team to the code So his tricks Sorry zipkin hyphen service Okay, and we're going to open up the zipkin. Oops, uh, we're going to rename this file to be bootstrap.properties Okay, i'm going to say a zipkin service application and i'll say at enable Discovery client at enable zipkin server Okay, we'll spin this up now by default you'd probably have a backing data store But in our case i'm going to use the default which is just in memory when this spins up It's going to start important 9411 Here localhost 9411 Or not what did I do wrong? I don't have the zipkin ui sign of a gun so let's see start.spring.io zipkin ui okay This bit right here Okay Take two There we are now We have a distributed tracing platform a distributed tracing service called zipkin as I say it's an open source project that originated twitter We uh, we have this running off to the side. It's not in the same process as any of our code We have instrumentation. Thanks to spring cloud sleuth Uh that or we have listeners thanks to spring cloud sleuth on all the common places where messages enter or exit the system And we then have an adapter a listener a specific type of listener For zipkin But you can write your own listener if you want to to capture that information Our listener Broadcasts all the trace information in the spring cloud sleuth abstraction To whatever we want in this case it broadcasts it to a zipkin. I'm using Http to broadcast so the zipkin the zipkin service is itself an http rest api But I can also use spring cloud stream So I can publish any trace information using rabidim q or kafka And then have it delivered to the zipkin stream server instead of the http rest api doesn't matter We have adapters for for both http and messaging The result however is that you get this pretty user interface. It's got some data You can then analyze the data by looking at the bits. So let's go ahead and see this in action I'm going to drive some traffic here on the left I'll uh, send a few curls and I'll say cat reservation name I'll send uh Dr. Who again? Okay minus h content hyphen type application jason Http local host and then it's a 99 9 9 4 slash reservations Okay Refresh that so now I've made some requests both read and write and if I refresh this I can see that it is aware of our two services through which messages have passed because it has its spring cloud starter Zipkin on the class path I can now click on find trace and there I see a bevy a plethora a multitude of different traces I can see that I made a request less than a minute ago. So let's click on that one for information It has five spans and when I do that I can see that the total request time was 22.6 milliseconds Right for the request that happens. Here's the nature of the request. Here's the breakdown of the individual hops in the request's journey The way I like to read this is that the request At the reservation client was going to the reservation names endpoint Which was then going to the names endpoint at the reservation client Or the names method rather which was then going to the htp reservations Endpoint from the reservation client going to the reservation. Sorry the spring data rest api on the reservation service Right, I can see timings relative timings. I can see that these timings are you know, steric so I'm not sure if you can see but There's actually colors here, right? This is Well, that's a very saturated Okay, well just take my word for it. These are both colored and by definition These are wider Than the blocks below them. So you can see relative timings. You can see when the message is started Versus when the original message started the original message started here But spring data rest didn't get the message until uh six milliseconds or so Later on okay, I can click on each individual trace Each individual span rather and I can get metadata Here for example is a request log. It shows me that the message has arrived And you know, it was sent from this node in this port on the reservation client then the uh, uh, Received on this node, etc. And we can see the flow. We can also see service specific information These are called tags. These tags are you know, this is htp based information I can also get the same thing for messaging, right? So if I go here And I say find trace I can do a search here for the messaging input channel And I can see that I made a request here for the good doctor sire I can see that that total transaction took 43 milliseconds And I can see that the message started at the reservation client going to the reservations endpoint Which then wrote it to the right method which then sent the message Here on the output channel And then eventually it was delivered here on the input channel now In contrast to the other traces. There's a bit of a gap, isn't there between output and input? right Can anybody tell me what that gap represents? Sorry Yeah, it's it's in the broker, right? That's the time it's transiting So here it left spring integration. It left spring cloud stream on the output channel Spring cloud stream delivered it to rabid mq. Then we lost track of it We don't have instrumentation in rabid mq Then it popped out the other end Over here to be delivered to the reservation service. And then we start the journey again We see the story pick up again, right? So we can see that that that gap represents the time it transited between between one service to another in the broker You know, it doesn't say that it's rabid mq, but it's pretty clear that that's what it is So you get visibility now you get systemic behavior. What's happening? You also get this pretty graph and i'm a sucker for for pretty graphs So uh reservation service is used by the reservation client. It's been used 16 times. I get this nice ontology. That's cool now One thing we should underscore. This is not for customer service. This is not to to figure out Uh, what did what did jane do on the website last year? Right, this is about online telemetry You're not going to keep most of this data most organizations don't keep more than a few days Maybe a week at max worth of data Twitter for example captures one out of every few million requests I forget the exact number, but it's more than a million requests one out of every few million They don't need to capture all of them It's not like pokemon so You can uh, you can specify the granularity by by configuring an object of type sampler that sampler can do whatever you want You can say oh, this is an htp 200 the six the request was successful There's no need to trace it or uh, it's a static asset like a javascript file or a css file Don't bother. There's no side effects. There's no reason to trace that for example by default spring cloud sleuth Which which then of course informs what spring cloud zipkin does captures 10 of all traces I'm doing a demo. So I have my configuration in the config server for zipkin in pursuit to be 100 But that's not a realistic Production deployment. You'll very quickly overwhelm your system You're not trying to do this for customer service like I say okay so So far we've looked ever so briefly at a how to build a a production worthy service that lends itself to agility We've looked at how to benefit from the elasticity of a cloud and scale out We've looked at how to do the right thing in the face of topology or service failures And we've looked at um, how to instrument both individual nodes and how to address observability now I've got two more demos. I'd like to show you Do you want them? Should I just give up? I can give up if you want We got I wanted to talk about security single sign-on between microservices and Okay, and then I wanted to talk about Um orchestrating messaging based microservices now the last one I don't really need to do too much At all because the one the only the amazing my friend and my hero Carlos will be talking about that right after me So he's not here So don't tell him I said nice things He can never know um Anyway, let's talk about security In a distributed systems world We have lots of different kinds of clients and lots of different microservices And we want to make sure that access to these different microservices is secure fair enough good good point right in a single In a single node system You have to protect just that one node and everything that happens there in to secure But that's not true anymore Now you've got lots of nodes that all need to be secure and they have to say they have to share the same notion of identity right, uh This gets more complicated when you introduce multiple kinds of clients Multiple different types of devices each of which has different guarantees that it can make for security So the the problems become much more, uh, sophisticated more, you know more Expansive extensive when you move to a microservices and a distributed systems world We need to have it we need to have a single way to centralize identity information and then protect all these microservices Quickly and easily with that information with that identity provider We also need to take into account the notion of a client Because it's no longer enough to say this is josh We have to say this is josh on his html 5 browser or his android device Right josh by himself isn't enough One thing we could do maybe is use usernames and passwords I'll have all the clients transmit in the headers a username and password perhaps doing html basic authentication But that that doesn't solve the client problem It doesn't tell us who it tells us who's accessing their service But it doesn't tell us which kind of client and it doesn't really help us because we've now made it so that all these different components Are now passing around usernames and passwords, which is bad. It only takes one broken link One insecure link for that password to escape So you want to reduce the time where that service that username and password is visible So what we're going to look at as an approach for some of these problems is it's something called oauth How many of you have heard of oauth? Oauth is a big way as it's built for it's its purpose built for this kind of problem exactly It's purpose built for distributed systems on the open web and by that I mean html based services It works great even in your data center be you know, that's not open as well But it's it's purpose built to incorporate the notion of a client oauth Has at its heart two things it has the idea of a username You know the user who is that making the request and the client and this is very very important This distinction is very important imagine you're on the internet and You see one of those One of those button, what's that called? Facebook sign in with facebook I don't use facebook I don't even have I don't even know many people that do That's why I'm so happy anyway, imagine you see one of those sign in with facebook buttons And uh, you click the button and it redirects you back to facebook.com In your browser and on your desktop And when you're on facebook.com it shows you the certificate and the key and it's htps. It's the server's, you know ssl It's secure baby And then you when you look at the page that says the page says Do you authorize this third party service? to You know see your email information or get your your profile information or post to your wall or whatever To it shows you a list of different things that it's asking you to approve It says do you do you want to spam your friend your friend your family and your loved ones mercilessly and endlessly? Until such time as they hate you And you say yeah do it absolutely So you hit okay And then facebook.com redirects back to the third party api and suddenly that third party service knows who you are welcome josh, right? If you weren't already authenticated when you went to facebook, it'll say you need to log in Right, once you've logged in then it'll show you the list of questions of the the permissions that it wants Then it redirects back At no point however in that flow. Did you enter your username and password on the third party website? You did it on the trusted facebook.com domain This is because that this is because facebook doesn't trust that third party service with your username and password That's fair. They want to reduce the surface area of a possible attack They have put a lot of money and a lot of effort into making sure that The only place where you put your username and password is encrypted and it's secure and it's super super secure It's locked airtight Once they redirect back to your to the third party service that third party service doesn't have your username and password It has a token. It's a string It's an arbitrary string that the api the third party service can then hand in for information They can make calls to the third party service to the facebook apis rather on behalf of you They bear the token On your behalf in fact, that's what it's called a bearer token Okay, they bear the token on your behalf. So you have three parties here. You've got the identity provider facebook.com You've got the multi the third party api and you've got you Right and you is actually two things and it's the client and it's you the user This is called three legged oaf Right, this is very common for third party services over which you have No idea no cut no control over the security for example And it's also common for things for for which you don't trust the client An html5 browser for example Cannot be trusted to keep a secret I can view source. So there's no point in making my javascript app Identify itself I can say I can have a javascript app that says my name is the html is the uh Reservations android or html5 device client, right? I can have that but there's no point in in further confirming that by making the the client itself Absent you know forgetting about your username and password. There's no point in making the client itself Provide a password Because you can just view source Right because the html5 browser can't keep as many secrets It can't keep it can't make any as many guarantees as something like a compiled signed Delivered on the app store marketplace iphone application Right because it can't make as many guarantees. Maybe you say oh the html5 browser the client has less Less freedom They can maybe they're only in read only mode Whereas the the device on this is multi factor and it's Thumbprint and it's compiled and all that in this case they get admin mode Right i'm it's a silly example, but the point is uh, you can have different clients with different permissions different roles different scopes That distinction is very important Imagine now you're on facebook.com or you're on facebook's app the app on the iphone On the android device It's their native client. It's the client that they built. It's not a third party api It's not some other website that's letting you log into their website using facebook's information. It's facebook.com basically Don't you think it would be kind of a bad experience if you went to the app on facebook on your on your phone and it said Do you and then it redirected you to facebook.com asking you if you approve of facebook having your facebook information Do you authorize facebook to be facebook? I mean that would be a really bad user experience. Don't you think? It doesn't make any sense either Of course, they authorize facebook Of course the people working at facebook authorize the native client developed by the people at facebook to access the information They can already access your information Right, so they don't need to do the redirect instead. They just give you a username and password field They're not worried about the password escaping they can secure that right So this is a different kind of flow for a different kind of trust level and a different kind of client Oauth can do all these kinds of flow and different interactions and many more besides All right, so with that in mind we need to answer two questions Who are the users and what kind of clients are they using to call to call this service? We're going to stand up A spring security oauth based identity server an identity provider We're going to go back to start that spring then I owe my second favorite place on the on the web And we're going to build something called the auth service I'm going to use cloud oauth Okay, I'm going to use h2 the in-memory embedded sequel database I'll use jpa the java persistence api because I make poor life decisions I'm going to use the config client and uica for service registration discovery Uh, and then that I think I'll do for now now. I'm going to go ahead and generate this And what we're going to do is we're going to do two things we're going to answer first of all the question about the users This is just straight up old-fashioned spring security. There's nothing oauth-y about this Spring security. How many of you have used spring security? Okay, so the rest of you have as well. You just don't know it Spring security is the most ubiquitous security technology in the world if you're building an application On the on the jvm. There's nothing that even comes closed in terms of the maturity and the robustness of spring security spring security has been around for 11 years In some form or another and spring is itself the most popular technology on the jvm Which which means that for the for half of the jvms more than half of the jvms lifetime spring security has been the only choice That is of a reasonable use. There's been nothing before that, right? They're like the the support for security in jave e is a joke. There's nothing there. There's basic htp basic Authentication that kind of stuff you have to use native proprietary plugins So spring security has been very popular and a lot of people use spring security even if they're not using spring for other stuff Like you know 10 years ago if you're using struts you could use spring security as a filter to secure your struts app right So what we need to do is to tell spring security About our application about our usames and passwords and spring security has at its heart the concept Of an authentication manager an authentication manager is a generic thing It says given a request we have to answer the question. Is that request allowed to go through? Very very simple It delegates to a series of authentication providers These authentication providers handle all sorts of different use cases and you can configure your own of course Some of them will say is there an x519 certificate in the request chain If there is maybe we can use that to figure out the user and if we figure out the user Then we can answer the question based on that user's permissions Or maybe there's a username and password and it's an htp basic You know Authentication or maybe it's a form login. You've got a username and password field on a website and you've posted it to some endpoint Or something else. Maybe you've got an oauth token Right given a request. Is that request allowed to go through and if so You have to have a way of taking that request identifying who's making the request and then Mapping that to some notion of a user. There's a specialization of this approach called a user detail service A user detail service is an interface And it has one job. It says given a username give me information about that user And so we're going to go ahead and just build a very simple domain here We're going to build a simple application a simple jpa based repository to manage A few simple records that i'm going to store in my little database here. Okay, so i'm going to say class account And i'm going to you know, i'm going to make it a jpa entity so it'll be at id wrong one at generate value okay And uh, we'll say at entity And then we're going to say private string username I'm going to say getter And we'll have a constructor and another constructor this for jpa and jpa only. Okay And then a two string now I want to be able to work with entities of that type so i'll create a repository as we've done before with jpa and their reservation and i'll Manage entities of type long account whose primary keys of type long And then i'm going to provide my custom account user details service Right, it's a very simple contract. So we're going to do that here. We're going to say implement user detail service and again spring security has a a a uh multitude of different implementations that you can already use that will talk to pam or cabera So siteminder or active directory or ldap or oh, you know anything, right? You don't need to use this home brew style bespoke Identity provider based on some table data that i'm going to store in jpa Right, you can use anything that you want. There's already an abiding for that But as I don't have really the time or the interest to set up a a SAML based Environment for you right now We're going to do this laziness trumps You know thoroughness, okay So final account repository Okay Now what I need to do is I need to say I need to answer the question. I need to say when somebody asks me For a user details object given a username I need to return a value or throw a username not found exception I can never return null. That's not part of the contract The user details object in spring security. It's just an interface. It has uh a collection of authorities or scopes or roles or permissions Or whatever it's random. It's it's arbitrary. It changes from one backing identity provider to another But suffice it to say that a granted authority is just a bucket for a string Which can be completely different from one system to another And then we have the username and password And then we have the these booleans and these booleans all mean the same thing Is this user allowed to to keep using the system, right? There are different ways of phrasing it but for our purposes it'll suffice to map that information To a boolean called active Oh Sorry password There's this i'm going to have a boolean here called boolean active Create a getter there we are So is active so we're going to map all that to the same thing uh Cheat it a little bit here string pw this dot password equals pw Okay, and uh Create a two string as well. There we go. Good. So now I need to create some sample data And we also need to answer this question. We need to answer this question. So let me create some sample data first account clr implements command line runner. So again, I'm just using this as an initialization hook I'm going to inject my count repository And I'm going to say stream dot of constructor stream dot of I'm going to create some records. That's me j long spring So my username my username is j long my password is spring p web boot d sire cloud Dot and I'm going to map each one. I'm going to say for each string s Split it in half. So I'll have a tuple basically a a two element array and then for each, you know, two element array I'm going to write some data to the database. I'll say account repository that save Nope, not delete save new account okay And uh, we're going to have to answer the question. I'm going to say that the username is zero And the password is one, right? And I should probably have a second a third field here for the is active boolean So let's go ahead and update our account entity here Where did I put it? Okay boolean Good, okay. So there we go. Now we have a very simple account object. We're saving it to the database. I've got a Uh, easy detail service. We need to answer the question now. We need to say when somebody asks We're going to have to find the user by username Find account or find by username rather than it. So we're going to create that method passing in the username and we'll have We'll have that automatically added. In fact, we don't even need What we need is an account an optional of an account okay So this is this is an optional is a java 8 ism It says that I have an object that has that may or may not have an object inside of it It's not no It's a nice clean way to functionally work with that that the presence or the lack of presence of that of that data So I can say if the data is there that i'm going to map from an account to a user details object And we'll look at what that looks like in a second or Or or else i'm going to throw a new username not found exception, right? So shruggy emoji good So okay now We can use A concrete implementation of the user details contract called the user and here We're just going to pass in the username the password The uh, and then the the last the next four fields Are oh, yeah the next four fields are basically all the same thing is active, right? So this this this this and then the final field is A collection of roles or permissions or scopes or whatever in in my case I'm just going to go ahead and give everybody an account It's like the best episode of opera ever look under your chair everybody look under the chair. You all get a valid account okay now Once i've done that I've taught spring security about my users and it's a simple adapter around this table database table I've got not a big deal, but it's there now. We need to teach spring security off about our clients We need to actually turn on spring security off. So that actually turns out to be fairly simple We're gonna we have to say oauth configuration Extends authorization server Configure adapter and the configure adapters just a a no op implementation of three configuration callback methods And you can implement whatever whichever ones you want. We don't need this one We're going to say at navel Authorization server and this is a configuration class, of course And we're going to tell spring security oauth about the users in our system by giving it a pointer to the authentication manager from spring security Right. We want it we want spring security oauth to just share the information that spring security already has Which may not always be the case Let's change this constructor here okay So We go down to the endpoints thing. We say endpoints dot Authentication manager this dot authentication manager. There we go And now we need to describe the clients and I can have as many clients as I want I could load this information from a data source. I could use another jpe entity I could do whatever I want right if you ever go to developer.facebook.com or developer.twitter.com or whatever You can register new clients There right well in this case I'm just going to have a static fixed list of Clients that I'm going to store in memory So I'll say with client and the client's going to be called acme Although maybe it should be called reservations html 5 client or whatever And then the password will be rather the Authorized grant type will be the password flow. So I'm not going to require users to To do the redirect back to facebook.com. I'm going to let them pass in a username and password like facebook.app on the phone The scope that I'm requiring The only allowed permission that I'm going to give out Is something called open id which is completely arbitrary It could be post to wall or tweet or you know access your emails up to you to provide a valid definition for a scope And here I'm going to say that my client has to present a client secret But again html 5 devices may not be able to do that So it doesn't make sense necessarily for every kind of client Okay, now The final thing we need to do Is we need to provide a token ui or token info ui So think about how this is going to work We're going to lock down our reservation client the edge service when the reservation client sees a request It's going to see if there's a token if that token is present It's going to then call something and say what does this mean? I've got a token. Is it valid or not? It doesn't know that it's going to call our authorization server Right, presumably the client has gotten the token from the authorization server So we we need to teach the authorization server how to translate a token for a principal a java security principal java.security.principal This endpoint is different from each for each oauth authorization service Facebook's token info token info endpoint is different from twitter's For for facebook.com the api. It's called forward slash me Right, but of course it has information about your friends and your timeline and so on that's very different from the information from twitter And it's very different from this information. So it has to be specific. It has to be, you know oauth implementation specific There's no standard oauth Info endpoint. So you have to provide it yourself ours is going to be very boring We're going to have a rest controller at rest controller class principal rest controller And we're going to say that whenever somebody makes a request to forward slash user We're going to return a principal and the way we're going to return a principal is just by taking in the principal That's given to us now. This is a bit weird at first blush very confusing What i'm going to do is i'm going to go up here. I'm going to say at enable resource server This this tells spring security Spring cloud security that whenever it sees an access token, it should turn into a java security principal Well in this case We want that principal so that we can turn it into json so that we can give it to the third party api So we're saying whenever somebody authenticates this Whenever somebody sends a request to the user endpoint on the authorization server If there's a token in the request turn it into a principal And all we want to do is give it back to the client so that it can then use that To figure out whether this user is allowed to access the reservation client or anything else I know a little a little mind-bending, but it's it's what we're doing. Okay So I have this application configured spring cloud dot config That ui equals htp localhost 888 spring dot application That name equals off hyphen service And we're going to rename this property file to be bootstrap dot properties Et voilà, okay Once this starts up it's going to look at the configuration from the config server and it'll start up on port 9191 And all requests to the off server are going to be it's going to have a context path of forward slash ua a Right, I've just done that in the configuration So now in order for the token exchange endpoint to work somebody would have to make a request to ua a forward slash user For example, okay, so this is up and running on this port Let's go ahead and change our reservation client now. Let's make the reservation client Um Protect itself. Let's make the reservation client smart enough to reject requests that don't have a valid token as they were vended from this authorization server So we're going to say spring cloud start to oauth 2 Same as before and this is going to lock down our reservation client or edge service the configuration Will be active once we say at naval resource server, so there's this So we say at naval resource server We're going to start the we're going to restart this and then we can go to this Browser plugin called the postman browser plugin and we're going to send a request to the edge service And ask for a token So i'm going to say localhost 9191 ua token And i'm going to send requests. I'm going to send a request That has an accept header of application json a saw encoded h3 basic username and password and then in the body of the request Since it's a post i'm going to send The password which is spring the username which is jlong. I'm asking for a certain type of flow to be authorized the the Uh in password grant type. I want the scope called open id The client secret is called acme secret and the client id is called acme Okay, if any of these things don't line up with what the the client is configured to give on the service Then there's not going to be a valid token So i'll hit send and we can see that the authorization server gave me this access token This access token Is a type of bearer And if we've done everything right on the client, we should be able to use it now. So curl minus curl hdp colon colon forward slash forward slash Uh 99 99 forward slash reservations forward slash names. So this should fail On full authentication is required to access this endpoint. Good Let's try passing in the authorization header bearer paste That works Right Now let's prove the negative. Let's prove that i'm not just getting lucky. I'm going to tamper with a token And now it says invalid token Okay, so i've now locked down my edge service the way that the edge the the edge service knew what to do Because if you look at the configuration here I've got this one attribute here that says Security o of 2 resource user info uri equals local host 91 91 ua user So when the token arrived at the edge service it said, oh, I don't know what to do with this But I've got this I've got this link that tells me to go to the authorization service and exchange it for a valid principal Which it does and then it uses that to figure out what to do Okay So now we've locked down our services and we can use this we can inject in our reservation client now We can inject the java security principal in any of our handler methods Now we can see that josh is making the request As opposed to just asking for a random collection of reservations Okay, good. So now we've got a very simple edge service now. Let's talk about uh orchestration 10 11 12 12 minutes 12 minutes Maybe 15 Are you okay? I told I I thought I told you to lock the doors Lock them in Okay so Ever so briefly, let's turn to the last thing which is orchestration of distributed messaging based microservices Earlier we saw that spring cloud stream makes it easy to build a system You know composed of small singly focused components that are connected by a rabid mq broker or an apache kofka broker or redis or whatever We can we've standardized or commoditized the transport of messages between services We don't we don't care about the the the way that these things are sending data Instead we care about the payload of the data not the actual transport As long as we're using htp as long as we're using mqp. It's all the same This this isn't this doesn't change from one service to another because If we don't care about that then we can do interesting things because we can Orchestrate higher level more complex solutions based on these small singly focused services in the same way that when I go to the command line I can say curl, you know food at you know cat food at txt pipe grep Get some data, you know and then out that to uh results dot txt, right? I'm building Interesting solutions by composing small little services and then passing them through a commoditized standardized transport in this case standard in and standard out as modeled by the pipe operator in the bash dsl We can achieve the same effect for our messaging based microservices We've got a simple one here that takes messages from the input reservations Thing and then writes it to our database very very trivial But we can now reuse that Right now I could create another service and use spring cloud stream itself But we have another opportunity we can move even further up of the abstraction stack and use spring cloud data flow spring cloud data flow is an operator's approach to to building interesting solutions stream processing solutions I I spoke earlier about the center network here in singapore that data is infinite. It doesn't stop, right? There's it's not like it's not like the streets just stop working It's not like it's a a batch job. It's not like the streets only work from eight to five As long as there are streets, there are going to be sensors and there's going to be data So that's not really suitable for batch processing This is something that's an infinite workload. There's always going to be new data So you can use spring cloud data flow to build stream processing solutions solutions that never stop Data flows through these different components and they get routed accordingly and because we're using spring cloud data flow to create arbitrary solutions and integrations We can reconnect these different components. Remember I showed you earlier The spring cloud stream binding is in the configuration I can change the configuration and suddenly the messages that are leaving one bit of code go to another one Right now their the reservation client is set up to talk to the reservation service But there's no reason I couldn't introduce a third thing in the middle Because there's nothing in my java code that forbids that it's just about properties in the property in the configuration server Right as long as I change that and restart restart the the flow is now different Right, so if we understand that approach and we understand the potential, let's go see it in action I'm going back to start that spring that I owe I'm going to build a data flow service. I'll say data flow server config client you weaker discovery and generate So Yeah Okay application dot properties Come on computer There we are spring dot cloud dot config dot uri equals hdp localhost 8888 spring dot application name equals What are we doing data flow service? and we're going to rename this property to be uh bootstrap dot properties And then we're going to open up the data source data flow application I'm going to say at navel data flow server at navel Discovery client And then we're going to start it up now this data flow service could use a database Well, I don't have a database right now so it's in memory But you can use the stored database and we use a shell a very similar shell to the bash Shell for example to talk to the data flow service so I've got a shell jar here. You can get your own shell. It's obviously In fact, you can use spring boot to create your own shell, but there's no reason to it It's just something you can download or brew install and home brew Here's the 1.0 jar for the shell I can just download it And then I can say java minus jar shelled up jar It's a spring boot app of course spring shell application and now data flow is connected to my service But my service has no streams registered. It has no modules. It has no apps It has no tasks either I can use data flow to orchestrate tasks as well So a task is a a batch job It's something that has a well-known start and stop Whereas a stream is something as infinite. It doesn't stop But you can orchestrate and describe and launch arbitrary compositions of these different tasks and streams from data flow from the shell So I want some I want some services. I want some apps. I have nothing here. These are there's no lego bricks I have no primitives on which to build interesting services. I could of course Model my own I can create my own spring cloud stream apps are the basis For spring cloud data flow apps, right? So they're just they're just apps that send data on well-known channels or receive data on well-known channels So let's see good github.com beautiful microservices. Sorry cloud native java Okay t df shell Nope beautiful microservices config t df shell Commands So the first command that i'm going to run is this one I'm going to import the maven coordinates This is just a url that has a text file that has the maven coordinates for a whole bunch of pre-built Components that you can use in your data flow applications I just like having them around because it's nice to have a richer toolbox But again, there's no reason you couldn't just register the ones you care about and ignore their others So now if I do app list I can see four types of components have been registered here, right? I can see that there are Sources processors and syncs and tasks a source is a spring cloud stream app That sends a message out on an output channel called guess what output It's literally that it's as simple as the one I built earlier A processor is a spring cloud stream application that has both input and output channels one called input one called output That's all and a sync is a component that takes data in on a well-known channel called input And I have some pre-built lego bricks here. I've got a for example a source that will monitor a directory for a file And then I can take the data from that file and then pipe it to another component And then filter it and then pipe it to another component in my case I just want to let's say monitor a directory And then divide the file in terms of the lines, you know Split it by the lines and then send each one of those lines to my reservations endpoint My messaging my messaging uh service So I can say here I'm going to create a stream definition that does just that so stream This says stream create the name of the stream is called files to reservations. The definition is as follow I'm going to use the file source The source is going to have a few different spring boot properties These are just these are just spring boot properties the same thing as you would put an application of properties in your own code Uh, and so I'm going to say that the mode is lines and the directory Is equal to users jlong desktop in and then i'm going to have that routed to the spring cloud stream destination Just like I saw in the configuration for the config server spring cloud stream bindings dot Output that destination equals reservations and then i'm going to deploy it. Okay now When I say deploy What I mean is that spring cloud stream spring cloud data flow rather has started a java process And it is monitoring that java process here That java process of course is the file source So I've got an actual thing. There are different spring cloud data flow Deployers you can run on on cloud foundry for example and instead of running local jobs. It'll start new cloud foundry applications So here's standard out. You can see that's just a regular spring boot application Running on my machine monitoring that directory. So let's go to that. Let's go to the desktop here And you can see on the desktop I've got now an input directory. I've got an in directory that was just created. There's nothing in it So let's open up a text file here And we're going to say, uh, I need some names. So we're going to say, uh jade sergio carlos josh um michael That's enough. That's that's uh Save these names here in the input directory Okay, I'm going to save these in the in directory as uh in dot txt or you know Pivotal dot txt. Okay Let me get up my browser here. So I'm ready to do this Before I save local host 99 99 reservation names. Oh, right. It's not uh secured Uh curl Where's my authentication token? I need the token from the endpoint So we'll go here grab this Before I save it. I want to get this all lined up. So curl minus h authorization Bearer paste htp local host 99 99 for slash reservations Names So there we go. That's working. Let's go ahead and now save this file Pivotal dot txt and then run the command again And there we can see it says jade sergio carlos josh michael Right So now instead of me writing low level java code I can reuse these little components. My messaging based microservices can now be composed together in arbitrary combinations And I can scale them out individually. I can scale them out as cloud foundry applications right each each Each shell each component and that stream definition can be scaled out differently, right? Okay Now we've only touched on just a few of the things that you can do With spring cloud today. I feel like if we had more time we might be able to really to get into some real stuff You know today was today was uh A little too quick a little too shallow. I don't feel like we covered anything really at all But maybe next time Maybe next time maybe with some more time we can we can really chew off some me topics now I hope you appreciated some of what we talked about today. Did you like any of it? Yes Very good. I'm happy to hear that makes me happy. I certainly like this stuff Of course I like this stuff. I'm wearing a spring t-shirt and spring underwear Of course that of course. I'm a big fan But you don't have to take my word for it right Remember the name of the game here is agility the name of the game is to expedite the movement the progression from product management all the way through the production How quickly can you do that? A lot of organizations have struggled with this some have struggled and innovated And some understand the importance of this and they've done whatever they can to go faster So there are organizations That are using spring boot and spring cloud right now They're starting new projects on it and there are existing organizations Some of whom are very successful and very big that are also using spring boot and spring cloud and cloud foundry to go faster There's a small company in los autos in in california called netflix Netflix is using spring boot and spring cloud at scale one of the largest websites on the planet They have talked about the use of spring boot and spring cloud In neighboring china next door china there's a small company there called alibaba Alibaba Had a single day last year where they had 14 billion yuan chinese yuan i think in a single day Billions of dollars of value In a single day 24 hours Forget about web scale China scale right They are using spring boot and spring cloud at the extraordinary scale and they've talked about that as well publicly by do Neighboring and also in neighboring china. They're a small search engine the third largest In the world after google and bing and of course bing powers. Uh, yahoo, so it doesn't matter, you know, they don't matter so so it's So by do serves something like 600 to 400 to 600 million chinese users every day And they're using cloud foundry spring boot and spring cloud There's uh, uh, if if if amazon.com is alibaba for the west Then certainly rackerton.com in next door japan is Alibaba for the further east Right, and they're using spring boot spring cloud and cloud foundry at scale Right these organizations and many others besides have the money the motivations the resources the brain trust the people And the need they have every reason to solve these problems And if they didn't have these technologies, they would have solved it themselves And indeed some of them have solved parts of it themselves It is by the good graces of netflix's generosity that we can all build upon some of the lego bricks Some of the foundational elements that they've open sourced and then we've integrated and made cohesive in spring cloud But even netflix is using spring cloud not just the constituent components because this is far more integrated and far more Uh productive in environment far more productive and experience Right and that's the goal here is to be productive to make short work of the non functional requirements that gate Our ability to deliver value to the customer Which is at the end of the the only thing that matters So i'm happy to take questions if you've got them now i'm going to put up that uh that slide Remember that slide i told you in the very beginning that you're going to want you should record I even described it as a life best in an ocean Well, this is that slide again Uh the uh the workshop up here my friends That workshop is a three-day lab It has instructions and exercises all in that open and that open source github repository that you can just follow along with In that workshop we covered most of the stuff that we've talked about just now today So you can now uh now that you understand the motivations and the why Now you can uh hone in on the specifics and how I'm not expecting you to remember everything that we've just done today I almost don't remember everything we've just done today But I expect you to know why Why these things matter and that it's possible Right that there are solutions out there that that work for this use case. Uh, there are also I should change this I'm going to change it right now There's also a great resource in spring.io forward slash guides there you go, so I would definitely record that as well Remove the links These guides are focused 10 to 15 minute long Introductions to one thing and one thing all the in all manner of different topics You'll edit link. I just want this to go away Okay, you'll find guides on on all manner of different things be it rest or messaging or big data or integration Or security or or anything and far more stuff than I've talked about today You'll find guides on circuit breakers and climb side load bouncing and centralized configuration and surface registration discovery And again so much more. So I encourage you to follow those those guides are meant to be nauseatingly simple Exhaustive is not the name of the game here. They're supposed to be super super simple The goal is just to see something work to to make it wiggle You want a simple rest api it'll show you how to create a simple rest api with one endpoint You can build the all singing all dancing demo from there and you've seen me do that today This stuff is iterative I've layered on support for different concerns as I needed to But you don't have to chew off everything at the same time. You can take pick and choose the bits you care about Okay, my friends if you have any questions, I appreciate it. Otherwise, let's make way for our next presenter and Thank you so so much for all your time today. I appreciate it. And also thanks for thanks. We've been for for making the AV work