 Welcome everybody recording has started again. My name is the audience retorty. This is me on Twitter very small So you don't have to follow me retorty are e2t or I? I I think I'm the nerdy enough in the family to own retorty everywhere Even dot-com. I bought using my dad's credit card when I was good I wish there was a famous single singer or something retorty that I So let's talk a little bit about microservices and True all we all read the marketing followers definition for microservices what it means what it does not mean and I want to Specific about more of the capabilities that we tend to believe make the things that we do microservice and For me these are mainly instead of oh, it's the main driven development It's the size of the application. It's the number of lines So I don't want to get into that but more of the surrounding or ecosystem technologies are on the application that we write and for me This is just ignore this thing for a while. Just this for me are are key to Getting very close to a microservice or interact tech which is service discovery lifetime load balancing fault-solving gateway and distributed configuration and a lot of praise through Netflix that they did this Implemented this like I think seven to eight years ago All the Netflix closed-test knowledge that people do using love today That was created seven eight years ago for a world without containers without Kubernetes without Docker where they using they were using Certainly essentially virtual machines for all of that, but they done all this they done all this And this was really great work that they did especially when they started to to open source that I know There are some guys from Pivotal here that they helped Netflix on that. So thanks for that They made our world better. So I'm just gonna go over and quickly on this concept here So we can have a baseline and then a goal So first thing service discovery like if you are from the SOA days I think you are probably old enough to be here on the SOA days. I heard of UDDI in the versatile database Universal discovery something I forgot what UDDI means, but in the SOA days There was a discovery mechanism where you would create an endpoint and we say oh, this is your service This is where service at so for service discovery in a microservices world. It's very similar you ask someone Where are you now? And this is important because people are used to let's say defining things are based on IP So what's the IP of the database, you know? Okay, what's the IP of that that pricing service? What's the address of that pricing service? How do I get there and? Problems that if you think from a cloud native perspective things change right today JPS one tomorrow should be a different one right and so if you ask if you want to know where a service orders is Probably it's going to be like zero one today and service order is going to be a different IP tomorrow So if you stick with IP addressing for identifying or for discovering where things are that's going to be a little bit complicated And if you expand on this if you're developing a service orders orders for dev It's one it's one place orders for QA It's in a different place for a pre-prod and it's another different place even for production when you say that you want orders That means another yet another service. It's still an order service right so service is carbon and what they One of the abnastic stuff did is that allow you to have like a registry of things if I want to talk to an order service for prod Where where is it? You know, so you would address things easy. So they Another thing is the load balancing. It's very easy to understand load balance. We all know this but the perspective that we see For node balancing is it's more from let's say a front-end load the past You're going to your accessing a website or some some front-end capacity and expect that load to be balanced across multiple service But when you're talking about service to service communication, you kind of ignore load balancing You know, you worry about the front end of the service the website the main page the global load balancer But once you reach that entry point you kind of forget about all the other services that will suffer under load So what they did is that they had a client side load balancing Netflix I stated side load balancing From the client perspective, so the client would know where all the nodes or the service for a specific service war So that they couldn't load balance the calls between those services. So this was good So not only they had the front door load balance, but also service to service call Now the problem with this technology the way that's like implemented as being a client sign technology It resided inside the application You'd have to have an implementation for Java for node. Yes for php for go for dot-net So there wasn't this one single service the client side or a load balance another Important concept for me is fault-tolerant. So this is a very simple scenario. Let's say you have your cart It's our cart. It's the shopping cart and in your cart you need to interact with two services before you place the First is to get the promotion and then after we get the promotions you You get let's say to an order service for whatever it's here. And the way I've seen this and then it's myself is that Okay, so service promotion. It's not respond, right? So what is the what is the the common the common pattern when that happens? Service promotion is all responding it's taking time and then So things go back, you know So let's say With fault-tolerant that it allows us to query a service a few times, you know For example, I'm gonna try that service three times and if that service does not respond I'm going to say that that service is not responding and I'm gonna keep calling it because every single request It's gonna have to wait for the time out for a service that I know it's down Smart, right? So then of course You'll be querying every now and then to see if it's come back back up, right? But this will be the concept of circuit breaker So once this fails this many times of after that many tries you open the circuit No more calls go to to your your service promotions And if they're not dependent on each other for your business case You can still complete orders without suffering problems and the promotion. We all got this this idea of circuit breaking The concept is actually fairly fairly The result will be like no more waiting for failure and this is a problem, right? You know your service is down and you're still waiting for something that's going to fail. So that's why circuit breaking is very important and another Important when if you think about history and boy how they are today they can much in between as a gateway I know there are people from that that contribute on history and voyeur. So they need me say like no, you're wrong but But they the advantage of the gate is that the allows you to implement And I'm more that they centralize many some of the fault tolerance Breaking and even load balancing capabilities that you would have on client side, right? So if we're to apply the same scenario, right now you have a gateway that can do load balance From multiple services. So not only service part is going to load balance directly Normally service shipping is going to go back directly, but it will also any case for some reason Service promotion that goes down the gateway. No that this thing in law and no anyone calling that service will say hey That's that services down. So I'm going to give you a like 404 I'm going to give you a not okay right now instead of having to give you a timeout 10 seconds later So this is this is and the difference is in extreme So and then configuration that's pretty obvious, you know, like what information do I need in order to run but in in a dev ops or ICD oriented model, it's more complex It's not only like what's the the address of the things I need to connect because as you move workloads across Multi-point environment, they should do the code should or the code base should always be the same But configuration that pertains to that particular environment should be provided by someone So as you move from prod to from QA to prod the code should be the same always The show months to be saying okay now you're supposed to run with the prod config and not Having to change code and create and run with a prod Question so far was it too fast? Yeah, okay So gateway implements the Capability some of the capabilities that I mentioned before And it's the pattern so the gateway path like load balancing and fault tolerance, but outside of the application So before and this is an essentially slain in like Netflix That will do this thing like no balance, but it's inside. Yeah Yeah, so outside. Yeah Yeah Yeah Yeah, it's the general gateway. I would say the general gate. We see that implemented in API gate But if you see gateways for other technologies, it's general gateway capability to do that You could yes You could have and it's it's a since since the they're often different companies that do them and different providers You normally see multiple gates In place even though they're not necessary Which is why I see so much value in an issue an envoy that you can get rid of five to ten gateways that you have In order to have this one So if you have an API gateway then load balancer then a service mesh gateway or a service mesh proxy What if they could be saying that's my view so the Netflix or excessive technology that created is awesome, right? But as I said, it's eight years old that's not not saying that old technology is bad, but It it can get better. So there is opportunity for evolution in that it's language specific So that means that the clients there are specific for a certain name. So there's a library for Java There there's been some implementation for other flat It relies a lot on the developer and if you have a company that's only like a are 10x developers That's okay. We can you can trust the developers to do that But the majority that I say the traditional nine to five developers It's a lot of responsibility to put in them to also care about the infrastructure level components of the application So if a developer forgot for example to apply or to add The load balancer fault tolerance It would how they have to do that during your CI CD pipeline to verify that during our CI CD or some other test but then still being it mostly client-side technology relies a lot of this talk and This is very specific about Netflix for us Then not everything that they do on this project that I mentioned It's under maintenance anymore. This technology that was great from when I was invented and it was still still very useful right, but not everything is immense and on the Istio community today that I attended in Sunnyvale, I think Six seven weeks ago. They were on stage saying manifesting their interest towards Istio and envoys like even there We're pretty nice like you did this. It was great. Would it help this? It was awesome but when it's time for something else something different so So Accomplishing like all the Netflix technology they're creating pretty good. Do you think I want to talk about so much new and we're in this so Most are bite bite by micro profile a little bit is that there's a group of companies IBM Red Hat By our company tribe that they want to create they are creating a specification for Java microservices application so again language specific that's from specific But somewhat of a say JEE back for microservices that will address also circuit breaking With with a standard that others would implement so with this back like both red hat and IBM and Then and the other vendors that have a JEE server or a Java server They would implement the same spec and users should expect the same the same so this is a micro profile that I owe the website and Again, this is specific for Java which were Java developer if you're essentially Java charge you have a shop pretty good Now this is this is one of the it's not there's it's under the eclipse foundation and this is So the the group behind micro file wanted more speed in an addressing new New use cases for the Java platform. So that's why they created this so it's not a JCP Yes, it's it's a set of API's and there are some reference limitation for the API as well good Now enters envoy, but Christopher, where are you? You know the exact color code for the list color. I go go there It's well, I mean according to my research does this one here I Then like enters and why I'm you have a like people that actually You know, there's You can make sure all communication went through this piece of software that can see and control everything And if you have not seen that client's presentation around explaining what envoy is I think he's explaining right now to another audience. I really recommend it was on the event managed by Richard from data wire the one-hour session that I believe everybody should Thanks the rationale behind and boy and for me what caught up most was that if you can't actually See it. You won't be able to do anything to operate it or to Provide support. They're very keen on on tracing calls very keen on understanding everything that happens on a call, right? I know that what's that you probably set left one more than a hundred thousand servers More than a hundred Microsoft It's a big number in terms of Servers not so big numbers in terms of the number of micro servers It's common like red hat working on the enterprise is common to get to a company and say like we have 6,000 app If you were to break them into 10 services each there will be 60,000 So so there are still things that I believe there they have there's opportunity for a lot of evolution in both Eastern and white is to handle more Not not necessarily load with more configuration So, yes, so this is the big brother. It's watching the envoy. It's like the big brother It's all traffic going through this one central place Sorry all traffic goes going to this one piece of technology that is spread across Your network or a match would say and it's yes is pretty much like a dog. So I decided to do this so in the Red big brother or 1984 the brother is watching you So that who controls if the past controls the future who controls the present It's like who controls the traffic controls the access who controls the routes control the traffic That was my parallel that I did with Envoy to try to explain that is it's going to control traffic means How many times can you do this operation? Who are you? Can I cross now for the route? Who should I send you now? Yeah, when I created this slide Man, this could be a very nice joke. No All right. So it's still a little bit of a show that I have like 15 years. Hopefully my demo Started by am Google and left things people crime the am Google and lifted are here a lot for also open sourcing this to the world awesome I love this and the the announcement was made on May 24 and other companies like pretty much I think they're all here except for Tiger and we've clouds that were all represented here That were part of the law so data wire was people toys here where it had was part of the law So yeah, thanks So you still what does it do pretty much? Similar to some of the Netflix was technologies, but in a different way so service discovery load balancing for tolerance It is a gateway has The pieces of you still are this one represented here The most important point I guess this is to understand that in every Container that you run on an every call It's going through a proxy all the time But if you're talking to a web from web servers web service to another web service or to a recent point The calls are made between the proxy a proxy talks to another product and then we shall so that is let's say What technical standpoint that is the main? I Think Architectural difference of the service mesh that the proxy it's in every communication and no Every communication if I'm lying, please let me lie alone. Help me on my life. Okay, so There are other components is still all responsible for authentication So we still today runs on Kubernetes. So it's going to read service accounts from Kubernetes And see if you can do something, you know Mixer is the There's a there's a new one that's a galley galley This story is also on the API that you talk to to execute the operation It's it's the place where I avoid goals to verify. What are the policies? What are the things that should be applied not? Pilot for configuration they added galley also for configuration So pilot was called manager, I think they just changed the name from One five For zero fifteen to zero sixteen But it's important there is a proxy in every communication There is a proxy never going to this is this is the key point here Okay, right and from a deployment perspective. I'm going to show you a little just a few minutes. What's happened? So This is the part what's going to get complicated I'm going to try to show you my the things I'm doing on my phone. Sorry. I'm gonna have to do this Alright, so of course I have my very nice script document So the demo that I'm going to do uses docker machine I'm going to do some from scratch. Well, not 100% from scratch already have a docker machine Create that have to recreate the docker machine So I'm going to be using a tubernet is 1.5 1.5 for the demo should work on 1.6 And it should work on on 1.7 as well the main changes. It's been on third party resources And to burn at it. I'm actually going to use open shift for this But you are that open shift is just to burn at it with more of things in it. Okay So all the command line you see I'm using open shift command line But you can be the exact same command line for that All right, so I got my docker machine on and I'm going to start an open shift cluster The version of that I'm using is 1.5.1. It's the exact same. You can be yeah, and it would work exactly the same on mini-cube It's me it's not mini-shift, but it could be mini-shift problem when you want to do like everything from scratch, you know I could have just have the environment running, but then First level down when you have like everything ready to go and just do this. Oh, you're out into the different versions But this hopefully will bring an open shift environment The version 1.5.1. Yes, it did I'm going to add a lot of policy. This is the thing. So there's a So the open shift by itself when you when you stop it or even when you run like open she's locally on your machine It comes with it comes close, right? So you can run root containers that require root It doesn't allow you to do crazy thing of the environment because that's what you're going to do You know they should have environments that are protected. So in this case, I had to allow a network policy on that It's going to Allow an IP table is execution so that every for all the traffic in the pod goes first Go first to the product No, so the the the IP tables that it's going to be written. It's inside the pod So the communication in the pod is going to be written inside the pod So the communication in the pod inside the pod It's going to say all communication in this in this in this pod inside the pod is going to go through this this other container this product So the same father to containers and I want to talk about a bit But it's good all these questions that are very good. So again more Everything is from scratch. Well added lots of service accounts to allow things to happen Now I'm going to actually deploy you see itself for me to use Grafana a service graph and zipkin I'm not sure I have time to show zipkin, but I'll deploy that as well but the first thing is he still and then Prometheus and then Grafana and then so I'll I had all the Docker images already Loading on my machine. So that's why I can see so if I come here and I see So this is my open chip cluster or kubernetes frustration. We would like to see so if like But Same thing, right? So see get pause has a little bit more data in it. So alright Not so well But it's the same thing sometimes I'm interacting with the same So the thing that I'm doing is you know open shift I'm doing everything on the project called default because in open shift. We have a concept of network Network name spaces and for multi-tenancy Traditional open shift environment. You wouldn't have access from one project. They're they're isolated tenants But the project to default on open chef can see everything outside tense anything on any project. So the Deployments that I have here are To deployments open shift has a Docker I just we also have a router This is our version of ingress, but we also have especially if I make it bigger, you'd be better Here so we also have this the plan. So these are kubernetes deployment They're not Deployments will feed as we call an open to their kubernetes deployment in this are all things that represent So the manager as I mentioned changes its name to pilot. So now I'm going to install or sorry to just to Install the demo application. So these are the things that you still need I'm going to start on our demo application and as you can see here more more deployments were created This is where it gets more interesting, right? So you can see you can see on the right Figure that you have more deployments. You have now a product page a reviews page Three versions of reviews a rate you can have so these are all kubernetes deployment and if you go to resources and applications and pods you see that Some Some deployment they are they're coming up, but they have two containers, right? So this is the thing when as I said all communication happens proxy to proxy So when you deploy an application to be a job application in this case, I think the reviews They are Java application, but everything else is I think it's an old application But inside your pod there are always two containers one is your actual application and the other it's the proxy So we can see this we can come to a pod. Let's take this one here for example We can go to the logs This is the review service This are logs from the review service and these are logs from the from the product But these are all things generated by We could open a terminal and them as well and do things, but I don't need to do that All right, we have auditing so and another thing that happened here and see if I can show this in the pod description So not here, but so Inside this pod description, it's very hard to see right now So there's there is this in each container, right? So in each container is something that was added reaching to you to Kubernetes It's it's something that it's run right after a pod is created So the first thing on the life cycle of the pod is to run the need container And this in each container is that's going to make the IP table call for that pod and say all Communication is going to the proxy whether you want it or not, right? So there's a need container. So it's a container that spawns up runs once Changes the configuration IP tables for that specific pod goes down and this this this pod has two containers one is the actual application container and the other is the Yeah, so I'm gonna need more time. I'm gonna go over a little bit some more hecky stuff here so what I'm doing right now is just a I'm using ingress and I'm routing the ingress part in Kubernetes to a node port So that means that my open-shift server my fructal machine is going to respond on the ingress part on this part that I just used here So this part is an old part. I'm not using DNS for any route. So if I reach that That IP I'm going to get the ingress router, which is also an envoy, right? So the first time I hit the servers for my application. I'm already hitting envoy. I'm already hitting proxy That means from that point on all communication is going to go already through the mesh And you'll be able to have first traces from the moment you get into the platform in the moment it goes up. Yes the ingress is So the ingress is for the cluster itself all ingress in the cluster Not just yes, so and could be in not just for a specific server So I'm getting the ingress to the cluster. So I'm getting in the cluster and from that The router will the end of our proxy will define what service you want to access. You see this This is this was just to facilitate DNS actually This was just to facilitate DNS. So I'm mapping I'm mapping the ingress the ingress IP to a To a node port. Did I answer your question? That's just easy for them for position. Yes under that that that makes sense but since I'm going to use ingress to the router and The router is going to then send traffic to the different multiple services that I have Router is a pod. I'm saying I'm sending a note or odd. That is the router Yeah, just make it. Yeah, I'm outside. So this is my Mac accessing the machine the server, you know I'm not as a sage into it. I'm from the outside of the machine. Okay, so 200 so it's working. So now we can test the application 801 So this is a demo app, right? Hopefully I'll be able to show more The demo app it's a very simple application that is using multiple services and it's getting Getting reviews from a different pod. So there's a there's a review service There are actually three versions of a review service and then each time I refresh the page I get a different version of the review service. So now I've got the review service that has colors Now I'm getting a review service that no review service now I'm getting Different reviews. So and this is the the route this route has been done by That in this case there are three services three pods under the same Kubernetes services And they are all being let's say that's the traffic load balance. There's no specific policy being applied that So the prox is just saying yeah, I just told balancing Service that are there and we're fine Now if you want to send for example all traffic to if you want Command here what it is later That Traffic going to be sent to be one. So whatever takes a while to refresh So whatever if you want is which is probably nothing nothing. I'm requesting all traffic to be so So there's a route rules for rating. Let me see what this So there's a route rules that were created So now I have that from the product page I have a route rule So I have all I have a route rule that's sending all traffic to view one of my rating service So this is for the destination me being rating service. I have Everything being sent to view one. So I changed the configuration all the process and it is working. There's nothing there I'm gonna do something different I'm gonna Change the rule and send all traffic for the traffic for a specific user with different route Create another rule. So in this case if I log in as user Think this is That means that logged in as Jason. I now have a rating service But if we go again and show the route rule the new route rules that we created now, there's this Regular expression that's looking something in the cookie if the cookie has the username Jason could be anything It's going to send that to V2. So the everybody that Whenever this matches is going to send that route to V2 Again, this is because there's a proxy that's controlling and managing all the communication from point to point It's going this way. Yes, they are there. It takes it takes it's not a it's not a Atomic distribution. So there will be some inconsistency between between The time you execute this command and to all endpoints get updated with that I'm sure I hope I would think there are ways to do it in atomic manner But then you've got into a consistency problems just like you get anywhere else Yeah, yeah, I It's less than seconds Because it's the time of following the mixer with the configuration and getting the practice updated I mean any API management tool has this Problem or characteristic today, you know And what you decide is that I and this is what say one of the things that Netflix would do is say like I think it's better to allow someone that doesn't have a service to see that That prohibits someone that has to not see something. So that's that's just how you You define your problem, but there are I mean I wave you over on my time But there are other things that I at least I want to mention That you still allows So it allows you to do fault tolerance. So the circuit break me safe to say I can wait seven seconds for this request if after seven seconds I don't get a response then the proxy will assume that it was not a So it can define that in a proxy level you can define also rate You can say this user for example can query three times Each second, you know Which is a lot related to API management and many many other things that Yeah, I'll just wait time for maybe like one or two more questions what I'm ready, but you don't like advice part of these invoice the proxy That does the control and sees the traffic Istio is a larger umbrella project that contains and boy as well that uses and boy as well. So and I have not but but Since let's say the use cases that I saw for envoy They were on I would say a fairly small number of services microservices that I've heard published a hundred number Lots of service though like but a hundred microservice That means that you likely need configuration for a hundred micro service for 300 if you have multiple environment for me This is still fairly small number and I think it will go through the same say Problems the Kubernetes went when they started to scale the number of objects in memory For things, you know, it's not not not the not the workload But the maintenance of the configuration of the Because the limitations in in our Kubernetes clusters today, it's not It's it's the configuration right so how much configuration can you handle and and how much consistency will play in that? You know, I know like just be aware that this is very new technology. There's the version the current version of these two is 0.16 It was like launched Eight weeks ago So you a lot to happen there. Yeah. Yeah. Yeah, I think that's the idea of anyone that's doing this I know Apigee's thinking about this. I know people from data power there are here. They're thinking about this Red Hat we are thinking about this as well, you know It just doesn't make sense to have lots of proxies, right? You have one That already knows things they can apply the same rate and limit that would I you essentially need a configuration tool on top of that to distribute user keys to Do billing to do metering and such but the proxy is already there. No need for another proxies Or another gateway. You should use this word. So No, so it's still today runs in Kubernetes, but uses Kubernetes to do service discovery Right. So if you have a Kubernetes service, that will be an issue of service Kubernetes services are DNS addressable although issue does not and it does not use The service IP it goes directly to IP of the thought Looked on But yeah, so service discovery it's done by a Kubernetes essentially it the service registry is inside so when you deploy And you have a service like the service review It's going to query your net and say like what are the pods that? have label selector for the review service and Then you still have a list of pods and sales. So these are the people that are responding under a review service So that's how it knows list of people like list of pause that are Yes, yeah, yeah, yeah, yeah, yeah, and and there is work to have Meshes network meshes that span different boundaries, you know, there's the boundary of the mesh that ends inside the Kubernetes cluster But there is work to expand my boundary meshes and to even there's that now There's a ideas to create mesh names faces for example because All mesh that I understand it's It's not namespace, so it's all part of the same you define everything using policies but there is not the concept of Mesh organization, so so the the applications inside this mesh can see themselves and they can look up themselves The organization the applications inside this other organizations and cannot see just like kubernetes has namespaces to organize I think there's 24 namespaces in meshes as well Especially just think of OpenShift online that each project is its own namespace Of course, if you had the mesh you wouldn't like people from other projects to see your your pods. It's very basic Hopefully I answered your question one more. Yes, I would say it's not a conflict is just work that needs to be done Because the technologies are new, but you are correct. So there's a cube Cube Proxy running in each kubernetes node that interacts to the IP tables from on a node level Right, but this was on a pod level just have the mind. It's a different level, right? So this was on a pod level. So the cube proxy does it on a node level And I do understand that there needs that that what does work needs to be done there The cube proxies actually not a proxy for example. There's no traffic going through the cube proxies. It's just a Management thing in the node Controls and talks to the node and reports back to the API server Good. Thanks. I know I went over but the questions were good. So, Shreeter next