 because, as I say, it's been a strange day so far. Myself, I'm a freelance consultant and trainer on cloud-native solutions. I'm a Brit living in France for about 27 years, so I'll soon be a Brexit refugee, no doubt. And I run a local Docker meetup group and Python user group. And this is Grenoble in the French Alps, where I live. OK, so first of all, so why wouldn't go from monoliths to microservices? But first, a bit of history. So it's interesting, if you look back the last 20, 25 years, it's been quite phenomenal the way computing has evolved. 25 years ago, we were still running bare-methyl servers and I guess if you were running virtual machines, it was because you were running IBM mainframes. I don't think there were many instances of virtual machines at that time. Then VMware came along. So then, virtual machines became the new thing, running on hypervisors, which led on to cloud, with a choice of hypervisors available. So cloud search more elastic systems. Then containers came along, which existed before Docker, but Docker made containers really usable. And that has really changed the system again. And the new FAD, I would say, I believe in it, it's real. The new thing I would say, new kid on the block, is serverless. So we're going to talk about microservices, which is a bit, actually, you could do microservice on virtual machines or containers or serverless. And one initial point I would say, well, the future will be hybrid. We will use all of these technologies depending on different constraints and important choice. OK. But why, still, would you want to move away from how wonderful monoliths have been serving us so well? Well, these monoliths, more or less, they're big bricks of software that are developed in some sort of waterfall development mode, where the new release is like six, 12 months out. So at that point, the official release comes out. In the meantime, if you need to patch, for example, it's a bit of a complicated process, both for the vendor and for the person deploying. And generally, you just don't have very much agility with monoliths, even when it's an end-tier architecture that you still have fairly large pieces of software that's difficult to deal with. And these are particularly ill-suited to large enterprise scale or web scale. And obviously, with the sort of applications we have on the web today from the likes of Google and Facebook, basically the GAFA, we're running at enormous scale. And so microservice really makes sense at that scale. So as I said, I mean, these are big pieces of software with components that are tightly coupled. Generally, the components are linked together with libraries. You can't separate out components very easily. Of course, those libraries could be reused, but there's still a whole sort of build process and packaging associated with that. And so generally, it's difficult to move, difficult to evolve, to innovate. Microservices, on the other hand, we're really talking about splitting up an overall application into components where each component does one function and it does it well. And that has several implications. That means that when you're working at web scale, for example, you might have some parts of the application, maybe a database, which actually might be just one instance, but other parts, which may be a front end web server, which you want to scale based on the amount of traffic coming into your system. One of the advantages of cloud, of course, why do you use cloud at all? What's the big interest? One interest is agility, ability to fire up new applications on the fly, but especially to scale out based on the traffic coming for the network. And that, in particular, allows to mutualize the resources in the cloud. So you want to be able to scale up some components when there is need for them and scale down when there isn't need to make efficient use of cloud or data center resources. So microservice architecture components, that they're lightly coupled. What that means is they're no longer linked together as a binary, but they're interconnected over the network. They can be scaled independently, and they can be deployed or upgraded independently if you do it right, of course. It isn't free. And being able to deploy and upgrade independently is really important to be able to quickly evolve your service. And you can even replace implementations of some components. Because the components, I'm saying, do one thing well. You might decide you might have an implementation of, let's see, a web server. It suited you well to do that in Python in the beginning. Now you want to go with Go, or maybe even C++ or Ultimate Speed. And with microservices, you can make that sort of change. So advantages, say, a component of a microservice does one thing well. And this means you can have small focused project teams working on these things. Who really are focused to just guarantee the functionality of this component. They might be working on several components, but each one they have some focus on. It's easy to scale these components, to deploy them, to test them, at least in the sense of unit testing. And so easy to evolve them. And also because they're separate components where it's just network communications, it's easy to compose new services from microservices. And so each component, as well, you can choose to re-implement in best of class technology, change the implementation language, or maybe change the implementation of a database, go from SQL to some no SQL or graph functionality, whatever. So are there panacea? No, of course. There's some cost. Of course, there's great complexity. Technically, you have more components and you have to manage more things. And you need to orchestrate these microservices. You also have greater complexity in your organization because you have more teams to manage. And then operationally, in terms of monitoring, debugging, and end to end testing, so for your overall service, these tasks are more difficult. Network communication now is critical. So you really need to manage that in terms of latency and detecting where problems are. Maybe doing what we call circuit breaking when a problem is detected to isolate problems from a particular area. So there's a sort of a mindset of design to fail but to keep working when there are failures in your system. And then be aware, microservices are useless if you don't adopt best practices. So it's really essential if you're doing microservices. You really want some DevOps processes. You want behavior and test-driven development. You need to be very rigorous about implementing the APIs around these components. You have to guarantee your interfaces. And you need to provide stable APIs with some degree of backward compatibility. So providing you stay with those best practices, then microservices are a good solution. So I mentioned one thing is that we need to orchestrate. So basically, now as these systems scale, your application scales across maybe a data center of hundreds of nodes, maybe even thousands, or maybe with a cloud provider. As there are more and more instances of each component, it becomes impossible for an operator to manage that. And you just wouldn't want them to do that. So you want to be able to place your components on different nodes to ensure availability. If a node goes down, for example, that your overall service still continues to work, maybe respect some affinity, non-affinity constraints, as well to guarantee the availability of the overall applications. And you may want to take advantage of specialized hardware, maybe a monitor which containers are not functioning and whether they're started and ready to go. And also, how are you going to upgrade your applications? Again, this is somewhere where an orchestrator, like Kubernetes, can come in. So we need orchestration. And I'm going to assume that we're part of the Kubernetes. I'm going to advance, I'm being a bit slow here, so I'm not going to present you Kubernetes. If you're in this talk, I guess you're aware that, basically, Kubernetes is a cluster management system coming from Google initially, based on their experience in-house container orchestration. The basic architecture has a set of master nodes, which is basically the control plane, and a set of worker nodes where your actual containers will be running. I say containers, but actually we run pods in Kubernetes. We'll come to that in a moment. OK, I'll skip over most of this. I want to spend time in it, but just to say that Kubernetes is a declarative system. You say, for example, you want to run an Nginx web server. You're going to say to Kubernetes, OK, these are my constraints. I want, let's say, three replicas, three instances of this. And it's Kubernetes that then we'll see what's the utilization of the different nodes, and it will decide where to place those instances. OK, I'm going to skip. Just to say so, a pod is a Kubernetes concept, which represents one or more containers running together. The idea is that you have one main container, which provides your functionality, let's say, web server, since staying on that one. And any extra containers, we call them a sidecar, which are basically providing ancillary functionality, helping that main functionality. So we'll see an example of that with Istio. OK, so as I was saying, I had intention of doing a lot of demos, but it's been a weird day. I will show just, nevertheless, that this is a visualization of, in fact, I'm running Docker for desktop. This was the environment I was going to show, but my Kubernetes cluster on digital ocean, it went away yesterday, so I recreated it. Today, I still had connectivity problems, but OK. Just to show, I was hoping to demonstrate across a three-node system, one master and two workers, but I would do the same demonstration based on Docker for desktop. We enlarge this. Probably can't see much. OK, is that readable? OK, so I'm going to just show how I'm going to basically deploy a Redis key value store. I'll tell you the right command. Basically, you do the kubectl create or apply of the YAML definition file for creating deployment. And what we're seeing here, this block is a set of Kubernetes objects that have the same run tag or app tag. And what we're seeing is a deployment. I'm saying, OK, deploy me one instance of Redis. OK, that's a bug. You see it twice. And we have a replica set, which we're not seeing. Sorry, and that's the third one is the part. It will become clearer when I run the example. I'm going to run a flask application, which would normally be a front end. OK, it was just red there as the container started up. If it was having to take time pulling images, it would have stayed red a lot longer. And I can do things like, I should show you. So the YAML file for creating those, basically you specify the number of replicas up there. So the initial version I did had replicas one, obviously. And now I'm going to rerun essentially the same YAML with replicas four. And we can see how it quickly creates several containers, several pods, rather. If I had my other environment working, then these would obviously be distributed across the two worker nodes that I had. OK, I'm going to move quite quickly. So I've got some slides just representing that deployment that I just did, but we'll skip over those. OK, so I've tried to organize the presentation by in two main parts, really, deployment strategies and architecture design patterns. Those might not be the best names for these categories, but naming is the hardest thing in computer science. So you'll have to let me get away with that one. So first of all, what do I mean by deployment strategies? So basically, how are we going to roll out our services to our data center or our cloud? And then how are we going to upgrade them and this sort of thing as services evolve? And so there's a group of things I'm putting under this banner. So there are deployment strategies for sort of evolving a service, upgrading a service. And we look at some of those examples. Some of these, most of them can be implemented by Kubernetes alone. The more advanced ones like doing canary rollouts or A-B testing rollouts would require something like an API gateway in front. So I'm going to look at service upgrade strategies, health checks, and what we call the strangler pattern. OK, so first of all, for service upgrade strategies, so really saying, for example, the flask and redis that I just deployed, how can I go about upgrading, let's say, the flask application in a way that it makes sense to the overall application? So for example, the first way of doing this is a thing called recreate, which is fine in development but not very good in production because you stop the old version of the application completely, and then you deploy the new one. So obviously, that's not exactly very production friendly. Nevertheless, it is a deployment strategy, which is fine for a developer on his laptop. The one which is used by default in Kubernetes is ramped, which basically Kubernetes will deploy a new version of an instance. Once that's running, it will stop an instance. So if you have, let's say, as I had here, four flask instances, it will upgrade one when it's ready. It will stop one of the existing ones, and so on. And so during the upgrade, we will see we actually have a mix of old and new, tend to say blue and green versions of the service. So that's one way of doing it. Advantage of that is that you have a service running all the time. It's not like the first one where you stop everything. But you get into a mixed situation, so you could have requests coming in that we treat by the old or the new version. And you might even have some incompatibilities of API to manage. That's something you have to design. OK, so I will show that one in a moment. Another way of doing it is blue-green. So blue-green, if you have the old version running, you start up the same number of instances of the new version. And once they're already, you then switch to the new version. So that's great. Sounds slightly ideal, except that, of course, it means you need double the resources during the switchover time. That might not be a problem. Canary is basically saying, OK, I'm rolling out this new version service. I'm not completely sure yet. Maybe I want to just allow it for 1% of the users, something like this. See if the phone rings and roll back. So try it and see. And for that sort of functionality, you generally need to implement that in an API gateway or a service mesh, which we'll come to. And the last one is AB testing, which is very similar to canary testing, but with the difference that you selectively determine which version of the service user will need. So if you have a service in production and you actually want to operate it just for certain class of users, for example, just with them you want to test, then you could do that. And that, as well, would require an API gateway or service mesh to implement that. So those are the choices. Now we'll show an example of ramped, which is the default Kubernetes way of doing things. So basically, I have another YAML file I'd already created. It's, I should actually show it. In fact, if I do, I'm going to be a bit strange. I don't have a copy base. OK, if I do a diff with the one I just used to get to four replicas, we'll just see that I'm using a different image in that file. I'm going to pass from v1 to v2. So I'm going to perform that update. And we should see, as I talk, there will be a moment where, on the right is the four flask pods. All in version one, we will see that we have more than that, more than four pods during the upgrade. Let's go fairly quickly. So we see new pods red until they're up and running. OK, it's all gone. OK, it's a shame I have a bug on my visualization because we would have seen the replica set. And what we would have seen is there's a deployment object, which is FlaskApp. And then we would have a replica set, which represents the version one of this FlaskApp. And it is the one that creates the pods. And so what we would actually have seen is still the same deployment, but we would have had two replica sets. The old one now with zero instances and the new one with four instances. And then these are the new FlaskApp pod instances, which you can see in version two. So just to be careful where I click. That was just to give you an idea, show you something real around one of these strategies. OK, so that's the demo I just showed you. OK, another thing to do then is to validate. So containers, yeah, it's fine just deploying them, but we need to be sure that they are healthy before we actually put them online. So Kubernetes will not want to root traffic to containers or pods that are known not to be functioning. And so how do you do that? You do that through health checks. And there are two types of health checks, liveness and readiness. So I have a definition. Liveness basically is detecting that the pod's containers are themselves just up and running, whereas readiness is detecting they are actually capable of serving traffic. OK, and there are three types of probe that you can do. So either a command to execute, check something about the container, or an HTTP request to try. So typically you would just have a standard URL but with a slash health or something like that to test, or a TCP request to try. And it's Kubernetes that's going to basically poll your pod to see if that health check passes or not. Likewise, there are so readiness probes. So the first one, although there are things like an HTTP request to try, this might be just a test to verify that the pod is there, that it's up. But not necessarily testing your application yet. With the readiness probes, the idea is that it's actually going to be testing the application. So it's a second stage of readiness. And these are the same probe types as for the readiness probes, same type, but maybe some different URL should be tested. OK, so the definition of a liveless probe, liveleness probe is something like this. So in this case, it's something that you exec on the container. And you can specify the initial delay when to start testing and during what period. And Kubernetes could determine that after a certain time this doesn't succeed. Then we should just destroy that container or that pod and create another one. And readiness probes are exactly the same. It's literally just the keyword, liveleness probe. It's replaced with the readiness probe. So I just wanted to say, OK, that's an important thing about the way Kubernetes operates in deploying pods. The other thing is, so how do you go actually from a monolith? That's actually the subject of the talk. So how would you do that? It's not easy. I've said it's a monolith. How the heck move to microservices. You can reimplement everything and six months down the line. Yes, we've got the new service. Exactly the same functionality as the old service, but you've lost six months. So you probably want to have some more incremental way of moving to microservices. And basically the strangler pattern provides a possible solution to this. The idea is that you would progressively take out or disable parts of your monolith application and reimplement them in microservices. And so you'd be in a gradual transition until you have lots of microservices and maybe just one small part of the monolith which is left functional, which is being strangled by the microservices. That's what the strangler pattern is. It's basically a migration deployment strategy. And this is something I took off the Azure documentation site just showing the migration from left to right, from lots of legacy in a bit of the modern microservice implementation through to having just the modern implementation where you finally you've strangled the original monolith. OK. The other thing I want to talk about is I grouped as architecture design patterns. OK. So I'm not talking about like standard component design patterns, Java design patterns, whatever you might have within your components. That's another subject entirely. Microservice themselves is worth mentioning. Microservice is a design pattern. Nor sidecar pattern, which we mentioned exists that you can have second, third containers within a pod. Now, we're concerned with basically exposing services and ingress functionality, so providing access to services within the Kubernetes cluster itself. I'll come onto that. And then externally around your cluster, the use of things like an API gateway service mesh and even more advanced API gateways based on what we call the API gateway pattern. And I would say this part is real war zone, that last bit of API gateway service mesh. There are several API gateways really well established, something like NGNX, HAProxy, and so on. But now we have service meshes coming from Istio, LinkerD, and so on. And there's quite a battle going on between the gateways themselves, between the service meshes, and between the two of them. OK, so that first part, looking at how we access our services. So we've deployed them as a set of pods, but so far we don't actually have a way of addressing them. Each pod has its own IP address and maybe is exposing some ports. But those are specific addresses to a particular pod. And the very nature of Kubernetes or these container platforms is that pods of containers are coming and going, they're dying, or they're being scaled up, scaled down. So you certainly can't depend upon the pod IP remaining valid. You could attach to a host with something like what we call NodePort or HostPort. But then you're exposing your infrastructure. Actually, the same thing if you were addressing directly the pod as well, you should actually be on an isolated network and that shouldn't be something you should be extroving into the exterior. So what we need is basically service endpoints, a well-known address we can use to address our application and then the platform will take care of routing that through to the pods that implement that service. I'm sorry, I've said most of what is on here. So the basic principle is we have this service of some sort which is going to access and load balance across the various pods available in our system. And our end user application, he should just need to know the service IP in port or host name. This is where you would have reverse proxies that map to a nice application name through to some internal address on your system. So there are various ways of doing this. NodePort and HostPort are almost the same thing. I don't remember the difference but essentially it's providing an access direct to the actual cluster nodes of your system. There's cluster IP which is an internal address which is addressable by pods within your cluster purely internal. And there's a load balancer. Typically load balancer functionality requires natural implementation of load balancer. So when you create a service on, I don't know, APS or GKE or whatever, if you create a load balancer service, it will actually, behind the scenes, it will be integrated into their infrastructure. If you run a mini cube, just sort of a Kubernetes in a VM, there is that functionality already implemented. If you're doing yourself on a VM you've created, then you'd actually actually have to add something like an NGINX to do that load balancing. Okay, so NodePort, basically, your user who's wanting to access your application will attack this well-known input and that will be mapped onto, if I, sorry, in this case, you'll actually address directly the IP and port of one of these nodes, okay. So that works, but you're exposing part of your infrastructure. Load balancer, on the other hand, you really have an independent endpoint address and it will take care of balancing across the nodes, of course, addressing to the specific pods. And ingress controller, so, in fact, ingress is the process of controlling access into your cluster. And an ingress controller would actually be an external element like an NGINX or HA proxy, something which will implement the ingress rules that you specify about how to access your services. I don't want to do a demo on that. There are slides just on creation of a load balancer service. Actually, what I will do, because I had prepared that, so I can add, if I were going to use this application, I would actually create a service on my Redis application here. And in my architecture, this would be basically used internally, it would be so that my Flask front end can attack the Redis database. And if I wanted to access my Flask application from outside, and again, I would create a service Flask application. Interesting, I now see the replica set there that wasn't appearing because of a bug. Okay, I don't want to spend long on this. I really want to get to, I'm running out of time, and I want to move on to API gateways and service smashes. So API gateways essentially is an external software which will be doing typically things like access control, load balancing, maybe reverse proxy functionality, and allowing to access services within your Kubernetes cluster without seeing the internal structure. And there are things that this can be doing, things like TLS encryption, and essentially doing so offload from your cluster. You don't have to implement these things in the microservices, but you run them at the entry into your cluster. And actually on the diagram, another functionality which is important is that it might be doing protocol conversion as well. You might be just wanting to expose an HTTPS API, but internally that's going to map on to REST, to RPC, to whatever. And there are many examples of API gateways which exist, I've just named a few there. Nginx, HAProxy, I guess traffic should be in there. And there's a newer generation based on Envoy, which are being created. I'll come on to that in a moment. Okay, I'm personally going quickly, so I want to get on to the next contender is service mesh. So one problem with microservice is, well, first of all, the network communication aspect is much more critical. So there are things like detection of errors or maybe circuit breaking or these sorts of functionalities, or maybe just logging or encryption. There are a lot of functionalities which could be common to all your functions. So you don't want to have to implement them in each microservice or these are going to be micromonoids and the size of those container images going to be larger. So the approach with service mesh is to say, okay, well, we'll create a cluster-wide capability which provides these services over the network, of course. And so this reduces a lot of the functionality you would need to do within the microservices themselves. Oh yes, and one point is initially with microservices, people tend to implement these sorts of things, but then in libraries. But given that one advantage or maybe a risk with microservices is that you can be polyglot, you might do one component in Python, one in Go, one in Rust, et cetera. But of course, if you're building with common libraries, nice to have, you might be having problems then with your polyglot environment. So that's something you don't have with service mesh. Basically, it's provided by a set of containers through external for your actual microservices. And the software functionality is provided through sidecar containers. Okay, so there are other examples. I'd say the two main service meshes around today, I certainly have a lot of traction, are LinkerD and Istio. It was looking for a while like Istio is just gaining so much mindshare. Basically, it's supported by, part of it came from Lyft already. I think they created Envoy Proxy, supported by IBM, Google and others. So there's a lot of traction behind it. But there's still LinkerD, which has been around for longer. And I read just a couple of days ago, they've secured some venture capital funding. So that's good. I think we're open source, we do need more than one solution for each of these functionalities. So very quickly, with an Istio implementation, what you would have is an Envoy proxy which is going to run in each of your pods. I think I won't have the time to show a demo or that unfortunately. But basically, when you install Istio, you can then enable a namespace within Kubernetes that any pods that will be created will be created in Istio mode. So if I were to recreate my flask application, I actually did it just earlier as a test, in an Istio enabled namespace, then we would find that the pod that it creates actually has two containers automatically. There is the Envoy proxy, which is inserted into our microservice pod. So in this way, basically, all network communication from our microservice is going to pass via this Envoy proxy. And so we've offloaded things like encryption functionality into the service mesh. Okay, so let me just say, service mesh, there's a lot of traction around that. The service mesh vendors are saying today, well, we still need API gateways today, but maybe not tomorrow. So service mesh is really about this east-west traffic between services within your cluster, as the API gateway is really up top north-south traffic into your cluster. But Istio, for example, I think about six months ago, they added first implementation of gateway functionality within Istio. So they're starting to north-south traffic. Nevertheless, there really is room for both. I'm gonna try to move on very quickly. So there's another sort of API gateway, which is coming out based on what we call the API gateway pattern. It's actually this guy, Chris Richardson, who termed the coin. And it's basically talking about an API gateway functionality where the gateway understands the APIs that are passing through it. And it also maybe understands the APIs of the infrastructure on which it is running. And so an example of this is a project called Glue. It's an open source project from company solo.io. And they're demonstrating how they can do hybrid applications where because of their understanding of different protocols and the APIs that are being used and the infrastructure API, they can actually allow sort of a more flexible way of federating across existing legacy. So monolithic apps, microservices, and then like the next step in a way, serverless, where serverless functions could be considered the nano services. So I really wanted to just, I've run out of time, but I really wanted to get to that and say that that's an interesting next step in microservice evolution. So I encourage you to have a look at what solo.io are doing. They're a small startup, been around for about 18 months. They have several tools for helping to migrate to microservices. Okay, I'm gonna, I basically said what's on this slide. To summarize, I mean microservices, they offer new deployment possibilities, really increasing the agility, the ability to deploy, to scale, and upgrade more easily. And they can also facilitate best in class implementations of components. Nevertheless, that's useless if you don't adopt best practices. And that includes not just the design, but in terms of your organization, the way these teams work together. You can do incremental rollouts with like the Strangler pattern, for example. Of course, you can do Greenfield deployment if you're really lucky. There are as well hybrid approaches like I've just shown you in this project called glue. And then there are, it's interesting to offload the functionality via API gateway and or service mesh. I would maintain that I think for foreseeable future, we will see both of these being used. Okay, thank you. It was a bit quick and sorry for lack of demo. Any questions? Okay, we are a little running out of time. So if you guys have any question, Michael will be here today, evening and tomorrow, all day. So please conduct him. So please put your hand together for Michael for a great time and a great presentation. Thank you.