 I've got two now, do I do a putty? So I was told that we have 20 minutes and the talk starts in what, 10 minutes? No? So can I grab the extra 10 minutes as well? Okay, I will try my best to keep it 20 minutes then. Is it projecting? No. Okay, now? Ah, okay. Okay. Yeah, I've got quite a few. Okay. Let me just... Who is the worst? Is there any way for me to see the screen from... I will do it live. Yeah, no worries. Okay. Okay, ready? Okay, so don't tell Michael that I got 197 slides. Okay. And I also need a little bit of time to do a demo. Okay, all of this in 20 minutes. Okay? Hopefully we're going to keep this secret. Okay, so who am I? I've got a really long name. So as long as you can be done, I'm happy with that. I work for a company called Learn Kubernetes. And basically what we do is we teach to other people Kubernetes. Jessica has got this logo. So what am I here to talk about today? I'm a little bit... I'm here to talk about, you know, why do we use these Kubernetes being... Why is it so popular? Why is everyone, you know, so interested about deploying containers or using containers for, you know, machine learning or web applications? So, but before we dive into that, let's have a look at a little bit of a background, right? What do we do when we deploy applications? Well, back in the days, right, we used to have large applications, right? And we used to build these large applications. But, you know, in the recent past, what happened is we basically decided that it was a much better idea to just, you know, break these large applications to smaller components than each component would have, like, you know, a single focus. We would have an API, a front-end, and each of those would be independent, right? And we thought it was a good idea because we could have quicker deployments, right? Everything would be separated and we could be released independently. We said, you know, each team could work on a single feature, so that's a good idea. And it's also overall less risky, right? Because if a component goes down, it's just that component. It doesn't take down the entire system. And, you know, as developers, we said, this is a great idea. We should keep doing it as much as we can. However, there is something else going on, right? So, there is, you know, some of us who are perhaps more into the infrastructure side of things are seeing these kind of things and thinking, well, it is great, but when I have a single application that I could just bring that single application and put it inside a box and that would just run, okay? But now we don't have a single entity anymore. We have, you know, these individual components. So we should run each individual component in its own box, right? What we call virtual machines, for example. But the reality is that as developers, we like, you know, we don't just break things in four. We tend to be, you know, a little bit more. You know, we take things to the extreme sometimes. And we end up with a lot of applications. So a lot of applications actually introduce quite a lot of challenges on the infrastructure side of things. And if we look at some of the challenges, the first challenge that we can think about is, you know, when we had this virtual, when we had these computers, right, these computing units, then those tend to be like quite large computers that could run a lot of things, right? And if you look at the alteration of things that were using the operating system and in a space which was usable for you for running the application, it was a tiny bit using... The operating system was using a tiny bit. The rest you could use yourself. Now, if you had only a few applications, then, you know, you would have this split. And overall, the memory and CPU used by your operating system would be a tiny bit, right? So if you think about it, 2 to 3% to run the operating system, it's okay, right? It's very small. But when we switch to something like, you know, smaller components and a lot of them, what happens is we tend to run smaller applications and smaller rich machines. And the ratio between what is used by the operating system and what's used by your application, your web application changes quite a lot, right? So we're seeing some shifts from 2 to 3% to more than double from 6 to 10%. And if you think about it, if you have a bill on GCP, for example, of $1,000, then you're spending $100 in running operating system. So some money that you will never see back, right? It just lost. The other challenge is usually when we build this application, we used to put this application inside these boxes and everything will be fine, right? We will have another box and another box and if the application was smaller then there would be just a little bit of gap, right? Which we couldn't utilize. And it was still okay. But now with this application that we're building, which are much smaller, much more focused, right? Compared to what we had before, we have, you know, all different shape and sizes. So if we deploy them in these boxes, then there is a lot of wide space, right? Which we didn't have before and now we do. Which is kind of a problem. And the last sort of issue that we get that we inherited from this sort of development that we now we do is that, yes, all these microservices, we like to pick languages, do we? Okay. But it turns out that we really, really like them. So if you go and look at these, you know, microservice architecture or, you know, if you go in a company you will find that, you know, someone is writing Go, someone is writing Node.js, there is, you know, I'm going to say some legacy in Ruby, but I don't want to offend anyone. Okay. But in general, more in general, is we have a lot of applications, right? Different languages. But even if you are in one of these companies, really rigid saying, we do just Java. Okay. Really serious. They still have the same problem, right? Because they have different run times. It's never just matches the same. So even if they pretend to be better, they are not. Okay. So what can you do? That's one solution. Okay. But let's take a step back, right? So we talked about, so all of these, some of these problems were originated by ourselves, right? So we pick these boxes, we call them version machines, and we place stuff inside. But what if we change the story, right? What if instead of starting with version machine, we start with something else? So it turns out that in the meantime, there were some features in the Linux kernel that they were quite interesting. One of these was control groups, which is basically when you launch an application, you can define how much memory and how much CPU that process is going to use. Okay. And then you can limit things like memory and CPU. And then the other things you can do is you can launch a process like your web application or your binary, and you can also set isolation. So what a process can see. Can you see other processes? Can you see network packets from other processes as well? So we just, you know, designing boundaries around our application. What kind of things you can see and how much memory can you use? It turns out that this sort of stuff is quite interesting, and someone said, what if I combine three things? You know, these control groups, name spaces, and a little bit of magic, okay? And they came up with this, right? What we call dockers, what we call containers. So just a way to launch a process, which has got some extra memory and some extra name spaces just to set isolation and groups. So generally, we don't say this, but let's say for the sake of the point we are trying to make, but docker containers look like lighter virtual machines, like lighter boxes that we've seen before. So before we had this, right, so we had the server and several virtual machines, several boxes inside. Now we basically, if we'd run containers, then we don't have these boxes anymore, and then we can run the infrastructure much more packed, right? So everything is neatly organized together, which is good. And the other benefit of these containers is that because these are just processes where we set limits, right, it's not as slow as a virtual machine. I mean, you must have run virtual box and waited, you know, for a copy and come back and still there booting, and there is no hardware emulation. This is just launching a process in the same way you launch your IDE, right? So that's the same thing. The docker container is the same thing. And then the best part is that when you launch a virtual machine, it's going to pre-allocate some memory, right, and CPU. When you launch a process, it's just going to use the memory dynamically. So these docker containers are basically just double-clicking on your IDE. And then the first part, which is also quite interesting, is that these docker containers, at the end of the day, what they are, are just archive, like zip archive. And what they contain is not just the code that you run, but all the dependencies. And dependencies, if you're running, for example, a JVM, it's going to be the runtime. If you're running Node.js, it's going to be in Node.js binary. If you're running Ruby, it's going to be the MRI. If you're running Go, well, you don't need any of that. It is just a binary. So we combine your runtime or your files all together with any other dependency you've got, and we just wrap them up in a zip file. And we call that zip file container. Okay? So that's the magic of containers and how they work. But the powerful thing is that if you think about it in the past, right, if you had to deploy anything like a Ruby on Rails application or Node.js, you would need to go to someone else and say, hey, can I have a box? Can you go inside? Can you go inside? Install a couple of binaries, blah, blah, blah. And if it was just one, that was fine. But when you had several applications, it becomes really, really complex, right? This server needs to have Node.js and Ruby and so on and so forth. But with Docker containers, we don't need any of that. Because we wrap everything inside a container, we don't care what they look like from the outside because they are just containers, and we can just deploy them as containers. So containers are great. I hope you share the same feeling. But the thing is if we have a lot of virtual machines and we switch to containers, we still have a lot of containers. Okay? We haven't solved the problem on how do we use these containers? So it's Christmas time. Okay? So I have a couple of wishes. You know, I'm going to have a wish list with some things I want to solve this problem of containers. So the first wish I had, what if it could be something that could manage all of these containers? And it could be something that ideally, I don't need to go there and manage the servers. It's just going to take everything, it's just going to take care of everything for me. So it's going to basically look at all of the things I've got and just make a single bundle. Okay? So I don't need to worry about where these applications that I built are going to end up. And then ideally, I want to also make sure that this is something that I can program because I'm a developer and I like to write code. Okay? And you might have used some of these tools in the past. So it turns out that this technology that managed several containers and have got these properties are called container orchestrators and the market has got a bunch of them. Okay? So you might have heard of these ones, but another example is if you're using AWS, it's called AWS ECS, which is the container orchestrator. And it's not a fair fight. Okay? So Kubernetes is de-factor, de-container orchestrator. Okay? So this is a fairly old graph, but the trajectory is, you can see the trajectory of Kubernetes. And so why this Kubernetes? Why is it so popular? Well, one of the reasons why it is so popular is because you don't need to think about your servers. You don't need to think about this computer that you have to deploy your applications. What Kubernetes does is you install it on one of those. The other nodes inside your infrastructure will join the master node. And what happens is the cluster is just going to absorb memory and CPU of all the nodes. Okay? And it's going to become like a massive computer, just a single one, which is much easier for us. We only deal with one single computer. The other interesting thing is about Kubernetes is that it's quite clever, but clever is not the right word. It's quite a nice Tetris player. So if you think that these boxes that we created, they come all different shape and sizes, right? Kubernetes, what it does is usually play Tetris with your infrastructure. As more containers come in, Kubernetes will just find the best slot to place your containers, right? Until the end, you get something like this, which is great if you like Tetris. And it's also great if you like efficiency, because as you can see here, we're trying to maximize the space that we can get out of our servers. And then the last one, again, I'm a developer by trace, so I like to program things and change things. So everything you're doing in Kubernetes is just an API call. So you've probably seen, this is a cool request, and I can query the API, and I can query the application that I deployed. And then there is also documentation, so I can just read the documentation and create more endpoints. So if I want to create an application, I can just say, curl, create an application. Curl, create an under node. Curl, provision, load balancer. So everything in Kubernetes is just an API call, and that's quite so cool. So do you believe me? I just made up everything. It's demo time. So before we do the demo though, just a quick description of what we're trying to do. So ideally, this is what we want to build. We want to build the red pages that people visit. That's what we do all day. However, sometimes these red pages, they become super popular, so we need two copies instead of one. So what do we do to distribute the traffic between the two? I hear a lot balancer from the crowd. Very well. Now, we might have more than one application because hopefully, you know, where we work, they don't have like a single product or a single microservice, they probably have more than one, like this. What if I want to route the traffic to both microservices? How do I distribute the traffic? I hear a lot balancer from the crowd. Not really, but that's the thing. Okay. So actually, we have two layers of load balancers when we want to distribute the traffic between several services in our infrastructure. So we have an external load balancer and an internal load balancer and then the applications. So in Kubernetes, we do exactly the same thing. We just like to call everything with different names. Okay. So instead of calling the internal load balancer internal load balancer, we call them services because you know in computer science that's a very specific term, the service. And then instead of calling the external load balancer, the load balancer, we call it the ingress. Okay. This is a little bit more specific, thankfully. And why calling the application or container? Container is where you can call them pods. So that's how we call them. Okay. So that's basically how we deploy application in Kubernetes. So let's have a look. So we discussed about three different things. We discussed about the ingress, the service, and then our pods. So the way it works is first of all, we create a container. Yes. Correct me. So a container, what it is, we usually create container from a list of instructions. So the list of instructions is usually called a dockerfile. And this contains basically, it's like a glorified ISO in VisualBox, right? You say, I want to start with Ubuntu. I want to start a couple of things and install my application. That's the same thing we do in containers. In this particular case, I built a hello world app in Go and just packaged inside the container. Then I can build a container with docker build, dash t for the name. I can call it hello. And then hopefully it will not let me down. Okay. So this is going to take ages, and we can move on. So what we do when we have this container, then we just define how we want to deploy it. Now, everything we do in Kubernetes, we actually write this very nice YAML file. So just to configure a description of what we want the end product to look like. And what the end product will look like is a single application. What's the name? Hello world. Oops. And what's the name of a container? Okay. It's called demo. I built the wrong one. But you get the idea. We're basically just describing what that should look like. Then we have the service, which is the internal load banser and the ingress. If you think it's a lot of YAML, yes. It is a lot of YAML. Kubernetes engineers are usually called also YAML engineers, because that's what we do all day. Okay. So once we have those, then I have a cluster running locally. Oh, by the way, this is just created a container. It's local. So when we have these resources locally, then what we do is we have a cluster, a Kubernetes cluster. In this particular case, I'm running a local cluster called Minicube, which is running on my machine, because I was afraid that the internet would not work. And usually what we do is we send these YAML files to the cluster. The cluster would read it and then create the resources on our behalf. Now to send these instructions, we use a command line called kubectl, which everyone pronounces with a different name, kubectl, kubectl, I say kubectl. And then we say, oh, my. And we just send the resources. You can see that when we create these resources, then the created resources, we submit the request, but it doesn't wait, right? It's just going to return immediately. But in the background, what happens is Kubernetes will go and create these resources for us. I can see hello word here and then the ingress. So I can see that it created all of the resources. Can I see my application? Hopefully it's going to work. Yes, I can. I can see the application. So you can see CKFGD. What is that? CKFGD. OK. Well, good. Now, the good thing about Kubernetes is that we generally have a single application, but that's, you know, we like things to scale. So generally what we do is we scale the number of replicas and scale the number of instances of our application. And the way to do that is by writing what we call a deployment, which is what I just showed you. Generally, the reason why we use deployments is because in Kubernetes, when you create a deployment, then the deployment looks after your pods, right? Looks after your application. And if one of those applications was to go down, then Kubernetes will bring it up. OK? So let's have a look at this one quickly. So what I can do, I can go in my deployment and I can change the number of replicas, which is the instances of my app from one to five. And I can see there are five new applications running now. So what if I go and delete this one? Then I do a get pause. There are still five running again. OK? This is because as one is deleted, Kubernetes will replace it with a new one. OK? So it's doing a lot of stuff for three for us. OK? But one of the things that we do very, very frequently when we deploy application is we deploy an application and then we have a better version. OK? We deploy version one and then, you know, a couple of days later, we just find a way and redeploy it again. Now, Kubernetes is quite interesting on this space because it automatically does what we call a rolling update. It's going to do an upgrade of your application with zero downtime. So how do you do that? Well, if you look at this particular deployment, we have three applications with a red page. So when you say I want to upgrade from version one to version two, this is what happens. Kubernetes will create a new version of the application. OK? And we'll replace it with the old one. Create a new one, replace it, create a new one, replace it. OK? Because it's doing it one at a time, then the traffic never goes down. Right? We never had downtime. So let's have a look at that as well. So what I should show you before we start is that the application exposes a version. OK? This is version one. So what we can do, hopefully this is going to work, nope. I'm pretending this is normal, but it's actually not. So bear in mind with me. I've got version one. You can see version one running. OK? So what I do next, I'm going to go and change this from version one to version two. What would you expect to see now? Is this is going to continue to be one all the time? What's your guess? Is it going to go to two? Yes, hopefully. I guess. Hopefully it works. So the way we check that is we do a kubectl apply dash f. OK? And then switch back to the slip. And we see what happens. Oh! More two. One and two, two, two. OK. Eventually it's going to switch to two, right? You can see now the whole traffic is going into version two. So we just did a rolling update, but just changing the version of the application in the specification. And that's basically, Kubernetes did a rolling update for us. What is my mouse? OK. So that's basically why everyone is so excited about containers and Kubernetes. It's quite easy to create deployments. It's quite easy to do rolling updates. And because everything is encapsulated, it's very easy to scale to several servers as well. So who is using this and why? So I think a good example of who is using Kubernetes for the sort of large-scale processing is this company called OpenAI. So what I do is back, I think two years ago now, they used to train machine learning models on Kubernetes and they had like this what we call now medium size. At the time, it was really large size. We have 2,500 nodes. OK. And what kind of things they did, they installed basically something designed to scale the system, right? So when the application needs to train more models, then it will just create more instances of this model and will just, you know, process more jobs. And then that was combined with another thing in GKE, which is the cluster auto-scaler. So when you run out of space, like in this case, it will just create more nodes and then keep doing this training of models. So it turns out that they were building a model to play Dota, which is quite cool. And I don't know if you've heard the news, but I think this year in July, they also managed to compete with pro players as well. They run, for competing with pro players, they run much larger cluster. We're talking about more than 5,000 nodes per cluster. So it's very cutting edge stuff. But at the time, the reason why I'm mentioning this is because at the time, they basically shared what kind of configuration they had for their own cluster. And this was the configuration for the 2,500 nodes. So you can see that they used 250 sites P10, P100, which I think are TPUs, and a bunch of various TPUs. This is everything in Google Cloud, everything on GKE. Can you guess how much money is this per hour? No? Well, it turns out that it's not a lot. I mean, it is a lot. Objectively, it is a lot. So we're talking about US dollars here. So it's probably 1,600, close to 2,000 dollars. But if you think about it, you're right. If you're a student, well, if you're a student, it's probably going to be too expensive. But if you're a small research center, you have access to so much power, which normally would be just for large corporation or large companies. Now you can go in GKE, click on two buttons, provision a 500 node, 5,000 node cluster, pay 2,000 dollars for an hour and just take it down. This is unprecedented. We never had such ability to create things so quickly and destroy them. Even if you're sitting in your room, you're back home just mining bitcoins or training models. So I think this is pretty exciting for me. So that's everything I wanted to show you today. I hope it was 20 minutes, more or less. So we talked about microservices, how things have changed since from where we started, from these monoliths. We talked about these containers as lighter VMs, so just processes that have got memory and CPU limits. And we talked about Kubernetes as one of these container orchestrators. So that's everything I wanted to show you today. I think we usually run workshops on this kind of stuff. We have one coming up, the 27th of November. But the reality is that we're going to do a repeat of the zero to Kubernetes course, which we have done in September. And we're going to do it in December. What we're going to do in this session is we're going to do the same three hours. We're going to go through building an application in Kubernetes and deploying it. So last time was quite popular. I'm not sure about this time, but yeah. I think if you're interested in these sort of days, it's on the 7th of December. I think it's on a Saturday. So join me. And I hope you enjoyed it. Thank you very much for listening to me.