 So today I'll be talking about starting your journey with Captain Cube, basically Kubernetes. How many of you heard about Kubernetes? Almost everyone, right? So a bit about myself, I'm Bhavin. I work with infra cloud technologies and my daily work is mostly around helping people to migrate to Kubernetes, get them working CI CD and similar things. So let's start with the story and this is the beginning and meet our application. The name is Pippi and it's a very basic application with minimum business logic you need. We have just started, the team is new, so less number of users and small team of engineers and basically any new project you're starting and the infrastructure you have, you basically if you have less traffic and you have very few people, you start with deploying things on virtual machines, basic thing, installing your tool or application in them and starting the VM and then comes the growth. So what happens is, you started getting more number of users, getting lot of hits and the traffic is huge and it's increasing day by day. So as users increase, the demand for new features increase, what you do then? Then what you do is, you have more number of engineers and the need is to start moving very faster. People are asking for new features, you have to prioritize them, start working on them and so on. But while these things are changing, what about the infrastructure and what about the way you deploy things? So those also change, the way you change is, you start doing continuous delivery or deployment using the same infrastructure you had. So you have VMs and there are two ways to do this. So few people choose one way which is building virtual machine images which basically will run your VM for application update. So whenever there is new update, you will build image and start the virtual machines and so on. Another ways to do this is using tools like Ansible Puppet and in this way what you will do, you will have your application artifacts which is built and that will be deployed eventually to your virtual machine. So you can see your CP application and there are some shortcomings with this approach. So what are they? First thing is time consuming. So VMs are time consuming when they are booting, if you have a lot of dependencies installed on them and lot of services running on them, building VM images also takes a lot of time. So if you are building VM image for each update of your application, it will take at least 10 minutes for building those images and when we say we are building the images, it also comes with the overhead of doing the network related settings, installing the packages and so on. Also there is OS update. So what happens when there is critical operating system update that you have to pass on to all your application deployments, then you have to build all the images from the new updated operating system and then you will update your application. And next is things like auto scaling, rolling updates. So when you are deploying your application on a VM, by default you do not get any resource consumptions out of the box. Let us say you are on AWS. So you will not be getting metrics about what is the consumption of your particular application instance. Either you will get what is the CPU and memory usage of that particular box, EC2 instance or that VM. So these kind of things will make it hard to take decisions about scaling. So let us say your VM is hitting the limit because of some other issue and not related to your application at all. Then it will still scale your machines and this is not what you want. You want to have your scaling based on your application's metrics. It can be a number of requests your application is handling, also things like rolling update. So these do not come very naturally when it comes to VM. You have to use some extra tools. You have to use some kind of hacks to get these working. Let us say you are doing rolling update. What you do? You create new instance. When it is working fine, you will delete the older one. So if you have five replicas, you will do it one by one. That is how rolling update works. But to get this working, you have to make sure that a new VM is running and it is working fine and then you remove the old one from whatever load balancing technique you had. So there comes Kubernetes. As we talked about, the meaning of this word is a ship's captain. So we will see how it helps and all. So Kubernetes helps you to manage your container. So these will be your application instances. And Kubernetes also takes care of a lot of things which are industry practices. So let us see what Kubernetes offers, networking, scheduling, load balancing and auto scaling. So Kubernetes comes out of the box with scheduling. There is load balancing which is inbuilt. Also auto scaling is there. So you do not have to worry about these things and these are implemented in a very proper way. So all the people coming from different backgrounds who know, who are running workloads in production, they know how to do it. So the logic which works behind all these tools and offerings which Kubernetes provides are really tested. So you do not have to worry about doing your own hacks and all. The next is Kubernetes follows declarative approach to things. So what you do is you say, I want my application to run with fire replicas and I want this configuration. And you say, you just post that configuration to Kubernetes API. And from there it will take care of deploying it and managing and maintaining those number of replicas. So you do not have to worry about what it is doing there. So it will take care of managing all those. That is how the abstraction is added over the underlying infrastructure. So even if you have underlying nodes which will eventually be VM, but those will be kept running as it is and over those VMs your applications will be running. And similarly there are a lot of new things happening in this ecosystem like we heard about VITES from SUGU. So similarly there are a lot of tools. So how it helps you? So you do not have to manage individual VMs for each of your application. You just have to manage set of infrastructure that can be VMs running anywhere. It can be in AWS, on-premise VMs or even it can be bare metal machines as well. As I talked about it follows common or industry practices so you do not have to worry about implementing those on yourself and there will be no vendor locking. So let us say you have one Kubernetes cluster which is running in AWS and you want to move it to somewhere else. So the basic offering provided by Kubernetes, the basic constructs will always be same. The reason behind this is the foundation, the organization which backs Kubernetes, which is CNCF, they provide certified Kubernetes offering validation. So let us say you have some, let us say you start your own business and you are providing Kubernetes as a service. So you have to go through that certification and all so that everywhere the minimum required same thing will be there in all Kubernetes distributions. So that is how they ensure that all the Kubernetes offerings be it you are using anywhere, those will be same always. Obviously there will be some platform dependent things but you will find some alternatives to those and that transition will be very easy. So my rating to Kubernetes, so let us see how that will start and what are the things which you have to take care of and all. So before going to that, you have to start containerizing your applications. So we were running in VMs, right? So we do not have any containers and all. So what is container? We can see our FPP application is sitting inside a box in a huge ocean and we will see what that box means. So what are containers? Containers are nothing but isolated processes and which are also packed with their dependencies. So as we saw the FPP with box, consider it as a box that box will be created on the machine itself and all the file system will be isolated and everything will be isolated and that is how containers work. So while you are getting into the containerizing things and all, let us see what are the best practices which will help you to keep things clean. One process per container, so what happens is people have multiple processes as part of one application which they are running in VM and they just do lift and shift from there. That should not be happened. So there must be one process in one container which is running and doing all its work. The reason behind doing is if you have multiple processes, let us consider you have a VM. Inside VM you will have system D which takes care of running the processes even if it fails, it restarts and all. So system D is there which is a component which takes care of your processes. Inside a container there will be no one to take care of that. So you have to always make sure you have one process and whatever the container runtime is sending to that process will be received by the process which is intended. Let us say you have one application, application ABC and it has another process running inside its container XYZ. So whenever the application has some issues it will crash, the ABC process will crash. So the entry point of your container should be ABC so that if it crashes then your container runtime let us say Docker will know that it crashed. If it is not that case then you will have some shell script which is running that process and it will be a complete mess for your container runtime. It would not be able to figure out what is happening. Try to keep images lightweight. So the main reason behind moving from VMs to containers is to become fast, move faster. So don't do direct lift and shift from VM images to the containers. So what people do is they take same operating system image let us say Ubuntu or Debian and they start installing all the packages they had and they install everything which was there in the packer script which is used to build the VM images. So that should not be done at all. Logging to STD out and STD error. So usually what people have is they log to some files or something so that should not be done. This is kind of one good thing to do and whenever you are logging to STD out and error those logs will be shipped to some centralized location by other applications. So that pipeline will be taken care by someone else so your application won't be investing in logging to some file or some location something like that. So are you ready to sail with CaptainQ but wait before that let's see some of the challenges. So when we say we have shared infrastructure that also means that there are going to be resource limits. So while running applications on top of VM you are like the whole box is mine I can do whatever I want. But in case of Kubernetes you have to take care of this because your applications might get killed because Kubernetes takes care of keeping the memory footprint for every application restricted so that it won't affect some other applications which are running alongside with it. The next is inter-process communication. So let's say you are running inside VM you have two processes as part of your application and they might be using any inter-process communication method to basically communicate between both of them. So when it comes to Kubernetes you can only do it by sharing a file or sharing over the network. So that's the thing you have to change you have to make this change and if you can split the two processes as separate applications that is well and good. If you cannot do that then you have to make sure that you run those two containers together the two applications will be bind together. Then in Kubernetes there is concept called POD. So you have containerized your application that application will be part of a POD. So if you have two containers like two processes which are tightly coupled then you can have two containers inside one POD. So that's how it should happen and the communication will happen over network or file system. Here are some of the things which you should not miss POD IPs are FMRL. So while running in VMs world you know that IP address won't change for long but in case of POD if it even crashes once or gets scheduled somewhere else that POD IP will change and POD IPs will get reused as well. Health check endpoints for your applications. So the way Kubernetes knows one particular application is ready and it is ready to serve the traffic and it is healthy is health check endpoints. So these can be commands which are run inside that container or these can be HTTP endpoints which will hit by the Kubernetes components to make sure that the application is running and it's healthy. Resource requests and limits for each application. While Kubernetes has strict policy of resource limits you should make sure that you have specified what is the requested resources like what are the minimum resources required by the application to run and what are the maximum like soft limits and hard limits. So this makes sure that your auto scaling works properly. The resource values are used to calculate the percentage of utilization of your application when you want to do some auto scaling. Let's say you want to auto scale your application at 60% of CPU users. To calculate that 60% you need some threshold right? So you should mention like I am giving this application. This application will request one virtual CPU and your application is using half virtual CPU then your utilization will be calculated according to that. So it will be 50%. Also use community Helm charts and operators. So Helm charts are the way to package your tools, package your objects which are Kubernetes objects into one package. So that's how you can package your application and ship that to anyone who ever want to use it. While you will move to Kubernetes you will need some standard toolings right? Things to do logging, things to do collect the metrics and all. So in order to install those tools make sure you use the community written Helm chart instead of investing your own time doing that. Because they will definitely have followed all the standard practices. So you will be fine with running them. Operators are another way to run your applications. So those are also available like Helm charts. Namespaces per application or team. So Kubernetes has a logical separation of namespace. So it's like let's say you have some team front end team and back end team so they can have different namespaces. The reason behind using this is you get some kind of logical separation. Also, when you have namespaces for application or per team, you can set the resource limits quotas. So those are called quotas. So you can say this particular team should not be able to use more than 4GB of memory. Their applications won't need more than that. Something like that. So we are all done. We have containerized applications. We have all the workloads ready to run on Kubernetes. But how do I migrate? So one way is to just lift and shift all the applications and pray to God that everything will work fine. And another is migrating gradually. So what does that mean? You will basically start with the stateless applications followed by stateful applications which need storage and all. And then you can move databases as well or use some databases which are made to run in Kubernetes environment. So start with doing this. Move only one application at a time and shape the traffic or divide the traffic using load balancer in your cloud provider or any NGNX or some kind of tools so that you have the traffic split between your old deployments and new ones so that you can make sure that everything is working fine. Another interesting way is to do is one API endpoint at a time. If your application supports this kind of traffic, then you can do, let's say, API slash V1 slash get is running on Kubernetes and API slash V1 slash post is running on old deployment. So that way you can move one API endpoint by one endpoint. So should I run my monoliths? Yes, you can, but just make sure that you containerize them properly and you will be good to go. And in fact, this is also a good thing to start with if you have a lot of applications. Now I have running everything in Kubernetes. I have my applications. I have all the containerized applications. But what about the cluster management? The underlying layer will be still there. Cluster will be there. So let's see how you can deploy the cluster itself. So you can start with using managed cluster solutions, like Google's Kubernetes engine or offering by AWS EKS and offering by Azure AKS. Similarly, there are multiple offerings by different cloud providers. And if you want something which is managed out of the box, the master in Kubernetes, you have master plane, which takes care of all the things, like API endpoint is in master plane and all data about your objects, Kubernetes objects is stored in the master plane. And using managed services will make sure that those master plane nodes are managed properly. The database, which is a key value store, which is used inside that is ETCD. It is managed properly. The data is encrypted. And updates are applied automatically. What you have to take care of is only the nodes you manage. And there are other ways as well. You can self-manage your cluster as well. That means you will manage your master plane as well as you will manage your nodes. If you have some use cases where you want to tweak your master plane components, so you might not get that in above solutions, which are managed cluster. So you can do that with self-managed cluster. But that comes with cost as well. So you will have to manage the master plane. You will have to manage all by yourself. But now what about backup and restore? So there are tools like Valero, which will help you to back up whole cluster as a one. It will back up all the objects. It can back up all the volumes, so that if some disaster happens, you can immediately restore that and get your things working. And the major and important thing is cluster updates. So Kubernetes is really fast-moving community. They have one minor release every three months. So you should always make sure that you are on latest version because they only give support for last two versions. So let's say 1.17 is later. 1.15 and 16 will be supported by the community. And older versions will have some lifecycle of nine months, I guess, about security patches. So if you are on managed services, they will take care of updating these things. But you have to make sure you are on the latest of their offering as well. Some managed services might be on older versions as well. So Amazon CKS stays two versions behind the Kubernetes community. So while you are done with moving your applications, there are a few things which you can check out. There is CNCF Landscape, which gives you a whole list of all the tools. There is a tool called Prometheus, which can help you to collect metrics like CPU uses, disk uses, and so on. There are a lot of them. There is a tool called Argo for doing CI CD. And NGNX Ingress, this is the tool which you can use to expose your services publicly over some load balancer or something. Now, questions. I have a question. So how easy it is to set up Kubernetes for the .NET-based applications? It's same. If you are on .NET Core, then it's similar with other applications. So you just have to make sure that the image you create is lightweight because what people do is they take SDK images. And those will be very larger, like 1.5 GB or 2 GB. So there is another way you can create one executable which is independent of anything. So you can just run that executable only without having an operating system at all. So they call it something called FAT binary or something similar. Thank you. Have you tested any service on scale in Kubernetes? So there is a lot of routing that happens. Like for example, if there was only an EC2 instance, the routing will be very less, right? Because load balancer to the EC2 instance, you can directly connect. But with a pod, there is a lot of internal routing that happens. Right. And at times I've seen that that adds up to certain latency, like for example, five to seven milliseconds of it to every request. Like have you tested any application at like peak scale or huge scale? So when it comes to that, I have personally haven't tested that at that scale, but I have tested the one which is offered by AWS. So what they have is, they have their own implementation for the network layer. So it is called CNI. So the next talk we'll be talking about all the network related stuff, you will get that there. So the network interface, the way network is set up in Kubernetes can be in different ways. So you have to make sure the one which suits you that you should choose. That way it will have less latency. I'm here if you have any questions. Yeah, there are some use cases which you might not want to move there. Like banks are not very, what I can say, they don't move easily because their applications are running since ages. They are running perfectly and right now I cannot think of any particular example but there are use cases where you don't want to move at all. No, so if you have a Kubernetes cluster where you are already running some applications, right? So it makes sense to move your application there itself. So you don't have to worry about managing all other things for that particular application only. So if you have one cluster running, you can run any small application there that is totally fine.