 Okay, hello. So I'm here to give a brief, I don't even know if it would reach the level of tutorial, but more like discussion on Kubernetes. So I have an audience here that cannot be heard in the recording, unless I say so. We talked about our background. So basically this would be appropriate for someone that sort of knows a little bit about what Kubernetes is, but maybe has never made a deployment or used it or don't even know when they would use it. Let's see, I'll come to my screen share here. So I've shared a HackMD in the meeting chat, which maybe someone can reshare again for the people that have arrived lately. If you would like to say anything to me or make some comments, please look in the HackMD where it's open and then just like make a question or comment and then I'll see and then we can discuss. Or you can use voice if you would like. Okay. And I just noticed that if you're logged in, I might see your name in the recording. So maybe you would want to do this without being logged into HackMD. Okay. So yeah, as you can see here, the demo, this is sort of a crash course. So the reasoning behind this is a few weeks ago, I wanted to make a deployment in Kubernetes. So someone far more clever than me had set up Kubernetes in computer science IT and they we've been using it for several years now. But I wanted to take and run a Zula bot in this Kubernetes cluster. And I didn't quite know what the right practices were. So I had the cluster and I needed to run something in it. And I didn't know how. So I wanted someone to give me a tutorial sort of the minimum about how to do that and why I would and so on. In the end, I sort of figured it out myself for better or for worse. And now I'm here to give you this tutorial that I wished for a little bit ago. So I've been using Kubernetes for say two years or so. No, not two years since 2018. So three years now. But always as sort of a user and not as a person that's been like maintaining it myself. So I know a little bit of the principles but not too much. And I see the person who I'd consider the person who set it up isn't here. So hopefully we can answer any questions. Yeah. Can you while we're waiting come here and write in the hack and the as an ice breaker right here what you would like out of this so we can sort of discuss. And while we're doing that, I can say I can consider two different kinds of services here. So one like the Zula bot I deployed it ran in Kubernetes and connected outside of the cluster. And the other one is something that runs like a web server which receives incoming connections from outside. So I have a demo of the first one. I do not have a demo of the second one. But we can try to do that live, I guess. Okay, so while people are feeling. So one is Kubernetes used. So it's a container orchestration system. So basically it runs containers and provides a coherent way to manage them. And this management is the important thing. So the containers can be Docker or different other runtimes. Because the Docker image format is basically the standard but there's different runtimes which we don't really need to worry about ourselves right now. But Docker is a one node kind of thing. So Kubernetes provides this interface where you can do things like say, okay, I need to run 50 web server containers at all time. There might be times I need to roll them over to a new deployment. And when doing that, there should always be 50 running at any time. I need 10 caching servers, five memcache servers, three database replicas that are all somehow working together and connecting. So this could be a sort of huge system administration task. But once you have Kubernetes, you can basically do this declaratively. So you'll make this YAML file that defines the whole thing you want. And then whenever you need, you apply a new version and it will figure out how to upgrade it and manage things however you would like. Okay, let's go back and see what people have wanted. Brief glimpse. Yeah. So, hmm. Yeah, okay. We can come back to some of these at the end if we have an answer. Okay. So let's see. So the power of this comes at a cost. So there's a lot of concepts to learn even for small deployments. And well, at least for me, it was really hard to figure out what you might need and what I even needed to do. But I think that if you have the cluster running, then it's not and you know how to do it. It's not that hard to make a deployment and run something within the cluster. And then it's sort of this coherent management. Kubernetes isn't just for big things. So when I got started, I ran mini cube on my own computer and was able to and it's like a single note thing. It just run one command that runs and then you have the API. So really, Kubernetes is defined by its API more than what's running in the backend. As in there's many different softwares that you can tell this these requirements to and it will make it run. Okay. Let's see. When do we use Kubernetes? I think I sort of already covered this. So in my case of some simple chatbots, it was a little bit overkill. Like I may as well run that on my computer. But when the alternative was running it on a university shell server, which gets rebooted periodically, Kubernetes is a little bit better. So let's take a look. I'm switching to this view to see. So let's say you have a Docker container that can run. So you would say Docker run the container tag and then you can mount different volumes for different ports and so on. And this is an imperative process. But I guess it's how most things are developed. So let's presume you have a container that does what you need. So the basic idea is you would convert this to Kubernetes YAML. So this is what it looks like. It starts with three dashes. If there's multiple declarations in the same file, there's some standard stuff. Like this is version one. This is pod. Everything has metadata, like labels and so on. And now we have a spec. And this is what we'll run. So the first concept here is a pod. I think the analogy is it's like a pod of whales or something like that. I guess because the Docker logo is a whale with containers on it or something. Okay. But a pod is multiple containers that are running together. So in this YAML file, we'll make it larger. We see a spec which contains containers. And here's a definition of one container. So I'm calling this name bot. The image is Zulibot, which is the Docker image name. And other things like should the image be pooled all the time or if not present, well, here we pool it if it's not present. And now we specify the volume mounts. So I'm mounting something called ZulibarC at config. It's read only. And now there's a bit of interaction here. So there's volumes where I've defined what's mounting inside. So notice that the name here matches ZulibarC mount to ZulibarC mount. And this is a secret. So secret is another Kubernetes object, which has been defined. So the secret name ZulibarCCSIT gets mounted here. And this secret has one key inside of it, which gets mounted as a file inside of this config. Okay, this is the basic idea. And we will see demos shortly. So Kubernetes objects, everything's organized into objects, which is really sort of like mapping to the API. So there's this YAML description. And as we know, YAML is basically the same as JSON. Well, something static like this can be represented as JSON. This is also the form that's queried to and from the API. And when you make an object, it basically takes this, converts it to the structured JSON or whatever format, sends it to the API, and then it creates objects like that. And there are many different object types. It actually took me a while to figure out where I can read about these objects. And that was basically in this Kubernetes API reference. So when I started doing this, there were lots of tutorials I didn't really like. But then I come to this API reference. Let's see. And I say, okay, I know I need a pod. So where's that under workload resources, pod. And here there's a bunch of definitions. So this is a pod, it needs these different things. There's a spec. This is the pod spec. And basically by looking at this and reading about what everything meant, I was able to create that Kubernetes YAML. Combined with some tutorials, of course, but this was the basic idea. So that was the first thing that I wish someone had told me. Okay. Yeah. So what are the different kinds of objects we may need? Maybe someone can go and write what I'm saying in the HackMD as I'm saying it since I didn't fully fill the cell. So the pod is the fundamental of unit of computing, it has different containers inside. The containers are basically like Docker images that we're familiar with. Through the specification, you can define all the different properties like the image name, environment, startup command, and so on. And you find that via this API reference. A deployment. Is someone going to be writing this here? Or should I write it myself? Or is someone writing? Okay, thanks. Yeah, if you just make some notes, I can fill it out later. So deployment basically orchestrates different pods running. So deployment is like saying, I want this number of pods running at the same time. And then it manages them running. So if a pod dies, the deployment will restart it, and so on. So I think usually people would almost always make a deployment instead of a pod because if you make a pod and the node goes down, then the pod is gone. If you make a deployment when the nodes get readjusted or a pod stops because it crashes or something, it keeps it running. And deployment basically has a template inside and that template is the pod template. Okay, there's volumes and volume mounts. So a volume defines something that can be mounted. And the volume mount specifies where it is inside of the pod. So this is sort of a little bit of indirection. So for example, in this Jupyter.cs deployment we had. At first, all the volumes were defined on pods. But then we made a separate volume definition. I noticed there's a persistent volume or something. And then we could refer to that from all of the other pods. So then to make a test deployment, we change one place and then all of the different mounts change at once, which is kind of nice. So secrets and config maps are configurables. So maybe I'll start with config map. So config map is keys and values that basically key value configuration stored within the Kubernetes API. Secrets are like that except their secret. And when we start exploring the deployment, you'll see an example of this. Yeah, and configs and secrets can be deployed or put into the pods in different ways. For example, as environment variables are mounting them as files and so on. Okay. So a service. So a service is like a proxy to a set of pods. So let's take our example where we had 50 web servers running. When a connection comes in from the outside, it has to connect to one of these 50 different pods running. So you define a service which has a label, a service defines which pods it's connected to. So any incoming connection gets distributed to one of these many pods that are running to handle the request. And we'll get back to services when we talk about incoming web servers and ingress somehow map incoming HTTP connections to services. So basically it says when a web connection comes in on this host name and this path, it will go to the service which will then forward it to a certain pod to handle it and then send everything back. Okay, so let's make an example here. So yeah, I'm not really going to use this here. I think I had an initial example, but now I have a better example, which is here. Let's see. Are there any further questions written in here? Nope. Okay, I'll leave space where I can see if someone writes with questions or comments. So here we are on my work workstation with access to the Kubernetes deployment. This repository I'll be looking at actually, I can give it to you. Okay, I'm putting it in the chat and in HackMD. So let's see how I've organized this. So I should say this is a tutorial and not necessarily the best possible practices. So this is sort of like the example I got a few years ago, which I worked with. There's different ways that this can be made better, like somehow make it automatically applicable. But for a demo, I think this is a good start for like an hour to get things going. So let's see what files that are. There's Kubernetes YAML. And this contains the different YAMLs of the deployments. There is this make secret script. And then there's a secrets directory. That's a Git sub module. So you have on GitHub the repository itself, but not the secrets that are included. So let's take a look at a deployment. Or do you want to look at the pod first? So this pod is no longer used. This was my first attempt. So we see there was API version, kind pod, some metadata, a spec. This is the Docker hub image I deployed. And then I gave this whole command here. And then volume mounts. So yeah, this is like it said. So here's the secret name. Let's go look at the secret and see how it was made. And that's in this make secrets script. So this script one run this kubernetes, krup control, delete, it deletes the secret and then it remakes it. And this secret is called zuliprc-csit. And it is from file zuliprc equals secret slash this. So zuliprc is the name of the secret. And this is the file name of it. Let's give an example of looking at the secret with the API. I will switch my view just to make sure this doesn't show anything. Yes, it works. Okay. So now we have our first examples of the API. So everything is done with kube control. And there's some kube config that tells you it basically says what API is this connecting to. And it connects to well, the API of the internal CS Kubernetes, it could connect to your local deployment or things like that. Okay, kube control, git secrets. And I'm doing this in the namespace zulip bots. So namespaces are a way to sort of separate all the different applications that are running. Okay, so we see. Okay, this is we have this many secrets in here. So there's kube control works by this a verb, the object type, possibly a namespace and then things. So after this, let's describe a secret. And I will describe the zuliprc csit. And we see here, there's the name, type, and data, it's 128 byte file, which is here. And what we've just done is basically the basics. So you can keep repeating these commands, kube control, git pod. So here, I've gotten the different pods that are running. So we see there are these different bots. We can get deployments. And here's the deployments that are running. And notice the names exactly match up to what we described in the API. kube control, describe deployment. Let's describe csit one. So it says not found that's because I forgot the namespace. And if we look this looks a whole lot like basically exactly what was in that Kubernetes YAML file. So um, yeah, I mean, it is, that's the importance of this API reference. Okay, let's go back to our directory. And let's actually start the deployment. Or so here's the deployments. Let's delete a deployment. Let's delete this csit bot. Of course, the namespace. Okay, so it's deleted. And now it's not there. Okay, so now here, we're finally at where we're really start. So you have the YAML file, how do you bring it up? kube control, Zulu bots apply from a file. Yeah, this was the name. So yeah, and we use apply instead of create because then the Kubernetes API will remember what was made last, and we can apply again, and it will transition to that new state without bringing everything down and bringing it back up. And you can specify things like, for example, there should be no more or no less than 80% capacity at every time or first bite this much or that much or whatever. So let's apply. And then immediately we'll get the deployment. Well, and it's back age five seconds. And it started very fast because the two pods started. Okay. Yeah, um, let's see. What else can we explore in here? This image contains a Docker file, which is what's being used by all of these deployments. So right now I build this image manually push it to Docker hub and then it gets pulled back into our cluster. Let's open up the YAML file. So here's the deployment itself. Have we seen a deployment yet? Possibly not. Someone's saying something, no. API version, kind deployment spec. So here in the spec, we see these match labels. So match labels is used so the deployment knows which pods go with it. So this has to match up with this. Okay. And well, all the other stuff we've already talked about. So yeah, let's go take a look at hack empty. Maybe let's take a little pause here. Let's ask a question. Please answer this question, everyone. Let's see how we're going. So do you have any questions? And what would you like to do next? Is there anything in particular you'd like me to elaborate on? So what I would do is try to create an ingress and run a connected to a web service. And this we would do live together. So I'm not hearing anyone. And that's always a bad sign. I don't know if it means you're getting exactly what you want. Or yeah, go ahead. Yeah, yeah. Okay, yeah. Okay. So the question was sort of what's going on behind the scenes. So things like you need this many servers and this much load balancing and so on. So in some sense, yeah, we haven't really talked about that because for the most part, it's not something that like it sort of all magically magically happens in the background declaratively. But let's take a look. So let's look at deployment and see what relevant things are here. So as I'm scrolling down, why not make this larger? So we see under deployment, there's replicas. So this would tell us the number of desired pods. So defaults to one. So when we need 50 web servers, we would say 50 deployments. And in my example here. So most of these deployments have only one running because there should only be one instance of the bot. This Zula pillow is defined with two pods in it. So it's two of two. Okay, what else might be ready? We see like strategy here. So you can define things like rolling update. So what's the maximum surge number of pods that can be made? What's the maximum pods that can be unavailable at any time during this rolling update, and so on. Um, yeah, so all of these kinds of things are like they happen. Kubernetes programmers have done a lot of work to make this easy to do. And I mean, once you have these yaml files that define things, it's pretty easy to define the different properties you may need. Let's see, I'll keep going down. So here's all the different API properties, which somehow relate to the above. So far, all I've ever used is this, these these yaml definitions up here. I guess another question is when you do things like rolling updates, how do you trigger it? So do you trigger it by manually running group control apply with the new file? Or is this automatic? So there's different things that you can run it directly from the Git repository. So you would take and you would say, let's see, what would you say? You commit to the Git repository, you push and then the CI will take it, verify it, then run it and then automatically apply the configuration to Kubernetes. And that's something that I haven't gotten to yet. And I think within our team, no one has gotten to yet, although it's something we would like to do in the future. Did that answer your question somewhat? Any other questions? Yes. Basically, it's all automatic. Yes. Yeah, you need to know how to declare what you want, but then it automatically happens in the background. And this yeah, so yeah. So let's make a let's actually let's try to go to this next step and do the load balancer things. So please continue commenting if you would like me to do something else, but I'm going to try to do some live development here. So it probably won't be very successful. But let's just see. So we have Zulip Hello, running here, which actually is in Zulip. It's some echo server. Okay, Kubernetes YAML echo server deployment. So this is running this echo server. So we have the deployment running. So first, we have to test the deployment. And I want to see can this container accept a connection on port 8080? Yeah, okay. There was chat messages about audio, but I think most people can still hear me. So I will just go on. Okay. So I know there's this kube control proxy command. Let's see. So I want to kube control proxy into this pod. This looks like it use port forwarding to access applications in a cluster. Yeah, I want to kube nodes. Yeah, I want to kube control port forward into there. Okay. So this is running some container. Um, I'm scrolling down to the good thing. kube control port forward. This does. Yeah, let's try it. I need the pod name. So let's forward to this one. There's a new question. Let's see. I think that's a good answer there. So like running locally, you just have different pods. But let's come back to that later. Okay. So what do you know it worked? So I guess this is running. Okay, so I was able to access this pod on port 8080 from my own computer. So we know the pods running. So what's the next thing? Now we need to make a service. So we go to the API. And let's look at the different things. I think service resources looks reasonable service. So we want a service. Okay, um, let's readjust our windows some find this service. Let's copy from here. Name. Okay, so now we need the spec. And this will be new to me. We scroll down, we see service stack. So selector. So this routes the service traffic to pods with these labels. So where do we get this? Well, we would get this from the deployment. So switch to the deployment window. So the pods are made with these labels. So I copy it back to here. And there's actually all different kinds of things for these selectors. Um, ports. And if you'd like me to stop doing this at any time, just let me know. Port dot protocol TCP. I don't know if that's needed. Let's just try it and see protocol. I'm just going to ignore that for now and hope it works. If it doesn't, then we get to learn something. Type. Um, yeah, so cluster IP means it's assigning a static IP visible from inside the cluster, which will be the accessible point of this IP families. Ignore that. Let's assume that most of these will be automatically assigned external IPs session affinity. Yeah, most of these seem like they might be relevant, but let's see. Do you want to give it a try? We need to apply with dash f. Okay, we see an error. And I've noticed with these errors, it can be quite hard to read. So here, there's like long recursive names that include. So where's the actual error in here? I think it's this it got a map instead of a string. But spec selector labels from a string to a string. Let's take a look. It's map looks okay to me. Let's go deeper and see if we can find an example to copy. Spec selector. We don't need an extra labels there. Yeah, ports protocol TCP. I think that looks good. Let's give it a try again. Got array expected string ports. If anyone Oh, yes, there we go. Thanks. This is why we do pair programming. So now we have a service. So notice we see it's assigned some internal cluster IP and port. There's a question in chat about the app.kubernetes.io part of labels. See, I think this is some sort of a convention like namespacing the labels. For something small like we do, I think it doesn't really matter. And as you saw in my previous one, but I guess if you get to a massive scale, this might start to be important. I think also part of it was there can be tools which can automatically help you view your cluster. And by using the standard namespace labels, it can automatically do a lot of stuff. Okay. So let's try port forward. And I actually don't know. Yeah. Well, let's look at the help for once. This is a lot of text here. Use resource type name deployment. Yeah, so we can do service slash name. Yeah, like this. This is what we're going to do. So now we're going to connect to the service. Okay, there we go. So now let's come back to our example. And now what do you know here it is it's running. So now we've got some basic load balancing here. This may not be exactly what you talked about, but we have one point where connections come in, and it will be distributed among two different pods. So now we can start scaling out and just transparently increase or decrease the number of pods without, without changing the service definition, and things will just work. Okay, so what would be next? So the final thing would be to make an ingress. So the ingress let's come back to the API. Could it be under service resources ingress? Yes. So here we would define things like what we see there suspect ingress spec, different back end names, rules. So with these rules, we can define things like the host name, the HTTP paths that will be forwarded. And I bet if we keep going down, there will be a backend. Yeah. So then you guys can have different back ends. So if the back end is service, it will forward it to the service of a certain name and a certain port. So I guess I could go further and try to take one of the existing services we have like Jupyter and then deploy it. Actually, that might be a bad idea because it would take down the other service. Hopefully you get the general idea here. So the ingress defines the incoming path, the whatever, and the service it gets thrown to. And then suddenly you can connect things coming from outside to pods. And the pods can be transparently scaled up and down using the deployments. And maybe I should stop here since it's been about an hour, and we can take a few more questions. Maybe I'll do one more demo. So here's our deployments. We see there's two pods running. Let's say you need to scale up. So let's change this to replicas five, and then group control. So we're going to reapply this deployment. And now we see two or five pods four or five. So we've just scaled this up to five. And now we can scale it back down if needed. And everything is sort of automatically being handled in the background. Okay, so here's HackMD question. I guess there's load balancing at all kinds of different levels, like at the provider level to different clusters at the service level to different pods, multiple different ingresses and so on. Yeah. So let's come back to our initial what we wanted to get out of this. So brief glimpse. So I think we probably have done that. Better understanding of when you would use Kubernetes. Yeah, so after this, I mean, I wonder when would I use this? So it's pretty easy to run MiniCube on your own computer. And then instead of managing Docker containers manually, you can define these things statically. I guess sort of like as an alternative to Docker compose or something like that. That might be a reasonable use case. But you'd probably only do that if you need to learn Kubernetes anyway. How does it relate to singularity and Docker and Slurm? So my guess is if you're doing stuff on a HPC cluster, then like with Slurm, Kubernetes is probably something that's not really relevant there. Although I was at one interesting talk about a year or two ago, or someone claimed everything will become Kubernetes in the future. So if you have any compute resources, you will have to have a Kubernetes API in it to run things. So Kubernetes does have concepts like jobs. So you can say I need 100 jobs that run this container and do this things and then these mounts and then it runs and could do your computation and save it. So if you really wanted to, you could use a Kubernetes cluster to do batch computing also. But the amount of overhead for a typical researcher to do this well, most are probably not going to be able to do it. Okay, let's take feedback. So possible traps, please. Yeah, go ahead. Yeah. Yeah. So the question was about the Zula bot itself and being surprised at how it was dying so often. So really, I think so I only changed it to Kubernetes about a week or two ago. About the time I said this, and since then it hasn't really died that much except it was, but there was. So now Kubernetes will restart it if it dies. A problem before is that the bot itself, the process because it's multi threaded when there was an exception, it didn't actually the process didn't terminate. So it didn't know it had died. And what I was using before it was supervisor on auto shell server. I transferred it to Kubernetes and then it still wouldn't restart because the pod wouldn't die when it was done. So I modified the code and I made it actually exit on this error. And now I think since then I haven't had to restart it. Well, much. So I think the main problem of the bot dying is not it not Kubernetes not being able to restart it, but the process itself not terminating. So Kubernetes knows it should restart. And there are actually ways that you can define liveliness tests like Kubernetes can periodically run something and pull a service and see is the service still actually alive if not kill the pod and restart it and so on. So yeah. Someone just said it can be like an HTTP check or executable running inside the container returning a status code. Yeah, please give questions, comments, feedback. Um, did this did you get what you wanted out of this tutorial? That would be a good piece of feedback. It's been about an hour now. So maybe we can stop now and see if there's any more follow up discussion and see. Yeah, I mean so the recording that yeah. So I'm still recording, but anything that the audience says will not be in this recording. So please like you can ask and I'll repeat the question unless it's unless you want to do discussion and then I'll just stop the recording. Okay. Yeah, go ahead. So is your question about how to link kube control to a given cluster from someone else or so from my understanding there's kube control which is the standard command line interface. There's probably more and then in kube control there's these configs which let me switch the screen to make sure there's not secrets in here. Yeah, I can't show you the contents of this file. It looks like it has secrets, but anyway to so what I'm running here is actually running on our computer science kubernetes cluster and um in this file there are things like actually I can tell you the fields in here. There's a clusters list and it says cluster server hdpskdsproxy.domainname and the port name certificate authority data and so on. So basically to connect to the cluster what I did was I well the person you're hearing here I'd say how do I connect to the cluster? You said okay here's this config file put it in .cube config and then you can connect and after you do that it's just completely transparent so it doesn't matter if it's running on your own computer or somewhere else so I think this is some instructions you would get from whoever has set up the cluster. They would tell you how to connect and then everything else I've said here would be the same and really to me this connection and the format of the kube config files are sort of magic so I know there's ways that you can specify multiple different clusters so you can like use different contexts and connect to both your own one and one running in a cloud provider and so on but um yeah I remember once our clusters what was it the certificate data changed and I had to go find the new config file to be able to keep connecting to it and I got quite a bit panic there but somehow I found something I copied and pasted and then it worked so yay I guess okay I'm not seeing many more questions I'm going to stop this other recording so for everyone watching thanks a lot and hopefully we can see you later. Bye.