 Okay, hi, welcome to this talk. I'm talking, I'm gonna make a simple demo of a Kubernetes deployment. So there's a very specific audience here. So it's people who roughly know about containers and research software and web services and stuff like that. And are wondering should or what is Kubernetes and how can it be used to deploy web services? I've already given introduction to people here. You have this in the chat. Please keep pasting it when new people arrive. Yes. Oh, also my OBS recording has my audio but not other people's audio. But we can see which audio we share later. Okay, so the roadmap. So once someone I saw on, was it masted on I guess? If someone posed a question, what's the easiest way to make, to like put some file and make it available on the web? And there were all these discussions of like easiest way to register a domain name, get a server, stuff like that. But my initial thought was, if you don't really care if that's the only thing it's GitHub pages, isn't it? Or GitLab pages. So that's sort of the idea here. Let's say someone has developed a web service and they want it to be available online. And someone else has done all the different infrastructure that you might need. So in other words, you have Kubernetes available for you. So I think Kubernetes is actually a pretty good way to do that because it handles a lot of the different sysadmin kind of things. And then you just need to focus on your own work right there. Okay. Yeah, so that's sort of the over, like the guiding principle here. So let's get started. My keyboard isn't scrolling. Okay, so that was this little roadmap here. So I think that Kubernetes could be useful when we're making these web services. They're not really that demanding therefore testing purposes and so on. And we have the cluster available and we can deploy onto there. Let's see. So I used, the question was really be ultra specific. So I'll be talking using the CS Kubernetes cluster as my example because that's the only one I've ever used. But I think a lot of what we're discussing is general and general questions are completely allowed. We have people from the Alto cloud team here and they know about these other commercial cloud clusters. And I think at the end of discussion will be how similar or different they are. Okay, yeah. So CS Kubernetes, I think probably is production ready. Let's discuss this at the end. Please add it at the bottom and yeah, okay. So yeah, some basic glossary part one. The one thing that might be confusing and I think this is how it goes image is basically the file system. Container is the running copy of the data and all that. Like the file system when it's running. So you have one image but you can have many containers running on that of that image. I don't know if this is a Docker terminology but it's something that's confused me. We've often, oh, okay. Now everyone can write on HackMD without registering. Good question. Okay, so step one, have your web service. So this is not a main part of the talk. So I assume that you already have the web service somewhere. So as an example, I made this quick web service which does a search of the SkiCom docs. So rski.com.alto.vsite. So the idea was, well, it was more a toy service to play with some ideas. I don't know if it's actually useful. You can see the source right here and you can test it by running here. So I'll just give a quick example. Let's try, try to account. And it looks like it's returning the right kind of information from the docs. Okay, that doesn't really matter that much. So the point is down here, meta search.py. So if I run this, it creates the big database and we'll start something on a port. And you can try to follow along with this if you want, but it doesn't really matter that much. Okay, let's let it finish. And when this is done, we'll be on. So it says it's running on local host port 8000. And we can, well, you can believe me, but I realized this should be on my other computer. Okay, so you have a web service. So what do we want? We want this online. We want it with SSL. We want it to have some domain name. We'd like it to be reasonably secure and we want minimum effort. And especially these things like with SSL and with domain name can take a little bit of manual effort. And that's annoying. So what's the first step? So it's containerize it. So again, this is not the main part of the talk. I assume that people here can figure out how to containerize a web service some other way. If we look, we can see the Docker file. It runs. This is actually a multi-stage build where first it builds and it builds the database. And then at the bottom, it copies just the search database and search.py into the new container. It sets the properties right. And importantly, if you're new to containers at the bottom, it says this command. And basically clearly says it's running the service with serve. It gives it the existing database and tells it where to bind. And then it tells it, okay, this port we expect incoming connections on. Yeah. Okay, so let's demo that. Yeah. So the question is, does it all need to be in the container? And basically, yes, because it's not running on the service. So as we'll see Kubernetes works by pooling containers from registries, so it has to all be contained in there. I mean, I guess you can do things like we have a Jupyter container for our Jupyter Hub where it mounts the user data in with NFS, which is useful in some cases. So I guess you probably can do a lot more, but the starting point should be stuffs in the container. Yes. Yeah. Yeah. Right. Yeah. Yeah. Okay. Yeah. So I guess your idea, so the question was, is it better to use an existing image rather than build our own? And that's actually a good idea. And I guess, I won't say the name for the recording, but the person who asked, that's a good point. So in here is only search.db. It's only search.db and search.py. So we could use a completely generic Python image and only mount those two things inside of it. That's a good point. Let's discuss more later. I mean, this is, you probably know more about this than me, what people actually do. Maybe I should start doing that more. Okay. There's also this build script. So whenever I was first introduced to Kubernetes, I saw things like these scripts, which would basically build the container with Docker with all the right arguments to make it easy to run. I don't know if that's best practice, but yeah, I guess you all can script stuff as well as I can. So how do we run? I was giving the example here, Docker run. So I'm copying this tag and we see serve database on this port. And I guess we can probably for Docker in order for this to work, we have to tell it to expose the port. So port, these kinds of things are what's required for testing. Yeah, okay, here we go. Yeah, so I was able to run. So it's running in Docker and I'm testing locally. Okay. So again, this is not the main part of the talk, but sort of the basic setup stuff. So now we have to share the container on a registry. So the registry is a service that hosts containers. And there's different ones. Docker hubs, probably the most famous one because Docker, could it be my CPU or internet, which is causing me to break up? How bad is it? Does it happen to be when I'm loading the web pages or something? Well, let's just hope it's okay. I don't think there's much I can do about it right now. I guess this is a reason to present from home and not from work with a proper computer instead of this laptop. Okay, so registry host container images. Docker hubs, the most famous one. So we put it on the registry and we tell Kubernetes the address of the container. So as you'll notice, this container I made is tagged as harbour.cs.alto.v slash alto RSE slash ski comp doc search colon latest. So interestingly, the image tags include the address of the image. That's how they started doing it. So I guess there's some benefits there. So this is the universal locator. Docker push, Docker pull, pulls things from registry. Docker login will handle authentication. Docker is not the only container runtime around now. I don't know if there's something better for building the containers these days. People can add in the notes themselves what's recommended. But yeah, so I have it all set up so I can Docker push this. Well, it's already pushed, but let's demonstrate. Yeah, it already exists. So Kubernetes handles pulling this. Is everyone videos off? I can actually turn off my video. Okay, let's see if this is better. So now we're getting to the actual Kubernetes parts. So Kubernetes is the container orchestration system. So we define what we want and let it figure out how to run these things. It's declarative. Ingress handles incoming web requests. Pod is the unit of scheduling. So I think a pod can have multiple containers working together. My pods have always had one, but you could consider like pods the unit of scheduling and replication. So if we need to scale up, we request more pods. The Jupyter deployment in Kubernetes has one pod per user, for example. Service is like a pointer internally. You can look at it as like the internal DNS in a very broad thing. YAML is a human readable configuration language. You'll probably know this. KubeControl is the local running command line interface, the thing that does everything. Actually, it's the API server that does everything. So KubeControl completely runs via API over the web. You configure your local KubeControl to connect to the cluster. So here, this is my work station. KubeControl is configured to connect to the computer science API server, which then controls everything. So basically I'm developing locally, running stuff locally, and it knows how to authenticate me and do things. It sends all the config there. Namespaces separate different users, provides at least things like permissions, resource limits and so on. Okay, so let's get started. So like I said, Kubernetes is declarative. So we declare what we want. Once we have the declared, we can do things like create this definition, update this definition, delete and recreate this definition and so on. It defines everything. So the image, what services run, what ingresses there are from the internet, what domains names it's listening on and so on, stuff like that. And here is an example. So the three main parts in here are ingress, service and deployment. And these are all separated as different YAML documents, which basically means things separated with the three slashes. So let's see. First up is ingress, which basically says, please proxy this HTTP from wherever into me. And I guess this is somehow sort of deployment or dependent on the cluster, like what ingresses are even available. So there's some common framework here. API version, what kind of object it is, this is ingress metadata, we give it a name, we give it the namespace. So there's all kinds of magic words in here like annotations. So this here tells the computer science Kubernetes cluster, give me automatic Lexen script SSL certificates, which is amazing, isn't it? So just by declaring this, we get SSL. Here, here we can get the, this is, okay, I should stop clicking, I guess. This tells us the incoming, this defines part things in the ingress itself. So it's telling it the maximum HTTP post body size should be larger than default. So this is a pattern we'll see often, we declare different configurations like this and somehow it gets sent into the container, which is running the ingress and tells us let us take a larger body size. Many of these things are magic to me, I find them somewhere and use them. Okay, now here we have the specification of the ingress itself. So it tells us we need incoming on these hosts, secret for the SSL, I guess. And this is where it's sent. So notice this is kstest.cs.altered.fi. This is an automatic wildcard ingress. So there's wildcard DNS, that will take any name here, and then it sends it in time. So I didn't need to do any DNS configuration, I could get this running as a test with SSL and domain name, by just writing these two lines, and then the rules. So from this host, we take the root prefix and we send it to the service called ski.com.search on this port. Okay. And these kinds of things I basically have one and I'm copying it over and over again. I'm not figuring it out from the start. I guess you'll probably do the same. Let's go on the service. So again, the basic starting point we see, the API, what kind it is, it's service, name, namespace, labels. Okay, and then the spec. So this tells us it's HTTP port, target port. Yeah. And then this defines what can come in on it and this defines where it gets thrown. So it gets thrown to an app named ski.com.search. Again, this is something that sort of pulled or copied from something that already exists. This gives us some sort of internal DNS. So the containers inside the cluster can access it on ski.com.search.rsc.internal, something, something, something, if it was needed. It's not needed here, but in Jupyter stuff, this is used. So yeah, you could look at it as like the internal interaction mapping from a name to something that handles requests. Okay, I don't, not seeing any other questions. Let's go on. So the last part, this is the actual deployment. So again, copied from something I had that already worked. Give API version kind, metadata, labels, and then the spec. Okay, what's the deployment? So deployment defines a template of a pod and then things like replicas, like how many are needed. So it sort of manages the scaling. So this deployment may say I need one copy of the pod. Other than tomorrow, we might need more. So I change replicas and deployment automatically scales to two pods and things like that. Or scale down when a course is over and I don't need as many things. Okay, so the spec defines the service itself, how many replicas, selectors. So here, this tells it how to find the matching pods and under the template, it tells it how to, it gives it these same labels. And we see a bunch of stuff which you can probably read as well as I can. So whenever a pod dies, it should always restart. We give it security context and then the white part containers. So the search, the image name, image pool policy. So every time it restarted there, container port, we can do things like put environment inside of the containers. So this uses a secret and will automatically mount some token inside, or no, it puts it as an environment variable inside the container. You can also mount secrets as files inside. And then some, and then there's some secret which is used to pull the container since it's not a public image. Yeah, let's see what else I had written under the notes here. So the ingress is probably cost-dependent services that deployment defines everything we need. It defines how to match it if it's already running so we can scale it up and down. And we see some of these features that I had talked about before, like the automatic SSL and the automatic hostname. We see an environment variable coming from a secret or we can find them here. A common pattern I'd seen with other containers that already exist, they define a container where all the configuration can be done by environment variables. And then you can use the upstream container and define the basic parts there. Are there any questions? There's a lot of chat going on, but I think it's unrelated. Yeah, yeah, mm-hmm, yeah. I guess, so the question is how do the different like containers running as part of the service know their identity or know each other? And I guess the idea here, as far as I've seen, when you make one single deployment, it does one thing and all of these pods are independent. Like they probably aren't communicating to each other and they don't really have a concept of who they are. It's like web scale kinds of things where you define these microservices and they're independent. Yeah, yes, exactly. So you don't see it here, but the Jupyter.cs under somewhere in the spec, it defines the, actually I probably resource limits like, so it will define the standard container stuff. So CPU request and limit and memory request and limit. So request means how much I think I'm using on average and use that for scheduling, stuff like to try to not overload a node and limit is the maximum amount I can use. So yes, exactly. And in the Jupyter one, we do this very well, but this is so small, I haven't done that yet, but I guess we should add that as an example. Okay, yeah, mm-hmm, yeah. Yeah, so the question was, is this best for things which function without side effects? And I mean, I guess that's sort of the idea it's been designed for. So each container itself shouldn't have side effect, but for example, you could have a different container that's running a database and that database has storage mounted to it and maintains the state across all the different restart stuff like that. So there's the question, what kind of side effects we've talked about. So I guess it would be things like, if it saves some information, well then it should save to a different database pod if it's modifying something else. I mean, I guess at the web scale kind of thing, there's the web services and then there's the databases and these are often separated out into different services. So database service runs with more stability and knows about things like replicas and all that. And the service itself is designed to just be completely stateless. But if we need to define state, basically we can go mounting in the storage there or whatever. Did this answer your question? Maybe we can come back and discuss more later. Okay, let's discuss more later. Okay, so do the deployment. So how does this actually deploy? So, so far we haven't touched group control but it's this apply command. So dash F met search. So if we deployed this, it says the service is unchanged. If it was new or changed, it would tell us what the updates are. And I guess I could go do a demo of the update but you can probably imagine how that works. So in this case itself, copying from the Jupyter Hub kind of things which I had seen, I made a script called redeploy which does everything, it builds the image, it pushes the image, it applies the service and then it runs Kubernetes rollout restart with the deployment name. I should have linked the script here. But this basically takes it and the rollout says stop and restart all the pods which causes it to pull a new version of the image. So here I use the latest image. I guess you could define a specific image version. So every time you update it, you would update that, that tag. But this is development. I didn't care that much. So yeah. Okay. And I guess by now you know how it works. It reads the YAML file, it parses it, it sends it to the Kubernetes API which is on some other computer and that does everything. So it pulls the image from Harbor using the pool secret. It figures out what needs to be run. It creates the ingress, it creates the service, it creates the deployment. It waits for the deployment to come up and then it's there. So the secrets themselves I've defined separately from the command line. So you can see examples here like, I made a secret with group control, namespace RSE creates secret generic. These are all kinds of things which I've copied from other places. Mentioning the service. So there's things like you can get the pods that are running. So here we see there's one replica running. We can describe this pod name which gives us lots of information about it. We can get logs from the pod. This is running on my workstation and communicating by the API to the Kubernetes master nodes which someone else has set up. Yes. Well, I guess I don't think I did group control login but I found a wiki page that said how to activate group control and copied that stuff without knowing exactly what it was doing which basically someone would tell us, here's how you activate group control to connect to this. Yeah, okay. And well, here's some different examples of things we can do and basically manage everything. So we see deployments, get there's the service. Here's the internal cluster IP which is managed for the internal network. So I think since I'm in CS we're all admin so we could fill the cluster with stuff but one of the points is that you can do resource limitations and probably what would happen is we would be allocated a namespace with certain resource limitations and then be expected to work within those limits. This is something which we would talk to the Kubernetes experts. So unfortunately the Kubernetes expert in CS just had a child so they're obviously not here right now but I'd say in general, every time I make a new service I do it but I go and I talk with our local experts to figure out what's wrong with it and make sure it's correct and get some advice on that particular thing. So most of these Kubernetes things I'd say we shouldn't try to learn everything ourselves but learn enough to get to the container point to push the container and then try to make a YAML file based on a copy and then ask for some advice for the actual deployment until we get good at it because we have specialization of labor for a purpose. So basically I just said we have specialization of labor so us developing services if we can get it to the container stage we can talk to people that are Kubernetes experts to get through all the other stuff and that's more efficient than other things. I've talked to the CS Kubernetes expert whose name I won't say many times for all these services and 15 minutes there saves so much time but I propose we move on to get to the Q and A so what else can Kubernetes do? It can scale so changing that replicas numbers we get more services there. If we need more hardware we get someone else to add hardware to the cluster whether it's locally or whether it is in the cloud and then we can scale more. We can request and limit memory CPU I guess probably time also and there's two tiers to how much we expect it to use and how much it actually uses. We can move stuff around. So in theory this YAML is going to be pretty similar and after we deploy locally for testing and we need to scale up we can make some small changes and then deploy to Azure or AWS or whatever. I don't know the practicalities of doing this but it's possible. We can also scale down, we can run locally. There's all kinds of things like mini-Q, micro-K, des, whatever that will run a Kubernetes thing locally so instead of running the Docker images you can test on your own computer. I think I am running one of these somewhere but I don't really know much about it or which one is recommended these days. Everything, yeah, yeah. So we just had a comment saying pod men feels better than these mini clusters which sounds good to me. So everything is declarative. So it's easy to make things from scratch. There's no, well if I move somewhere else I have to figure out what I did to set up the web server and that kind of things. The clusters are shareable. So for example, we can take the CS cluster and get RSC access to it and then we can start running stuff there. I don't know exactly how that works and of course you can do things like deploy from Git which I've never done but obviously people do that. What are the limitations? I would recommend doing this if you have someone you can consult with but luckily we have that. So what's my proposal? We work on our stuff. We develop containers for our services. We try to make initial Kubernetes ourselves and then we work with Kubernetes experts to do deployments and now we've got lots of Q&A here. So I propose I turn on the recording. Does anyone have questions but you don't want to be recorded on YouTube? My local recording has been excluding everyone talking remotely but I can turn that on and we can have a general discussion if no comments. Okay, from this point on if you say anything it will be in the YouTube video. So be aware. Q&A. So I'll be discussing here other people please go writing your proposed answers to these questions directly and we can have multi-threaded discussion. So is CS Kubernetes production ready? Okay, to use bit heavy calculations. So it's in pretty heavy use. There's like hundreds of CPUs and maybe close to a terabyte of total memory there. So I think we can use it for semi-heavy things and it's being used in production for things like the CS educational technology and Jupyter Hub. So it needs to stay running but for like really long-term stuff maybe we should look at other things. I don't know if this is a policy decision so other people need to talk. Could CS Kubernetes provide runners? I guess it probably could. Yeah, it seems reasonable. What provides the security isolation? So I guess it's the container runtime that would do that. So it will be as secure as the runtime is and that's a question which other people can answer more than know about this. I always hear the idea is it should be pretty isolated but then also I hear Docker and so on isn't designed for security isolation. When is this the right model for a web service? Well, I think I've explained when I think it's right and you can probably use your own thought to think is your service matching this model? So the big question is things like this next one here. So for AI models, loading all the weights can be the slowest part. Like when the container images get really large so you're dumping say many gigabytes of like the AI model weights inside of there then I'd start wondering is there a better way to split out the data from the container itself? I mean, there's might be ideas like we put the model weights elsewhere. Yeah, so on NFS for example and mounted in or local storage on the nodes and mount that into the container. I don't know. Other people please give comments here. How different are the different implementations? That I actually don't know. Maybe some people from the also cloud team can write in here that answer or speak up if you wanna be recorded. So isn't it heavy to use Kubernetes for a small thing? So I think the idea here is yeah, you wouldn't want to set up Kubernetes for some small little thing, but if it already exists, it might be easier to use it than to try to set up your own web server some other way. So that's the big idea here. If we can outsource the Kubernetes things to something big enough that it's worth knowing about it, then I think it makes a lot of sense. This is outsourcing both the management and some of the knowledge. How do you put secrets? So I don't know. I mean, I've seen environment variables used a lot RCS Jupyter Hub has some mounted files in there. I mean, I guess it depends on how you can write the application. So I wrote the little web service to use environment variables in Jupyter Hub. There are some things that had to be files in there. So we mounted them as files. So no, it's there. Build your own images. So here's an example. So we have a thing, a test service running. That's the HedgeDoc, which is like a HackMD, an older open source fork of HackMD. And for that, it's running in our cluster. It uses the upstream image and it's set up where by using nothing but environment variables, I can do all the configuration that's needed. So there it's using the generic upstream image. And I think that makes perfect sense. You might want to fix it to a certain version. You might want to always tell it to use the latest one. Yeah. I mean, I guess we want to do the least amount of work possible. So thank, do we really need to build our own image? Best practice for auto updating containers. Oh, I guess a reason for building your own image is that then it's usable without Kubernetes also. Like it can be converted to a singularity image or something like that and run on the HPC cluster and so on. That's something to consider. Best practices for auto updating. Well, that's a good question. I mean, I don't know if I really have much to say. If you're asking that question, you probably know what the trade-offs are. As the fun joke, so on Christmas Eve last year, the, one of the Zulub chats went down because it was set to automatically pull updated images, but there was a problem with SSL on some other service. So yeah, Christmas Eve, I was debugging why a service wasn't coming up when it was set to automatically update. That was fun. People, so to complain about the complexity. Yeah, I don't know. I mean, it is super complex. It's something that with the declarative syntax, you can do anything. I mean, if you ask me now that we have someone that actually runs the thing, it's not that hard to use. You just need the scale where you can specialize and let someone else know about Kubernetes and you know what you actually need to get done. So that's my take. Yeah, I mean, this is a relatively simple YAML file here. Imagine what it can look like when it gets more complicated and the jokes can write themselves. Is it easy to run a service in Kubernetes but external database? Yeah, I mean, it's a bunch of network stuff. So you can tell your service running inside that connects to a non-Kubernetes database outside. And it does have a single IP address to allow connections from. I think it's, or doesn't. I think this is something which can all be managed. So take our Kubernetes expert and I really wish I could acknowledge their name by saying it. But I mean, there's so much we can do. If we say I need all connections coming from this single IP address, I'm pretty sure that can be arranged. Yeah, so I mean, this kind of thing I'd say, probably possible, come bring, come talk to our expert. Okay, any more questions, comments? I'll scroll up and see. Yeah, so thanks to everyone that's been making comments and questions here. Let's go back together and try to update the notes as a reference for someone that wants to follow this talk later. There's some good clarifications which should be added up there. And yeah, this has links to all of these different things. So you can basically copy it for any new services you might do. I'm gonna add a Creative Commons Zero license to the YAML and that kind of stuff. And yeah, I guess it's almost time we should get going to lunch somewhere. But we can talk together about how to use the CS Kubernetes for this or the public cloud Kubernetes. So with that, I will stop my recording. And yes, thank you, bye.