 Alright, next up we're going to have Jaguar from K-pop to stay us updates on Rocket's integration with Kuban edits. Thank you. Yeah, thanks, Stefan. Yeah, so I'm going to talk about Rocket. Basically, our project is about the update, so what's being new or relatively new in Rocket and also about his community integration with his Rocket list. So, first, there's something about me. So, I live in Berlin, which I really like. It's a first on the list. And I'm one of the co-founders of K-pop. So, at K-pop what we do is we're basically a software team that, you know, companies hire us to do development on their products and we help their teams with that. And we do a lot of work on Rocket with ProRes and then on Rocket Live after that. And, yeah, I'm currently a maintainer of Rocket and Rocket Live. And, yeah, you can see my K-pop there and my career. I don't really tweet much, but I retweet a lot. So, yeah, the plan for today is I'll explain a bit what is Rocket. I assume that most of you will have heard about Rocket, so keep it brief. And I'll explain a bit how Rocket works, some interesting internal details. Then I'll do the same with Rocket Live, what is Rocket Live, how does it work. And then I'll talk about what's new and then what's missing. So, future work. Yeah, Rocket is basically, that's a tagline of modern, secure, composable, engineer runtime. And we say modern because we try to take advantage of the latest technologies in Linux currently. We say secure because we try to make it secure for default. So, by default everything is locked down and if you want a disabled style, you have to do it explicitly. And composable because we try to play nice with the rest of the system. We try to reuse components that are already there. So, basically, Rocket is an engine of the API spec, which is now not really actively developed. But, yeah, I think it kind of triggered future movements like the OCI spec and all this stuff, all this standardization that happened in the containers. So, yeah, the app says an open specification. It defines some concepts like the image distribution and things like that. And it has a much bigger tooling. So, there's a lot of things in the app see organization of GitHub. But, basically, it defines four things. The image format, which is the ACI image format. And that defines what the application consists of. So, basically, a container is just a toggle with some associated metadata that's usually a manifest. And that defines how this toggle should be executed, like which namespaces, what security features should be enabled and who stuff. So, that's the image format. The app see also defines the image discovery. And this is how you can go from an image name to an image artifact. So, this toggle I was talking about. And, yeah, that's pretty cool because it reuses the DNS system. So, if you have a container's name, corporation.com slash whatever, it will go to that website and then that website will have some metadata that will tell you where the actual artifact is. And that's cool because it's distributed. It does not centralize ready to be and so on. And that defines the concept of pods. So, in rocket containers, it can be grouped where it makes sense. So, if a pod needs, if a container needs access to some other container, it makes sense that they're all together. So, yeah, the pods define that concept. It's very similar to the Kubernetes pods. And then the executor. This is just what the application should expect when it's running. So, the environment you run in front of you. So, in the end, rocket is just a CLI tool. It doesn't have any demons. It's just a CLI tool that you run and then your container is running. It's running in GoLang and it's, yeah, Linux-specific. So, we don't try to report previously or macOS or all this stuff. It's pretty self-contained. So, it just has some simple dependencies to the GLC and the typical dynamic libraries you have in your Linux system. And it's in its system and it's diagnostic, although we optimized this to run on the system needs. So, this is basically the typical 3D diagram of rockets. So, something will start rocket. It might be system need. It might be batch. It might be our unit. You name it. And then, rocket will start a pod. So, you see there, there's three stages. Stage 0, Stage 1, Stage 2. So, Stage 0 is just a rocket binary CLI so that does image fetching. It renders image in the pod system. Stage 1 is the actual container. So, it's the environment that pods will live in. And it does project lifecycle management and it contains the resources. And it stage 2 is just your application. So, it's whatever you run. The cool, wonderful thing about this stage 1 is that it's comparable. So, there's different limitations. The default one uses the new things, Paces and C Groups, which is what people used to call the Linux container. And it's based on system need and spawn. And inside that container, we run system need again to manage all the applications in the pod. There's also different implementations. There's KBM implementation, which it uses hardware for qualification instead of Linux container. And it's based currently on QMO and system need. So, that's pretty cool. If you have a full VM for you, you can run your applications. And then there's fly, which is something that has no isolation. It's just a skip through it. And it's there because to take advantage of rockets image distribution mechanism that helps when you want to run just an application without isolation. Okay, so, now I'm going to talk about a bit about Kubernetes. So, probably most of you will know this, but I will give the introduction. So, Kubernetes is just an orchestration system. And it provides the developer with an API and the developer can just submit and finish that API. It doesn't have to care where those containers will run and which nodes. And the thing I want to focus here is this cubelet thing. So, this is a component that runs on each node of a Kubernetes cluster. And it's the one that's actually tasked to execute the containers themselves. So, in the beginning, in Kubernetes, everything was Docker. And that was not ideal. So, when we tried to put rocket into Kubernetes, we discovered all these assumptions in Kubernetes that were there. And that led to the creation of the CRI interface. And, yeah, thanks to the CRI interface, now we have two different implementations for container runtimes in Kubernetes, like Cryo, like CRI continuity, or like Rocketlib, which is what I'm going to talk about next. So, the CRI defines some interface that container runtimes need to implement. And these are these methods, like rompods, sandbox, to create a sandbox, that's empty, no containers there. Then you can add containers, and you can then start containers and stop containers. So, basically an interface that abstracts away the container operations. And then the cubelet calls these methods via ERPC. And then you have something on the other side of the ERPC connection that implements that. And, as I mentioned before, CRI continuity is an example of Cryo and Rocketlib. So, how does Rocketlib work? As I said before, the cubelet is there. It sends a JRPG request to the Rocketlib daemon. And then what the Rocketlib daemon does is it will start a system unit per pod, and then it will add and remove containers if the operations require that, communicating with the rockets running a system unit. So, yeah, it's pretty simple. And then, yeah, you take advantage of this in these supervision capabilities. So, to implement the Rocketlib, we had to do some design changes to Rocket. Because on Rocket, we have this concept of immutable pods. So, your pod is defined from the beginning, and then you just run it. But the CRI defines something different, influenced by the way Docker works, because in Docker there's no pod concept. So, you just have containers, and then they're linked together. So, we had to implement immutable pods in Rocket. And to do that, we did this, what we call the app experiment. And, yeah, basically, we had an instance of command for Rocket that creates sandbox, and then we added more commands to add containers and remove containers and start containers and stop containers. Since, as I said before, we're running a system unit inside a container, this was pretty easy, because this maps pretty well with system D concepts, like create a new unit file, or remove the unit file, or start a unit file, or stop the unit file. So, that helped us. And, for now, it lives under a variable, so it's not really visible by default. And, yeah, that's the name of the variable. Another problem we had was logs. So, Rocket used a journey inside the container to manage all the logs. And, the CRI doesn't really like the binary format of JournalD, of course. And, the CRI wants text format, like plain text, very simple. So, the initial solution we came up with was kind of a hack, because we just add a sidebar app to the pod, and that app will translate the journal logs to the CRI logs and write it in the right place. But, that was kind of unreliable for the resource consuming. For example, the image was, I don't know, 500 max, which is not very ideal. So, the proper solution is, we did this experiment attached. And, it's basically creating this component called IoTTYmax that sits in the middle between the outside of the container and the application. And, just intercepts all the standard outputs and the input of the application. And, it can write the output and the error to the file directly, without having any translation. So, this is the diagram type works. So, you have your structure there. And, you have system inside the container that starts the IoTTYmax component. And, then the output of the application is connected to IoTTYmax. And, that's being written to the file there. And, yeah, that's the form, a simplified version of the format for the CRI logs to just update the stream type, and then the log line. And, this also allows us to attach to the input and output of the application, which was not possible before. So, yeah, it allows us to implement something like rocket attach. So, those were the most difficult parts when we tried to iterate rocket with Kubernetes. So, now I will talk about the reason for the development in rocket. Yeah, so, this is some kind of a random collection of new things that happen in rocket. So, we've amended the sharing IPC namespace, because, as I said before, Rocket uses Nspan. And, Nspan doesn't have a concept of only share this namespace, only share that namespace. It does all the namespaces, so, what we did basically was add some hidden variables to system.dnspan so we can actually share namespaces independently from each other. So, yeah, and then we've amended the rocket part. And, this was done for some high-performance, high-performance computer computing client that needed the IPC namespace to be shared between containers. Yeah, we did a lot of fixes in the rocket to make rocket-led possible. A lot of fixes on the rocket experiment I mentioned before. And, we added integration between these attached experiments, the iot21-ymax component in Kubernetes. And, also, we switched default KVM flavor from LKVM to BMO, and that was because LKVM was kind of unreliable and it had some problems with mounts. So, QMO is much, much better in that regard. Yeah, and a reason the element that happened was documentation. So, documentation is important. And, thanks to the CNCS, that funded us to actually write a bunch of documentation. And, you can see there that we closed 31 issues after November of last year. So, that was quite a lot of work. And, yeah, we think the documentation state of rocket is pretty nice now. And, we added also some documentation to rocket-led, which was lucky. Okay. So, what comes next for rocket? We need to update the CNI version, because rocket is using version O3, and some features were added later that people are interested in. So, this is something we need to do, but it's kind of hard because, yeah, there's a lot of assumptions in rocket that depend on the creepy behavior of CNI. We should stabilize the rocket experiment because, yeah, it's not nice having an emperor builder to do these sandbox kind of thing. So, we should just remove them and make sure they work fine. There was some experimentation of using RunC to set up Stage 2. And, this is all because what we've been doing now to start applications in a pod is run them in a system unit file, which has a lot of primitives, like saccom, you can enable non-privileges, you can enable all kinds of stuff. So, usually they translate well with the CRI specification, but sometimes some things are slightly different, so that's kind of hard. So, the idea is to use RunC, which is what which is defined by the OCI. So, using RunC we will have basically the same options as other runtimes. One cool thing that my colleague Albon was experimenting with is sync. I'm not sure everybody can know what sync is, but it's basically rsync plus content addressable, and it makes you, it's a very efficient way to transmit images. And, there's a talk later today by my colleague Albon at 3 p.m., I think it is, and if you want to know more about that, please attend that talk. So, there's, yeah, right now Rocket has a big image support, but it's not ideal because it's basically integrated in the Docker support of Rocket. So, what we do basically is we take the OCI image from Docker registry and then we convert it to OCI and then we run it in Rocket. But, there's a possibility of doing it natively, so you don't have to do this conversion step. And, of course, there's always bugs. So, you're talking about the caching tests, there's some graphs there, so the red thing is the amount of data transmitted using caching and the blue thing is the amount of data transmitted without using caching. But, as I said, there's going to be a talk on this, so you will get more details there. So, what comes next for RocketLib? So, we don't fully support the CRI. There's some things missing and the most important things are QCTL attached and port forward. That doesn't work at the moment. Although, i2dymax allows us to do that, but we have to write the code for Rocket. Yeah, CRI Container Stats is a new interface where the public can ask for stats to continue running instead of doing C advisor and checking the C groups manually. It's kind of ugly. So yeah, that's not implemented there. And we can improve performance because we've just been focusing on making it run and we haven't really looked at performance yet. So, we run the end-to-end performance tests in Kubernetes and we don't pass them all. So, currently we pass 130 and we fail 15. Some failures are kind of easy to fix. Some failures are not so easy. Yeah, as I said I work at Kipfork and we're a consultancy so we cannot just work on this unless we have a client that requires us to, or that pays us to do this work. So, yeah. And, yeah, I had a demo of the markdown which is kind of screwed up here. But the demo is also screwed up, so sorry. The demo was a Kubernetes cluster using RocketLift and running on Kipfork, which is a tool we built to run clusters that don't run on VMs, so sorry, that run clusters on your local machine that doesn't run on VMs but run on Kubernetes. But, yeah, the connection here is kind of shitty and I have to download things so that didn't work. Yeah, so I have one minute and a half for questions. Any questions? Yes, yes, you can. It's just a runtime option so you can just run in problem. Any other questions?