 All right, welcome everyone to OSS Summit. This is the Exotic Runtimes Talk. I have a list of keywords up here, and I'm curious which keywords have brought the most people here today. I'm going to guess that it is probably this one, more than anything. I may be wrong. Can we get a show of hands? Who's using Kubernetes? Okay, that's good. Who's using GitOps already? Most of you, cool. And who is actually using Ruby? A couple, all right. How about Ruby and Wasm together? Nope, not yet. What about who's building a Kubernetes operator? Got one, two, cool. Kubernetes operator with Ruby? No, didn't think so. Okay, cool. Someone tell me that these were original thoughts that I put together and I thought that was cool. So I wanted to know a couple things from the audience. All right, so I already established nobody was here yesterday, but yesterday I presented this as a lightning talk. Today I get to slow down a little bit and we can talk about some things that there wasn't really space for in a 15 minute session. So that's cool. I'm not going to assume that anyone has already seen the session from yesterday. So I'll repeat most of that stuff today. So this is me. I'm Kingden Barrett. I am a Flux maintainer. You can find me on these social media platforms. And I'm a YouTuber. I am a open source support engineer at Weaveworks in the developer experience department. And I work on Flux, mostly in a community capacity, but I've also been, had my hands in the code as a Flux V1 maintainer and I'm trying to contribute where I can to the code of Flux V2. I have a weekly event that's called the Flux Bug Scrub where you can find me and other Flux maintainers if you want an audience with a Flux maintainer. If you have issues that you'd like to have addressed that they've sat in the queue, you can reach us directly and talk to us face to face on a Zoom meeting. And it's the forum for it. Like we have a regular dev meeting also, but okay. So as an open source support engineer, I am focused almost entirely on open source. Big surprise. And I lean into open source solutions as much as I can. We have other presentations also here at GetOpsCon related to Flux. If you scan this QR code, you can find out about all of them. And just a little bit more about me. If you go to my YouTube channel, these are the types of things that you'll find there. One of the things that you will find there is me live coding a Kubernetes operator in Ruby this morning to show off my Wasm bits. So, okay. So what are we here for today? Wasm and Ruby, I think primarily. And what is untrusted code and why do we want to run it? Well, I will tell you a secret. I do not trust any code. That's probably false. If you look at what I do from day to day, I'm probably trusting lots of code. But in a sandbox, that's what Wasm is built around is a sandbox that is for running untrusted code, which is very helpful if you are on a web browser or if you're on a server. So we're primarily gonna focus on a server today. And I had a question when I started preparing for this talk was can you run Ruby in WebAssembly? Because that was what it seemed was on the tin. And it turns out, yes, you can, but I'm not sure that there's really a good reason to do this, at least in a server context. So we'll get back around to that. But why WebAssembly? So we wanna build on a secure foundation. If you haven't heard of the bytecode alliance, they have a great page where you can read all of the details about why WebAssembly. For me, portable artifacts, that is what I'm after. And I would like to have language independence. I cannot sell you on Wasm. If you do not know why you need it, it's gonna be a tough talk. If you take it with you, I will get nothing. There's no commission. I don't know how Wasm makes their money. But let's find out together. Okay, so why Kubernetes? For me, the answer is, I'm a flux maintainer and you have to have Kubernetes to run flux on it. So if you chose Kubernetes, you probably already know why you did. But if you haven't or you're thinking about it, this is a really brief summary of Kubernetes and GitOps together. You have declarative versioned immutable artifacts that describe your desired state and your infrastructure is self-healing. It's constantly reconciling on a path towards putting those things towards your desired state, the way you described it. So these are some compiled languages that you can put into WebAssembly modules. And if you are not interested in Ruby at all, I promise there will be some value for you here today as well. So here's a little bit about my history as a Rubyist. I have been using Ruby for a very long time thanks to a friend on the internet who helped introduce me a long time ago. And I also, my first permanent job was primarily focused on building Ruby things. We used a lot of other languages, VBScript, various things, probably more shell script even than anything else. My second permanent job, I was really focused on Ruby and Ruby on Rails and I was an application developer there with a lot less responsibility. Better pay though. So for me, why Ruby, I know it better than any other language. And for you, whatever language you're choosing, that's probably the language you want to use because you know it well. For me, the benefits of Ruby, comfort and familiarity, debugging experience, can't be beat, bundler, handles gems, dependencies wonderfully. I haven't found an experience like that in practically any other language. And fibers, which are relatively new in Ruby, they are great for concurrency. So I'm using fibers. Minimum viable product is the goal. When you're writing Ruby, you want to write fast. You want to pull in as many dependencies as you can, probably, and don't write anything you don't have to write. If it's your core business dependencies, things that you really need to know inside and out, you probably want to write it yourself, at least once. But once you know it, you'll probably want to lean on whatever's the best in class library. So what are we using Ruby for most of the time? To run a website, I guess, or connect to a database or scrape some content from an internet or maybe build an IRC bot or something like that. Lots of different things you can use Ruby for. There's no compiler. It is duck typed and object oriented. Those things are all great. Okay, so why not build a Kubernetes operator in Ruby? I think that's a great idea. And I'm not the first person. Someone put out a nice library for that. We'll get to that. So WebAssembly and Ruby, here are your entry points. You can use, these are the things I tried, at least, Wesmer and WasmTime. I liked Wesmer a little bit better. I thought their examples were more approachable and more comprehensive. But I did start with WasmTime because that was the runtime that Spin was promoting more heavily first. I'll get Spin also. So what we're gonna do with these is run WebAssemblies in Ruby, not run Ruby in WebAssemblies. And what is a WebAssembly is a runtime format, is binary format. And it is also, besides being able to run applications as WebAssembly, you can build libraries and include them in other programs, like I said. So if you wanted to bundle Ruby interpreter inside of Ruby, I would suggest that you consider not doing that because I couldn't think of any benefit, even though that was the first thing I tried. Like I said, I only wanted to use Ruby. I'm not sure why I would use a Ruby in a WebAssembly for Ruby. There's actually, we have a question? I did. Well, with WasmR I did. Yeah. So there was really no benefit that I could tell other than I won't have to use another language. But for various reasons, the top of them probably being the WebAssembly size. So the Ruby interpreter itself as a compiled WebAssembly is like 30 megabytes. So if you actually intended on distributing these things to people running them in a web browser, you may have made a terrific mistake. So we're gonna try the other thing. And so this is the type of thing that I think WebAssembly users are doing based on going through all the tutorials. And if these are not the things that you thought you wanted to do, keep watching, hopefully they will make sense as we go on. So you can export functions from your WebAssembly and call them from Ruby, or you can, well, from a language that you run your WebAssembly in. You can also export functions in the opposite direction or import functions into your WebAssembly so that you can call them from inside of your code. That's a nice way out from the sandbox if you need to provide a way out, say for fetching or something like that. And the examples were really focused on the use of a compiler, which is kind of the point where I began to infer that maybe this was trying to trick me into using compiler. So, and then there's the system interface. That's the penultimate example that is good for all of the things that you can't easily do in a sandbox. So what is the system interface good for? Input and output, file system, if you wanna import some files so that you can just access them from inside of the WebAssembly, read them from the disk. Those are system calls. And if you wanna make connections to the outside world in a protected way, there's really no network inside of the WebAssembly itself, but you can still do it with something like Wezzy. This is not the only way in, by the way, I should mention, but it's the one we're gonna focus on most today. And there's an extension called Waggy. If you're not familiar with all these buzzwords, maybe you've heard of CGI. Waggy is like CGI, so you take your standard input output and you put a little header at the top that says this is the content I'm accepting or expecting and it's your connection. You don't have to manage connections now, so that's pretty cool. You just get a request and it comes in standard input. So these are some features that you won't find in WebAssembly directly. There's no string type. That's kind of tough, especially for Ruby. I found that if I wanted to get a string into WebAssembly, I had to allocate some memory and then keep the length of the string around and I was getting flashbacks to see and I don't think I want to do that. So I was able to avoid doing that thanks to Wezzy because we can just pass the string in that way and then parse it and pass the result back out through standard output. But as far as actually passing a string to a function, that was the first thing I tried after I tried running Ruby and Ruby and that also did not work for me. So I had to find a different way. What I like to emphasize right now is that all of these constraints, as I found I was running into roadblocks, there are other roadblocks I haven't mentioned yet, a C extensions was probably the primary one. If you use Ruby gems, most of them don't have a C extension but probably somewhere in the realm of 10% of them do. So if you wanted to use NoCogiri say to parse some HTML, you might need to find pure Ruby. If you were still thinking about bundling your Ruby in the Wezm, which I suggest you do if you're interested, but like I said, we're gonna focus on the other stuff. So I was trying to aim towards solving a real problem with Wezm. I needed some string prams and return values. I would assume I would need those things. Okay, so I started with spin, which actually has great docs. And if you're trying to learn Wezm for the first time, I suggest that you start there as well. They will tell you about your language, what level of support there is for that language, what types of things you can do. And there are great examples or links to good examples from there. I did find a nice Ruby example that showed me how to pull in gems. And I think that was in the spin kitchen sink. We'll look at some of those examples later, but so what is spin exactly? It is kind of a serverless framework. And if you know what serverless is, it's kind of not really a thing. It just means we're gonna run some code and then we're gonna shut it down and reclaim all the memory and wait for another request before we use any resources again. So spin is useful for testing locally also. Spin itself is running as a server that will, like I said, help you shuttle connections to your process. It kind of behaves like a router. We're not gonna run on Fermion Cloud because that is not open source. But we are not gonna run on Hippo Factory either because I don't think that is a serious idea. And I've already been sold on Kubernetes and if you're telling me now we need in addition to Kubernetes we also need a Hippo Factory. I know you're pulling my leg. So Hippo Factory is the open source version of Fermion Cloud. So we're gonna use Kubernetes not only because we already have it but because we wanna use Flux. And these are the things that we hope to gain by adopting WebAssembly and whatever shape that finally turns out to look like. We wanna get testability, reusability of our code modules, type safety between languages and I'm also gonna add in here now smaller payload size. We want our code to be smaller when it goes to the users so that it can launch faster because that is one of the primary goals of WebAssembly if you read all the marketing copy. And also we want our polyglot teams to be able to work together. If we've got teams that have specializations that depend on the libraries that exist in particular language we want them to be able to interact with each other with as little friction as possible. So I did build some things for this talk and I followed all the examples that I mentioned so far and let's see what we did. I found a problem that was big enough to require some code to solve. My boss goes to the webpage for each of our GitHub packages on GHCR every day and she doesn't really do it every day but it takes a screenshot and then feeds it through an OCR and figures out over time how many downloads have we had? I mentioned I work in developer experience so how many downloads, how many users are important questions for us? And it's one of the few ways that we have to track our users because we don't like tracking users. Many of our users are in an air gap. We know that there are limitations no matter what we do so this is one of the things we like to know is how many downloads there are and plus it's a really big number so we like to show off big numbers. We like graphs that go up and to the right. So what we do is we visit this page and we see that there is a number there and there doesn't seem to be an API endpoint for that number but it's updated in real time and if you hover the mouse over you can see somewhere deep in HTML there is an actual real number, real whole number of downloads you can fetch. So I built EKS cluster to run this stuff on and if you go to this GitHub repo you can find all about the EKS cluster definition. It's a multi-arch cluster since we wanted to demonstrate the platform independence and it bootstraps flux onto the cluster as part of EKS if you've never used EKS cuddle it's a really nice tool. I actually used it for the first time well recently I'm sure I used it once before but this is the first time I've used it in anger and I deployed some stuff to it. I tried to figure out how to deploy so all right forget Ruby how do we deploy any WebAssembly at all? Let's say we're happy to lean on spin so I came up with a solution for how to deploy a spin application that existed already called Bartholomew onto Kubernetes via GitOps and for me I thought the easiest way would be probably not Helm but I did decide eventually that Helm would be easy enough because I found this great library to help me build a Helm chart, this library called Helmet. It's basically you write a one liner in your templates and then you expand it out with values you just say what is unique about your deployments and whatever else it needs to depend on inside of values.yml How many people are Kubernetes users by the way? All right so these might be familiar topics to everyone. So at this point I began to understand a few things and it's gonna be tough to cover them so I won't be able to go into detail about all of those and show the code I'd rather show the code but Fermion if you know the history of Fermion who's leading SPIN and a lot of the bytecode alliance stuff leading the pack there in the least in my view. They're the inventors of Helm. Matt Butcher is the CEO of Fermion and he invented Helm and they helped develop Azure Kubernetes service after Deus was acquired by Microsoft. So I was pretty miffed when I pulled up all the documentation I found there was no Kubernetes example they didn't show you how to deploy on Kubernetes at all but I did become a little more understanding about that when I tried it myself it was painful for a number of reasons. So let's step away from that for a minute I also built this series of examples moving towards a Kubernetes operator and the Kubernetes operator has a design to solve this problem where we have a project CRD that has the name Flux CD in it that project CRD goes to fan out and see how many packages are there in Flux and then it creates a sub resource called leaf for each one of those packages that sub resource goes and fetches that page that I was talking about that has the number on it saves it in a file passes it to our WASM module which we'll also talk about it's inside of this repo and then the WASM module returns the number parsed from the HTML as a string since we said string return types was a difficult thing and WASI helps us solve it, this is the proof. So once we've got the number back in Ruby we parse it put it back in a number we store that number in the CRD status there's a field defined for the status that says how many downloads count and we'll come back later we'll build a Prometheus exporter so we can scrape this, feed it in a Prometheus we don't wanna store this data long-term but we'd like to have it for a couple of weeks so we can export it regularly and then put it in a spreadsheet. How did I build a Kubernetes operator in Ruby? Well actually it was really easy this project on GitLab is inactive if you look at it but it's based on a nice library called kube-client that is actively developed so with this example you can read a single page of code it's really a very minimal code all it does is it registers whatever CRD that you should be listening to and then it registers two functions upsert and delete it manages your finalizers for you if you don't know what those things are I saw, I asked when we started but how many people are developing Kubernetes operators? We have more people now than before more than a few, all right there's still very few so these might be really unfamiliar concepts but the finalizer is the thing in your custom resource that tells Kubernetes there's something that has to happen before this gets garbage collected if you get a delete signal wait for me to come around and remove the finalizer then clean it up so I wasn't sure what I was gonna use that for but it's common in the operator pattern and if you've tried to learn the operator pattern you probably reached for the controller runtime book which is a bit longer than a page this was very approachable I thought this was a much more easy way flux is based on the controller runtime probably there are a few other controller runtime like things out there I'm not gonna try to name them but here's Kube Client the Ruby library I was talking about has some nice features like it uses server-side apply if you don't know about server-side apply that's another relatively new feature in Kubernetes that flux uses gives you some very nice capabilities like the ability to wait for the resource that you just created to become ready before you return from your function call that said create this thing and if it fails to become ready or times out or anything else goes wrong you can get a signal back instead of just assuming asynchronously that it went okay so even if there's an admission controller or something on the Kubernetes API itself that would prevent that perfectly valid resource from going in you can get that back as a signal and that's great so if this was the lightning talk we would be at the end but we're not and these are some other talks you'll be able to see I just wanna mention them before we forget and run out of time because we're gonna leave the slides and start looking at code so yesterday when I presented the lightning talk the operator wasn't finished yet there is a link if you download the slides you'll get a QR code later if you wanna download the slides or you can just go to my YouTube channel and find where I did finish the functional parts of the operator this morning it actually does work hopefully we'll see it I wrote it in about three hours total if you think that sounds fantastic Ruby man okay so this is also I didn't mention this but I had to write a Rust app because I said well putting Ruby inside of a Wasm probably doesn't seem like a good idea I think it's trying to trick me into a compiler maybe I should let it so I learned a little bit of Rust I learned how to parse HTML and Rust this is my first Rust app we're gonna start with that and then we'll work our way back out to see how do you deploy Wasm and Kubernetes which may be what most of you came here for today and we'll come back to this if we don't forget oh, crashed if we can, all right okay so here is that stats tracker repo that I mentioned I think I want a different here, this is the better one set of tabs each of these repos that I linked by the way in the slides has a tremendously well-developed readme to explain what I was trying to do a lot of the things that I have explained here today are gone over in more depth here things like performance questions does this actually perform at all are started to be addressed in here there are some gaps still in those numbers because I like I mentioned I just finished the operator this morning so it's tough to collect performance metrics right now but what I found most of all was the fact that I kept being forced to make my problem smaller originally I said I wanna do this all in web assembly I wanna do it all in Ruby and that would have been easy except for a web assembly so each time I found a constraint I said okay we can work around this my C extension, no Kogiri there's probably a Gammo pure Ruby library that I can use to parse HTML then when that didn't actually solve my problem I said well how can we solve it let's find another HTML parsing library in Rust and so I did eventually get that to be quite small here's a nice graph going up into the right we'll hopefully see some performance numbers in the future but in each of these example directories there's another readme that shows some of the thought process and some of these things that I found difficult how to walk through them step by step we're not gonna have time for all of that today but let's actually look at that Rust program that I wrote can everybody see this a little bigger it's good, okay so in this directory you can find the Kubernetes operator as well and the lib folder is most of the code and if you go into the lib folder and I'm gonna run make clean here is a file that shows that we are trying to run a WebAssembly in Ruby let's look at that okay so like I said I developed this and I switched it over from Anoco-Giri to Gammo so there are different implementations of current download count in here they all do basically the same thing they start parsing some HTML and then they use a big honkin selector here's the selector way up here I was a little bit nervous that the HTML parsers were gonna have a hard time as I went to more and more obscure HTML parsers but none of them had a problem with that big CSS selector that was fine and this is the entry point for whoever is gonna call us they don't care that we're parsing HTML they really just wanna know the current stat and the time together so we wrap an HTML client cause I thought I'd need that we have a project and a repo and an image which we'll get from the operator structure that I explained so we can get all those numbers and collect them and we'll call this like this so let's make sure that Rust library actually works oops, cargo not found, oh no wait, I know what to do okay so it's doing a few things first it is compiling with cargo it already has the dependencies so that's pretty quick it then strips the wasm module that spits out so we're using this target wasm32wazzy so the compiler knows about wazzy and the strip makes the web assembly much smaller then wasmopt I found that in a guide I'm honestly not exactly sure what that does but I think it is optimizing I think that's what that stands for and then running Ruby we see we actually do get the output we were expecting from the Ruby app with our number that came from Rust and it's been parsed and there it is if we run it again we would see it as in fact a different number that number changes every time so there are people, well not every time but there are people downloading Flux all the time okay so that's cool let's look a little bit more of the code so how do we get our operator to run and let's point at a cluster so this actually works so there's a CLI uses Thor this is really simple let's look at lib CLI so we see how that works Thor lets you define commands like I want CLI controller to do something and let's say we wanted to run with a different organization than Flux CD we could pass it in here I wasn't gonna do that but alright so this is also a very simple code we can delete some stuff that we didn't use I thought I would need a loop or something but that's not how the Kubernetes operator gem works this is it so fiber scheduler basically this just makes sure that these two parts can run in parallel so they're not really running in parallel Ruby is a single threaded language but they do have capability to understand when the system calls weight and send the execution to another fiber so that we can run these things together so that's cool so let's see if that works um okay use local cube config what we see here I'm gonna scroll back up because it's moving really fast we've got our two fibers running trigger action for Flux CD it found the Flux CD resource that had already been created on the cluster let's see that resource looks like it's doing some stuff looks like it's working so that top level resource is called project see we've got one here and we see that it has an empty status don't know why we'll see why in a minute um I wasn't really interested in the project status I wanted to see the status on the leaves so and I like I said I had limited time so let's look at those leaves so it did fan out and find all those packages uh there's a piece of code we didn't see in lib called project reconciler that does that all in ruby I don't need those things to happen in parallel it's it's just one fetch to find all that stuff so I thought that would be fine to leave it this way um we can delete all of these resources as long as we make sure that operator is running it needs to clean up the finalizers like I said let me put that comparable size so you can see so what we should see is it's receiving delete events and it's removing finalizers one after another this is great this was super slow in my hotel wi-fi it's working great here so they all got deleted successfully uh project okay delete project flux cd that one can also get deleted I just want to show you all that we can do it again and it should work and it's actually going to run the wasm module and we'll see the current values so this is not a finished operator there's a list of check checkmarks on that front page read me all the things I had intended to do and it's really more important that I get up here and show some working code I think so alright so we create the controller and here's what happens when there are no projects and no leaves it starts the operator and it registers uh those functions that we set and there's no log output so then we can go to kubernetes operator so here's the code for that kubernetes operator by the way this comes from your gitlab and there's no license in this code I wish that I was able to get in touch with this fellow I I sent him a couple emails to just let him know that I was using it and I really appreciated they made it uh but just to be forewarned there is actually no license in this code um I assumed it was on gitlab so it was okay to fork it but you may you're mileage may vary I'm not sure I just asked nicely if you asked me to take it down I will um okay so in the examples directory here's the example operator this is a lot more approachable than the one I tried to build because it only reconciles one resource but you can see this is a page it's not complicated you register your crd and then you uh create you know you create a kubernetes operator from the object and you tell it what crd it's supposed to listen to you attach it to a logger and then uh there's some event helper I'm not sure what that's for um and uh you register the absurd and delete method and then you run it and that's it and that's what we did in our two fibers each one gets run if you pass uh follow the execute execution path you'll find that run method getting called um are we uh close to time I think I have arm not over time got five minutes four okay great all right so in the example directory there's a manifest directory this also if you're using controller runtime it will help you generate your crd's this does not help you generate your crd's so if you go into there's two example crd's this is a really basic crd definition and it's just slightly adapted and and here's what I wanted to show you uh the properties on the status so this is why our status message if you find the status message in the code didn't actually get reflected because I changed the crd definition there's no message field in our status properties so there's count and last update that's what I thought we should keep track of on both crd's so on the next crd this this one at the top is our projects that's our flux cd and then at the bottom is leaves those are customized controller source controller all of the various microservice components of flux and anything else that we've published as a package like the flux cli um some required fields so there are the three parameters that I mentioned uh get passed into the um scaffold for the uh wasm module and then here we have count and last update again let's make sure we run it before we're out of time okay so here we see triggering some actions all right it's doing some stuff we are going to see some errors in here I haven't tested every single resource but we are going to see also um it's adding events for each one it looks like it's processing most of them without errors I got a couple 409s I don't know what that's for okay all right so it's created all these leaves that's great how many downloads does the flux cli package have 57 million thanks wasm two minutes okay uh let's see what else was I going to show we have two minutes I want to make sure you know about this directory this is where flux is bootstrapped into by eks cuddle and in this here we have examples some of these things I depended on I'm not going to name each of them um I had hoped to show you scaling to zero with kata but the operator itself is not going to scale to zero uh and then taking bartholomew I think we're not going to get to actually run this deployment but I do want to make sure that you know uh that this also has a dedication here for my wife Catherine I'm going to help her build a blog we've tried this before didn't always work out I think this one is going to stick um do we have any questions you have a question uh for me I was not able to get there for me what I need to do is actually package the spin runtime in a multi-arch container image and then deploy that with helm and from there pull the wasm module uh using one of the tools so there are two tools they're available for that flux has its own oci artifact feature and um there's flux build artifact and flux push push artifact which I'm familiar with because I work on flux and then I found a similar feature was added in the spin uh I'm not sure if it's spin push artifact but it works exactly the same way and it's for packing up your wasm modules and then pulling them down at runtime so that's that's pretty much how this works um and if you go in and see the docker file let's look at the docker file this is really simple here we're just downloading spin 1.0 and there's a platform script to detect which spin platform this is a multi-arch container uh build so if you look at the google github workflows here's our consolidated build and it builds using buildx so we get multi-arch image out that we can run our wasm module wherever we need to go so um no I don't see it's going to replace right away but maybe long term there is a tool called kwasm operator if you go look at that that is going to use a field called runtime class that you probably didn't know was in deployment and pod and that runtime class allows you to actually replace your image with some other thing than a docker container it can be a wasm module and then it'll get run by whatever binary you pass in so kwasm operator is not for production um right now if you look at the readme it says right on the front not for production because it uses a privileged um access to put that spin binary in place so that container d can use it but I think we're out of time are we having more questions okay thanks