 All right. Thank you, Candace. So we're going to be talking about how Wasm simplifies streaming data pipelines. First, a little bit about myself. My name is Tyler Rockwood. As Candace said, I'm a senior software engineer at Red Panda. I love all things distributed systems, databases, and programming languages. I'm the current technical lead for web-assembly-powered data transformations at Red Panda. There's a few of my socials will be at the bottom. They'll be posted in the chat and then also at the end of they'll be posted again at the end. If you want to get ahold of me for any reason, I'm happy to talk all things Red Panda and web-assembly. So briefly go over our agenda. We're going to start out by just giving you an overview of what is web-assembly, how a little bit of details of the VM, how it works, and how you can interact with it. Why would you want to use web-assembly for stream processing? And then we'll talk about the integration challenges of putting a web-assembly runtime into Red Panda itself, and then talk about how we overcame some of those challenges and some of the cool technical things that we got to do. And then we'll end with some Q&A. First, we're going to start with just what is web-assembly? Maybe you've heard of it. Maybe you haven't. We'd like to start with just a little poll to gauge your level of experience with web-assembly. Yeah, maybe you've never heard of web-assembly and you're here to find out some more. Maybe you have heard of it, know it's some cool, interesting technology. But yeah, have it interacted with it. Maybe you're somebody who uses this all the time in production. We'd love to hear what your level of experience is in the poll. Cool, it looks like answers are coming in. So I'm going to go ahead and get moving forward here. Thanks for your responses. So jumping into what is web-assembly? So web-assembly, also known as WASM, is a binary instruction format for a stack-based virtual machine. So maybe you're familiar with virtual machines or not, but it's designed to be a virtual machine that's a compilation target for many different languages. So you can take all of your favorite languages, such as Go, or Kotlin, or Rust, and compile them down to web-assembly. And then you can run them really anywhere. Initially, it was built for the web and for the browser. But today, as you'll see, they're being used in message brokers and server applications as a function as a service platforms. It's really, really taking off and being used all over. So the main things that make web-assembly appealing to use in these situations are that it's efficient, it has very good compatibility guarantees, and it's safe. We'll touch more on all of these, and you'll get a better idea of them as we progress through. But efficient in terms of the byte code itself for the VM often maps directly to common hardware instructions. So for things that are doing compilation of web-assembly into native code for speed-up, which we'll talk more about, it can be a fairly easy mapping. It's also open and backwards compatible and versionless. And there's new standards that can be incrementally added into web-assembly, which is key for if these things are running in browsers for a long time or in hosted services. And then safety is also key pillar of web-assembly. So it happens in a sandbox execution environment. You have lots of controls over almost every aspect of memory and CPU limits that you can do. And we'll talk additionally about a standard that gives you the web-assembly the ability to access OS-level primitives. Cool. So we'll talk about the main major pieces of a web-assembly VM. So the main one is probably the compiled module, which is this blue box here. And is the compiled version of that binary instruction format loaded into a VM. So that represents things like what functions are available and all the instructions required to execute those functions. And then specification for how much memory it needs and something called a table, which eventually effectively gives you the ability to do pointers to functions so that that's defined in a sort of portable format. And then once you have that compiled format, you also need to take the actual host memory, give it to the VM so that the web-assembly guest code can interact with it. And then also you need an instance table to store all of those different function pointers and things like that. And when you take those three things and combine it together, you can get an actual live running instance of a web-assembly VM. So next we're going to dive a little bit more into functions and the execution of functions. So earlier I mentioned that web assembly VMs are a web assembly is a stack-based model for the VM. So we're going to go through an example of just adding two numbers just to sort of give you a sort of idea if you're not familiar with VMs or what a stack-based VM looks like, just sort of how the execution works and how these different instructions. So these are three actual instructions from web assembly and we're just going to walk through the execution sort of like you're stepping through it with a debugger one of the time. So this first instruction is representing a constant for a 32-bit integer. So there are four main types in web assembly. There's a 32-bit integer, a 64-bit integer, and then the same for floating point, there's a 32-bit floating point and a 64-bit floating point. Types, there's another standard. Other standards have come along and added more types such as SIMD for sort of vectorized instructions and also references to functions and things like that. But those are the first four, the four main core types. So this first instruction, this constant instruction just pushes a constant value onto the stack. So we're pushing one onto the stack and then the next instruction pushes two onto the stack. And now we have two items on our stack and we get to our add instruction. And basically all these instructions consume and produced onto the stack. So these constant instructions, all they did is push things onto the stack, they didn't pop things off. Add pops two things off the stack, adds the result of them and pushes them back onto the stack. So if you see we pop one and two, add them together and we'll push three back on. So that's sort of the general mental model of the stack-based machine and how it works. So generally when a VM runs, it needs to convert this sort of stack-based machine into actual hardware and instructions that are ran. So there's a variety of ways of doing this and they all have their own trade-offs. So I have sort of here a spectrum of compilation time. So how long it takes to take the WebAssembly bytecode format and turn it into something that's executing on a CPU. So on one end, you have really fast. So an interpreter has very little startup time needed. But on the other hand, there's execution speed. So the more time you spend optimizing and converting the bytecode into actual machine code, you can push performance faster and faster. So the sort of spectrum I'll talk about is there's an interpreter on one end, which just sort of natively is a big loop that walks over every bytecode instruction and evaluates it as we walk through in our example. And then on the other end of the spectrum, you have a compiler, something like the Clang compiler, LLVM compiler suite can let you build something that would take all those bytecode instructions and actually turn them into hardware instructions directly and doing all the different optimization passes that a compiler framework like Clang can do. And then sort of, and then in the middle is something known as a just-in-time compiler. If you're familiar with JVMs or anything like that, it's very popular, the most popular JIT compiler. Basically, you do the sort of compiler step but you do it incrementally and on the fly. So instead of doing everything at once and doing a big optimization passes, you'll just do a single function. And there's sort of a spectrum, even within just-in-time compilation, where you do sort of what's known as a baseline or non-optimizing compiler that will just basically directly map individual instructions to individual bytecode instructions to hardware instructions to sort of the goal being to do compilation really fast. Whereas the other end of the spectrum you have an optimizing JIT compiler that can do things like profile code as it's running to sort of figure out the right optimizations to apply and can do a lot more expensive and spend more time doing compilation for your hot code paths. So that sort of gives you the spectrum. There are WebAssembly runtimes on this full spectrum and some individual runtimes implement multiple tiers of this. So they have an interpreter that runs and then you can sort of upgrade functions to using the baseline JIT or even the optimizing JIT. And there are some VMs that just use ahead of time compilation to really squeeze out all the performance that you can during runtime. So this is sort of similar model to like, for example, the JVM I mentioned, it actually has a similar model. This is a initial code runs as an interpreter then there's the C1 and C2 compilers and C1 compiler sort of represents the baseline JIT and the C2 compiler is sort of the more optimizing JIT. And this is sort of a standard framework for that. So when you start to think about integrating WebAssembly into a live system, you got to kind of work through, what's my use case? What are my trade-offs here that I want to take on this in the spectrum and help you can pick it? There's a variety of different runtimes out there can help you pick between them. So that leads us into the next point, which is why I use Wasm to do lightweight stream processing. So you may not be familiar with stream processing. So I just wanted to throw up a poll and sort of gauge your level of experience here. Have you heard of stream processing before? Are you somebody that does this all the time? Is your day job? Yeah, I'd love to hear, get your thoughts in the poll. And we'll give it a couple of seconds for everyone to put in your answers. Awesome. And we're sort of jumping to talking about stream processing. So stream processing allows you to, as events get processed through like a message broker, such as Red Panda, you can sort of transform those on the fly. So one very common use case is sort of the stateless transformation. So we'll give an example here of there's a stream of JSON click events coming in from your website or whatever it is. And you want to run a transformation because other parts of your software stack use Avro instead of JSON, which is Apache Avro is another common format used in data streaming applications that has schemas and things like that. So you want to do this transformation for downstream consumers of this stream of events, these stream of clicks to consume them in Avro. So today what you do, if you do this, you have these events coming into Red Panda, your message broker, and then what you do is you set up this whole other distributed system. Apache Flink is a very common system for this where you basically, it's a great framework for setting up these big streaming pipelines where you can read in the data from the broker, do your transformations and you can write it back into the broker. Now, Flink can do a lot more than that for this, but for this simple use case, there's a lot of extra overhead of additional VMs to provision management. You got to learn the separate programming model of Flink and manage it production and monitor it. So sort of the idea with WebAssembly within Red Panda is that we can just integrate these sort of stateless transformations directly into the message broker. So the broker itself as it receives these JSON events can run your WebAssembly module to perform these different transformations and write it back out to a separate topic within the data streams for other, these other consumers. So this eliminates a lot of overhead. We do not ping ponging data all over between these different distributed systems. There's less moving pieces to manage and it's just a lot simpler to get started in going when you go from just message broker to okay, now I want to do some stream processing and do some small transformations of my data. So that's sort of the main idea here. And the benefits of using Wasm over some other sorts of programming language or plugin system is, I mean, some of the advantages that I talked about from the very end, you can really use any language. A lot of languages, most languages can compile down to Wasm. And I think that supports only growing. So if freeze users, you can meet your users where they're at and let them kind of program in their favorite language. And then also, especially in Red Panda, you can strictly control the sort of runtime of the VM. So you can strictly impose limits on CPU usage, on memory usage, and things like that. And then also we're gonna talk a little bit about the Wasi standard, the WebAssembly system interface, which basically gives a sandbox POSIX environment for people to give sort of, you can expose common programming patterns to people. So it doesn't feel like they're writing in some weird alien environment. So yeah, these are the lists of some of the languages that you can use. There's some languages that are even only compiled WebAssembly, Grain and Assembly script, or examples of that. These are both languages that were built directly to write, compile down to WebAssembly. And then a lot of other popular languages can run in WebAssembly VM as well. Awesome. So some of those runtime limits, like I said, very important for Red Panda, and we'll talk a little bit more about that later in the presentation. But here's just some example of the CPU and memory limits that you can give. So for CPU, there's a concept known as gas metering, which basically lets you stop execution after some number of instruction counts. So for example, if somebody accidentally has some weird edge case that triggers an infinite loop, that's not just gonna hog up CPU in our broker. We can say, okay, you can put some time limit on, say every transformation can run for X amount of milliseconds. And if it runs longer, then abort it. And there's other techniques as well for supporting, for like other threads, aborting the runtime and things like that. So lots of different controls you can give. Also for memory usage, you can put bounds on how big, how much memory you can use. It's important in Red Panda, because we have a lot of strict requirements on how much memory we have. You can put bounds on those table sizes and those are the two big usages of memory. And then also a lot of the VMs themselves have pluggable allocators. So if you wanna have different allocation techniques, this is common if you're in like C or C++, you have different allocators that you wanna use. Maybe you wanna use a pre-reserved pool of memory for your VMs. A lot of them support being able to plug in different allocators. So we're gonna switch gears and talk about WASI. I mentioned WASI is great. It's a system interface. All these common things that you're used to interacting when you're writing code or things like accessing a clock, accessing environment variables, having access to random data, standard in and out in a file system, or sockets or things like this. These are all things that WebAssembly core doesn't have a concept of. But again, there's a standard for sort of, it's important for upstream languages and tool chains to be able to compile it down to some standard. And then basically it happens it's up to the host that the VM is running in to implement all these methods. So you can do things like only grant access to parts of the file system or only certain file system commands you can do or disallow socket traffic or things like that. So there's a lot of nice things about having this sort of sandboxed, sort of like over the OS level of primitives that you sort of take for granted, you can sort of have full control over within WebAssembly. So as somebody who's embedding WebAssembly into your software, how do you interact with the VM? So basically you end up defining an application binary interface. So this specifies essentially there's some functions that both the program that's running expects to be exposed. So the guests that runs within the VM can have specific methods that it knows it can call into and it's gonna expose certain methods that it knows will be called. So for example, we can do this for every time there's a new record that comes through on our data stream we can call a specific callback. And there's a limited set of types that you can use for this, which is just basically the types that are available within WebAssembly. You can have integers and pointers. And there's also a standard that lets you pass arbitrary host data that's sort of like an opaque pointer but not all runtimes have support for that quite yet. So, but if you wanna pass more complicated data, generally the pattern that you use here is I32 values can be interpreted as pointers into an offset within the module's memory. So if you wanna have like a more complicated structure some binary encoding that you pass data through like JSON or whatever it may be you can write that into the VM's memory and then pass a pointer to the start offset of that data and that's sort of the mechanism you can use to pass back and forth. So we're gonna walk through a very specific example within the WASI spec of just reading some random data. So that top signature is the signature that says if I am going to run in a WASI environment I expected to be a function called random underscore get that is exported that I can use. And if we translate that sort of that signature into what a C function would look like you can see you will return some int 16 that's an error code of yes no if they're getting reading from the random source works and then there's a buffer and then how long that buffer is which in the WebAssembly land just are a couple of integers that get passed around but as you see within to the guests those are actually pointers into the its own memory and then how much into how long of a buffer within that memory you should write random data into. So hopefully that gives you a good idea of sort of what that interface between the two guests in the host running look like. So we're gonna talk a little bit about downsides to WebAssembly it's not a perfect thing it's getting better all the time but there's still some downsides. One is there's an upstream tooling that's still maturing a great example of this is Golang there's a separate compiler for Golang called TinyGo that compiles down to WebAssembly using LLVM and that can use wazzy but upstream wazzy upstream go the main go compiler can only compile it cannot compile to go that runs in that wazzy environment that I was just talking about. Although it's coming in the next release that I believe is maybe already out or coming this month and also things like interpreted languages. So if you wanna run JavaScript or Python or Ruby inside of wazm you need to package the whole interpreter for that language which ends up getting quite beefy just to run your interpreted languages. All right and then yeah so feature support like I said there's WebAssembly has been built to be modular and backwards compatible so all these new features that have been added to it are sort of like added on to the side and what support is there is like constantly evolving very rapidly even. For example there's like GC and this is needed for languages like Kotlin and Java to be able to do garbage collection very efficiently inside the VM. It's right now only supported in V8 and I believe it's behind a flag although maybe it's an origin trial now and then there's also a big change coming through called the component model. You can read about that but there's it basically changed how you link in the interface between different WebAssembly modules because you can have multiple of these that you upload sort of like shared libraries and access and talk to each other and then the current wazm support is based on a preview snapshot from 2020. So that has changed a lot and made a few different iterations and yeah it's continually evolving. So yeah, if maybe those downsides don't work for you or what are the other alternatives? So there are some languages that have better support for like plug-in experience. One that comes to mind for me is I've worked in a lot in JVM systems is Java has support for sort of loading jars dynamically and other interpreted languages can usually dynamically load code as well. So that can be a good option for some people although usually with those options there's like little to no sandboxing. So essentially when you load that jar it can do sort of anything you want. So as long as it's highly trusted in highly trusted environments you can also kind of push performance a little better because you get all the same performance. There's no like extra VM wrapper in there excuse me, within the JVM. So and that can be an alternative. And then there's also some more specific or niche solutions such as Google's common expression language. So I used to work at Google and on the common expression language itself and it's a great language for things like writing policy or very small snippets that specifically built not to be Turing complete and execute very fast. So if you have certain use cases that don't need the full overhead of Wasm although Wasm VMs are very lightweight generally compared to like something embedding a full JVM into a system you may even want something more on the lightweight version of the spectrum which something like cell is a great use case for that. Cool, so we're gonna switch gears here and talk about the challenges about integrating a WebAssembly VM into Red Panda. Sort of we've been talking about generally what is WebAssembly and sort of how to integrate it into a system and I'll get into sort of the background and lessons we've learned from Red Panda. Before we jump into that I wanna talk about Red Panda. And before we do that I wanna just briefly ask a poll of how familiar are you with Red Panda? Have you heard of it at all? Is this the first time you've maybe heard about Red Panda? Are you using it in development or are you running a production cluster of Red Panda? I'd love to get your answers in the poll and yeah, we'll wait for those answers to kind of come through. Wonderful. And I see there are a few QA questions I will answer those towards the end. Keep those coming in. Great, all right. I think we're good, we'll move on. So now I wanna introduce, yeah for those of you who are not familiar with Red Panda just sort of in 60 seconds give you a sort of idea of what Red Panda is and then we'll talk about sort of the interesting constraints that it has as a system is somebody building and hacking on Red Panda itself. So Red Panda is a high-performance streaming data engine. If you're familiar with Apache Kafka we are compatible with their API. So you can produce and consume records but we are a more efficient and faster and lower cost of ownership version. So we'll talk through some sort of the key tenets that make Red Panda very fast and efficient but if you're unfamiliar with Kafka and sort of the paradigms there Kafka basically breaks out into there are partitions that you read and write records to and each of these partitions are logs essentially and there can be several thousands of these per cluster and each log or partition is a raft group itself. So we run raft usually three members is sort of the de facto number there that replicates each of these logs. So there's a single leader elected all the reads and writes go through that leader and then the leader then will replicate that data to the two followers. And then one of the other key things is we're super easy to get going play with manage and run there's not a bunch of individual microservices to run everything's packages a single binary which really helps with getting running in production. So that's Red Panda in 60 seconds we're gonna walk through sort of some of the more well and also data transfers data transforms of what the scene. So we talked about those two different logs or those partitions. So the main thing about these web assembly data transforms I highlighted this very briefly earlier on when we talked about Flink but you have these individual squares or records within the log they get written and written by clients and then basically web assembly is something that will pull from one log do some transformation and then write different records into a separate log sort of as like a background process running so this gives you the ability to really easily sort of like write data in a different format for example or you can do things like strip out PII or do data scrubbing for certain other parts of your application there's a bunch of very common use case within these big streaming distributed systems is doing these very simple straightforward tasks. Cool. So we're gonna talk a little bit about Red Panda itself sort of and what makes it so fast what makes it so efficient what are sort of the things that Red Panda does how is the system sort of work at a high level? So one thing is that the system's built to be entirely asynchronous. So there's no blocking system calls and we run a thread per core architecture. So this is something that's common in a common sort of architecture in high performance computing environments but basically we run this reactor loop on each individual on a thread that's pinned to a core so that thread never executes on a different core there's no context switching or anything like that always runs on that core we can in sort of Numa architectures we can co-locate our memory and our core together to sort of squeeze out all the performance we can from the hardware. We use a shared nothing model so no memory is shared between each of these threads on startup we'll divide the system memory equally between all these threads and then the threads communicate themselves using single producer single consumer message queues and then it's built using this a lot of this functionality comes in this framework called C star which is a really great framework that we use that handles a lot of this basics for us. So I talked a little bit about this reactor loop that runs so one thing I said everything's asynchronous so everything that runs needs to run for a very short time it uses cooperative multitasking you don't have these big threads that are running that you context switch between and you rely on the kernel to sort of make sure all these threads are making progress is if a single task within a thread runs for 30 seconds that's known what we call as a reactor stall and because during that 30 seconds that runs or even 30 milliseconds really for us is doing this 30 milliseconds no other IO can happen no client request can be served so all of these stalls impact the P99 latencies because there's nothing else that can happen on that core during that time. So what happens as a result of this any long running computations that run need to be broken up into smaller chunks of a millisecond or two to prevent blocking really important IO work that is happening so such as servicing client traffic or reading from disk and things like that. We use a shared nothing allocator so again we talked about we split up the memory for each of these different cores on startup and then we use a buddy allocator on each shard and there's no synchronization between different threads so a lot of the very advanced if you've studied allocators a lot of the advanced allocators do all these interesting thread local and try to reduce synchronization and we sort of do that by design by just saying no thread share any memory at all they all get their own portion of it and then there's no pagecast so we use direct DMA controller writes direct IO to write directly to disks and don't use the kernels page cache at all so it allows us to have full control over the best ways to cache and manage memory ourselves but it's on us to do that, right? The kernel has a lot of great optimizations that have been put on it by many people but we get full control to sort of eke out those last couple of bits of performance. Another interesting note especially for within the context of WebAssembly is that generally when you're writing a C program you can malloc and ask for memory from the OS and it will never really directly fail. It'll sort of slot like soft fail by as you ask for memory in typical situations it'll just performance will start to degrade as you use that memory and page tops can happen and things like that and eventually as you use more and more memory the umkiller can run in but C stars allocators different in that if you all that memory that we divided up on startup that's it that's all you got. So if you, if we have too much fragmentation or exhaust memory we'll throw exceptions and it's sort of on us to either handle those or abort the process or whatever it should be. There's a lot more constraints and each of those generally we run we usually say like at least two gigabytes of RAM per core or per thread that's running so it's generally the sort of model that you run with so it's not a ton of memory that you may normally be used to running with. So we're gonna talk next about sort of how we address these challenges so there's all these interesting constraints about that make red panda really fast but it also becomes a challenge when you're integrating a VM that may not be specifically designed to meet all of these or designed under these sort of constraints, right? So one is like one thing that we did very early on is we built a software development kit to sort of hide the complexities of some of the techniques that we did to users. So for example, one thing is like I said memory usage is very limited and we try not to do large allocations that are contiguous because those can cause lots of fragmentation. So what we do is we do we'll do a lot of things within the VM for users to manage memory well such as pooling buffers and reusing memory when possible and things like that. And we handle that sort of transparently inside the software development kit and give users a sort of idiomatic experience. So this is an example of our tiny go SDK the sort of functions, the snippet of code that you would write to transform or something says you get a record that comes in as XML and you wanna transform that in the JSON. This is sort of a snippet of how you would do that. But yeah, the main thing is like all the complexity stuff of our ABI specification that's all handled and done by the SDK. So the fact that we're, you know writing into some place in memory and you need to call this very specific function that the runtime expects to be there that's all handled within SDK. So users just get this nice single function to write their logic in. One thing for memory like I said because like I talked about fragmentation and sometimes as a long running process goes memory may be fragmented enough that if a VM needs to get 10 megabytes of memory we may not have a contiguous range of those 10 megabytes of memory and all the VMs need the sort of large contiguous chunks of memory. So what we do is we pre-allocate all this on startup for each cores will basically, you know each core has two gigabytes and will reserve some configurable amount of memory that then the VMs get to use themselves and run. And then the other thing that we do that is really neat is there's, well, there's two versions of this. One is some functionality that we expose. So one example of that is we have a registry of what schema your data is currently at and how we implement that within Red Panda is the data for that is sharded across all the cores. So we don't have all these requests going to a single core because again we're doing message passing if you need data and we're not using up all the memory on one core to service all these schemas. So they're in memory and what we'll do is we'll is if a wasm transformation needs to go look up one of these schemas is we'll, we run each of these VMs on a separate stack and what we'll do is we'll I'm using sort of if you're familiar with stack full co-routines is sort of how they work but we'll pause the VM by when it calls into our host function to grab that if we don't have the data locally on our shard we'll pause the VM, suspend it and do other sorts of IO work, client facing traffic, whatever needs to happen within Red Panda for that core we'll hop over to another core, request the data and when it comes back we'll resume the VM with the data now and that's all transparent within the VM there's no async stuff that needs to happen within the VM itself and sort of if we had to do things like network calls and things like that other sorts of async work because Red Panda is an async first system we would use this sort of technique to do that but again this can all be transparent you don't have to write highly asynchronous code so if you're writing something like C or TinyGo you don't have to, it's a very familiar to like a co-routine sort of programming model it's you don't have to burden yourself with understanding like when things sink and have all manage all these callbacks so we do this with stack switching it's really quite fun so we'll talk about, so it's sort of like the challenges that we face with Red Panda and sort of some of the ways we solve them is like how are we going to do some of these asynchronous work without blocking a reactor and causing all these stalls and hide latencies so we're going to talk about what other system other systems that use Walsum and what they use them for so one example is the Envoy proxy so it's sort of if you're familiar with IngenX or Apache these sorts of different network proxies is Envoy is sort of a cloud native first one and it allows you to upload web assembly to do request filters or change requests or route them different ways or do all sorts of custom logic within you can upload a Walsum module to do that another example of Walsum being used is in the open policy agent so this is a, there's a language here called Rego which allows you to write policies of authorization for example of what things are allowed to happen and those can get compiled into web assembly modules so then they can run sort of anywhere within a web assembly VM so you can have custom, these custom policies that are ran without needing to build an SDK and an interpreter for Rego and every individual language Cool, all right and the key takeaways here is yeah, so web assembly can be used to simplify data streaming and you don't have to set up another distributed system within Red Panda and it's lightweight and efficient and then the Wasi standard is a great way if you're not familiar with that to expose as familiar sort of POSIX environments to developers, you wanna hide details that users don't wanna worry about such as what your ABI contract is and how you call into the VM and manage that and then the Walsum ecosystem is constantly evolving but there's already enough to like really build production ready systems in it and integrate this on the server side platforms. Awesome, thanks for joining, we have some Q&A now there's already a few questions at all that are already posted, I'll read through here and answer if you have others, feel free to type them into the Q&A box and happy to answer them. So what is your language of choice to use with web assembly? So one of the languages that we use a lot is Go, we think it's got a great TinyGo specifically right now but we hope to switch to the upstream compiler soon. TinyGo in general is a great language because it's very simple to use and it's efficient because it's compiled but it doesn't force all of the things that you need to do such as manage memory and things and if you're using C++ or Rust so it's like a nice middle ground of like developer ergonomics and also performance that has pretty good support for web assembly. So that's what we started with our SDKs is using Go so you can develop these different streaming plugins and transformations using Go. So next question, are there any projects using WebAssembly extensively that you might know of? So I mentioned two there, those are the two that come to mind besides Red Panda. The next question, can you use Lua and JavaScript with Wasm? So yes you can but again, like I mentioned very earlier is you need to, when you're using these more interpreted languages you have to build an interpreter into WebAssembly and then within WebAssembly VM that's running there's a interpreter that's running that runs your JavaScript code. There may be ways to take your JavaScript in Lua and ahead of time compile it and then run that directly as Wasm but I don't know any efforts for JavaScript specifically and not for Lua. All right, next question is Red Panda SDK takes care of the Wasm API presumably means you have to maintain SDK in each language that you want users to write transform in, how toilsome is this and how hard it is to change your ABI? Yeah, so this is a great question. So it is fairly toilsome, right now we just support Go so it's not that toilsome, we're just trying to hammer out the experience there and we'll support more as time goes on and yeah, there'll be a porting way. I think the main ways that I've seen to reduce the toil and maintaining these multiple SDKs is to try to mirror them as much as possible outside of like language barriers is you basically write them all very much the same so that you can kind of look at your JavaScript and look at your Go and the code looks almost exactly the same is one way. Another technique you could use because again, Wasm everything sort of compiles into Wasm you could potentially write all these in something like C and then integrate them all that way as sort of in Go you see Go well it ends up just all being WebAssembly at the end of the day and then changing our ABI is a great question. We sort of initially or from the start have built in a protocol to sort of manage versions within our ABI so one of the functions we export and assume that a guest function is going to tell us is what version of the ABI you are so we can sort of boot up, run the VM and sort of understand what version they're running at and then we can make sure the right support is there for that and use different host modules to sort of version our protocol as we evolve it so we have not yet had to make a change but it's built in such a way that should be straightforward we will have to support the old ABI's for maybe forever but at least we have a way to make progress and encourage people to upgrade. Okay, next question is Wasm strict runtime limits only apply to Wasm instructions are there any host environment function call have to do their own gas metering? How do you make this work with Wasi? It's a great question so host environment calls do have to do their own gas metering some WebAssembly VMs support attaching a instruction cost to a given module or to a given host function so after you run X number of instructions then you call into some host code and then it will automatically decrement the host code count is like 20 instructions or whatever it is or whatever your gas cost is which is pretty cool. So that's one way to do it. You can also a lot of VMs end up letting you directly access the sort of place where the gas is stored so you can also just in your host function if you have like a dynamic cost you wanna associate you can increment or decrement the gas as it runs but again this is sort of specific to what VM you end up using. Cool and then let's see next question how does Red Panda store and retrieve Wasm code? Do you have pre-execution validation or similar checks before modules are loaded into the database? Yeah, that's a great question. So we store them internally. We have sort of a mechanism for storing things metadata that we store and then we also store the code itself on disk and replicate it so that it's all safe to use and you can access it from any node in the cluster and then yeah pre-execution validation so every time somebody uploads a WebAssembly module we compile it and run it through a few of these checks the ABI checks, make sure we can specify the ABI version and yeah, things like that as they're loaded in that's a great question. And then last one here is what is the compilation model you're using for Wasm? So today we are using a JIT compiler in Red Panda for a variety of reasons. One is the ahead of time compilation would be great for our use case because we sort of expect you to upload something once and we're gonna run that a ton and we want that to be really fast but there's developer experience to sort of consider here when you're also picking a compilation model because if you're uploading this already ahead of time compiled machine code you can have a very easy attack vector there of like you need to make sure you can validate that machine code is only executing within the sandbox constraints and things like that. So that's the reason we're using just in time compilation you can get some of the performance benefits of compiling to machine code but you're doing that on the server and just in time as you need it so there's less like you need to compile this for your right host and architecture and some of all these concerns you can just deploy some raw Wasm from your M1 MacBook and then your Linux x8664 machine can just compile that to the right thing so that's one of our concerns there. Cool, what Wasm runtime are you using in production? So currently we're able to use a couple of different ones we've experimented with VA, Wasm time, Wasm edge of the main ones we sort of like took a little bit VA for a number of reasons doesn't work super well in the constraints that Red Panda has in terms of memory and thread per core architecture. So we've been working with both Wasm time and Wasm edge and are sort of like working between the two. All right, and then another question here. Great, I love all these questions. Does a data transform explicitly materialize the transform log or does it support fuse change transformations? So yeah, that's another great question. So transformations right now are written to that separate log. We don't have support for like ephemeral change ephemerally changing data as it's written or read at the moment although this is something we're actively exploring and looking into. But yeah, so currently the model today is you write into one partition within a topic and then that partition gets transformed into an output topic. And you can also chain these into make sort of like a DAG of transformation. So you could take one, you could write things into one topic from your producer and WebAssembly could change that to another and then you could have another one that changes that back although we don't support cycles because obviously then you'd just be looping data all around through the system. All right, I think we're running close to time here. These are great questions. Thanks everyone for attending. Feel free to reach out here. Walk through all my socials and contact. Feel free to reach out to me and ask some questions on Twitter or LinkedIn or you can email me. Happy to talk more about Red Panda or WebAssembly at all. So again, thank you very much to Linux Foundation and I'll hand it back over to Candice. Thank you so much Tyler for your time today and thank you everyone for joining us. As a reminder, this recording will be on the Linux Foundation's YouTube page later today. We hope you will join us for future webinars and have a wonderful day.