 All right, okay, so I think it's time. So let's get started Well, thank you all for being here. You know, I'm really glad to see so many people are interested in this topic and You know cube counts have you know earlier deadlines for the for the talk So when I submitted the talk I want to talk about log processing But when I thought about it when I write up the Slides I saw you know, it's really about stream processing not just limited logs about other things So, you know, my name is Michael Yuan and I'm the maintainer of CSCF's Web assembly runtime project called wasm edge. So that's a GitHub link if you're interested, you know With us on GitHub, you know, there's essentially all our activities happening over there. Hmm. So As you can see the title, you know, there's two seemingly unrelated technologies that have been mentioned in the same sentence wasm and ebpf, you know If I if I ask people most people would just know they're both Lightweight virtual machine formats, but hardly related, you know, that's that's you know We're gonna dive into that and we're gonna even go further, you know, we're gonna towards the end of this talk We can also talk about things like, you know AI and machine learning which seem to be the the the topic of this conference, right? How does that relates to say wasm and ebpf and the whole idea of stream processing, right? You know, so there's a lot of topic we're gonna cover. I do have some Some demos, but I don't think we all have time to do the live demo here so I'm gonna give you the link and And you can you know go home and do that from your hotel room, you know That's I think it also would be more friendly to the to the conference bandwidth, right? so You know From the conversation that I had yesterday and the day before that seems like a lot of people Not really familiar with say the cloud native wasm idea, you know So web assembly sounds like something you use in the browser, you know, how's that how's that related with? You know cloud native computing and what do you mean? What do we mean by? you know wasm on the server side, so This this whole idea goes all the way so I added a couple slides at the beginning of this talk, right? So this is one of the slides that added so The idea goes all the way back to the origin of the cloud computing right, you know, so the cloud is really You know sharing resources with other people, you know on the on the larger computer You know, so there's no cloud that's just running programming other people's computer, right? So the whole idea of this is that the enabling technology really is isolation and virtualization. That's why you know the You know the clouds grow up with say AWS, you know That's we and where you know this generation of companies that provide virtualization and isolation on workloads and the evolution of the of the isolation virtualization goes you know From our point of view goes through three phases the first is the virtual machine era So where you have you know hypervisor and you know things like that and that is where The cloud computing first come up, right? You know you rent VPS virtual machines from say, you know ISPs or you may AWS, right? you know, that's a lot original cloud provider using this technology and After a couple years Docker Camelon and container Camelon, you know, that's people find you don't need to wrap Say operating system and then give people root access to install multiple applications You can containerize each application in their own container. So deploy a lot of those and that's I think that's the Cross of cloud native, right? You know cloud native is to say there's a lighter weight and the more flexible way of running workloads that in isolation So that's the container error, which you know is I Think the error we are currently or though towards the end of it. I think you know, so that's To make application and workload in the container, right? So what's the next generation after container because container why why tell people container is a heavyweight solution Most people are very surprised, you know, especially in cubica people say containers are lightweight solutions You know, you are you know, that's because they're comparing that with virtual machine, right? But if we compare come container with a new generation of virtual machines, you know We we call application virtual machines though, which happens not at the application level isolation But as the function level isolation, you know, meaning that I no longer just isolate an entire application But I isolate a function in the application, right? You know, so that's what we call the wasm area because that's where the the the lightweight virtual machine that is Even lighter than the container orders magnitude lighter than the container can play an important role So that's so if people ask what's wasm on the server side wasm on the server side is even ever smaller Granulation ever small fine agreeing isolation level for applications You know, we go from the whole computer to an application on the computer to a function in the application, right? You know, so wasm would be able to You know, I provide security and isolation for those for those very small workloads, right? so the whole idea of You may think, you know, that's okay That's you are describing microservices. Although traditional definition of microservices is not that traditional definition of microservices is you know Application service that wrapped in the in the container, right? However, if you look at How those today's microservices are being wrong in the in a data center you would find that's you know, a large percentage of the virtue the Utilization is spent on things that are at the function level this things that are transit, you know, that's things are very short-lived but with Container paradigm you have to start a container for those, you know If you if you think about how you set up a Kubernetes cluster, you know There's everything is a container even the weight action doing nothing and weight is a container, right? You know, that's so for all of those you need to spin up the operating system the virtualized environment and all that so, you know To a lot of people this is this as microservices goes More and more finally agreeing it becomes more and more problems, you know So we have worked with large internet companies that has 50,000 microservices, you know, it's not 50,000 the machines is 50,000 different services Each service has a team to maintain and has its own machine provision, right? You know, so it's the you know the the spread of microservices has really put a lot of strength on the on the infrastructure, right? That leads to You know a very interesting Exchange I think from last year. I mean, I'm hoping two years ago, you know, so when Elon Musk acquired Twitter, you know, that's You know, he was Looking to very aggressively cut cost and you know, so he was thinking a lot thinking out a lot at the time A lot of people thought he was you know, this was crazy, you know, that's a you know You can't Move from microservices environment overnight to to something that is not a microservice But you know, that's although you know The Twitter as we know it is still exist still exist, right? And this hasn't crashed yet, you know So, you know, it's a it's a from outside a lot. There's a lot of You know people have a lot of Resentment in terms of you know, how the microservices paradigm is being abused, right? You know, that's to putting everything Every minute function into a container and then make it As part of the heavyweight infrastructure is You know is I think a lot of people the wrong way, right? You know, so that's why he had this to it and to say that he gonna Turn off 80% of those microservices and you know, the website needs to still be functioning, right? You know, that's it was it was very interesting because at the At the time, you know, you would see Twitter has random failures for instance the two fact You know the the two factors indication would fail because there is a microservice that's responsible for sending out the SMS Right, you know, and he shut that down that you know, the two factors indication was suddenly not working, right? You know, so it's gradually to fix all those problems, but you know, that's it is one of those very high profile. I would say You know objections to this whole paradigm of having too many containers and too many services that's that clutter up the whole infrastructure, right? so We so in in the in in the language of the cloud native computing, you know We position wasm as another container on time, you know, so we we often score wasm as a container You know, that's although it's not really a container. It's it doesn't give you operating system It has a very confined set of functionalities that you have to write your application to that specification or to take advantage of it, right? However, if you do that if you use you know, if you write applications in To the wasm API and then compile to wasm the benefit you get is first of all is very lightweight You know, it's a as I will demonstrate in a minute It's about two orders of magnitude smaller than the comparable container images, you know That's so we are going from as I will show in a minute tens of megabytes or even hundreds of megabytes container images To ones that measured in kilobytes. Okay, so a database application can now become Kilobytes in wasm, you know, that's so When we had those screenshots on Twitter, you know people were very shocked, you know That's so so we have a three-tier web application that has a mysql background has a web server and the total size of the application is 700 kilobytes. Okay, you know, so it's so but exchange you have to use, you know Maybe a new language or rust or maybe a new SDK, you know to rewrite your application this way, right and It's gonna go a lot faster than spinning up a container You know, that's all spinning off spinning up spinning up a Linux container. So it would be Again one or two orders of magnitude faster not at a runtime But at startup that allows it to scale to zero meaning you know a lot of container settings Because it takes like close to one second to start up a container in order to ensure that Response always happens in milliseconds. You have to have the container running all the time you have to keep it in the in the In the memory or or have a dedicated CPU to it. It's called keep warm, right? You know, but with wasm you can truly scale to zero meaning you run nothing when when this task is not needed, right? You know, so it's it's it's truly start up on demand, right? Because in wasm, it's you know in the host environment is just starting up a thread, you know There's no additional overheads that you have to load all those memories all those libraries and you know things like that and Wasm can arguably be more secure because it is a it is a new security paradigm that that's that's That's what they call capability based the security meaning that you have to explicitly declare what you Watch this container with this application have access to right? It's unlike say a regular container I'm kidding and as we all see we want to talk about ebpf is that a lot of things asked for the route permission, you know So, you know, you don't really want that, you know That's so in the in wasm. There's more finely green way for you to for you to do those things and perhaps More interesting and very interesting today is that wasm application is truly portable Traditional container applications are not truly portable because when you look at the container images, they have x86 They have arms, you know, I'm CPU, you know, you have at least those two distinguishings Which is okay if you only have two CPUs However, more and more workload. We see today runs on GPUs and other specialized hardware. So If you look at Pytorch Docker image, it has different image for different version of CUDA and different version of CUDA And so there's hundreds of images and each of them has a work has slightly different, you know You know external driver version that is that being versioned to external drivers, right? That's really one of the big challenges in today's Claw native computing is that Kubernetes is designed to distribute binary artifacts It's not designed to recompile your source code on your deployment machine because it assumes That's the binary artifact is somewhat portable and with the new That's why I want to talk about AI in a minute That's with new AI and ARM workloads that promise has been broken, you know, because you know say if I develop Even if a Python application you have to specify whether underneath it you want to do the CUDA driver or something else, right? You know, so if I develop something on the Mac, there's no guarantee that this same application gonna work on A video machine in the cloud. There's no guarantee that this machine this this application gonna work in say the AWS Specialized inference chips. That's called inferon. I think, you know So you are going all the way back to pre Java days 25 years ago Where you developers have to recompile and attest their applications on the deployment platform, you know, so I think that's It's also one of the features that wasm significant features wasn't gonna give us is that it's truly a portable You know, it's just one binary. It's always the independent CPU independent with machine learning and AI workloads It's GPU independent. So, you know, you can just you can truly run anywhere. So those are the Some of the features. So, you know, so those are the things that I talked about, right? You know, so if you I know it's a little hard to read but, you know We have You know around that time we we we published the I wouldn't call the benchmarks, but you know some some numbers of you know some applications that we built, right? You know, so those applications is some kind of web service It has a has a you know, it runs a web server So you can access from outside and it has some kind of database on the back end The database could be my sequel could be real S radius could be postgres and if you use the Traditional way to Let's not say Java because Java is hundreds of megabytes But even if you use go to write or us to write those applications and wrap wrap around in a Say a one-two image the the the container size would be in the tens of megabytes But here we can see that's a you know The wasm size the entire application is about I think 700 kilobytes or 800 kilobytes You know, it's that you know a sub one sub one megabytes You know, so it's that the difference is huge the difference is at least one order magnitude often times two orders. So Well, it's not just us that says that, you know, so the since if any survey said What did they say all containers are the new normal and the web assembly is the future I think that's why you know, that's why they say it, right? You know, so So a little word about you know, the wasm age project wasm age project is a cncf And it's only a wasm runtime project in cncf You know, so there's other wasm application project like wasm cloud is actually the application server that built on wasm right, you know, so But wasm age is the actual runtime, right? You know, so it is a what do we call a cloud native purpose build a web assembly runtime for for server side applications and It has a lot of features that are important on server side, but not important in the browser, right? So for instance, it can create HTTP servers and in order to create HTTP servers, you need asynchronous connections because you can't you know Node.js is a single-threaded HTTP server you can have multiple connections coming at the same time But they're on the same thread, but they can share time because they do not block each other, right? So we do the same thing with with wasm age starting with HTTP servers in that right, you know So we support database connections We can connect out to MySQL or Postgres and you know where I have different databases as we have seen from the from the previous tweet, right, you know So build 3TR architecture applications the server the web server the middleware and the database, right? We support a lot of AI workloads, which we're going to talk about later You know, it's including PyTorch, TensorFlow, Lama.cpp and you know things like that to provide a cross-platform AI experience for the for the you know for our applications and Perhaps the more interesting is that we support existing container tooling meaning that's within today you can use Kubernetes or Cryo or C wrong or elements of the or even Docker desktop to directly manage wasm age applications as if they are Linux containers and we're gonna show that in a minute, right? You know, so that's one of the advantages of playing this ecosystem playing the CNCF ecosystem is that we get other tools to support, you know, deployment of of wasm applications So let's see how it works in Because this is the question I get a lot, you know in Kubernetes As we have just mentioned the Kubernetes container Linux container for Linux containers are not Not truly cross-platform. So you need to give it a CPU architecture, right? You know, it's x86 or arm 64, you know, whatever. So there's a new CPU architecture called web assembly wasm 32 Okay, so when you when you have a wasm binary artifact, you build a wasm application to build a wasm binary Bite code application sort of like Java bit code application, right? you can upload it to Docker hub and give it the CPU label as wasm 32 when the Kubernetes tools like C wrong or container D plus plus a shame plus wrong wasi Pulse that artifact from the Docker club. It checks the CPU architecture CPU architecture to see if it can run locally now. It sees is wasm 32 It's there's no corresponding CPU that can run wasm 32 at that point The tool knows that it needs to use a wasm runtime to run it, right? You know, so at that point it's start wasm edge and run it so for Developers it is a fairly straightforward experience, you know, is that you can run Linux containers and the wasm wasm applications side by side in the same cluster So you have a pod that has maybe three Linux containers running as a servers and the two wasm Applications that running as a serverless functions or something like that, right? So it's This experience of course still needs to be it's continuously being proved, you know So we have you know the partners that we work with like liquid reply You know, they have a k wasm project that has has a Kubernetes operator that's automatically installed wasm runtime on the cluster And you know things like that. So there's lots of things that's that happen in this space But the general idea is that we define the new CPU format called wasm Sorry to that doesn't it really exist, you know in the real world But when the Kubernetes runtime sees that it knows to use the wasm edge runtime to run it, okay? So here are the some of the database connections that we support, you know That's we're gonna go over that. So, you know, that's With all the introduction about what is wasm and you know things like that, you know So we we can finally talk about some use cases, right? you know, so the first use case we want to talk about is wasm for streaming functions and You know, this is somewhat a hot field. So, you know, I think only a couple months ago, you know Well, it's that 222 but I think this may be early release. They have a new updated release. So Streaming engine Red Panda has declared that they would have wasm to build a building to their product You know, so the idea is really to have the messaging queue and processing Live on the same place because used to be they're separate the Kafka server and the flink server are two separate servers One manage the messaging queue as a process the managing messaging queue, right? But with the wasm server is function bedded into the messaging queue You can do lightweight processing right there. So one of the use case examples is that so for instance There are a lot of incoming messages, but some of them have sensitive data like containing password You know things like that or your worse Bitcoin private key, right? So you could have a wasm program that detects those features and the discard those messages not lock them, right? You know, so right there without having to spin up a flink server on the side, right? You know, so do that right there in in the in the messaging queue, right? You know, so that's a red panda's work, but the thing I want to really talk about is We can go a step further to say we can we want to use wasm to do log processing in the in the In our say large department of Kubernetes clusters. So The current state of art or the current state is that's the ELK stack. I think many of you probably have Heard of that and played, you know struggled with that, you know, it's a stack of elastic search lock stash and K-bana Those are three JVM's those are JVM applications And of course you can't directly deploy JVM applications in the in the in a cloud native cluster You have to wrap them around in Linux container the VMs So that blows their size to you know hundreds of megabytes and you know So it's a it is a very popular stack, but it's also in my opinion very slow, right? You know, so this is actually one of the huge bottlenecks if you look at Leading internet companies. So, you know, one of the guys that we work with They have 2003,000 the machines processing locks. Of course, they have 700, you know, they have over a billion, you know, and may use using their platform, you know There's a but you know the the cost that's Inherred by just processing those locks and in the new era of you know People want all their data to do machine learning and you know things like that It's even more costly because you can't really because you want to save those that you want more deeply processes of status, right? You know, so it's a it is a it is a challenge. That's that's that is very big. You know, that's sorry. So how does Wasm help here is that instead of having those three containers for its JVM's and the process data Can't we have lightweight containers? Reading wasm that is much fast that is much faster and much smaller to do Something mundane like processing locks, right? So this is the project. That's exactly what we did with one of our so By the way, I'll give a shout out to Linux foundation Mentorship programs that's you know, so they get graduate students from all over the world to work on You know Linux foundation host data projects. So this is one of our graduate student interns He came up with the whole solution and And so it's sort of like Google summer code, but did the did the whole project, right? So in this project, you know, as you can see He wrote a wasm application that start with the Database bin collector or the pot lock collector, you know, so it collects a lock Hookup with a database or hookup with a log file in the in the in a different in a different container in the same pot, right? collect those Those locks and then with those locks Implemented components to pass those locks and aggregate them and then process them with each step There's a way to plug in your own logic, right, you know, so wasm is ultimately application development platform So it provides You know, if you think in Java terms is the abstract class that dividing here So you have to provide your own implementation, but in rust terms, it's the trades Right, you know, so you implement those straight and compare it with with the existing application You add new features to it, right? So, you know, that's he did all this in rust So he so this so this is the overall diagram. That's you know, that's allows the application to start a very lightweight Wasm application wasm container and then use it to process all this data, right, you know, so the you are the The GitHub repository to this, you know, it's listed here. It's under the wasmage organization it called log flex and If you look at it works out of the box So it has a configuration file That's that you can just simply use because by default it just collects those logs and send them to a database, right? You know, that's so you know So it collected and being locked from my sequel and you know So it's a configuration file the application reads and then it specifies where to collect the data Where to how to combine the data and what's the format for the output to to send out, right? You know, so so I won't have time to really demonstrate this but you know, that's But when you see that you you get the idea, right? You know, that's get data from my sequel and all to put the radius or out for the Kafka, you know something like that, right and But like I said, the most interesting thing about it is that this this project if you are a developer, you know this allows you to Define your own logic in terms of how the data is being processed so an important thing here really is that you know, you implement this trade and You you you compile this trade with the With the wasm log flex project you get a new wasm file and then you deploy that you you deploy this wasm file into Your Kubernetes cluster now you don't need those the ELK stuff, right? You know don't need the three JVMs plus three containers plus operating system and everything you just have one You know very lightweight Perhaps two three megabytes size, you know Wasm application where you know, you can you can start in your in your your pod and and start to collect all those data right, you know, so this is one of the examples and I'll go a little quicker because I have eight minutes left. So we talked about how to get data and how to You know Process the log. This is one project So I want to move on to the next project, which is also done by one of our Linux Foundation interns so she's also a graduate student and what she did is to solve the problem of How do you get the source of the data? You know because the the the source of the log data typically come from say, you know like in the previous example, it's come from the The Mexico green lock right, but a lot of system level data Increasingly they come from ebpf. So you have a ebpf program You inject into the kernel in the that runs the host offering system, right? You know and then start to collect networking level data so that when that data get emitted you can have the the stack that I talk about instead of the okay, but I have wasm file to process that but to Deploy the ebpf is Is one of the pain points that we we have identified that's you know, that's we we are working with some you know on cloud native companies that's That are doing this work, right? You know so today without wasm There's really two ways to deploy a dbpf one is to integrate integrated control plane You know, that's essentially you start a container to deploy it, right? You know, that's and the other one is that you have a sidecar you have a ebpf demon. That's you know things Project like one would be and things like that. So those are the two existing Deploy models, but they all have problems, you know So for the integrated deployment you have to start container with the operating system Just to deploy a ebpf agent and manage the life cycle of that agent just think about you know Whenever I see those I think about the waste it generates, right? You know, it's you know, all you need to do is deploy something But instead you have to start a container to do that and the container need root access because you know, it's Because the Linux security model is that you know nothing or all you can't have finally grain access You know, you have to if you want to inject something to current you have to have that root access, right? So this is one of the problem with integrated deployment where you know, you have you have a dedicated Linux container to manage this process, right, you know, so Then on the other side is a sidecar deployment sidecar deployment is Because that's the process you have to run in the pot. It's a lot less flexible It's a lot more Intrusive to the to the Kubernetes system itself, right? You know, so I'm not saying there's anything wrong with those two deployment I'm just saying there's room for improvement, right? You know, so the idea of rooming improvement is really to have The ebpf deployment container not be a Linux container Okay, but a wasm container something like this. Oh, sorry. Oh, I think I'm missing a slide. Okay. Well, yeah So that's the the red one should be a wasm container that has so what we call was the extension for ebpf Meaning that's there's a system a set of system cause that's specifically scoped for deploying ebpf agent and collect data from those agents right, you know, so The project itself is is is is here So there's a library and the tools, you know You have your existing ebpf program and you use the The tools provided here to compile them into wasm, right? And then you use Wasm edge plug-in which provided those host functions into the into the wasm runtime into wasm container And then the wasm container would be able to take that wasm file Strip out the ebpf stuff and inject that into the kernel, right? You know, so it's a complete the whole process, you know, I'd really Again, I'm running out of time, but I really encourage you guys to try this I think this is a really nice way to you know to to deploy ebpf as you know, that's that's you know That's that is lightweight and that's more manageable. Yeah so, you know, so here are some of the You know Benefits, you know, that's it's either faster, but you know, where they knew that, you know That's so we keep I talked nonstop for like one and a half an hour to say why this is better faster either, right? You know, so There's One last thing we want to talk about is that there's another way of streaming processing and streaming processing is called AI You know, the AI applications are just data streaming if you think about the AI applications They're just you know You inject prompts and then you get some data back and then use that data to call function You know something like that When wasm edge first started a lot of our customers use AI on the edge meaning that's the AI in the camera where in the car You know, those are all data stream of image data's right, you know So you do processing over those and the the idea is I would skip forward So, you know, so we have a project called llama edge, which is an application that build on top of wasm edge that's allows a True cross-platform comparability for those applications meaning that's I write towards a wasm API and once I Once it's compiled I can deploy it to any device be it Intel bit AMD bit and media bit Mac bit arm, you know Anything the wasm runtime figures out what is underlying runtime you're gonna use and what's the GPU? It's gonna use right so it's gonna so the there'll be one binary that being concentrated across the entire each network and the the the cloud network by Kubernetes so Yeah, so there's there's I'm gonna share that in a slide So, you know, there's something you can go go to the hotel and try it You know, that's it's gonna download a large language model. We're on your own on your own computer and you can try it right, you know, so Again those the third time those are also built by our interns. Thank you So, you know our interns has to build a pytorch extension to wasm edge meaning there's There's not no Python in there But the pytorch C library and the pytorch C API build that into wasm so that from from wasm You can use rust or go or JavaScript to call those Pytorch functions They build TensorFlow they build open. I know so it's not done by the same internet down by Over the span of I think one and a half a year by Over five or six interns, you know, so some of them build open CV FFM peg You know, so now we can support the the meta models the media type models, which is a Google set of you know Vision and audio models and even things like the yellow five object detection models And you know things like that. So, you know, I think again running of time But there's lots of examples that that you can see for this type of stream processing not really a lot anymore but image data audio language and you know, so you can you know You know every single one of them has you know, a CI CD attached to them So you can see how they run in a to get up environments so you can run it on your own environment and you yeah, that's so Think thank you. Yeah