 Welcome to the tutorial session for creating and deploying a lightweight microservice in WebAssembly. So we have a lot to cover today. But first of all, I want to introduce the panels of the speakers. So my name is Michael Yuan. And I'm the maintainer and the founder of the Bottom Edge project, which is the WebAssembly runtime in the CNCF sandbox. And here we have Vivian Hu and Hong Yin Tai. And you can call him Hai Dai. And I'm based in the United States. They're both based in Asia. So they flew 15 hours to be here to give this talk. So thank you. Thank you. So the agenda of the talk is that we're going to spend, I think, maybe 10, 15 minutes to talk about the background of the whole thing. Why do we need WebAssembly? WebAssembly has been a hot topic for, I think, at least for this conference. The opening keynote, talk about maybe the next thing would be WebAssembly. So there has been a lot of interest. So we're going to spend a couple of minutes to talk about, especially, how WebAssembly fit into the typical use case scenario of a microservice, what we see, what our users and our customers in our community have been doing with WebAssembly and microservices. And then we have a series of four tutorials. They are all hands-on. And if you have your computer here, you are welcome to follow along. They are not super complicated. They are something that you have all seen before in the world of Java, in the world of Node.js, and in the world of other languages that you have used for microservices. We are just going to have a new stack that we're going to use WebAssembly for that. And I'm going to tell you why and how to do that. So we can have a series of four tutorials. And each of them takes about, I would say, 20 minutes from start to finish. So we may not have time to cover the number four. Although I think the number four is quite interesting because it really showcase why you need WebAssembly. And it combines hot topics like chat, GPT. So if you don't have time to get to the last demo, you are very welcome to try it yourself. And so all the resources, including all the demo steps, is available in that GitHub repository, KubeCon EU 2023. And this QR code is a bit relink that goes directly there. So just go ahead and check out and follow along. But if you just want to sit back and enjoy it, because this is the last day of the conference, and I'm the talk before lunch. So you are more than welcome to follow us. We're going to go through all the steps on the stage and explain what each step does. So that's the agenda of the talk. So I'll start with the why question, the so what question. In this industry, we keep reinventing the wheel. That's why do we need another thing to run WebAssembly. Is the current technology good enough? So there is a, at least from our point of view, there is a distinct trend that's happened, I think, opposed to pandemic, is the rise of lightweight microservices. What does that mean? I think one of the tipping points is about this guy. His rocket just exploded yesterday. But I heard it was a success, although it exploded. But people have a lot of different opinions about him, but I think this series of tweets really gets a lot of people thinking about the microservice architecture. So he said, when he took over Twitter, he went there and said, he didn't realize there's over 1,000 RPCs. What he really meant is web services, is microservices. Within Twitter, if you think about Twitter, it sounds like a fairly simple application. How is it possible that needs 1,000 microservices? But that's actually not far from the norm. We have worked with large internet companies, both in Asia and in the United States. We have routinely see applications that requires 10,000 microservices or more. Those are 10,000 different services, not 10,000 machines. Each service requires a couple of machines. So there's really explosion of microservices. How to manage those microservices and how to make some wrong efficiently has become a huge challenge. So you have patterns like the service weaver and things like that. There are lots of solutions out there trying to do that. But one of the most direct thing to do is to make those services much lighter and much faster. So that's where this whole idea come from because I would say, I hate to use the word traditional because Java may be traditional, but microservices, a lot of people consider it's a fairly new technology. But if you look at how microservices set up today, you typically have a language runtime. It may be a go runtime. It may be Node.js or it may be Python or maybe JVM. You have a language runtime. Outside of a language runtime, it's wrapped around with a Linux operating system. And outside of that is the container or it's a micro-VM, it's a firecracker or it's a Linux container. And after that, there's the Kubernetes pod and things like that. So when you combine this all together, if you are talking about a very simple function that all it does may be sending out an email. So one of the microservice that Masq turned off and caused a lot of trouble was the service to send out SMS message for two-factor authentication. And then suddenly people can't log into Twitter anymore. So those are the things you see. They are very, they do one thing and it's not very computational intensive. What they do is call another API. Maybe a Tivolo or maybe a Telco API or something like that. To start a whole Linux system and then start the JVM inside it just to do this one thing. And it's not very complex not computational intensive. It's very, I would say it's very wasteful. So in the way of the new jargon it's a zero interest phenomenon meaning there's, people are washed with money. So they don't care. Just throw more machines at it, right? But I think we are in a different era now. So we are looking for efficiency. I think that's the primary driver of the so-called lightweight microservices, right? So I think we have just to talk about those pain points. There's the heavy weight. They are very slow, especially at startup time. If you have used things like AWS Lambda is one of the biggest problems that they have. Is that the startup of the micro VM, Linux and then JVM takes hundreds of milliseconds, if not seconds. But you can say, okay, it's startup slow, but it's okay, I can keep it warmed up. But as soon as you keep it warm up you are wasting even more resources because now you are idling, right? So you can't scale back to zero. So you have a lot of issues like that. And there's a general trade-off between security and overhead because if you look at how Linux Container is used you look at projects like, what's that project? I forgot, it's kept in my mind. But there are different projects, a runtime project that aim to secure Linux Container but they add weights to it, right? It's in general not portable because Linux Container, you need to differentiate what's running underneath it. If you look at today's cloud providers they all have at least the ARM architecture and the Intel architecture. If you have more complex workload there may be the GPU and TPU. And if you look at, there may be also RISC-5. So there's lots of different architectures that's in play today, right? So there are many pinpoints to do that. So why is WebAssembly a good fit? So I'll make some very bold claims here and Hidai would back me up with more details in his section. Is that the first, for a comparable task, so for say, I want the service to babysit a database. It's very typical, say a Java pet store a type of application, you know, Herd, CURD, right? It's typically the 101% of the size of a typical Linux container applications. So instead of tens of megabytes or even 100 megabytes this thing is measured in kilobytes, okay? So, you know, you think about that a web server plus my SQL clients with a connection pool everything added together is measured in kilobytes. And startup time is one millisecond, two milliseconds. So it's not in seconds, right? And this has near native runtime performance because there's a lot of compiler techniques that we can do to make it compile ahead of time not just the JIT like Java does, but AOT, right, ahead of time compiler. And it's secured by default because it has a very small attack surface. It doesn't have an operating system in it. So, you know, there's very limited ways you can attack it. So it's secure in that regard, right? And it's completely portable across platforms. So this is, a lot of people compare WebAssembly to Java. You know, it is a bytecode format that is abstract away from the underlying CPU architecture. So, you know, so you can have, you know, you can have one WebAssembly bytecode that goes across, say, ARM, Intel, and the RISC 5 and all those things, right? It's strived to be programming language agnostic as well. You know, that's actually an interesting point and that's also a bold claim that, you know, the community wants to make. But, you know, we are still getting there. So, you know, meaning that's any programming language supported by the LVM compiler would eventually be able to compile into WebAssembly. That means CC++, Rust, and, you know, and in the future, in the near future, I think we would have go. And a variety of different interpreted or, you know, high-level languages like Python, you know, VMware labs, and VMware's, you know, Wasm Labs has done a lot of work putting Python to WebAssembly. And we also have done a lot of work putting JavaScript and also Shopify. You know, there are a lot of companies in this space that are trying to make, you know, the everyday programming language more accessible to, you know, WebAssembly environment for developers, right? So it plays, and also very importantly, and that's why we are in KubeCon, is that it plays well with, you know, a service mesh and Kubernetes. Unlike the JVM who has the same aspiration, but the JVM cannot be directly managed by Kubernetes. We all know that, and it's not a secure container. So, you know, that's, but WebAssembly is different in that regard, it's designed from ground up to be, you know, to be a secure container. So that said, nothing is free. You know, that's, I'm making some very bold claims. I said it's a hundred times smaller, it's a thousand times faster, you know, and as if there's no trade-off to be made, but there are trade-offs to be made. So there's no free launch. The biggest issue here, you know, that's why, you know, we run tutorials like that. So to get developers, you know, getting developers started with WebAssembly development. It's the most important thing to remember, it's not a general OS environment. A lot of the benefit it has is because it's not Linux, but that also means if you have a Linux machine and you have locally tested Linux application, you would not be able to directly run in WebAssembly because, you know, the architecture is different. So there's new SDKs to learn, you know, that's to do similar tasks. So there's, you know, in this ecosystem, there's has been, you know, I think a explosion of, you know, open source projects and also commercial projects, right? So there's a couple, you know, I really want to call out, you know, like Spin by Fermia, you know, they are, they have a really nice, you know, JavaScript-based SDK and also Rust-based SDK that allows you to write, you know, HTTP servers and, you know, those type of applications. Running that inside, right in those languages and running that inside WebAssembly, right? You know, so they have their, they have a set of SDKs that's APIs that you can use and the Wathom Cloud, which is our sister project in C and C of Sandbox, it's run by the team at Cosmonic. They also have the idea, you know, a similar approach, but they also have their own SDKs, right? You know, to solve, you know, a different set of problems. And then you have Wathom Cloud and component models. This is, you know, WebAssembly's standard body, which is W3C and bytecode alliance, wants to standardize a lot of those external components to WebAssembly. It's sort of like relationship between the JVM and the JRE, right? You know, that's, you know, at least I'm old enough to remember that. Maybe some of you guys too, you know, that's a, it's a core runtime into the library around it. So, you know, so there's also effort to standardize that. And in this talk, we can also talk about our interaction with Dapper, you know, because Dapper, you know, as a microservice site car, can bring a lot of value into WebAssembly. We're gonna talk about that later. You know, so with all those, those are new SDKs, you know, new things that's to develop new applications, what I would say. However, you know, one of the focus in our talk is, what if you have existing microservices that you use, that is written in established frameworks? So for instance, you have a Rust developer. You are likely to have a microservice that's written in Tokyo, right? You know, or use Tokyo framework to do, you know, networking and to do database access and, you know, things like that. Say if you're a JavaScript developer, your application is written in Node.js. Is there a way to run them in a WebAssembly runtime? So, as I said, it's definitely yes. So that's, you know, something that's, you know, we want to explore in our four tutorials. So, in the next couple of slides, maybe we can demo from here. We're gonna talk about how Wasm Edge enables that, enables us to run those, you know, existing applications, existing microservices that you've written in existing, you know, libraries. Hi, I'm Hai Dai. And today I'm going to talk about how Wasm Edge's approach, you know, to support its team projects. So, let me introduce Wasm Edge first. Actually, it's just a Wasm runtime, like maybe you have learned from, you know, the Wasm time or the WMR or such of Wasm runtime or Wasm 3, right? And we have different focus. For example, we are focusing on how to support the networking socket and web service. What does it mean? Actually, in the current days, you can find that there is some very limited socket support in the Wasm spec. So you can only, for example, you can only accept something socket. However, you cannot bind, you cannot do the dominant service checkup and you cannot do the TLS things. So we are focusing on how to bring these, you know, these powerful networking socket applications to the Wasm's world. And also another focus on us is that we want to support more and more database because that's a model of microservice. They will communicate with the database, you know, you have to rewrite data. So let's, you know, are none of us. And the third thing is that we all know that, you know, the AI inference is very big today. For example, the chat GBT, right? So how to leverage those AI models in Wasm runtime is the very key features in our implementation. For example, we already support a TensorFlow. We already support an open vinyl, and more and more AI frameworks in the near future. Also, we have already integrated into the Instinct Cloud native infrastructure. So you can use Wasm Edge with the RunWassie or use the Docker, use the Kubernetes, use the continent B, you know, to manage your Wasm application. And then we have learned that the JavaScript community is very huge. So, you know, we want to help them to acquire their current project into the Wasm's world. So that's our very big difference with the other, you know, the other Wasm runtime. So how do we mean the right way? We have two examples, and one is the Redis application, and the other is PostgreSQL client application. With these two applications, it's already run into the dark plus Wasm technical preview. So you can go to the Docker's website and download the technical version and try to, you know, play with these two demo. And you can find out that the total application size is pretty much smaller than the user case. For example, you only have 0.7 megabytes here and 0.8 megabytes here, and it's pretty small. That's why, you know, we want to help the current container with providing a new way, which is the more lighter way, you know, to manage some, you know, some lightweight services. So how we bring these features to, you know, to the Steam project? Let me just, I mentioned that the Wasm stock is very limited and is not very useful for the current project. So we enhance with the following important features. The first thing is that no one like Blocking, right? If you are running a Blocking stock here, then lots of, you know, the data-intensive application will stock your whole stack. So we enable with the number of Blocking stock here. So now you can do, you know, concurrently the HTTP request and database query. Also, one very important thing is that somebody just told us, hey, why the Wasm stock can only accept IP address, you know, when I deploy our service on the Kubernetes and we will get lots of your domain name. So the one important thing is that how to support the domain name. For example, the name service lookup from Wasm's back. So we're just porting from the library called get address info. So you can now retrieve lots of information in a Wasm label. Also, TLS is good, you know, DomainSack is good. There are lots of features. It's because we extend the current API so you can do a lot of stuff. And with this as our, you know, as our infrastructure, how we bring these features to the esteem projects. We just customize, you know, customize those guest SDK. For example, we fork Tokyo, we fork MIO and just, you know, switch the Wasi target to Wasm Edge Wasi target because we want to add those, you know, those powerful functions in layer version. So the only thing you need to do is just change your name from Tokyo to Tokyo Wasi, then that's done. We just want to, you know, simplify the program's life. So the only thing you need to do is just change the name. Okay, and we also create lots of folks for the database client. So you can just, you know, switch your roster credit and just use our version, you know. So that's how it works. And now in our demo, you can find out that if you have the Ross Tokyo best client or you have the, you know, the Noda address client, now you can work with the Wasm Edge Plus, the Wasm Edge Wasi socket, yes. And we already suppose the following database including MySQL and MariaDB, Postgres SQL SQLize, and, you know, lots of all the database. If your favorite database is not on this list, we are very happy to have progress or we are very happy to hear that. So we are, if you have any, you know, recommendation, you can just talk about us and we will try to, you know, just implement your, you know, your favorite database client. Yeah, we can cooperate with that. And enable socket is not just allow us to do the database staff. We also want to go beyond the database. So with our, you know, our improvements, you can have just, you know, use the hyper framework, use the request framework. Or if you are using the Kafka, you can use, you know, the Rust Kafka to integrate with us or Redis server or Rust or Dapper, okay. And if you are using JavaScript or we have already, you know, porting some NPM and it's in, you know, not just functions in our ecosystem. For example, if you want to use the React SSR feature, yes, you can try it. And if you have any interesting, which is not this on this slide, just talk to us. We are very happy to help with and we want to bring, you know, we want to enhance the whole community with, you know, more and more features. And we already integrate lots of the tooling. So in our, you know, in our following demo, you will see the darker one, okay. And if you are familiar with the container D or Kubernetes, you can use them with the run was suppose. And if you are using your part man or you are on the Fedora, you can try serum plus the wasn't suppose. Yes, the, all these on this slide, we already have integration and we are very happy to hear that you guys to try it. Okay. Now I think we will have told lots of non cold staff and actually everybody's now is running to, you know, to see our demo. So let's do our first tutorial. Let's welcome Vivian. Hi, I'm Vivian. I think we have learned why we need the WebAssembly. So it's time to do something. I will do the first tutorial, create a complete three tiered microservice with Docker plus wasn't. You can find the source link and you can also scan this cloud code to access all the resources in this tutorial. In this part, we only will use Docker last year in KubeCon Detroit. Docker announced is supposed for WebAssembly. It is a partnership with wasmage. With Docker support, we could run wasm containers side by side with Linux containers, which I will show in the next. With Docker support, we all one more interesting is that with Docker support, we don't need to install Rust and wasmage. We just need to install the Docker desktop. So here is a typical microservice architecture. I think most of you here are familiar with this image. We have a web server for the front end and we also have a database. In the middle, we have our own business logical. It includes an HTTP server to connect to the web server. And we also have a MySQL client to access the database. They are all running in Linux containers. But with wasmage, this architecture has changed. We can see the green part. The green part is running in wasmage containers. We can also have our own business logic functions. In this wasm-based microservice, we can also have HTTP server to connect to our web server. And MySQL client to access the database. So this is the steps to create an interior microservice. The first step is to download Docker desktop. The version should be 4.15 or above. In the GitHub repo, we have four steps to reduce the demo. So the first thing is you need to download Docker desktop. So if you have a laptop in your hand, you can download your Docker desktop right now. And the next thing is to get the source code. I already opened my terminal. I forgot, I already get cloned as a demo project in my computer. So next, let's open the folder and then we will just use one command line to build and run this microservice. We just use Docker compose app. It may take a minute to build and run this microservice. I will ask Michael to explain the code we use. Okay, yeah, so the microservice itself is written in Rust. So I know maybe many of you are more familiar with Go or JavaScript, but just bear with me for a minute and I'll show you how simple Rust can be for a microservice environment. We are just to emphasize what Vivian had just said. As you can see, we just downloaded Docker desktop. We did not install Rust, we did not install Rust. We did not install WebAssembly, we did not install anything. Docker takes care of all that because it's a containerized build. So it downloads the correct container to build it. So can I make this better, I think? Okay, so this is the code. It looks fairly long, but really the important thing is that at the end, the main function, it's like Go or C, it has a main function. The main function has two parts. The first is to set up a connection pool to a database. If you are familiar with Java and it's a very familiar process, basically you build up a connection URL to the database and you give it options, you have a database connection pool, it's a five to 10 connections and sitting idle in there when you need it, you get one from the pool. So you set up the database connection pool first. And then the second part of it is to start a server. Okay, so this piece, you start a server and bind it to the local IP address. The server is running on 8080. And when a request comes in, so here it's a little bit Rust syntax. It's an asynchronous function so that when a request comes in, so you can handle multiple requests from here. So when a request comes in, it starts a kickoff as asynchronous function. And what the asynchronous function does is that it calls the method handle request and pass in the HTTP request and the database connection pool, okay? So that's really the whole setup logic. Essentially, you create a database connection pool, you create a web server and when a request comes in, you call a function called handle request and pass in the request and the database pool. So the bulk of the application is actually in this handle request. So this asynchronous function in Rust, so it's handled by Tokyo. And if you read into that, it's actually very simple. So it's really detects what's the URL elements that's in the incoming request. If the incoming request is slash INIT in it, it would just say, okay, I'll get a connection from the pool and I'll send this SQL statement into this connection to initialize the database. Here, it's just a create table connection statement, right? And if this comes in as a create order, I would just, you know, because the create order request has the order information in the body as a JSON object. So I'm gonna pass out the JSON object, I'm gonna map it, you know, Rust has very nice, you know, strongly typed mapping between JSON and the Rust objects. You know, so I would just map that and I would create the SQL statement to do that. And, you know, so the Rust is just a current application, you know, so you already, you know, create, update, read, read, write, you know. So here's, you can create multiple orders, you can have an array of JSON objects and create them, and you can have update order, yeah, you have update, oh, you have the U here, right? And you can have orders, you can get all the orders back in the JSON object. Basically, it's a select statement, SQL statement for the database, right? So, you know, it's all very common, you know, stuff, and it's just a syntax look different. So I keep encouraging people to learn a little bit of Rust, you know, it's like all those years ago, I really encouraged those pro programmers to learn a little bit Java, you know? It's, you know, so now I'm encouraging people to learn a little bit of Rust. I think, you know, Rust may seems very intimidating, but it's not, you know, you can see this, everything here is really, you know, it's just, you know, what do we do on an everyday basis, right? It's a little bit error handling, you know, so you have, you know, you have the main function that's set up all this, and then you have the asynchronous handle request. So let's see if this has, Subirian, do you want to continue? Yes. Sorry, I'm too, I'm a little nervous, so I forgot something. I'm sorry, installing the Docker desktop, you should enable a better feature in Docker desktop. So click on the sighting, and you will see feature in, sorry, feature in development. We should select use continuity for pulling and storing images. This adds the ability to run wasmic containers. This is a must-have site. If you made something wrong, this may be the problem. Okay, let's continue. I think the containers are running, so let's go back to Docker desktop and see what happened. First, we can see we have three images, and the first one is called demo microservice, and it was always a wasm label. This means there's a wasm images, and we also have MariaDB and NGX image. I want to highlight on the sides here, as Hadad just talked before, the size of wasm image is very small. It only has three MBs. It's much smaller than MariaDB and NGX, so wasm-based microservice is lightweight, fast, and secure. And we also can see that we have three containers. We have the client, we also have a DB, and we also have an observer. It is also labeled with wasm, means it's a wasm container. So with Docker plus wasm, we could run Linux containers set by side with wasm containers. Next, let's test if this demo works. Let's open the web URL. Let's click on add an order. I will fix some information here, and click on add an order. Here we can see we already added an order, and we also can delete this order. So there is no others. Does anyone have any questions? Okay, no questions. Next, Hadad will dive into this code. Hello everyone, it's Hadad again. Okay, so maybe you don't just want to use black box, just install Docker desktop, and don't know how it's really work. So I will show you if you don't have the Rust desktop, how complicated thing you have to do by yourself. So let's go, the first thing is that if you are not use Docker compose, so the first thing you will have the Rust program, right? And you have use the Rust tool chain to compile the program into the Watson application. And be careful, if you are using Kaggle build, just a command, you forget to specify the target to wasn't 32 WASI, then it will just not work because it will be compiled to the native code. So this is the first trap here. And after you have the wasn't application, you have two way to execute it. First one, if you don't want to, you just want to run it, then you can skip the compiler part. You just run the wasn't with the wasn't with correntine. And after the wasn't edge correntine is up, and we will figure out which plugin is used. For example, we all know here, we want to leverage the database connector. So we will have the wasn't edge socket extension, which is included the finding function, and the NSLuga and lots of, I just mentioned before, those feature is enabled. Then it will open several socket, raw socket. So it will try to connect to the MySQL database with the URL, and you will provide the HTTP socket so you can have just the website to communicate with your application. And if you want to improve the performance of the current wasn't application, actually wasn't edge provide a compiler, a hell of time compiler, which you can compile, you will, we will assemble the application into, just with a section, provide the native binary. So you can execute this binary format with a faster speed. So it's a little bit complicated. So if you want to try it, because you have to install MySQL database and start it, so I will suggest you to use a container, to install those staff then. If you don't want it, it just deleted the container, then everything is gone, right? Okay, so first thing is that, let's go back to our GitHub. So you have to install the Rasa Toolchain, install wasn't edge, and install and start your MySQL database. So I have a short script here, so if you have your container, you can just copy and paste it. And please, you may need to change your password with MySQL secure installation, and we will use the username and password in our following steps. So after you are done with the installation, I just use container in Mac OS. Sorry, let me clean it up. Okay, so first thing you will get the source call, and I believe you already get it in our previous tutorial. So the first thing is that we will need to build the application with Rust. So let's go back here, and cargo, build, release, target. And because I don't want to waste your time, there are lots of people, so I already compiled it, so you can find it's very quickly here, but I don't want to remove the build folder and do that again, it will take a couple of minutes. So if you are running it, actually it will be a couple of minutes left. So I just give it the part. And after you build your Watson application, you can find your folder here, here is your target. So the Watson file is in this target folder, and here, okay, so this way. If you build it, actually you are download lots of queries you use in this project and try to compile all of the Rust code into this Watson file. Then we have the second step, which is try to improve the performance with our AOT compiler. So just copy this one and pass here, and actually it will take some time, maybe a couple seconds to do the optimization for your host architecture and you just compile, verify, and optimization, now it's trying to do in the output file. Let's just wait. Okay, it's done. Actually it's more faster than you do the cutoff build, okay. So the next level is we will run this application, just copy it and pass, what's my password? Hold on. Let me check my history. Oh, collaborative, okay, okay. Now I remember my password now. So after, you know, it's cute this file, it will show nothing. Why it will show nothing? Because it's a server, you know. So the other thing is that you can use your browser just to find out your link with, for example, the order. Okay, so I have just put some, put some data in here so you can just see the details before, you know, we are building the database sheet. So when you run it, the first thing is that you will need to init, okay. So just copy here and pass here and init, you will say that the status is true, which means you already need it. And the second part is we want to, you know, we want to create orders with this order file. So you can find these orders just in here and we just want to add more and more different order. For example, let's change those in and change here to cloud native. Okay, you can do anything you want with this order, okay. And then just copy this line and it will try to, you know, insert those orders into the database. Okay, the status is true. Now we can check all orders here with this, this URL. And maybe it's number three T, so you can, you know, you can try to use the json pp to make it since it's more beautiful, right. So you can find that's the order list here. We just add a new order called the, oh, sorry, I spelled it wrong. It should be cloud native, okay. Then if you want to update something, you know, you can use the following command to update the order, okay. We can also, you know, change the updated order here, cloud native. This time is right, right? Okay, and we just copy here and past. Now it's updated, so we can, you know, we can just try to call the orders again and you can figure out, now, yes, the cloud native is correct. Okay, then you can use, you know, if you want to delete something, you can use the delete order by yourself to do that. So because we have seen the website before, right, everything you do in command line, actually you can write a simple website to control it. So, you know, you can just go to our client folder and use the simple script, just, you know, create a simple HTTP server to rewrite the data. So, if you use Docker compose op, then you will start an Android server and do all the stuff for you. But if you want to, you know, go deep into it, you have to do all of the stuff by yourself. So now there's a starter server. Here, we started in nice awesome. Okay, there's a lot of orders we just, you know, we just insert and update here. And now I need to, you know, remove the wrong one, right? Not good native, we'll delete it, okay? Okay, let's hold off, you know, hold off the demonstration of, if you are not using the Docker tool or any, you know, fully integrated tool, you have to go this step one by one by yourself. So you may be very comfortable with it, but we are happy if you can, you know, you can try it. So, let's go to our tutorial street, right? So, let's welcome Michael again. All right, thank you, Heidi. And I know it's launch time, but I promise what I'm about to say is more interesting than launch. So, it's to do more with Dapper, you know, there's a, you know, I know what you're thinking. When you see the demo, the first two demos, it's a simple card application. Okay, it looks cool, but feels like a toy, right? You know, that's, so, you know, you are setting it up with a database you already knew, you know, my SQL database and, you know, there's, you know, it's something, well, I would say a lot of microservices are simple functions like that. So, even in that context, I wouldn't say it is really a toy, but, you know, it's a, like I said, you know, when Elon Musk wants to get rid of microservices in Twitter, you know, the one that he turned up is to send out SMS messages, you know, how complex is that? You know, so, you know, so there's a lot of microservices that, you know, that are at that complex level. However, you know, if you really have complex services that has, you know, need to access things like, you know, very, we already can access the Kafka queue and, you know, things like that, but you may need, say, a MongoDB, not my SQL, or you may need something else, you know, that's, so there's lots of options out there. So, how do we take the next step to make Wasm services, you know, interact with essentially all the infrastructure services that are commonly available in the microservices environment? That's how we get Dapper to help us. You know, Dapper is also, I think a CNCF incubation project. So, you know, so they are designed for microservices. And so, let me take one minute to introduce the Dapper SDK for Wasi or Dapper SDK for Wasm Edge. You know, so the way Dapper works is that it is a sidecar, you know, so in a Kubernetes pod, you can have a Dapper sidecar running in that container and then the microservices application running inside another container and they communicate with each other. So, the Dapper sidecar would provide services, you know, like a secret store, health check, database, messaging queue, and you know, all kind of nice things that you may have in the microservice environment to the microservices you are running, right? So, we are building the bridge between the Wasm microservice and Dapper, right? So, we have a Dapper SDK. It's currently written in Rust. However, we are putting it to JavaScript as well. So, what it does is that from the Wasm application, when you need to access Dapper services, for instance, key value store and you know, things like that, you would just call Dapper. You don't need to know what's behind Dapper. So, because Dapper itself connects to over a hundred different services, you know, like all the common databases, all the common key value stores and all the common messaging queues all have integration or connections with Dapper. So, that really allows us to expand the scope of, you know, Wasm-based microservices, right? So, that's the introduction. So, let me give you, I think this is a fairly complex demo. So, you know, in a minute, I'm gonna do it. I'm gonna show you it's running in GitHub Actions because that's the most reliable way that I know how to run it. But, you know, it's consists of three services. It's an image processing application or microservice. So, it has two image processing functions. Each of them is a microservice. One is to turn an image into grayscale, okay? And, you know, we can see a gazillion use cases for something like that. And the second is image classify. What image classify does is that it runs a TensorFlow model in a web assembly to recognize what's on the input image and spit out a text, right? So, both of them are individual microservices and they each have a diaper sidecar connect to them. And then there's another one that's called event service because, you know, we need analytics and observability from those two image services. So, what we do is that instead of having an image service directly written into a common database for logging or common files, I have an event service. You know, that's fairly common usage pattern, right? You know, that captures the event from those image services on the front end. And then it writes into the database or write into a key value store, right? You know, so it, so the three service, you know, in a, I would say in a simple or in a naive environment, you would have the URLs for each services hard-coded into each other, right? You know, you would have, say, the grayscale service is aware of the, where the event service lives. It's IP address or it's DNS address. So, Wains needs to record something to just, you know, make an HTTP call for that API. But with Dapper, you can simplify this a lot because each of them connects to its own Dapper sidecar. And this Dapper sidecar allows, you know, they communicate with each other. So you don't have to know the IP address or the URL of the service you want to use on that end. You just need to give it a name, right? You know, that's, so Dapper would look at it for you. It would set up the HTTPS connection if it's necessary. And, you know, because the Dapper sidecar can expose HTTP interface as well. So you can connect to the Dapper sidecar directly from the outside. And then, you know, so for instance, you can connect to the Dapper sidecar that's attached to the event service and send it the image and say, I want to recognize what's on the image. So the Dapper sidecar knows how to communicate with other three, other two Dapper sidecars to send that image to the image classified service. And the image classified service would run the TensorFlow model and get the results and then record the event in the event service and then returns that result to the service caller, right? So it's a really, you know, you know, we, you know, in a microservice context, we really love that approach because I think it's a really good fit for WebAssembly because WebAssembly still has this, you know, on the, it's still a baby, it's growing up. You know, it's, it can do a lot of things but it still has a lot of things that's, you know, even look at early days of Java, it's the same thing, lots of libraries are not available, you know, that's why I want to do X, a very particular segment is not available but in all likelihood, Dapper would allow us to do that, right? So let me go into the demo where what I'm going to do is that, sorry, where is it? Okay, here. So I'm going to kick off the CI build because it takes, I think, 10, about 15 minutes from the front end to the end, so you know, so I'm going to kick off the build and then I'm going to go through the CI document to show you all the steps, what the build does, right? So what we, you know, as you can see, I already successful run, you know, that's, I did my homework, right? I know it runs but I'm going to show you so I'm going to say rerun all jobs and what it does is that it's going to kick off GitHub action, it's going to get a Ubuntu server and it will just start to build it. That's one, two, three. I mean, is that really that slow? Okay, it started and we can see now it's installing the Linux packages. So let me show you the configuration of what we do here, right? You know, so it started the first task. It's really to say it runs on Ubuntu 20.04 and then it installs all the packages so it does upgrade and it does, you know, things like Wget, Kerlget and, you know, things like that as a common developer package that you would need. And it installs and runs MySQL, you know why? Because you see the diagram I did show you before. The event service has MySQL database that's connected to it, so it writes into that MySQL database, right? And it installs the Rust target for Wasm. It's what Haida has said, you know, you need to install the Rust target for Wasm32Wazzy, you know, that's so that Rust can compile into WebAssembly because by default, Rust compiles into the whatever's a CPU that happens on your machine, right? And it installs Wasm Edge, so we have a script that allows you to install a very specific version of Wasm Edge, but there's an interesting aspect is that, there's TF extension, you know, so many TensorFlow extension. In Wasm Edge, we have a plugin system where, you know, we keep the core runtime very lean, but you can have those plugins that you can install into the runtime, so for instance, the whole TensorFlow stuff and the PyTorch stuff, you know, they are all, you know, so if you want to do a model inference on PyTorch, you know, you need the PyTorch extension because it needs the libraries to access the GPU and you know, things like that. So, you know, there's, you know, we have a lot of extensions in Wasm Edge, that's one of our strengths, you know, is that, you know, people can come in and develop, you know, people would say, okay, I can't do this in Wasm, I would say, you know, just go develop Wasm Edge extension, you know, and you will be able to do that in Wasm. You know, that's, you know, the same conversation happened a lot during this KubeCon, you know, people come up to say, I want to do X, but it doesn't, but you know, but I can't do that in Wasm, you know, that's, I would say, you know, just, sure you can, you know, just, you know, do an extension. So we have, actually, we have a Google Summer of Docs program that we have, you know, several documentation writers to help us, you know, polish the technical documentation for plug-in development. But anyway, that's a little long. So, you know, so then install an init dapper. So let me see where it's now, so now it's already starting the dapper service now, you know, so let me, so, we have gone through all the steps, right? We have gone through the install, the Rust compiler target, Wasm Edge, and the install and init dapper is really just one command, it gets the dapper binary and then starts it, right, you know, so, and then you start dapper services like a central service because, you know, those dapper sidecars have to discover each other, right, you know, so let's close all this because, you know, they do have a lot of locks. Now I'm at a step where I would compile and run my application, right, you know, so here, right here, run grayscale. So this four commands, really just three commands because the first is just enter that directory. The cargo build is to build a Wasm binary out of it. Then the Wasm-HC is optional step, which I just talked about, is to AOT compile the Wasm binary into a machine-native binary on that machine. You know, that's, in some, in many cases, it's dramatically improved performance, you know, that's, so, but in some cases, say if you just have a microservice that needs to make a HTTP call, you don't need this, you know, because the vast amount of time you are spending is waiting for the other side to respond, right, you know, so, but if for compute intensive applications, like, you know, image processing and, you know, things like that, it's really nice to do this. And then it's just a dapper. You know, dapper run, it's an interesting command, you know, it runs dapper at a port and it's then runs application that attached to dapper in another port. So they form a pair, you know, that's the sidecar in that application, right? You know, so now the sidecar knows how to communicate with each other, but in this command, you know, when you do dapper run, it's actually starts dapper with the, you know, with it's attached application. And the process is slow because the cargo build, because, you know, as you know, one of the criticism of Rust is that the compiling takes a very long time, but for good reason, because Rust compiler really does do a lot, you know, it's find all the potential problems for you, you know, that's, so I think it's time well spent, but we'll, let's see where they are now. And okay, so it's still running greyscale, so you see it's still compiling and it does, you know, the wasm-hc, it's the local compiling. And then do the exact same thing with the API classifier, you know, it's another image processing, and you know, that's the second image processing of microservice that I've shown in the graph, right? It's take an image and then recognize what's on it and return the English name for that image, for that object, right? And then you run the event service. The event service is also, let me go into the source code of the event service and then the classify service. So the event service, the source code is also very interesting, very simple, it's just the main.js. What it does is, in the main function, it starts a dapper client, so it's no longer starts, say HTTP server or anything like that, you don't need that anymore, because dapper handles that for you, right? And then it's, oh, it does start, sorry, I misspoke, and dapper use HTTP to communicate with it, so it does need HTTP server just like before, but it's, and it also connects to the database, so both pieces are still there in the event service. However, what dapper does is that it's, you can see the, when it starts the MySQL connection pool, it no longer has to hard code the password in it. You know, when we went into the demo, the user name and password was hard coded into the Rust code, I don't know if you notice that. Well, had I did the demo, it passed as an environmental variable in the command line, but in dapper, because dapper has a secret store, it's like a vault, right? So you can put the data in the vault so that it become a dapper service. You can go ask dapper, what's my user name and password, right? You know, so you would never ever forget it, and it's also making it much nicer in terms of automation, you know? So, you know, this is the first dapper service that we use, it's a vault service, right? You know, so you go and ask the local store, ask the encrypted local store, what's my user name and password, and it will tell you. You, essentially you connect to the sidecar and ask for that information, and using that information, you connect to the database, right? And then, you know, there are things like, you know, say if I want to create an event, I do the SQL stuff which we have shown, which we have seen, but we also do the key value stuff. Let me see, where's my key value stuff? Insert, where's my key value store? This is the SQL, right? But, maybe not. Okay, so, you know, and I remember, you see another service. Let's go to the classified service. Let's see how dapper is used in here. Oh, by the way, yeah, yeah, let's do that. So, go to the, when we go to the classified service, we open up the main.rs. Here, you still have, you have the same thing, you know, that's to connect to dapper sidecar, to open HTTP port, now you don't have a database, right, you know, because the database is now handled by events. So, when something comes in, how do you tell the events to save my data? You see, it becomes a dapper client, invoke service, event service, you know, so now you only need to give it's name. You don't need to know where, which machine the event service is running, which port it's running, you know, that's things that you are likely not to remember. And how do you make authenticated HTTPS connection to it, right? You know, so it just needs to know that the event service and the function it calls on that service is called create event. And then you send the event data that you want it to save there, right? And then you can also save it locally. And, you know, you keep, here, we have some events saved on the event server, but also have some, you know, data locally because here we want to prevent people from abusing us. We don't want people to sending, you know, visiting images and consume all our computational resources. So we log the IP address and their time. So, you know, each IP address would only be able to send in like one image per minute, you know, whatever, right, you know, so you can have a lot of things you can do here. And here we use the key value store. You use, say, you know, unclined, which is a connection to the dapper sidecar to save state, right, you know, so you can save any state you want at key value into your local store. You know, all those are shared in the entire environment. But I know, you know, that's maybe something that you are also very curious is how do we do image recognition? It's really just this, those lines of code, you know, super simple, you know, it's, we extract the body from the HTTP post. You know, the body is an image. And then we come, we use a special function it's called wasm edge TensorFlow load JPEG to RGB8, you know, so meaning it's load the JPEG image and turn it into a tensor format so that the TensorFlow model can consume, right, you know, so different model had different requirements. So here is what do we call image preprocessing and a lot of those, you know, in traditional machine learning tasks, those are done in Python, right, you know, so, and then we start a session and then we do session, we load the image data, we load the model data and we add the input and then we just say, get output. That's it, you know, that's, it goes to TensorFlow and does the inference. If you have GPU, it does it on the GPU. The output is array, you know, it's a list of numbers. The, those numbers are probabilities. They correspond to the predetermined set of objects that model has been trained on. So the model may be trained to recognize a thousand objects. So it would give you an array of a thousand numbers and each number correspond to the probability of one of the images, right? You know, then you pass through those, the list of objects and match whose, who has the biggest highest probability and then return the result. You know, that's it, you know, that's, you can see, you know, people say Python is easy, but you know, this, this whole thing, I can do image recognition in half a screen. You know, that's, so I have to argue, you know, and, and the much, I don't need the Python runtime, I don't need all the dependency libraries that Python may bring in. I don't even need Linux. You know, that's, you know, it's, I think it's a lot lighter approach to do, you know, to do inference on production, you know, that's, you know, so, so that highlights one of the, one of the benefits. And I also show you, so that you can remember the image that I sent to the, you know, in our demo, the image I sent to the service to recognize. Okay, so remember, so this is the image we sent to it. Let's see here whether it's finished yet. No, it's still doing the event service. Okay, so let's see, I think it's gonna finish in about two minutes. Okay, so, but, so let's go and see what are the tests that we run. So we build those applications and then we compile them and we start them and now we run tests. The tests are those curl statements, you know, so we run a bunch of those. Okay, so essentially the first is we do innet. What does innet do? It's because the event service has a database attached to it, right? You remember, so, you know, so we have to initialize the database. We either do that in code or we ask the user to do it. Here, we do it ourselves. You know, the user initializes it. So we innet the database and we get the events. It should come back with an empty array and then we test the grayscale service. Then we test the image recognition service. Okay, so it's food.jpg, it's not demo.jpg. So let me go back to look at the food.jpg image. And then we look at the events again because we call it both services. They should have saved something into the events database and we would be able to see that in my SQL database, right? You know, so that's a, you know, fairly simple thing to do. So it's in docs and food.jpg. Oh, okay, same thing, you know, so that's the image we feed into the models to see if it can recognize that, right? So I think, okay, it's almost down. It's running the AOT compiler. You know, the AOT compiler, like I said, it compiles the WebAssembly byte code to native code for the machine. It's currently running on, you know, that's, I wish Java has that, you know, because that's really improved performance by orders magnitude. But I think the Grail VM now has it. So maybe Java is getting there as well. But you know, this is one of the things that's nice things when you have, you know, compiled languages like a Go and Rust that you can do things like that, right? You know, because it's easier to translate directly into machine code. You know, that's with WebAssembly, of course, we have the sandbox that wraps around it. So that you can't, even with machine code, you can't escape the sandbox. So, well, you know, that's, it's still doing the compilation. You know, in fact, in this particular demo, we can, we don't have to do that because the bulk of the performance issues, you know, that's what happens in TensorFlow, you know, not inside this wasm application. So, you know, so it takes the image and it's passed off to TensorFlow and runs on GPU and then come back, right? You know, that's a, although in this case, because GitHub Actions only have CPU, so it runs on CPU, but, you know, the bulk of the TensorFlow operation is not down here. So I don't have, I don't necessarily have to do the AOT compilation. You know, the AOT compilation may improve the performance by two X here, you know, but it's not like 10 X or 100 X here, but, but since I'm already, I have this image script, so we'll have to wait until it finish. So, let's wait for another minute. Let's see if there anything else I want to show you here. So, yeah, so, you know, you can see this is how you load the model, and this is the label file I talked about, you know, so it's interesting about TensorFlow is that, because, you know, really what goes into it, the PyTorch is the same thing, what go into it and what come out of it, all the numbers, you know, so how to map the numbers into English words or into something that you can interpret is what the application does, right? You know, so, you know, so you would, typically it would be a Python application, but here we have a Rust application, you know, so it prepares the data and once you, oh, here it's done, okay, so, you know, so let's see the run tests. The run tests, let me just show you the result because the result all jammed together, you know, it's here. So the first, what? Okay, cancel, okay. So the first is status true, remember what's the first thing we did is the init, right? You know, is that I create the database table. The second is the empty array, okay, so I do the events, so it come back with the events of the database table I just created, which is empty. The third is hotdog is detected with the confidence of this, right, you know, do you remember the picture you see? It looks like hotdog, right? So it's, and it's detected with a very high confidence, you know, it's, you know, in those, you know, a thousand item array, this one is almost, you know, 1.0 and everyone else has very minute probabilities. And then, we go back to the events database and to query the, you go into service to query the database again. So now it has two events here. Event number one is the grayscale event. I didn't show the output of grayscale because, you know, I can't show that in a console, you know, that's, but you can see that in the grayscale event and it knows what the input size is and what the timestamp is and the event number two is classifier event. So this whole dapper application, I think shows, you know, it's a fairly complicated setup, but in a lot of microservices, I think it's a fairly typical level of complexity that people are dealing with. You know, that's why we need things like sidecars, right? You know, so it's a couple of services that will depend on each other. They have, you know, and they need to discover each other and they need to share a common set of, you know, infrastructure services like in this case, the key value store and the, you know, the secret store for the database access, right? So, you know, and you can do a lot more complex with dapper, you know, that's, you can do messaging queues and you know, things like that and I'll leave you to discover all this, but you know, the GitHub repository is here. You know, it's a, if you don't want to mess up your own machine by setting up all this, I think you can just fork it and then run in GitHub actions yourself, you know, and see all the output and you can, you know, you can change things and see if the GitHub actions change. For instance, you can change the image, right? You know, that's a, this image, this docs, docs, slash, docs, slash food, dot jpeg, you can put another image here and see if it's gonna recognize it. I encourage you to try it. You know, that's, it's quite amazing, you know, how accurate it can to, you know, to recognize stuff. So, I think that's, that's this demo and I think we have one last one. We have 15 minutes left. So, you know, we may not really have time to do this demo, but I really encourage you to, because I really want to leave some time for QA, I really encourage you to try it. What it does is that it's a bot that does PR review for GitHub pull requests. Why do you do that? It's because we are all open source communities. We are as maintainers and all as senior developers. We spend a lot of valuable times on reviewing PRs and pull requests. And sometimes we are, we are slow and the community is not happy, you know, there's all kinds of things, you know. So it's a very expensive and very slow and very, you know, an irritating process for a lot of people, right? You know, so to speed up PR review, I think it's important for R&D management, especially for open source communities. And so in the Watermage repository, we have a, we have a GitHub bot that build on Watermage itself. And it's when you submit a PR or you say a magic word in the PR comments, it gets all the files from the PR and send it to the chat GPT4 and ask to review it and ask to come back with recommendations. It's gonna come back with a really nice written set of things that it thinks that may be problem with the code. So I just to give you an example, because I would like you to try this yourself. So I would give you an example to entice you to do that, right, you know. We have a PR that was submitted by a JavaScript developer and he, in one of the files he changed, he added a function called check prime, right? You know, check the prime number. And his implementation is to set up a loop to start from two to square root of N, right? The input is N. And to test each number whether this can be divided by, you know, divided by N. And if any of those numbers can be divided, it would just return to say this is not, N is not a prime number. And if none of them can, it's gonna return to say N is a prime number. That's pretty reasonable, right? You know, that's, in fact, that's how I would write it. You know, that's a GPT4 saw that and the GPT4 said, you know, the function gives the correct results. It's true, but it can be improved. How to improve it? It says, because you don't need to try even numbers over again, right? You know, so from two to square root of N, you don't need to go one in step. You can go two in steps, right? You know, or two and three and then do all the odd numbers instead of trying even numbers all over again, right? So I think this is, obviously, you can go along that line. You can even ask more questions to it to say, can you write the code for me? Or can you further improve it? Because now I understand multiple subprimes that I have discovered or none of them need to be tried, right? So is this one of the examples? What are we doing? Okay. Yeah, so we have that integrated into our repository and it's, to me, it's eye-opening because it finds a lot of issues. It finds a lot of issues that humans tends to overlook in terms of programming issues and programming bugs. So do you want to do something here? Oh, okay, yeah, yeah, that's, you know. So for all the things that I said, it gave me one sentence, the check prime function can be optimized further as it checks the divisibility of even numbers up to two, which isn't necessary. It blows my mind. It finds a lot of issues like that. Let me tell you. And why this is a good use case for Wasm as a microservice is because in this function, Wasm does very little. The microservice does very little. Most of the time it's been waiting for GPT-4 to respond. So to spin up a Linux container or a micro VM just to do HTTP connection and wait for, you know, because GPT-4 could take three minutes to come up with a complete answer. You know, that's, it's super slow. To wait for that long to get an answer, I think it's very, it's extremely wasteful. You know, you are wasting like a half a CPU core just for that, right? You know, that's, but with Wasm, you can scale to zero. You can have something that is wait there that consume very little resources. And once it's done, it gets completely shut up. So we thought that's a really interesting use case that's, you know, that we use ourselves in our community. And if you'd like to set it up, we'd love to help you set it up. And because this is all written in code, this is written in code that compiles WebAssembly. So you can have all kinds of ways you can customize it. You can change it, you can change the prompt. You can tell it, you know, say certain things, but don't say certain things. Look for formatting errors or the code I'm gonna give you or JavaScript. You know, just do not consider other languages. You can do a lot of things. And yeah, that's, do you have anything to add with this? Yeah. No. Here are some resources to follow up with us. We have a disclosure. And if you are in since have a Slack workspace, you can join our Wasm Media Slack channel. Is there anyone has any questions? Yeah, thank you. Yeah. So enjoy your lunch. Enjoy your lunch. And if you have questions, we're gonna be here. We're gonna skip lunch. You know, so if you have anything you wanna talk about, it's real fast, gonna be here. Thank you.