 I'll try to make it interesting and finish on time so that you can go out to the reception. So I'm the maintainer of the project called Wasm Edge and in this talk, we're going to talk about a new way to do microservices using two CNCF projects called Wasm Edge, my project and Dapper. So we're going to talk about those projects in a minute. If you want my contact information, it's in the barcode above. So if it goes to my GitHub profile where you can see my Twitter handle and email address and things like that. All right. So some of you might have heard about WebAssembly. So it's what we call Wasm, it's WebAssembly started as a browser technology. So why is it important and how it's connected to microservices? I would refer to a CNCF survey that was done about this time last year. So it asked a lot of IT professionals around the world, what's the technology that's coming next? So the key findings, by the way, so this survey has only one key finding, there's no other findings. The key finding summarized into one sentence, it's containers as a new norm and the WebAssembly is the future. So as we all know, CNCF, the organization was built on the idea of containerization which gave birth to the entire Cloud Native movement. So fast forward about 10 years now, there's wide use of containers and Kubernetes and related technologies. So what's next? So I think there's a consensus among the practitioners of the Cloud Native space that WebAssembly is the future and we're going to talk about why. So that's the Wasm Edge project. Wasm Edge project is only WebAssembly around time in CNCF ecosystem. So the other project we want to discuss today is called Dapper. Dapper is incubated by Microsoft and then get donated to CNCF. It's a new way to do microservices as you can see from the community metrics. It's anonymously popular. It's one of the largest project in CNCF. So there's lots of contributors, just who's who in the Cloud Native space and lots of get-up stars and all that. So this is one, I wouldn't say a new paradigm, but codified a lot of best practices in terms of how to do large-scale microservices. So those are the two projects we want to cover. So the agenda of the talk is fairly straightforward. It's first, we want to talk about why you want to run microservices in Wasm. So lightweight microservices in Wasm. And the second, how does Dapper improve that experience and why it's necessary to use Dapper together with Wasm. And the third, we want to introduce a Dapper SDK for Wasm Edge and the fourth, we want to talk about the serverless deployment and I will also give an example. Serverless deployment of both Wasm-based microservices and Dapper-based sidecars. So Wasm as a microservice runtime. So this is a plot that I often show. It may be a little small, but the y-axis reads from slow, it's about performance. So it's lower, it's slow, higher, it's fast. The x-axis is footprint. So on the left is heavy, on the right is light. So we can see virtualization technology has gone through multiple iterations over time. So when we started, when cloud computing first started, what is cloud computing? Someone said, I like the code very much. To say cloud is just somebody else's computer. You are running your code on somebody else's computer. That is called cloud computing, right? So the key characteristics of cloud computing is isolation. You will be able to run your code free from interference from other people on the same hardware, right? So that is enabled by a virtual machine. So when Amazon have EC2, when those cloud computing providers first come along, they were all based on virtual machines, VM-based. However, people soon find that the virtual machine is a sort of heavy, it emulates the network, all the hardware, the desk and all that stuff, right? And it's also fairly slow. It's a large performance overhead. So around the time since they have come along, when Docker was really popular, container becomes the standard for cloud computing workloads. So that gives birth, like I said, that gives birth to the cloud native era. So if people ask me, what is cloud native? Cloud native is container cloud. Is that all the workloads are in containers and then they were somehow distributed on the cloud and then run, right? So the container era is, I think, lasted at least 10 years until now, right? So now cloud computing, cloud native computing, becomes so prevalent that people want to use this paradigm, this type of isolation, anywhere. So for instance, on the edge, so for instance, in the car, we have heard a lot of talk about AGL today, automated grade Linux. One of the key requirements is isolation. You want to isolate the workloads that's in the car. Can you use container for that? You may be able to do that, but it is a very heavy proposition. So for instance, if you want to do machine learning, something like that, the PyTorch container in the car is four gigabytes each, right? That consumes a truckload of disk space and memory space. So it makes it very, it's not very economical to build special purpose containers for those applications. And also, one of the other issues a container has is, while it's portable across operating systems, it's not portable across hardware. So for instance, a container image that built for ARM processor is different than the container image that built for the x86 processor. And let alone all the hardware accelerators that today we have on the edge, like the GPUs, like TPUs, like NPUs, there's all kinds of hardware accelerators that we have available on the edge that's not properly abstracted by the container. So I think container has brought us to today. That's where there's a prevalence of cloud native computing paradigm. However, it's difficult by using container alone to push this to even more computing devices. So that's where WebAssembly comes in. So WebAssembly is something that is, I think, orders a magnitude smaller and potentially faster than containers. And it's also provides true hardware probability, meaning that the WebAssembly application can run on any hardware device that supports the WebAssembly runtime. So we will see some examples in a minute. So that's what makes WebAssembly a good candidate for running microservices. Because by definition, microservices are small in a, I've seen companies that has tens of thousands of microservices, not tens of thousands of servers, tens of thousands of services. And those are large-scale internet companies. And each of them perform a very specific task. And you want the runtime for those microservices to be small and then to have those microservices to be able to move across different machines, different CPUs, even different clouds, sometimes on the edge, sometimes in the data center. So Wasm is a good choice for that. That's why people think Wasm is the future. Container is there. So just to give you a feel about, so those are the two tweets that we sent out. So we built your typical database application and compiled it to Wasm. The application was written in Rust. So one of the applications is a Rust client for Redis. So you have a web server. And this application provides a web server. And the application provides business logic to interact with the Redis server on the back end. So it's a very typical microservice setup. So you can connect to that service through HTTP and make requests to the Redis database. So the whole thing, including the runtime, so as analogous to the container image, the whole thing is 700 kilobytes. So going to the Wasm world seems again being measured in kilobytes. Because if you do that with a container and managed by Docker Kubernetes, the container itself would be 100 megabytes. So it's fairly standard, say, a 1 to Linux container size would be 300 megabytes. And there are some slim Linux distributions that are smaller. But with WebAssembly, we can get all the way down to less than 1 megabytes. And on the right side, it's the same thing. We have a Postgres client application. So it's a very normal web service. And when you make HTTP requests against this Wasm application, it connects to a Postgres database on the back end. So it can do the database read, write, delete, and just the regular operations. And the whole application is 800 kilobytes. So we are going back to the world of some megabytes. And also, the applications like that are completely portable across all the hardware. So it doesn't matter where you want to run it on ARM, on RISC-5, or if we're going to see in a minute if you have a machine learning or AI work or you want to run it on GPU, CPU, anything you want, that need to be the same binary. And it can be orchestrated and it can be developed that's with ease. So that's the thing about Wasm. So this is the Wasm Edge project, our GitHub repository. So an interesting thing about Wasm, why Wasm Edge is a CNCF project is that by positioning Wasm as the next generation of application runtime, we need to integrate into existing tooling systems. So Wasm, through our partnership with Stalker, our project has partnership with Stalker, partnership with Container D. And now, using those apartment red hat open shift, using those traditional container management tools, you can not only start Linux containers, but you can also start Wasm Edge runtime. So your workloads can run side by side with your container image. So your Wasm-based workload can run side by side with your Linux container. So Wasm Edge has several, I think, quite unique features that distinguish it from, because WebAssembly is an industry standard. So Wasm Edge has some features that distinguish itself from other WebAssembly around them, so to speak. So it provides rich support for application-level SDKs, so meaning that you can build high-performance web servers and high-performance web clients through asynchronous IO that's being built into Wasm Edge. And you can have integration with infrastructure services. We have just the same, you have Wasm Edge application that connect to a Redis database, and also Wasm Edge applications that connect to a Postgres Redis key value store and Postgres database. So there's a variety of different databases and storage engines that we can support by having their SDK ported into Wasm and have them accessible through a Wasm Edge application. And perhaps more interestingly, I would have a talk about this tomorrow afternoon, is that increasingly, we've seen use cases where to use Wasm Edge to run AI workloads. Because without, what's the traditional way of running AI? It's Python and PyTorch. And PyTorch, and you bundle it into a Docker image, right? So it has all the problems that I mentioned, because PyTorch Docker image itself is four gigabytes. Then you have your application on it. And on top of that, the image has to be tuned towards the underlying hardware. So if the underlying hardware is a media GPU, it's very different than, say, it's an MPU or something else. So you have a huge amount of work and a huge amount of asset that you have to manage by running a Python-based inference. And through Wasm, by integrating with popular machine learning frameworks and pre-processing and post-processing frameworks that are associated with some of the most popular machine learning models, that we are able to run things like the Lama 2 large language model, the ULO5, the object detection library, and the media pipe libraries. So there's a variety of different AI and machine learning libraries that you can run in Wasm that has very low footprint. So the whole thing about running the whole setup of running Lama 2 models, for instance, in our case, the runtime is about 20 megabytes, and the application itself is about two megabytes. That compared with gigabytes upon gigabytes of space that's required for PyTorch. So those are some of the typical use cases for Wasm applications. That's why we want to run on microservices, because those services are small. We want it to run anywhere. We want it on the edge, in the car, on the router, even in the door camera. So we want all those use cases, and we need a small runtime that can accommodate that. And Wasm Edge is one of those things. Then enough about Wasm. So how about Dapper? How does that improve the Wasm experience? So what's still missing? I talked about how you can build microservices, you can build database connections from Wasm. But what's missing from it? What's missing from it is although we are connecting to all the databases that, in my previous slide, we have all those logos. But that's not nearly enough, because in the cloud-native ecosystem, if you just look at how many projects the CNCF has, how many companies there. I think the CNCF, the silver member and up, there's over 1,000 companies now, and each of them have a product. So for a microservices framework to be truly useful in that environment, you really need to integrate with a lot of those stuff. And also, the common microservices design patterns are not yet codified in Wasm, meaning there's very established ways how to develop in the wrong microservices in languages I could go. But in Wasm, those common patterns are not yet implemented. So enter Dapper. So I want to spend a minute to tell you why Dapper is so popular. Dapper codifies a very powerful microservices architecture patterns called the sidecar. So the sidecar pattern means in the Kubernetes pod, you have application runtimes that runs in containers, right? And then you have a separate container that acts as a service provider for all those application containers. And all the connections through the infrastructure services goes through that separate container. So that separate container is called a sidecar. So the application container deployed over here, the sidecar deployed over here, infrastructures over there. So Dapper provides a unified API for the sidecar. So that allows us to decouple the business logic in the microservice from the infrastructure logic that needs to support the business logic, right? So it provides a clean API that provides access to all those components. So for instance, the distributed lock, the key value store, workflows and all that stuff. So all those are services where your application written in different languages can use a GRPC API or HTTP API, connecting them through the Dapper container. And the Dapper container itself has all those services but you can also, by connecting through its service API, to have other cloud services to connect to Dapper. So for instance, the key value API in Dapper was backed by over a hundred different cloud-based key value stores. So you have key value stores in AWS, you have key value stores in Google Cloud, and you have all that, you have key value store in Dapper itself. So no matter which one you want to use, you can have a service API that connects to Dapper so that from the microservice point of view, all those are the same. You just need to know the Dapper API and you just configure Dapper to go to, you just tell Dapper that I need to save this key value pair. And the Dapper would read this configuration file and figure out whether it's gonna go to AWS to save it or local file to save it, or a secret file or vault to save it. You know, something like that, right? So it's basically codifies and implements a sidecar pattern and it's a very powerful way of isolating the business logic in the microservice from the infrastructure logic that's the cloud provides, right? So the Dapper value proposition for Wasm is really because Wasm is light and fast and portable, right? But it lacks functionality because it doesn't have 10 years to develop all those cloud native ecosystem yet. So what Dapper does is that to say, don't build drivers for all those individual database and key value store and cloud services individually. Just build a connector to Dapper and through Dapper you're gonna access to all those other cloud services. So that's the collaboration between, I started with our two projects about I think a year and a half ago, right? You know, that's the, so we talked about that. We also, you know, because for the Dapper project, they want a new lightweight approach to raw microservices. And for us, we want our microservices, Wasm-based microservices to have access to all the infrastructures that the cloud has to offer, without having to build hundreds of database drivers and connectors ourselves into Wasm, right? So it is really because of this separation between the business logic and infrastructure logic that it makes it possible, right? So again, there's lots of Dapper components. I think I talked about that, so it's divided into categories and each category, each of those components has a lot of backing services that you can choose and pick, right? So a real-world scenario is say, so it's high throughput messaging processing. So that's a typical use case is to say, if you have a financial services firm, where you have say, market signal from exchange and also news and other signals from another source and you combine them together into a data stream and then you have to process the data stream in high speed and then have the result and then make trading decisions to go back to execute the traders. For instance, some bad news happened with the company and now you detect that. So you need to go there immediately and short that stock, right? So there's in high-speed trading scenario where you typically have a setup like this where you need a messaging queue that to process those messages at real time and once you process those messages, you need to get them back. But instead of having say a Kafka or a messaging queue or fully-featured messaging queue inside your water application, you can connect to Dapper and have Dapper connect to a PubSub service that would be able to process a message asynchronously, right? This is one of use cases for one of our customers. So then, so the next step becomes very obvious is that now we want to, because we've seen the benefits of connecting Wathom to Dapper. So what is a piece of software that actually does that? So the software is called Dapper SDK for Wathom Edge or Dapper SDK for Wathom, right? So it basically provides, it is a Rust Quake or a library where you can import into your microservice project and when you compile your microservice into Wathom, it would, this SDK would be compiled into it. It would give you a high-level API where you can manipulate the objects by instantiating a connector object to Dapper and then call the functions on top of it, right? So this is a Dapper-side sandbox project, meaning that Dapper is a large open-source project by itself. So they have their own incubation program that has a sandbox, so we contributed this particular driver to their sandbox project. Yeah, so that's basically the same plot that I've shown that all the components in Dapper would be supported by this SDK. So I wouldn't actually run this demo application because this demo application, it takes like 10 minutes to start up and actually end fully run. So, but this would be a very typical microservice application. So you can see what this one does is that it's an image processing application to utilize the AI capabilities in Wathom Edge. So it takes the AI model and create a web interface so where you can upload a picture. And then by using the AI model to process the picture, it's gonna tell you what the picture is, right? But at the same time, it's gonna generate, say, observability data or logging data, events data, that would be saved into database, right? So what we do here is a very, I think a very common microservice pattern is to decouple those features. So the image classifier or the image grayscale, those image services are in their own microservices. But the event logging is also in its own microservices and the events logging service connects to the MySQL database that saves the logs, right? And the image classifier and image grayscale services connect to the AI models, right? So what it does is that you can see all those because imagine if you don't have dapper in the middle, if you don't have dapper in the middle, you would have n times n complexity, n squared complexity because each service needs to call the other service. So you need to know where's the IP address or the port number that the service is running and then know what's the interface you are expected to deal with, right? So for three services, it's fine. It's only nine combinations. But imagine, like I just mentioned, there's a large internet company that we work with has 50,000 microservices in their system, okay? 50,000 microservice. And each time you load a new page on their website, it's hundreds of microservices get triggered. Some of them doing predictive analytics, sometimes some of them doing advertisement serving and things like that. That, if you, the n square of this is a tremendous number of potential combinations and where to find those services, right? So what dapper does is that now instead of each service looking for its consumers or, you know, so for instance, image classifier needs to look for events and events needs to look for image classifier. We need none of those because image classifier is connected to a diaper sidecar and each of the microservice connects to its own diaper sidecar. And the diaper sidecar automatically forms a network on its own. And so the image classifier just need to tell its own diaper sidecar that I need to find the event service and send some data to it. It's diaper sidecar would be able to find the event service diaper sidecar and communicate with that, right? So that's the way of decoupling and isolation is that all the communication and all the finding, binding and so the dirty work, so to speak, was handled by the sidecar. And then the logic in the microservices becomes fairly simple. Microservice just say I need to find the event service. And then the diaper sidecar goes to actually find it. Where does it find it? It does it on its own, right? So if you're interested, there's a GitHub repository. There are instructions on how to set it up because as you can see, you need three diaper containers and those green ones are watermage containers. So you need a cluster of six containers and also the MySQL database, that's 7th container. So you need to set up all those and have it around. That's why it takes 10 minutes to compile and start up, right? So we have a GitHub action that you can go there and see that's all the steps that's involved in deploying something like that, right? So if you do get it set up, you can provide a web interface like this where you can upload a picture and it would use an ImageNet model to detect what it is and then tell you what you can hear, obviously, give it a rust crap. That's but you can have, say, a hamburger or a hotdog, whatever. So it is pretty interesting to see the machine learning work that's in progress. So that's the complex story. But there's a silver lining. There's an easier way to do all this. What we call a serverless wasm and a serverless dapper because I think going forward, microservices become increasingly serverless, meaning that you don't really need to think about where it's deployed, how it's get done. You just send it to some machine and some service and ask the service to run for you, right? So fortunately for us, wasm and dapper now both have serverless platform available for that. So to run wasm as a serverless function, it's something called the flowstone network where you can just upload a wasm function and it will run it for you. And for dapper, it is a dapper catalyst. Dapper is a company behind dapper. They are essentially they're spinning off from Microsoft. So basically they provide dapper as a service, meaning that they provide all the key value store API, the PubSub API, the things that we talked about as internet APIs. So you can set up a key value store in their system and then use the access through the API. And by configuring what's the back end key value store should be corresponding to API, you would be able to save it to AWS or something else, right? It allows you to do the multicloud and edge cloud type of thing. So both of those services are serverless. And so I want to give you a very quick demo in terms of a whole application that we built in a couple of minutes. So the GitHub repository is this catalyst, the Telegram OpenAI. What it does is it builds a Telegram bot that answers, that takes user input. So the service is a Telegram bot. So when you type something in your Telegram bot, it sends to the service. The service use OpenAI to generate the answers and then use dapper to do state management. So let's look at how it works. It is a little small. So the first question I ask is, what is the container? So if you ask what is the container to a regular person, the person is going to tell you it's a glass box, something like that. But it's not. It actually gave a very elaborate answer. What is the container in the context of a cloud native computing? It tells you it's a software container. It's a software runtime. Why? Because we have trained the model to say that. So I ask another question, can I get it in the store? And the answer is no. You can get it in the container repository. So the model is very fine tuned to answer cloud native questions. And then the last question I asked is how it's related to Kubernetes. So it's also given a very comprehensive answer. So in my opinion, it is obviously a fairly useful bot. That's because, as you can imagine, it can answer a lot of those questions. So how did we build it? I'm almost done. There's four screens here. So the first really is that all the code is written in Rust and compiled to WebAssembly. And if you scan the PR code, you can get to the GitHub repository. So the first really is to, the undeploy is a function that defined by the serverless function platform to say when the function is first deployed on the platform, call this function. What it does is it's listened to the update from the Telegram API. When it receives a message from a bot that is being identified by the Telegram token, it will call the handler method. What the handler method does is that retrieves the message that is associated with this update. And then it goes to the next. Then what it does is that needs to, because this serverless function connects to a Telegram bot, there may be a lot of people using the Telegram bot at the same time. So it can get all kinds of different questions from different users. There has to be a state management element on the serverless function side, on the application side, where you can associate this user to this thread. So this is actually done through the DAPR API, through the Catalyst kind. So there is a key value store where I get the chat ID from the Telegram, which basically identifies the user. That's sending this message. And I keep that, map that to an OpenAI thread ID so that on the OpenAI side, I know which conversation thread I'm running with this user. So this is a very, very simple example. So if I don't have that set up already, I create a thread ID, and then save it. If I match it to a thread ID, I would just take the thread ID and go to OpenAI, and then ask the OpenAI to come up with the answer. So this is the OpenAI SDK that we write in Wasm Edge. So the message request is sent to the thread ID. And the other interesting thing is that you can see there is a system ID. It's used the latest features of OpenAI, where you can create a special knowledge assistant, meaning that you can customize chat GBT for specific domain to give it a special prompt and to give it special context by uploading some data. So what we did is that we uploaded some Kubernetes related content to this particular system ID so that whenever it sees a question, it tries to answer it as a cloud-native question. So the example you can see is that what is the container? It doesn't tell you what's the common sense definition of a container. It's to tell you what's the cloud-native definition of the container is. So that's to ask OpenAI assistant. It's just a bunch of API calls. And then once you get the answer, it goes to the key writers store and then find out where the chat ID is, and then send it back to the Telegram API so that it completes the entire application. And if you want to try the bot, you have a Telegram. You can just scan the bot code. And it's in a GitHub repository. You can chat with it right now. So well, that's the last slide. So here, we've seen running microservice as using Wasm to run microservice and how Dapper helps connecting those lightweight Wasm microservice, giving more features to those microservices to all the cloud-native infrastructure components. And also, we've also seen the Wasm microservice can also connect to a large language model like OpenAI. However, there's another dimension to it. Is that Wasm Edge is actually one of the leaders in fast and portable runtime for open source LIMs, meaning that you can use Wasm Edge instead of Python to, as a runtime container, to run a large language model like Lama2 or stable diffusion or a variety of different TensorFlow models like we have just seen, image recognition models and things like that. So those model applications could be written and compiled into a Wasm-based microservices. So here are two links that we showed. There's a large variety of different language models. So once you run those in Wasm Edge as a microservice, you can provide it as a service. You can provide the LIM function as a service to other microservices in your ecosystem. And how do you do that? You can use Dapper for it, because each AI model and each large language model could have a different API, could have different prompt templates. There are lots of things that are different. We can have a unified API like the Dapper key value store to hide away all those complexities of what the underlying model is actually doing. So you can have a new Dapper API that is called chat or large language model and have any model open source or proprietary be able to plug into that and plug that as a Dapper service as well. So yeah, but that's obviously has not been done. That's something that's well-do that's perhaps in the near future. So yeah, that's it for my talk. And I know we're going to have the reception in a minute, but I really do hope that you can go out and try Wasm Edge, try Dapper, and create your own microservice to create your own chat bot, and have some fun. Thank you.