 Hello, everyone. How do you do? I'll start now. So today, my topic is optimizing microservices with WebAssembly and Linux containers. So this is my GitHub handle and Twitter handle. And below GitHub repo is our open source project called Wasm Edge. So I will talk about the following several topics. And first, a little bit about me and Wasm Edge. And second, microservice architecture and problems. And WebAssembly containers has an alternative to Linux containers. And the container tools that support Wasm containers and cases when Wasm containers are preferred. So a little bit about me. I'm a DevRel informed member of Wasm Edge. And also, I've been an ambassador for Cloud Native Computing Foundation. And I organize different meetups and conferences. So below here is a meetup group I organized. It's a Wasm and Rust meetup in Beijing. And I give talks at the CUBECon and the other conferences. And also, I write some technical blogs and articles. Also, I do some translation for technology articles. And also, I commit those documentation on GitHub. So it's a bit low, I guess. Yeah, I'll just do that. So Wasm Edge is a WebAssembly runtime. So when you think about WebAssembly, before, I would think that it's usually used in the browser. But we are using it on the server side. So it can be a very lightweight sandbox for Cloud Native Edge and decentralized Cloud applications. And it's an OCI-complained runtime. And it powers serverless apps and embedded functions and micro services and IoT devices. Also, recently, we started to run AI inference across devices. You can even run it on your Mac and IoT device. So these are the partners, I guess. I will just be really quick. So recently, we were accepted to NVIDIA inception as well. And I'm not sure why, but our integration into Open Function has been forwarded on Twitter by a lot of Japanese friends. And yeah, these are some programs that are paid, and you can join. So the first one is Linux Foundation mentorship and also Google Season of Docs and Google Summer of Code. You can contribute really easily by looking. So all these different programs have different tasks. Or you can just find the good first issue by searching our GitHub issues. It's really easy to start. So yeah, I will get into micro service architecture. So I guess I think for the Cloud Native Edge, we are really familiar with the micro service concept. It's independently deployable, and it helps the team to shape software quicker. And it can be really resilient. So for small teams, they can all work independently on different micro services, and it's really fast. So the scaling can be really easy. And also maintenance would be very easy. And so for large and complex applications with millions of users, like for Twitter app or for TikTok app, they usually have tens of thousands of micro services behind it. Actually, last year when Elon Musk just took over Twitter, he said, why a simple app that just allow you to post 140 characters a tweet would have tens of thousands of micro services behind it? And he said, I want to reduce it by 80%. But it's just how the more complex application works so that the teams can work on different services and iterate fast. Also for smaller teams, the organization can use micro service architecture to quickly adapt to changes and new technologies. So this is a typical micro service architecture. So like Twitter, it might have a mobile app and then a browser, the web app. And it's like that. Actually, this one is more like an e-commerce app. So they have inventory and shipping database and also account database. So I would use second hand e-commerce app, FireMouse, as an example, to talk about it. Because we have a client who have been using Wathom containers to get running side by side. So Wathom containers run side by side with their Linux containers for different programming. So why they choose the micro service architecture is also because they can choose the optimal programming language for different services. And so when traffic enters through a gateway, it's distributed to various services. And they can be written in different programming languages. And each micro service would exist independently. And it has its own storage and caching system. And the services communicate with each other through RPCs or messaging cues. So that is decoupled. And this is the architecture. They deploy this on cloud. It can be AWS or it can be other cloud. So the e-commerce app I talked about, the e-commerce platform, so they also have a web page, a website app. So it's grew from a small startup with 10,000 of active users, 200,000 active users. And they used to use virtual machines and then use cloud technology and the ECS. And then changed to EKS so that to be better integrated to the cloud ecosystem. And the reason is that in this way, they can reduce costs and increase efficiency and do autoscaling. But with the current micro service architecture, there can be a lot of problems. So the first one is the heavy weight. So right now the container, the Linux container would have the container and the Linux operation system and framework, also the application. So it can have a long startup time or need to be warmed up. And there would be trade-offs between security and risk and security and the cost because you would have a very long supply chain and big attacking surface. So that's why we would need OpenSSF to deal with these kind of risks. So it's a very hard balance for security and cost. And also the portability is not good enough. So it's, for example, like Docker, it can be run across different operations system. But it's not completely cross-platform for different hardwares to reduce costs. So the e-commerce startup I mentioned, they want to transition some microservices to serverless. But serverless also introduce the new challenge that is also the startup time. So if the service is like a serverless function, it's idle for some time. You still have to load the runtime environment and initialize the function and then execute the code. So it can be really slow. And in order to make it not so slow, you have to run the all the time in order for the users to have a very good experience. So that would greatly increase your cost. So why hybrid containers? So that's when we introduce a wasm container. So compared with the Linux container, the hybrid container can help leverage the advantage of different containers. Like, for example, the traditional Linux containers like Docker can have a very mature ecosystem. And WebAssembly containers would have much more lightweight and fast startup and higher performance. So let's look at the evolvement of wasm. So it started from in 2010. And that is aiming to improve JavaScript performance. So later, WebAssembly has been adopted as W3C standard. And later, fast forward to 2019, the WebAssembly system interface is released so that wasm can access systems. And the server side wasm is going mainstream. And to 2022, the Cloud Native Computing Foundation has did this survey called the state of WebAssembly in 2002. And the key finding is just one. That is, containers are the new normal. And WebAssembly is the future. So this is last year. No, actually, this is this year. So this landscape is launched several months ago by CNSF Foundation. So this is a dedicated landscape, especially for wasm. So you can see there are different projects for the languages, runtimes, application frameworks, edge, and AI embedded functions, et cetera. So the combined funding of all this project have reached $58.4 billion. So we would think wasm is to, in some cases, replace containers or run side by side with traditional Linux containers, especially in edge use cases. So we would think the first generation of virtualization would be hypervisor VMs and micro VMs and also like AWS Firecracker. And the second generation of containerization would be more like Docker on the cloud. And we would think the third generation of virtualization would be the more high level language virtual machines like V8 or WebAssembly. And this is a quote by Tyler McMullen. He's a Fastly studio and Fastly is the world's number one CDN provider. So he says, a virtual machine is emulating a computer. A container is emulating a private operating system. And the WebAssembly instance is emulating a process. So this is CEO of Docker. So when he saw in 2019 the release of WebAssembly wasm is a WebAssembly system interface. He said that if wasm and wasm existed in 2008, we wouldn't have needed Docker at all. And WebAssembly on the server is the future of computing. So yeah. And then last year he also said Docker. The reason why the tweet on the right side is because Docker has integrated wasm into it. So Docker desktop would have wasm runtime already integrated inside it. So when you run some wasm file inside Docker, it will automatically use a runtime like wasm edge. So he saw the Docker integration with wasm. And he said that it makes perfect sense and we no longer live in a single runtime world. So there are many containers, windows containers, and wasm containers. OCI can package them all. And I should be able to build and run them all with Docker. So we would think wasm is an alternative to Linux containers. And here are some advantages. So it's 100 sides of a Linux image and 1,000 times faster startup time. So for this statistics, there is a paper on IEEE. And you can look it up. And it's especially fast with AOT. And also it can reach a nearly runtime performance and secure by default. So because it has a really small attack surface and it's fully portable across different platforms. And it's compatible with Kubernetes and other service mesh like STO and distributed runtime like Dapper. So the container would usually require namespaces, cgroups, and file system. But wasm doesn't require them. And these are some more detailed comparison. So the disk footprint startup performance and the host OS, et cetera. So I guess I'll make it quicker. And then also safety and security and isolation. So for some of them I've already mentioned. So like the software supply chain security. So for WebAssembly, it must be clearly declared that certain files can be accessed. So it's signed modules. And also for management and orchestration, it works with Kubernetes perfectly. And also for orchestration on the edge, some edge use cases or IoT use cases, there are also some orchestration tools like cube edge, super edge, et cetera. And it can be managed by container tools. Also ecosystem like you can see it's, wasm is WCC and OCI standard combined. So this is a test by ARM. The link is below there. So it's not actual production code, but it's just for tests. So you can see that's the Rust code in between. So this page is the first test. It compares the size of a regular Rust binary with a WebAssembly binary file. And one is 1.8 megabytes and another is 0.8. So this is a really simple one. So it doesn't really, like in production, it usually would be much more complex. So that's when a use case that is much more complicated, the size differences would be much larger. And also the boot time, you can see, the startup time for wasm is 25% of a regular container. So this is also a survey of 2023. So the reason why devs would use WebAssembly would be this, like loading time and everything. And also you can see out of 100% of people who are being surveyed, 34% have heard of wasi and planning to use it in the next year. And also 34% have been currently using wasi in their project. So the following are some container tools that support wasm containers. C-ROM has added wasm-edges support. And it can recompile C-ROM and add wasm-edges. And C-ROM would decide based on the image annotation whether to start the wasm runtime or Linux container. And also there is an out-of-box version of wasm-edges on Fedora. And you can also use container dshim to start the wasm container through run-wasi. And container d would check the target file form of the image, whether it is a wasm-SAT-3 or x86 or ARM. And so this is what I mentioned already. Docker desktop has integrated wasm-rontimes. Not only wasm-edges, later they also have other wasm-rontimes integrated. And tool support for packaging images is Vuda and Docker. So the use case in production. So we used Kubernetes to manage wasm containers and Linux containers. So when wasm is needed, so for the serverless use cases, we would use wasm containers to store data, receive the front-end into MySQL. And in JSON, MySQL runs in the Linux container. So the yellow part is running Linux containers. And the green part is run in wasm-rontime. And you can check out the repo of this demo. And it's very optimized startup time, response time, and memory usage. And also for log processing, it syncs MySQL binlog to Kafka using wasm container. And while the yellow part, MySQL, binlog, and Kafka, still run in the Linux container. And so we all know that large language models have been really popular. So wasm is also really, really good for running AI inference. And it's actually not only a large language model, AI inference, but also more traditional like TensorFlow and PyTorch. So it's passed the request received by the front-end to the AI inference function. And the AI inference function run in wasm containers. And wasm would return the inference result to the front-end. So I will show you a very quick demo for running LLM. It can be on your Mac or on any IoT devices. It's only four commands. So why we do that? Why we not use Python in the traditional way? Because it can have automatic GPU detection. And the inference app is only two megabytes. So there's no Python dependencies. And it's completely cross-platform. So these are the four commands I mentioned. So you just do these three steps and just do it in your terminal. And it can be really easy, especially in the cases that services like OpenAI can go down. And you can have your own self-hosted large language model service. So it's not only that, but you can also have an OpenAI compatible API service with that. So it's also really easy, just the three commands. And you can have your own OpenAI compatible API server. So I will show you quickly how it works. So this is running the menstrual 7B instruct. So this is to download the chat app. And you OK, let me explain this right here. So the first thing you install wasm edge. And then step two is download the models gguf file. And the step three is to download the portable wasm file for the chat app. So what you see on the demo is downloading the wasm file for the chat app. And then you can chat with the model by talking with it in your terminal. So the question is give a brief overview of Chinese cuisine. So the device is, I think, is an M1 with 32. Also, that's the speed. The speed would be 20 tokens per second. And the device is M1 with 32 gigabytes memory. So it's locally run on your Mac. So this is to Japanese developers who tried it out. And yeah, it's really impressive. And so the link they gave is a tutorial for those commands. And you can find it on the seven states blog. And also, we have this script to allow you to run with just one single line of command. So it's completely automatic. You just enter it in your terminal. And it would allow you to download the GGUF file. And then also first, what imagine, then GGUF file, and then the chat app. And then you can talk to your, oh, you still need to choose among the many, many models. Now we have about 20 open source models, 25 open source models. You have to choose among them. And then you need to download the model GGUF file you have. And then when you have finished all the downloading, you can start to chat with the AI inference app on your, the larger language model on your Mac. So the outlook, yeah, I'm about to finish. So the entire ecosystem, because Wathom has been really a new emerging tech. So the ecosystem would need more work and open source contributions as well. So the Rust language is, even though we say that Wathom supports multiple languages, like Rust, C++, and even Python JavaScript, but it's, so in order to better utilize WebAssembly, because Rust language compiles the best to Wathom file. So for many developers, if they want to get started, it might be, for them, they might be needing to study Rust. So that could be a huge barrier, because they would find it really hard to learn. So this is a small tool for developers who want to learn Rust. It's powered by a GB4. You can try it out as a chatbot to help you learn Rust. You can also try it on the web. And then the container tools are not the most optimized for Wathom containers, because right now it's more for the Linux containers. And yeah, I guess that's it. This is my Twitter handle. And I will also be sending the CNCF booth later. So thank you very much. If anyone has questions, I think we have some minutes. Hi, thanks very much for the presentation. Like probably many other people in this room, I have not heard of these types of containers before, so it's very good to learn. I have a question. If I could ask you to go back to the slide where you had the survey results, there we go. My question is very specifically on the 2% on the right. I'm interested in whether or not you have any insights. And it leads to a broader question, which is you spent most of your presentation talking about a lot of really good reasons for why we should use this. What are the specific reasons why we should not use this? I think my colleague would have a better answer for that. Yeah, so yeah, Wathom is very good compared with the Linux containers. But there is no free launch. If you want to use a Wathom container in your production, you may need to rewrite your application. Rust, Wathom can only run a Wathom bytecode. So it's not like Docker, Docker can run everything. But for Wathom, you can compile Python easily into Wathom code, and then you run a Wathom container. So that's why you will not use a Wathom. So Python can easily, so you're saying Python can easily compile into a Wathom container, and Rust easily compiles? Python is not easily, because Python has a runtime. We can't compile Python easily into a Wathom, but Rust and C++ is a better choice to compile them to Wathom. So in summary, if you are writing something in C or C++ or Rust today, you are a better candidate for this. OK, thank you. Also Go, Go recently either was a support. So if you are a Go developer, you can try to compile your Go application into Wathom. Thank you for your presentation. And I have a very basic question, because I'm also quite new to this web assembly. So you have mentioned several concepts, like Wazi, like W-A-S-I, and Wathom, and also Wathom Edge. But sorry, I was a bit confused about those conceptions. Can you kind of clarify the difference between them? Like what is Wathom Edge? I wasn't very clear throughout this presentation. Wathom Edge is the runtime that runs Wathom file. And so web assembly system interface, Wazi, is allowing Wathom file to connect, to talk with external services. Like it's an interface to add more. So web assembly is standard. Right now, it's managed by W3C. It's W3C, sorry. It's the fifth standard for W3C. So web assembly is standard. And Wazi, web assembly system interface, it allows a Wathom application to access file systems. So you can use web assembly on the other side. So there are several web assembly runtimes with different names. Wathom Edge is one of these. And if you use V8, V8 also is a web assembly runtime. So it's just different concepts. Web assembly is on top. And then we have different proposals. Wazi is one of the proposals. And then we have some different runtimes. So like Sony is using Walmart. It's like WAMR, right? To run on their IoT devices, Wathom files. So to give their customers the best experiences. It's another runtime like Wathom Edge. But with different advantages. And also there are more. There are Wathom time and also Wathom. There are different Wathom runtimes. It depends on your specific needs, I guess. Like for Walmart, it's better if you want compatible. Maintaining on Wathom Edge, you saw my answer. You should choose Wathom Edge. But it depends on your use case. Walmart is much smaller. It's kind of run on a very, very small device. But different runtimes have different advantages. It depends on your use case. But you should try Wathom Edge first. Also Wathom Edge is hosted by SynCef. So it's donated to SynCef. Hello, thanks for your talk. So previously you mentioned you are changing from serverless to from microservice to serverless. So could you talk about what do you think the difference between microservice architecture to serverless architecture? What are the main difference? I guess you can still do microservices not in a serverless way. Because serverless is like you start up a service when it's used and then it's shut down when not needed. But I guess microservice is more like you run it on the server. And sorry, we don't have enough time. But I think it's a concept on a different level. So microservice is more like you can also run it on the server and keep it running all the time. But for service, it's more like you auto-scale and then you scale it down. Yeah, I think it's like that. Yeah, we can talk more after this because we don't have enough time now. OK, thank you.