 My name is Angel and today I'm here with my teammate, Rafael, to talk to you about how to create Istio filters with any programming language. Before starting, let me introduce the team we are working on. We both are part of Wasunlabs, which is a small thing inside the VMware office of the CTO. We focus on creating and contributing to projects that showcase the possibilities of WebAssembly. We also create different tools and other projects to increase the adoption of developers for this technology that we are really excited for. Apart from that, we also create 13 experiments. We would love to do that. And today we want to show you one of the experiments that we recently did. One thing that I think everyone of us that work with services and infrastructure knows that proxies are everywhere. Proxies are a critical piece of infrastructure. We can talk about proxies even in single node deployments where we have a reverse proxy that redirects the traffic to our services. We can think of big clusters with more than 100 nodes and a lot of different proxies collaborating together to serve the traffic, redirect things, and provide the actual content that our users want to see. There are a lot of different ones. We have envoy, engine X, traffic, there are different types. So today we are going to focus on envoy and specifically about installation that are based on envoy. If we look how a regular deployment looks these days, we have a lot of different applications that actually share some common resources or the same box, we can say it could be multiple nodes but they are part of the same infrastructure. Then we want to expose them outside so we create get ways and everything so users can access to them. And in some cases we need more advanced connection between the different resources and the different services and we include proxies across all the different services so they can connect together and they can perform other advanced actions. This is not something that it's easy to manage and to deploy. Fortunately we have projects like Istio that allows you to easily deploy all these different proxies without any hassle, just put them where they should be and connect the services as we need. So now let's talk a little bit more about how we can extend all those small proxies or those small dots that we have in our infrastructure. In talking about extending a proxy, what we are talking is about adding custom behaviors that it's tailored to some specific use cases. It could be something that everyone implements on services like observability, traceability and this kind of filters, sorry, of custom behaviors or it could be something that is more specific to every infrastructure or every company. For example, you may need to have something that reports traffic to a centralized service for later analysis. It could be something that blocks a specific request. There are different behaviors. Everyone needs to implement something specific and this is why it is important. But why is it useful to do this kind of extension at a proxy level and not, for example, inside the services? If we look at the same picture that we had before, one thing that I didn't say is that we have eight services in this example, but every service is different. It could be a Ruby application, Node.js application, Go service, JavaScript, there are many different services all together in this infrastructure talking and providing content to users or to other services. If we have to ask every team behind every different service to implement a third in custom behavior, it takes time and it's not only that, but it's not easy to give that up to date. You have to ask the team to implement one specific, I know, block some specific request or Ruby, you need to provide maybe an SDK that everyone can import so they don't need to do that, but in the end, it's a complex process. But if we just focus on those small dots, proxies are all the same and all of them are managed by Istio. So it seems they are really good spot for placing third time behaviors that can be actually reused by any service because you do that just before they receive the traffic. And this is why we think this is really, really useful. So now we have the why, but let's think about the how. How we can extend proxies. Different proxies offers you different mechanisms to extend them. So if we talk about specifically about Envoy, for example, it could be that it provides an API, but if we talk about a different service, like EngineX, for example, it provides you a different way. When we talk about extending a proxy, we usually refer to two different types of extensions. We have models, which allows you to actually access the internals of the proxy. So you can go into the API, the internal API of the proxy. You can do much more advanced things like providing new languages into the proxy, doing specifically anything that the proxy can do. However, this is, it provides you more power, but it's more difficult to implement. Then we talk about filters. Filters are a much more easy way to extend the proxies, because they are just a source code or a process that runs between the request and the responses. So filters take requests, process them, continues with the response, receive that information, and then goes back to the user. This is more models are pretty common in every proxy, and usually you have different ways to create them. Those are the small ones. So focusing on filters, if we talk about Envoy, the way or the classic way to create those small filters are with the C++ API. If we go into that direction, it requires you third time things. The first thing is that you need to know C++, which is something that at least I'm not very familiar with, so it's not easy to implement those. Then those are compile filters. You need to actually compile the filters, then compile Envoy with that filter, and then distribute a custom version of Envoy, so it means that now your team needs to maintain that Envoy just for one filter that you created. Envoy comes with a lot of default filters. That's true, and all of them are amazing, to be honest. But when you want to create that, this is the kind of path that you should go. Another option is Lua filters, which are also available, because you can enable a model that actually allows you to create Lua filters. But that also requires you to know Lua. But today, I want to present you an alternative way to extend Envoy. So, imagine that there is a way in which you can take different languages, languages that you even know, and you can create filters that then will be load by Envoy. You don't need to recompile anything. You don't need to do any extra step. Just get those models and then run them as filters inside Envoy. And this is where WebAssembly enters the scene. So, with WebAssembly, you can compile those filters into a way that can be load by Envoy, and then you can implement those custom behaviors directly in a more, let's say, easier way than actually coding them in C++ and then introducing them via compilation. And to talk about that, my teammate, Rafael, is going to show you about WebAssembly and proxy wasm. Thank you, Angel. So, first of all, what is WebAssembly a bit more into the details and how these two pieces can fit together? So, the first thing is that if we think about these several programming languages, we write, for example, Rust, C, or Sieg. There are other programming languages, such as Go, as well, that can be compiled to WebAssembly. And so, instead of compiling our program to X86, for example, or to Arch64, we can compile that to WebAssembly. So, the binary that we get is a specific file that is standalone. And so, this file can be reused across different environments. And so, it's not only that this is standalone, it's also safe because it runs in a sandbox. And the idea behind WebAssembly is that since it is an environment in which it doesn't have access to the external world because it's running inside a sandbox, it can only get that much. So, there is no system interface, as the libc, for example, when you can access the external world, you can do syscalls, things like that. You cannot do that from WebAssembly. You only have the module that is on what you were compiled with and you cannot get out and get out of it. Also, the memory is completely isolated from the outer environment. And so, we have this binary. So, it's running in a sandbox. It's safe and it's also universal. So, we can run that anywhere. And we have this single binary. But we need something in order to run that. And that is the virtual machine. So, we need a WebAssembly virtual machine that will interpret this WebAssembly and will run that on our machine. And so, if we think about that, we see that from JavaScript, we have browsers that already implemented JavaScript itself. And then, from there, they were extended to also run WebAssembly. And so, from there, we went into the service with Node and so, we have JavaScript and also WebAssembly running over there, but it's even more than that because there are other runtimes that now are very specialized in only running WebAssembly. They don't run JavaScript, they just run WebAssembly and they are super optimized to run our code. And so, we have this file that is standalone and then, we run that instead of a virtual machine. But how do we put all the puzzle together? And the answer is that Envoy is able to start our Wasm virtual machine, our WebAssembly virtual machine. And so, we see that our requests are coming, they are getting into the virtual machine and they are getting back out. But what is the answer here? What is the piece that is missing? And the answer is Proxy Wasm and this is what we are talking about today. So, Proxy Wasm was presented in February 2020 by a joint effort by SoloIO, Google and the Istio community. And so, Proxy Wasm is actually multiple things. So, it's an ABI in that it specifies what is the interface that the guests, so the WebAssembly guests, can talk to? What are the functions that they can call? What is the functions that the host can call inside of the WebAssembly sandbox? And it's also another thing. So, this is ABI. Then, we have SDKs for the different languages. So, right now, if you want to write a filter and this will be on the guest side, you implement your program using this SDK and then you are able to write your filter. And in this case, there are SDKs for Rust, for C++ and also, there is another one for Go provided by the ttradelabs team. And also, that is on the guest part. Then, the other part is the host implementation of it. So, there is a reference implementation as well that is on C++. And so, this is the part that the employee uses in order to run WebAssembly, start the WebAssembly virtual machine and then run the WebAssembly inside of it. Okay, so, you can create your first filter. We are going to see just a very simple example on how that can be done. But you see that there are lots of different languages. Rust, C++, C++, assembly, script, Go. Recently, Go, the official compiler is able to build to WebAssembly directly without the need of any JavaScript. We also had a tiny Go before that. So, we were able to compile Go inside, to compile that to WebAssembly. And in this case, what we are going to do is to show some of the work that we have been doing regarding this. But we're going to run it inside the envoy because it's just for the sake of showing how we can extend the proxy itself with WebAssembly. And you can see that everything that is beneath this proxy wasn't in this case. So, if we take, for example, the Rust code, it's very simple. There is more code, of course, here, but the important takeaway is just this, right? So, in this case, we have a function as that is defined by the SDK and in the end by the ABI. And so, it's on HTTP response headers. So, as Angel was saying before, this is sitting in the middle. So, it gets the requests and it is also able to intercept the answer from it and adapt it somehow. And in this case, we are intercepting the response headers of the request and we are able, in this case, to add a new header to it. And so, this would be kind of a first filter. But we have seen that all the programs that we have been talking about and other programming languages have been compiled. So, what about interpreted languages? And in this case, and this is what our experiment has been about and it has been specifically for this talk, we have JavaScript and we have PHP. And also, Python could be added as well, but the examples that we are going to show today is exactly about JavaScript and PHP. And so, this module that we have on WebAssembly can be run on any other platform that supports WebAssembly, not only the Envoy proxy. This is kind of a different environment in which where you run the WebAssembly binary because you need to adapt to the ABI of proxy wasn't. And so, let me show you an example of PHP and then Angel is going to take the JavaScript part. Let's see, I need to change to mirror one second. Okay, before we get into the demo, so you have the repository proxy wasn't spec where you can see all the specification, what are the things that you can do with this in the guest part and also on the host part. So, if you are interested, have a look at that. It's proxy wasn't spec in the repo so you can have a look at that. And so, in this case, as part of our work, we ported the PHP interpreter to WebAssembly and so it is able to, you are able to put the interpreter and run that with WebAssembly runtime such as wasn't time. And so what we did here was that you have Envoy and you are able to provide the PHP script that you can run and then you load the PHP binary here on the local file name configuration part of Envoy. I already did the startup because it takes a little while to start. As I said, this is an experiment done for this talk so it's not optimized at all. This is something that we would love to do so if there is interest, we would love to know more. So as you can see here, we have, first of all, an echo on it. So this is the script that is being executed and if you see on the left-hand side on the bottom, we have the logs that are coming out of it. And so this happens and this is able, this can be done because there is a system interface in reality that is able to echo and to basically write to a standard output. And so you can see that. So on every request, I get a new standard output being written and we also get the header being set by PHP. So adding a new thing here would be as simple as that. So you just save that. You could call, for example, to the time function and then I can just run that. Since it takes a little while to start, let's look more of it into the details. So we did the work to port PHP to WebAssembly. And so part of the things that are missing, if you try to just run a data envoy with the proxy wasm host part is not going to work because there are some imports that are missing from the system interface. And here there are things that we have to implement on the host side of proxy wasm in order for this to work. And the PHP interpreter to be started by WebAssembly. This is something that we can talk with the community if there is interest in doing something like this. But yeah, we can get the ball rolling there. So this is one of the things that we had to do. And then the other one was on PHP itself because as you saw, we are reading the PHP code from the envoy configuration. And this is something that proxy wasm allows you to do. So you can configure the proxy, the proxy configuration and then you read that and then you load that. So when we start and I'm going to go through the C code here, you have the main on the bottom. When we start the module, we basically start the PHP virtual machine. And then on every request here on response headers, we are going to call to the already initialized PHP virtual machine by saying, just read this script and interpret it. And so this is basically what this does. And here the minus error option of PHP is basically I'm going to provide you this script via the standard input and then you just read that. And this is what is going on. We just do this call on every request that we get. Of course, this only works for proxy on response headers. As I said, this is just done for this purpose of this talk, but this could be extended to support the whole proxy wasm specification. And so if we go back to the change script, we see now that I added the time here ex wasm header at, and now we get that properly written over there. And this is actually calling to the time function of PHP. So yeah, that's it. So now Angel is going to talk about the last script part. Okay, okay. I think now it's okay, now you can hear me. Okay, so yeah, so the same thing that we had to do for PHP as Rafael showed, it's all the steps that we usually need to do when we want to run interpreted languages into the WebAssembly. So first, you need, of course, an interpreter that can run that code. And then we need to build that bridge between the interpreter, between the WebAssembly host, let's say, and then the code that it's running inside the interpreter because you want to write JavaScript, you want to write PHP, you don't want to deal with the C code that we showed and you don't want to deal with this RAS code that I'm going to show you now. So as we were focusing on this experiment, in the case of PHP, we were working directly on the C side for making it work with JavaScript, we took a slightly different approach. Instead of having to do it from scratch in the interpreted directly, we took advantage of other projects that are already in the WebAssembly ecosystem to create this was a module that can actually reproduce the proxy was an API and run JavaScript code. For that, we use the proxy was an SDK for Rust directly. And then we use the quick was a quick JS, sorry, bindings that the Shopify team created for a different project. So quick JS is a super small JavaScript interpreter. I think it's, as far as I remember, is I think 500 lines of C code or something like that. So it's pretty small. So it suits really well this kind of use cases as they are small models and they are really fast to run. Here we have the initial configuration that we didn't show before. All this is done at the Rust level just because we want to configure the filter and pass the source code via configuration. Then we create the HTTP context which basically allows you to connect to the HTTP at the HTTP level as Rafa showed before to get the headers and that information. And then we start the virtual machine, the quick JS virtual machine that will run the final code. If we move to the final quick JS bindings, here at one point is where we connect those words. We have the host calls which are these ones for the proxy was an SDK that allows you, for example, to get a value from the headers. And then we move that and configure a new function inside the JavaScript interpreter that can define the header specifically. But this is not something that actually look like a proxy was an filter because you have to actually call this method and this is not something that you could do with other proxy SDKs. So let's build an actual API in JavaScript that looks like proxy was an. So here we have these bindings that actually gets these imports that we define it for this specific experiment. And then we create JavaScript classes that actually maps exactly what proxy was an expect. So you can code the similar, super similar to how others SDKs work. If we now see the void configuration, it's pretty similar to the one that that graph I showed because for every WebAssembly model is always the same. You define an HTTP filter, you put that it's type was some, then you provide the file name, in this case, quick JS filter, and then you provide via configuration the actual JavaScript code, which as you can see is just a class that extends the HTTP context that we define. And then we subscribe to the on HTTP response header and provide that information. Here we have an MV proxy running with that specific model. And then if we go here, and the core dash dash B, I think it's 80, okay. We get that header. So we successfully configure and created a Envoy filter using JavaScript and even PHP. So now let's go back to the presentation. So let me extend. Okay, so yeah, this is the experiment that we wanted to show you. As we said, our goal was to try and try to demonstrate that this is actually possible. It may make more sense for some use cases, but something that we actually got any time we talk about WebAssembly is that people want to run the same languages that they are used to, because one of the barriers of using WebAssembly is that the best languages that now target WebAssembly are things like Rust, C++, that not all people are used to work with. Actually, I had to learn that one year ago. So I know that process. So one thing that you may think is that now that we have these models, how we can configure all our infrastructure to ensure that all those filters are run properly. And this is where Istio enters the same. So when I was trying that in Envoy, it was super simple as I showed you with the configuration. But now when we enter into the Kubernetes thing and we have not one proxy, but we have 30, 40, 50 proxies, then maybe creating those filters could be difficult. The good thing is that Istio already provides you a way to extend the different proxies with wasm filters. You can configure them directly with a YAML configuration, point them to the specific place that the proxies can find the model, which could be an OCI regular resource, or an HTTP endpoint. You can push your models to your default register that you have in infrastructure, and then you will be able to pull them and configure that in the different proxies. The good thing is that this proxies works, but they can be composed. You can configure multiple ones, not only one, and you can even match specific selectors so you can select what are the applications that are going to have a specific filter or a group of filters. And this is how it works. You configure it in the YAML, you drop that configuration, apply it to your communities cluster, and suddenly all the different proxies that matches the specific selectors, they are going to get their filter and they can start the traffic. This is how it looks like. So we have a custom Kubernetes resource, which is an extension for Istio. Then the kindness wasm plugin, we configure the metadata, we set the selectors, we put the URL, the configuration, and that's all. All the different proxies that matches it will get that information. Just a quote that we used to say like, void requires a filter to be present in the file system and Istio is the wizard that makes that happen. And before finishing, I would love to talk a bit more about one of the properties that I really love from this specific project for proxy wasm and from WebAssembly in general. And it's about portability. So three weeks ago we went to a different conference in Barcelona in Spain, which was the wasm.io. And there we attended a talk in which the con team presented the proxy wasm, yeah, the implementation of proxy wasm in NGINX and in the con platform. Since we were already experimenting with this, we quickly contacted them and we started to talk about this. They got excited about having different languages for filtering and it's good to talk to them because they mentioned that this is something that some customers already asked them, like, hey, is there any way we can use all the languages? Not the one that you are mentioning. So con announced a new project called wasmx some months ago that actually does it. It adds the proxy wasm filters in NGINX for extension in the con platform. The great thing about this is that once we had the, for example, the JavaScript filter with the WebAssembly model, we only had to send that specific model to them. We didn't care about recompiling. We didn't think about architecture. We didn't think about anything. Just the same exact model works in con, in envoy, and in anything that implements the proxy wasm interface. And this is something that is general for WebAssembly, but for this case, it was really nice to see. And yeah, that's all that we wanted to show to you today. So yeah, thank you very much for attending this session. We hope you like it. We recommend you to come to our blog because we have more experiment like this and more projects. Thank you. That was an equation. We have some time, so feel free. Whoop. No. Yeah, I don't know. Okay, I think now it's ready, the microphone, so. Yeah, I mean. Okay. Okay, so the question is, could you share some insights about memory footprint or CPU footprint comparing to approach with, let's say, C++? So we didn't still did the comparison with the C++ native filtering, but we compare the different wasm models from the different languages because it's different, for example, to have the REST filter, which is kind of compiled natively to WebAssembly than this middle step with the interpreter compile and then the source code. So the memory footprint really depends on the WebAssembly model because WebAssembly needs to put some old information inside a linear memory, and that will be the actual memory that it's only for that sandbox, but it depends on the size of that model. So for the JavaScript model, it's, I think, a few megabytes, not much more than that, and the CPU consumption is really, really low. For the case of PHP, since the interpreter is bigger, it needs to load more code to, for example, encode more all the functions provided by the language and everything. The footprint will be much, much bigger. The good thing is that when this happens, the Envoy proxy initialize first the WebAssembly model. So it's not like every request requires you to initialize the model. It's just that you are calling the functions. So once the initialization is completed, as Rafael showed before, that it takes some time for PHP, the footprint is already done there, and replying questions, it's just a matter of sending and replying to that information. We did the test between Rust, JavaScript, and PHP, and the time is mostly the same. So they don't change at all. Welcome. Thank you. Thank you. Hi. Yeah. Now, is it working? Good. I have a similar question, though. More on, like, not only on the footprint, but on the stress test, like on throughput or anything, like, have you tested how much RAM can you allocate for the Wasm VM? Like, if there's any limits to, or if it gets bottlenecked at some point, or if you're using too much CPU, then the Wasm filters start to get like less throughput, or does that anything happen at all? Thank you for the question. Sorry, good question. So we, oh, sorry. Think of the question, that's a real good question. So we haven't done any measures because it has been, like, just the experiment for this talk and showing that it's possible for people to reuse the software or the languages that they already know. But I think it's interesting, and I think it's something that we should do at some point in order to compare the different options that are there. But I don't know if you want to add something, Maro? Yeah. Yeah, I like. Okay, okay. Okay, I think, yeah, I think here. So, yeah, just to add that, that it's true that it's something that, from the WebAssembly, let's say ecosystem, is still working progress. So as you mentioned, this is a question that we did, not only for Proxy Wasm, but for running in general WebAssembly workloads that currently there is no way in the runtimes for limiting memory or CPU. It just goes up to the requirement for that specific model. For Envoy, I didn't check the specific Proxy Wasm and C++ specifications, so I don't know if they have a specific throttling or anything like that. But yeah, that's something that if at any time we put JavaScript or PHP filter, we must do before to understand really how it affects the performance. Cool. In terms of difference, I don't know, but there is a way also to say like field or gas to the WebAssembly runtime. So it's also possible that if you have an endless loop or something like that, you are able to either kill that or reduce the field that it's able to do in order to keep running. So that would be killed. Cool. All right, working again, right? Yeah, cool. And then the second question is like, have you ever had to take care of states inside those Wasm filters? I was like, not only on state that you maintain as a counter for rate limits, but actually to download state from somewhere else and something that a config map just wouldn't be able to work. Yeah, so there is on this spec, there is an API to actually perform HTTP requests, outbound HTTP request, and there is a whole bunch of things that you can do in order to get a state from the outside world, let's say. So this is something that it's also, but as we were saying, since it's a sandbox, you need to specify what kinds of things you are allowed to do within the sandbox. But there are kind of functions that allow you to do this kind of thing. So you can make HTTP requests, but because the Percy Wasm specification allows you to do so. Yeah, Mike, it also has relation to, how often does your VM get restarted or anything like that? Because if you want to download a gigabyte of states just to keep it in your Wasm VM, and then suddenly every few seconds your plugin gets restarted, you may face problems with downloading so much stuff. If it gets reloaded at just every config map change or anything like that, then maybe it gets better. But if suddenly every time you receive less requests it shuts down your VM and then spins it back up, it could face problems, right? Yeah, I mean, totally. If it's something that constantly, I think that it only reloads at the time that you actually get an error inside the VM, so it traps the error and restart the virtual matching. So as long as the proxy is live, it's alive, it will keep. Yeah, we need to stop. It's finished so we can continue the conversation. Thank you very much. Thank you everyone. Thank you.