 Hi everyone. Today we're going to talk about extending Envoy Proxy with WebAssembly. My name is Idid Levine and I'm the founder and CEO of SoloIO. And I am here Yuval Kochavi, the chief architect of SoloIO. And we will share with you about WebAssembly and Envoy. So let's start giving a bit of an intro. A little bit about the control plane and data plane separation. Just as it's important to understand the moving parts here. So if we go to the next slide, we will see that the way Envoy is structured is that Envoy itself is configured through a different component called a control plane that's responsible of processing user configuration and updating Envoy in real time dynamically in an eventual consistent way of that configuration. And in this slide you can see that we have two examples, one of Itzio and one of Glue. Both of these use Envoy as the data plane component, right? The data plane is where user data, HTTP requests flow through and they are the control plane component for Envoy. The control plane is the channel where configuration messages go through. And if we'll move on to the next slide, we can see sort of this visualization where a user configuration in Glue's case, Glue processes Kubernetes CRDs and one of the CRDs is a virtual service goes into Glue and then if Glue processes it through a Glue list of plugins and the user configuration, it gets translated to Envoy configuration and delivered on to Envoy. And now the reason for this extra API because the user configuration that Glue provides is use case specific and simpler to understand than the Envoy configuration. Glue also can automate a lot of the parts that Envoy needs. For example, cluster membership and endpoints with making Envoy easier to operate. Now if we'll continue, we can talk about how the data path looks like and we can see that an HTTP request comes into Envoy and that's one of the key parts that make Envoy very useful as a cloud native proxy, the ability to extend it, right? The request doesn't just pass through Envoy on to the upstream but actually goes through a series of filters that can modify the request and impact the behavior of the proxy. So in this example, we can see that the request goes through Envoy's filter chain first hit in the external filter where it's consulted with an external server whether or not to authorize the request moving on to the rate limit filter where it consults a rate limiting server to allow the request for rate limiting purposes and finally traverses to a GRPC transcoder filter that allows you to provide a rest interface externally but converted to a GRPC interface internally. So your microservices can talk GRPC which is very common in the Go microservices ecosystem while your front end, your external facing API will be a REST API that's easier to consume from JavaScript. So that's kind of how Envoy looks like and one of the nice things about Envoy is that it's extendable. So if you'll go in the next slide we can see that we can also insert custom extension points, custom filters onto Envoy and the rest of the talk we're going to talk a little bit more about these custom filters and how Wasson fits in. Now, when talking about extending a product usually we have a checklist of things we want to get out of it. So we want to be able to use a language of our choice we want this extension to work fast we want it to be safe that if we have an issue in the extension it doesn't bring down the whole proxy where a lot of other traffic potentially multi-tenant traffic flows through and we don't want to now we only want to build our code we probably don't want to rebuild all of Envoy just to extend it and we want an easy to use experience and with that in mind let's I'll hand it over to Edith to talk a little bit more about that. Thanks Yvonne. So basically we'll take an example of the two control plans that basically extending Envoy and managing Envoy and basically see how they decided to go about extending the data plan. So the first thing that we will look at is Glue and Glue is an API gateway based on Envoy it's focusing majorly on the edge that's the use case and when we looked at how we can actually extend Envoy we decided that because we more care about the performance we want to make sure this is basically on the control path on the request path so latency is extremely important we decided that we want to make sure that those filters going to live still in the proxy and not going to do any round trip but that means that it's hard because the way to actually extend those filter is by writing a C++ asking a filter code and then recompile Envoy. So by us choosing to decided to take this banner on us in that case the solar company solar company basically we make it extremely you know it's extremely to the user itself we make sure that it's extremely fast right because it's in the control path and it's built into the binary basically of Envoy it's extremely safe because you know there is no round trip there's no network involved or wire involved in this but of course it's not the easiest thing to do as I said C++ code is not something easy to do we are basically recompiling Envoy for the users so there's nothing easy about that we chose this we're taking the law for us so it is easy for our customers but in the natural for us it's a challenging thing and you know and there's not a lot of people who can do that STO took a different approach and when STO started basically their idea was we really care about extensibility we really want people to be able to take that and write it by themselves and not be depending on us and in order to do this they basically came with Mixer and the idea with Mixer is basically that every time that the request is coming to to the proxy it's going to basically take the request and route it to Mixer and Mixer basically would get all the requests and would be able to do whatever you want from it telemetry or maybe you wanted to do some you know any adapter that people might want and basically return the request to the proxy after it the problem with that is you know there is a lot of advantage with this right potentially right now people right be able to use their own language because Mixer can communicate GRPC you know honest to God most of the people were using Go it's not fast right I mean that's the majority problem that the people talking about or talked about stability in STO that was the majority of the problem because there is a full request constantly going on the wire it is safe right I mean it is going over the what you know again there is communication between two components here it's not at the same binary so potentially there is always a problem but you know relatively it's safe you definitely don't need to compile language which is a very very huge advantage and it's easy you know I don't know it's a complex problem to solve but it's easier than write a C++ asking and recompile so you know when you're looking at this you know the Google guys and the local community understood that this approach that the STO community took is a little bit problematic there was a lot of problem with that and tried to figure out what different way we can go so as an community we decided to basically to explore the WebAssembly so let's kind of like overview a little bit of what is WebAssembly and why we thought we thought that this would be a good way to leverage that technology so first of all WebAssembly funny enough it's not an assembly code right it's a binary structure format and it's not only for the web as you know because we're bringing it to Envoy so it's a little bit misled but the idea would actually creating WebAssembly the measure reason for its existence is to extending the browser right that's why people did it and the idea was that they wanted an ability to be able to extend the browser with but you know with that they needed to have some use case to make sure that for instance it's portable right because browser can run everywhere so we need to make sure that it doesn't matter which operating system you're using they need to make sure that it's secure because the last thing that they want is that the browser because some was some extension that is not you know you know that there's some error prone or something like that it will take all the browser down that's something that we definitely don't want to make sure that so they created a sandbox we want to make sure that it's fast because again performance is extremely important in the browser as well as the handboy and you can actually see that it's very interesting is that if before if you know before the was and people were using JavaScript JavaScript is not going to be extremely fast and the reason is because there's just more things that should be you know a process while actually it's running with WebAssembly it's basically much more a slim version of what should be happening when the thing is running so you only need to you don't need to pause you don't need to do a lot of stuff it's already basically package very minimum the last you know the other thing as you well mentioned before you probably prefer that the customer will be able you know your user will be able to use any language and for that in Web in WebAssembly they created a very interesting you know nice interface that basically mean that you can choose the language of your choice and there is some representative intermediate or representative that can go to whatever process like 86 or on and the last one that was extremely interesting when they came in is the ability to actually run it outside the browser itself and by creating this interface was the that's what basically allowing us to eventually bring it to endpoint so all those functionality of fast secure portable and as well as as any language it's what basically make that attractive technology extremely attractive for something like endpoint and this is what we decided to bring it because if you think about if you will leverage it then we can use any language it will be very fast because it's very close to assembly code it will be safe because it will be on the same binary there is no need to compile because it's basically extended and without needed to actually compile it and it will be relatively extremely easy or easier than do the rest so I will end out to you well to explain you how exactly we do WebAssembly and how we do the Envoy to get over the community thank you Diet so let's talk about that as mentioned before the Envoy has a series of filters that can act on the request and change it and essentially the way wasm is integrated onto Envoy is as a custom filter right so as far as Envoy is concerned most of Envoy core is not aware of wasm at all but rather wasm is implemented as a native Envoy filter as we can see in the next slide that filter actually talks with the Envoy runtime and runs wasm code so most of core Envoy is not really aware that it's running wasm and that also has the advantage of those filters having almost the full power of a native Envoy filter and that is accomplished using something we call the ABI so the way the workflow works is that wasm-vm is built into Envoy so we have V8 we have Wavom and we have an LVM that's not really wasm but more for development purposes and testing purposes when you want to use a wasm filter in Envoy you create an Envoy filter or you instantiate in Envoy config an Envoy filter of type wasm and tell the wasm filter to load your wasm code with a specific wasm-vm the wasm-envoy native filter communicates with your wasm code and that's what we call the ABI and in the next slide we see sort of an illustration of what this ABI looks like it's all open source on github so you can check it out yourself essentially it's a definition of C-like functions that pretty much correspond to the Envoy internal interfaces today and allow you to do the same request modifications for like external or like transformation all of this a wasm filter can do it has the advantages that you can load it dynamically into Envoy you don't need to recompile Envoy you don't even need to restart Envoy it's all loaded dynamically as regular filters can be loaded dynamically and it has the advantages of you being able to externally write and develop it if we go in the next slide we can see that today there are various languages already supported so you in order to essentially write a filter you can write it using the raw SDK but it'd be much easier to use a language specific SDK that brings the native language concepts to your development flow so we have a list of them in the script and Rust there's an additional tiny go and we have a list of them all in our Wasmere repo and we'll refer to that later and using the SDK allows you a much easier development experience than the raw ABI because they abstract much of the low level details of Wasm and the way it works today is that you use a language specific SDK to build your Wasm filter and then the Wasm filter that you wrote communicates with the Envoy Wasm native filter using the ABI in a Wasm VM of your choice where currently V8 and Wavim are supported we can summarize and we can see that we achieve almost all our requirements from before and the one thing, the one point of friction is how to get the Wasm module from your developer laptop into the cluster and I'll let it talk a little bit more about how this developer experience is solved using a tool we created Thank you Rao. So basically when we looked at this and we understand the amazing power that it's giving right now to the user, I also saw this tweet and that tweet is from Solomon Haik the founder of Docker and basically Lin Cloud is one of the leader of WebAssembly ecosystem basically announcing Wasi, Solomon Haik mentioned that if Wasi and Wasm was existing in 2008, we will never create a Docker, right? And that's basically gave me kind of like the connotation of the fact that as you well mentioned before it will be extremely hard, like it's really nice technology but it's extremely how to go about doing it and when you think about it, this is what happened with the container ecosystem because back then, you know, Google did an amazing job, Google and the ecosystem to basically created the Linux container but to make the Linux container as popular and use it including us, we needed Docker to make it very simple to use. So when I saw that I realized that this is something that we can help with and there is a lot of similarity between what happened with Docker and what happened right now with WebAssembly and that's why we decided to create WebAssembly Hub. So what is WebAssembly Hub? So it's very familiar it's basically giving you the Docker like experience that you're bringing to Envoy platform. So in the natural is very, very simple interfaces it's a command line and it's a registry and I will let actually you well show you that instead of me actually talk about it but by actually bringing this experience as well to the user we kind of like checked everything everything that we wanted on the list, right? It's use any language is fast, it's safe and you need to compile and now with WebAssembly it's also extremely simple. And now I will add it to you Val to show us quick demo. Thank you, Edith. Let me share my screen and I will do a quick demonstration of wasmi our tool for developer experience. All right, I hope everybody can see my IDE we can make it a bit bigger. Let's start with the demo. So in this demo we're going to show how we can use wasmi to create a nice developer workflow of creating, building and distributing and running our WebAssembly Envoy filters. So the first thing we'll start with is with wasmi in it and wasmi in it essentially has a template of a filter so we can you know have something to start with, right? Kind of like the React app template and when we run this command we choose the language that we would like to use, we'll use assembly script and we choose the platform we want to support the wasm ABI, it has multiple ABI versions so this step explains to us which version of a glue initio this filter will support. Once it's done you can see that it has extracted our wasm template and we can open a bit of code right now inside of the new filter folder the index.ts in the assembly folder and you can see this is a assembly script file, it's very similar in the syntax to type script and now just for fun we'll change this a little bit so what this filter does it's a very simple filter and on a response headers right whenever a response comes into Envoy it appends additional header onto the response and now I can just change this from a low world to a low asm a wasm and save now do note that this what we'll see is how we build and inject this filter onto Envoy so the next step would be to build the filter using wasm build takes an argument what builder type to use in our case assembly script and the image tag to use and this is a very similar concept to Docker images and in our case the image tag is WebAssembly.hub.io slash uvall which is my user name and then the image for name add header and a version and finally dot to build the local directory and now it pulls in a build container with all the build tools you need so you don't need to have npm re-install and builds the filter for us and let's give it a second to finish alright as you can see it created a filter with our image tag and that filter has our wasm already in it and now we're going to use the wasm.hub.io that is image listed as an image that was just built now in order to distribute this we can do wasm push now this is very very similar to how Docker push work and in fact we even use the same code and wasm push will push this to WebAssembly.hub and this is our portal to essentially help you discover, deploy and pull and publish all your extensions and as you can see let's explore this a little bit let's view more repositories you can see that there's a lot of existing filters already from users in the community just sign up it's free to sign up and push your code we can filter by label to kind of help you in this case if you look for example at the security label we can see this filter that has a nice description to it and then it also gives you the command you need to pull it so you can do wasm pull to pull this filter from WebAssembly.hub locally into your local laptop now let's log in just to show you how my folder looks like and you can see that I'm also a member of the Soli organization and these are filters published to the Soli org but if I go to my own user you can see the AdHeader filter that we just pushed alright let's continue with that demo so once the filter is all built and all pushed we can deploy it so wasm deploy here I'll just deploy locally to my PEST and let's run this command deploy we currently support a glue and ITCL or local envoy what it will do is essentially pulling the image if needed and then run it and essentially give you a very quick and easy way to get your filter loaded and running now let's see that everything is working hello wasm header on the response so let's do a head request just so I can get some headers and you can see here the request was sent successfully and the response indeed contains the hello wasm response header that I just created in this filter and that's it that is our demo for wasm the tools that allow you to build publish and deploy your web assembly and voice builders take it away for some final words thanks so much Yuval so guys as you see we are extremely extremely passionate about everything related to wasm and the extension of service mesh and voice specifically we think that this is a great tool and we worked very hard to create a tooling to make it extremely simple to use it and we really hope that everybody will use it running as an envoy in production for a lot of big huge enterprise our customers I can tell you that this is so valuable and extremely desired look everything that we're doing most of the stuff that we're doing is open source including glue and service mesh all the work that we're doing is almost open source so if you're interested in this you can go to the web editor go visit our repository and specifically the wasm repository there you will find everything that solo is doing including a spec for wasm OCI image so please check it out and we would love to take you you know to get any feedback thanks so much