 Hi, my name is Yaro. I'm a software engineer at AdShade and Envoy Contributor. My co-speaker today is Takeshi, who is also software engineer at AdShade and contributor to ProxyLasen. In this session, we'll present you the work that we've been doing to enable you more accessible way to develop extensions for Envoy. So here is our agenda for today. First, we'll explain why WebAssembly out of all the things. What is the problem? Then, we'll talk specifically about WebAssembly in Envoy. And next, we'll introduce you Envoy SDK for Rust that we've been working on. And our primary focus will be on lessons learned because they are not limited to Rust and Go. We've picked those items that are universal and with minor changes can apply to every language. After that, we'll demo a practical, ready-to-use extension that has been developed using this SDK. And then Takeshi will take over to update you on almost Envoy SDK for Go. We'll finish with key takeaways called to action and Q&A session. Let's be clear about the goals we set for the session for ourselves. We want you to walk away out of the session knowing what WebAssembly in Envoy is all about, how to get the most out of it as of today and what other lessons learned by the community so far. So this is informational component, but there is also inspirational one. We want to inspire you to give WebAssembly a real try, not in a year from now, when you might think it will be finally production ready, but today. So give it a try to extension that you've been developing and that we'll show you later in the demo section. Try to develop Envoy extension of your own. Try to develop Envoy SDK for your favorite language. And ultimately, join the community and help us to develop all these things together. So as attendee to this conference, you're probably familiar with the status score of Envoy extensibility. You have to develop custom extensions in C++, statically link them into Envoy binary, maintain a custom build of Envoy, keep up with upstream changes, keep up with security releases. Overall, there is a lot of burden involved when you decide to extend Envoy in such a way. That's why Envoy community has been asking for a long time for a different extensibility model. Where extensions could be loaded into Envoy dynamically and ideally, it would be possible to develop extensions in languages other than C++. And WebAssembly is an answer, is technology that make it all possible. So WebAssembly is a virtual machine and compilation target for general purpose programming languages. What it means is that you can develop applications in programming languages like C++, Rust, Go, but when it comes to compilation, compile into WebAssembly code instead of machine code. Envoy plays a role of WebAssembly host. It's responsible for loading and loading WebAssembly waste extensions at runtime, for exporting APIs that such extensions can use and integrating such extensions into its regular request processing flow. The final piece of the puzzle that you need to know is ProxyVasemEbi. It's a specification of a low-level interface between Proxy and WebAssembly-based extension. As its name implies, it's not limited to Envoy and it's meant to allow the same extension to run inside various proxies compatible with this pack. Very ambitious goal. So the next question is why Rust? As we mentioned, you can develop WebAssembly-based extensions in any language that supports compilation into WebAssembly code. So how do you choose one of these languages? We prepared a table and we suggest you to try to make the decision to try to answer the following questions. Can you use features of this language, all features of this language and all its standard library if you're going to compile it into WebAssembly? Will you be able to reuse other libraries in the system and will the language retain its original performance characteristics if compiled into WebAssembly instead of machine code? Well, as you can see, C++ and Rust are the most obvious choices. However, don't be discouraged about AssemblyScript in tiny go. Just set your expectations right in advance. So to be fair, there are other SDKs available for Envoy. C++ SDK, AssemblyScript SDK, and even Rust SDK. So why did we decide to develop yet another one? So after looking into all these SDKs, we came up with a slightly different set of requirements. First of all, we want our extension model to be independent from rock proxy wasm EBI. As we mentioned, proxy wasm EBI tries to be compatible with all proxies out there, which means that it's not really aligned with Envoy. So when we develop HTTP filter often for Envoy, we want to call it HTTP filter and not HTTP context. When we develop HTTP filter factory, we want to call it this way instead of root context. And when we develop access logger, we want to call it this way instead of, well, there is no even equivalent in proxy wasm. Next, we want to develop all types of Envoy extensions. And we want to use the same SDK for all of them and the same model for all of them. Next, we want to have seamless migration paths for existing extensions from C++ into Rust WebAssembly. And finally, we want source code of our extensions to be unit testable. So here is a quick glimpse into Envoy SDK for Rust to give you an idea of how the source code looks like. This is Hello World HTTP filter, where we handle request headers. Next, this is factory for this HTTP filter responsible for creating new instances and initializing them with some shared state. This is an example of how to use APIs provided by Envoy host. Yeah, like in this case, stream info API that you can use to find out more information about current request that is being processed. And finally, example of a unit test where we create something called fakeEnvoy, set up listener, send request and assert response. So the primary goal of the session is to share experience. So let's work through the issues we've met while developing this SDK, the lessons we learned and some random notes on the current status of ProxyVasem API and Envoy. Error handling is a crucial to every application. This is why the very first question you might want to ask is what happens when things go wrong? So what happens when extension fails at time time? Let's try to find out by introducing intentional error like index out of bounds error. So what happens if you try to run this code? You'll find out that until very recently, Envoy was completely crashing after such a bug in extension code. So apparently, the assembly sandbox wasn't a true sandbox. Right now, Envoy behavior has changed and it doesn't crash anymore, but it doesn't work either. Once error like index out of bound happened, the filter stops, completely stops working. So if you look into the cost of this error, it happens that when panic is happening inside the assembly code, its execution gets terminated abruptly without doing stack unwinding. It means that memory on heap and logs are not released and consequently, it's no longer safe to use not only a single HTTP filter instance where the error has originally happened, but the entire wasmbm. In other words, when error happens, it affect not only a single HTTP request, but rather all requests that being processed by Envoy in parallel. So what solutions can be there? First of all, without proper support for stack unwinding that should be implemented by WebAssembly engine and your programming language, it's not possible to handle this station gracefully. Until then, until such support is implemented, avoid panics in the source code of your extension and your SDK. As an example, we've designed Envoy SDK for us to never panic. There are no calls to panic and rep-expect in our source code and all APIs must return as old type instead. The next question, so once you eliminate all the panics, the next question is, how do you handle errors? And here is an example of, once again, example of our source code. This HTTP filter and callback, which allows extension to observe and manipulate request headers. This method itself return result type instead of panicking. And inside this method, we called one of APIs provided by Envoy, get request headers. The one thing that you need to know about calls from extension into Envoy, is that all such calls can end up with an error and you need to handle them in some way. However, in most cases, those errors are not recoverable and the only logical way that you can do just to propagate it up. Rust provides very convenient syntax, I like this question mark. Operator that allows you just to propagate error up. But in the end, there must be some code somewhere that actually does error handling. So our solution to that, that we move all just error handling logic out of your extension into SDK itself. So as long as your extension returns error and not panic, we will be able to provide proper error handling logic. So for HTTP filters, we will reply with HTTP status 500, internal service error, and for network filters ideally, we would close the connection entirely. However, it's not possible right now because Proxy Watson EBI doesn't support closing connections just yet. Next about error handling, you need to know that WebAssembly doesn't support taking a stack trace. So if we take a look into our array index out of bounds error once again, and the log, and the line is logged that it leaves, it doesn't have the entire complete stack trace, but rather only a single line. And often one line is not enough to understand the entire context that led to this particular error. So this issue is not unique to Rust. It's inherent to WebAssembly itself. And in order to fix this, there is a proposal in Vasee, which is a system interface for WebAssembly. Until it gets fixed properly in the meantime, the only workaround is to enrich the context for your errors manually. For example, by wrapping them before propagating them up. Watch out for string type in your programming language. In C++ and Go, string is an arbitrary byte array. While in Rust and Assembly script, string must be UTF encoded. So the problem is that when you, with things like HTTP header values, are not required to be UTF-8 encoded. So anyway, we'll happily pass these values into your extension. And your extension will crash while trying to treat them as UTF-8 encoded. And the problem is that it's security vulnerability. Attacker can crash your envoy or make extension non-operational by sending specific request data. So as a solution in NO SDK for Rust, we're using a custom type by string to be very specific, very explicit about those places where UTF encoding is not guaranteed. Also, we'd like to ask ProxyWasn'tSpec to be more explicit about encoding of strings in its APIs. Because using by string everywhere is not a solution. It makes things more complicated when you need to pass such a value into next string-sanctuary API. Let's move on to unit testing. There are two approaches how to unit test code of your extension or SDK. One of them is to compile into WebAssembly code and run it inside a real WebAssembly host. Second option is to compile into native machine code and run it without any ties to WebAssembly. And the problem here is that to be able to run unit test inside a real WebAssembly host, you need at least VACI support which is a system interface for WebAssembly. However, there is an issue in ProxyWasn'tEbi in particular function named ML log that makes it impossible to use it with the Rust tool chain. So at the end time, we run our unit test by compiling them into native machine code and running them without any ties to WebAssembly which is not bad, just not perfect. Next, we want to notice that we've gone a great length to support unit testing of extensions. To test individual methods functions in isolation, we provide some fake API implementations like fake stats, fake HTTP client, fake stream info. On top of this, we also provide test framework that simulates and where request processing flow. It might seem too much and it probably is but it allowed us to catch some blanks in ProxyWasn'tSpec and some bugs quirks in real envoy implementation. The next things to watch out for is that ProxyWasn't to be aware of is that ProxyWasn'tSpec emits details in certain cases, leaving no other choice but to learn how to use API by looking into invoice source code instead. For example, so-called mysterious proxy get property API, which under the hood uses some ad hoc data encoding. So after reverse engineering and resources, we were finally able to offer much more friendly API for the users of our NO SDK for us. Like in this example, when you, we are now able to provide strongly typed API to get from envoy information about single request. For example, connection ID, request ID, plugin name. And finally, final thought about ergonomics of SDK. The SDK should make a straightforward how to use APIs correctly. In other words, ProxyWasn'tSpec should not be on the learning parts of every extension developer. So to make it possible, we have to be very opinionated in our SDK. We deliberately abstracted away from the low level ProxyWasn't API. To me, our API is more idiomatic to the language and more intuitive to the user. And finally, it's demo time. So I've been telling you how great Rust is, how great Envoy SDK is, but now you've probably wondering, you're like, okay, show me the code. You're just, are there any real extensions that has been developed this way? And this, you're like proud moment where we can finally share with you some work that we've been doing at that trade. So at that trade, we care a lot about enterprise use case and so-called legacy software. Those legacy systems have been around for decades and we'll probably stay around for another decades. So instead of fighting them off, we embrace them. So we've been working on something called, can be called enterprise suite, which consists of extensions and extensions that are not cloud native, they're not hot technologies, but still they can be quite useful in the price environment. As an example, it's SNTP filter and LDAP filter and mobile follow. So in the demo, I will show you SNTP filter, but if you prefer LDAP filter more, you can check it out on your own. The use case for Envoy SNTP filter is to bring visibility into how your applications make use of SNTP. So let's see how Envoy can help us with that. We'll first set up test environment. We'll run SNTP server convenient for use in development. It has some use interface we can use to make sure that messages indeed arriving at SNTP server. And we have an example jail application that will be every second, we'll be logging an error and every error will be sent by email. So let's run this application and we should see messages start arriving. So let's take a look next, what's going on the wire. You can see that SNTP is a simple text-based protocol where client is sending commands and server responds with replies. So a typical email transaction consists of client filling in fields like from and to and providing the message of email. Once SNTP client is finished sending message body, transaction is email transaction is considered committed and now it's time for server to either accept it, accept email for delivery or reject it. In this case, SNTP server accepts email message and client can now quit and close session. So let's see how Envoy can provide visibility into this flow. First we'll reconfigure application to send emails to Envoy instead of sending them directly into SNTP server. And then we'll start Envoy with our extension, preconfigured. So we're using GetEnvoy tool to start Envoy with WebAssembly-based extension in it. GetEnvoy is one of the tools, open source tools we developed at that rate to help you get started with developing WebAssembly-based Envoy extensions. So let's see metrics. Okay. Okay, application is now started. Yeah, we can see that Envoy is now collecting metrics about SNTP traffic. At high level we see number of emails that Envoy has been seen, send by client, number of email accepted by SNTP server for delivery and number of mail rejected. We can also have course-grained metrics about commands in general without going into details which exactly commands. So this is another use case, just to demo. Like any other extension, our SNTP extension supports configuration and one of the options is how many metrics to expose. So let's reconfigure and let's restart Envoy. So we activated, so we want to see more metrics, in particular, metrics for every single command we saw in this flow. So for every single command like mail, recipient data. So yeah, now you can see that we have is metrics for every single command. So this was completely successful flow where all commands were successful. Let's do something different. Let's change configuration of SNTP client to so that SNTP transactions will end up with errors and see if Envoy will be able to catch this. Yeah, you can see it is counter, counter that start increasing. It's a number of failed commands. If we now go back to Wireshark and capture traffic one more time, we can see that the overall flow has changed. So right now, you're just, email transactions are failing at a step where the SNTP client is trying to fill in two field with Envoy that value. Finally, let's take a look into the source code of Envoy SNTP filter itself. As you can see, it's indeed developed in Rust and it has a structure well, resembling a structure of native inverse tensions. We have configuration, we have stats, we have filter itself and we have its factory. And a filter, it's a network filter and we override callbacks to handle new connection, downstream data, data from SNTP client and upstream data, data from SNTP server. Since SNTP is simple text-based protocol, so we chose to implement protocol parsing completely ourselves, but in case of LDAP, we would use third-party library for parsing LDAP which is binary protocol. So that concludes our overview of Envoy SNTP extension and now it's time to Takeshi to update you on SDK for GoLang. In this part, I'm gonna talk about Go SDK for proxy wisdom and its current status and challenges in the future plans. My name is Takeshi from TechRaid. So let's begin with why Go for proxy wisdom. Go is one of the most widely used languages in the crowd-native world, right? And being able to use existing Go library or packages is a very good developer experience for wisdom extensions. And also the number of developers writing Envoy extension will increase dramatically which is really great for both for Envoy community and the Go compiler community from wisdom perspective. That's why I started working on this project. And so let's talk about compilers. There are two compilers out there which comply with Go language specification. And speaking about proxy wisdom, it's not, the official compilers cannot be used for producing proxy wisdom compatible binaries yet because we cannot control export sections and also it assumes that the running host environment is kind of Go specific JavaScript environment. So that's why we choose TinyGo as a compiler for our SDK and the TinyGo supports was the target and this is my major contribution to TinyGo and also TinyGo allow us to control import and export section. So, and also speaking about binary size, how the word binary, binary, the official compiler's binary is much larger than TinyGo as you see. So this is why I think TinyGo is the way to go speaking about proxy wisdom. So Go SDK, there is some repository named proxy wisdom Go SDK in Tetrate Labs. It started as my personal project and then moved to the Tetrate Labs and it depends on TinyGo. And also all the examples in C++ and Rust can be re-implemented. So it's usable and so please give it a try. And also it supports envoy host emulation for unit testing. So which means you can use Go test command for testing your filters or extensions without running onboard processes. This is really great for developers because we can develop our extension just like you do with the native applications. So what are the challenges we are facing? And the first is that some of the existing libraries are not supported yet. There is several reasons, but some of the system cores are not available in TinyGo or proxy wisdom C++ host. And for example, you cannot use Crypt run package or you cannot use OS.getM and also time.now. These are not available currently, but they will be supported. And also reflection packages is not fully supported by TinyGo. So you cannot use JSON package, but this is highly demanded. So I'm gonna work on this. And also Gavage collection is re-implemented in TinyGo, which is totally different from the official Gavage collection. And it uses simple conservative backend sweep and it needs to, I think it's necessary for us to assess performance impacts on proxy wisdom. Maybe we got to develop our own GCR algorithm tailored for proxy wisdom. And also, GoRoutine is almost unavailable. TinyGo uses a little VM called routine, but no schedulers runs by default in the event driven and the thread log of VMs in Envoy. So you could use GoQ order, but no one can predict when this GoRoutine will be run or things like that. So what's next? Contributing back to TinyGo and the reflection package implementation or other system core support in order to be able to use existing Go libraries. And also we need to do some GC performance analysis, have been working on this. And also maybe, as I said, we may be, we have to have custom GC for proxy wisdom. And also GoRoutine support. In order to do that, we must answer questions like how to deal with GoRoutine in a thread log of wasn't VM executed in the event driven manner. So that's all. That's all. And let's wrap up. So start wisdom today. Let's start developing wisdom extensions. There are four languages out there. And also you can contribute to the community by developing SDK for your own favorite languages. And also there is gotembo project that you can use, that you can easily get started with wisdom extension. We accumulated lessons learned. So please give it a shot. And also join our community and give us your feedbacks. Thank you for coming today. Let's get onto to a session.