 Right. Hey. Good afternoon, everybody. Welcome to listen to our talk about Unvoy Compression Subsystem Evolution. In our talk, we give the story of a long but very successful project to design the new Unvoy Compression subsystem with some great community collaboration and then discuss some of its features and future plans. But before getting started, let us introduce ourselves first. My name is Mikko Ylinen. I'm a software engineer at Intel's open source cloud software engineering team, working on device enabling for Kubernetes and cloud native applications. In this project, I've mostly been coaching Dimitri who did all the heavy lifting, but I've also hear Unvoy contributions myself, mostly on the open SSL side of things. My name is Jose Nino and I am an engineer at Lyft, where I have worked on networking related technologies for the past four years. I was part of the initial open source team with Unvoy and worked on server side technologies for the first two and a half years that I was at Lyft. And then, for the last year and a half, I've been working on a related project called Unvoy Mobile, which we'll discuss a little bit today. Next, I want to introduce Dimitri. He's not in the presentation with us today, but his contribution to the work that we are discussing today deserves a special recognition. Moving on to the story, how the Unvoy compression rework got started and what we learned from the process. But let's get started with some common terminology. And there are lots of variations used across different code bases. So we wanted to explain the ones used in this presentation and in the Unvoy code base. Within the compression subsystem, overall, we have both compressors and decompressors and they compress and decompress data, of course, naturally. And then next, let's take a look what the HTTP GZIP filter in Unvoy does. And on the request path on the left hand side the client sends a request. The filter on Unvoy looks for matching a GZIP except encoding header. And the filter deletes it and forwards the request to the upstream service. And when the filter receives the response data, it compresses the data using the GZIP algorithm. And finally then Unvoy proxies the GZIP response back to the client. So this is how it works. And basically it is limited to only response compression. This all works well if you are interested in providing compression based on just like one existing one implementation of GZIP. But then you might ask what if you want to modify that implementation, say using hardware accelerated GZIP for instance. Or what if you want to use different compression algorithm altogether like broadly compression algorithm. Or finally what if you may also want to decompress the data at the proxy layer. These were all the questions that remained and which we kind of asked ourselves. And that's another reason why we got started with the work that we did here. Well, let's look at some of the existing advantages in Unvoy. And these were kind of the things that allowed us to start thinking the redesign a more comprehensive way and design the comprehensive compression subsystem for Unvoy. First of all Unvoy is open source and in our team basically our motto is upstream forest. We wanted to propose changes to the upstream and get them accepted and with open source this is easily they should be easily doable. Second thing is Unvoy has a very clear abstract interfaces for instance for compression. We had these very basic interfaces for compressing and decompressing data and that is just provided through the data buffers. And then third and perhaps the most important point Unvoy has a very mature extension system that allows people to implement their own logic for very well defined extension points and we thought that perhaps compression libraries could be the first class extension points for Unvoy. So with that we just took the time to prepare the initial pull request to generalize the existing GC filter into a compression algorithm agnostic filter. Originally we were expecting very smooth process in upstreaming maybe with just a few comments review comments that we needed to address and then we should be done and complete with this with this contribution. But it did not go like that. Everything did not go as we expected and at some point as you can see a long time since we originally started the pull request to work simply got the stalls and we did not see a path getting the contribution accepted. And myself and Dimitri we were like pretty puzzled so why is it so hard to get this accepted especially because the contribution the implementation was meeting all the needs that many others also had asked in the past and at this point we were even like considering going with the plan B and implementing what we wanted for the hardware accelerator just using custom external filters that meets our needs. But then luckily the work was spotted by Michael and Jose from Lyft somewhere I think in February time frame this year and they had seen the work and they also had the need for Envoy decompression in Envoy mobile to keep their public APIs with compressed responses and then do decompression on the client side. So it's not always just open source that makes these contributions possible but especially for bigger features like this contribution that we had it's really about the community and how the community collaborates together to kind of complete the last mile and getting things accepted upstream. And what we learned from all this during the way to while getting this accepted and merged in upstream so one observation is that sometimes it's best if you just share the big picture the kind of the the design idea in a shared document to make the conversation possible and then eventually create the kind of consensus about the changes and the way forward. Before investing a lot of time in working on POCs and then scratching the work and starting all over. But this document it also helps to build the community behind the work which is super important as we just discussed previously. And one other thing we also learned that sometimes all these open source contributions they need a lot of patience and that don't give up too easily if there's some delay in getting your changes reviewed and accepted. Good. So next I think we have Jose going to talk about generalizing the ANVOI on more compression. All right thank you Miko. So now that we have that background context of what existed in the ANVOI compression space with the specific G-SIP filter. Second what were some of the use cases that were not possible with that previous implementation. And then what advantages existed that could make a general compression system possible in ANVOI and how all that work was possible thanks to not only a cross-company collaboration between us at Lyft and the folks at Intel but really a cross-continent collaboration with people in the United States and Finland. So let's dive into what we actually did in generalizing this compression system. So the first obvious thing when looking at the G-SIP filter is that while this is what it looks from the outside if you look under the hood what you have is a filter in two parts the HTTP filter part and then the compressor part. So the if we zoom in here the HTTP filter part deals with the HTTP level concerns like looking for the accepting coding header like Miko was saying or replacing the data frames with the compressed data and then the filter itself uses the G-SIP compressor which it itself is the one that is compressing the data buffers using the G-SIP algorithm. So going back to the existing advantages what became evident when looking at the filter this way is that there was nothing specific in the filter that required the compressor to behave in a certain way to use G-SIP to compress because it used this very simple interface that we had seen before a compressor only has one function it compresses. So there was a very clear delineation of where we could divide the filter and then plug in a compressor library regardless of the compression algorithm that it used. So now that we had this generic compressor working within the generic HTTP compressor filter we could raise the compressor itself to be a first class extension in Anboid and so this is done using the classes that we see here where we have a class that derives from the config type factory class and then every particular implementation of a compressor library derives from that in turn. So now our G-SIP compressor or perhaps our broadly compressor in the future derived from the name compressor library config class and then the compressor filter uses this compressor factory class the generic compressor factory class to create compressors with which to actually compress the data. In other words we achieved the first of our goals we are now able to have any number of different compressor schemes in Anboid without each having to implement an HTTP filter themselves. We can just plug in any of them into the generic compressor filter so whether it is a G-SIP compressor or a broadly compressor or like our friends at Intel hardware accelerated G-SIP. All right so on to our second goal what about a full compression system one that can both compress and decompress data. Well that was easier now that we had a generic implementation of a compressor filter. We used the same principle of abstraction using the generic decompressor class and created a generic decompressor filter that used a possible generic decompressor thus creating a fully pluggable HTTP decompressor filter. So parallel to the compressors as first class extension points in Anboid we created decompressors as first class extension points in Anboid thus we paved the way for having not only a G-SIP decompressor but also having any number of decompressors to parallel the compressors that we were adding. So now we have a complete generic end-to-end compression solution powered by Anboid where service B can send a response and then the HTTP compressor filter can compress that response which arrives to service A and in turn its HTTP decompressor filter can decompress the response before proxying it to the service itself. So as you can see compression is completely done in the network layer without having to involve any product engineers, owners of service A or service B. All right so this is what the Anboid compression extension tree looks like. The common directory implements the generic compressor and decompressor factory base classes that all compressors and decompressors strive from and then each particular compression scheme has its own tree where they have a compressor and a decompressor and potentially a common base class that derive from some scheme specific functionality. So in Anboid currently we have the G-SIP compressor and decompressor but as we have been saying the beauty of this first class extension is that any number of compression schemes can be added whether added upstream to the main Anboid repo or in your private build of Anboid. All right so that's a whirlwind tour of how we generalize decompression but most exciting we want to present some of the case studies that were enabled thanks to the to this generalized compression scheme. And the first one, oh sorry Mikko, go ahead. Yeah so I just, thanks Jose. So I was about to say that our first case study talks about implementing hardware accelerated compression for Anboid. So some are a little bit about the rationale for why doing hardware accelerated compression. Typically hardware acceleration can bring several benefits to your application for instance as you know being very time consuming process it might make sense to offload all this compression processing to a dedicated co-processor. Sometimes even like hardware that is completely optimized for this particular task. And then the added benefit with that is that then you can pre CPU cycles for something more important that runs on the main CPU and then the co-processor, the accelerated co-processor doing all the work. In our case we have implemented a reference implementation using this Anboid compression subsystem for an Intel Quick Assist technology card. It's gonna dedicate card optimized for crypto and compression processing. This implementation adds QAT co-compressor. We have it implemented in an external repository as Jose mentioned. It is also with the Anboid extension mechanism possible to add your custom implementations easily living in your own external repositories. The QAT compressor we have implemented uses a framework what we have called QAT zip and then the QAT zip then directly uses the QAT hardware abstraction layer and the driver and sending out all the compression requests to the dedicated co-processor. We have it right now we have it only for compressors. Basically we are able to compress the data and what we are working on right now is that we're just adding looking to add almost there adding the decompressor piece for the hardware acceleration as well. Based on our experiments this piece of compression in addition to the fact that we are also able to free CPU circles for something else. All right so thank you for that case study. There are some published results and we will share the link with the presentation slides. So now let's talk about the Anboid Mobile use case. I want to first give a quick overview of the goal of the Anboid Mobile project and what it is as a baseline context for this case study. This diagram shows an increasingly common topology in network distributed systems these days. I'm sure many of you at your companies have set up similar to this where Anboid is deployed as a universal network primitive where it deals with the majority if not all the network traffic in a company's architecture. However my team at Lyft recognized that we had left an important hop outside the ecosystem because traditionally we've treated mobile clients as independent from the backend infrastructure and we have built unique solutions to what was assumed to be a unique space. So we identified a technology gap because in spite of all the work that we had done server side we saw that three nines of reliability at the server side is meaningless if the user of a mobile client is only able to complete the desired product flows a fraction of the time. So what we want client side are the same guarantees that we had server side. So that is what we created Anboid Mobile. We propose that we don't need to treat the mobile clients any different at least from the network perspective as we do with the server side infrastructure. So this is what my this is why my team built this over the last year. We have built bindings into Anboid that allow us to run the code base as a native library in a mobile client application and so we have great documentation and a ton of material about the project so if you're interested in the project in itself please feel free to take a look. It's also all open source but today we are here to talk about compression and in particular show you with how with Anboid Mobile in the client and with Anboid at the edge of our infrastructure and with this new generic compression subsystem we were able to do some things that would have been too resource intensive in the past. So compression at the edge is particularly important because payload sizes can lead to performance degradation under adverse network conditions which are usually experienced by mobile phones making public API requests to your infrastructure. So as I mentioned before lift compresses all responses at the edge and we wanted to have at least the same behavior when we migrated off our old client stacks to Anboid Mobile. So in the past before Anboid Mobile implementing compression at the edge required three different code bases one for the edge proxy one for the android client and one for the iOS client. So this might be okay if we just wanted to put you know one compression algorithm in place and just leave it be. But my team started asking questions very similar to our colleagues at Intel what if we wanted to squeeze better compression and and does better performance with different compression schemes what about bi-directional compression where we not only compress responses but also requests. And this questions quickly become intractable if for every variation that we want to test we need to implement the same code three times server android and iOS. And that is really one of the main reasons we embarked on the Anboid Mobile project to begin with because by using the same code base everywhere with Anboid Mobile in the client and Anboid at the edge we are able to have only one implementation for every new technology we want to experiment with affording us on parallel consistency and extensibility at a fraction of the engineering hours required in the past. So in other words now we do have truly a true universal network primitive that we can experiment with. And why we became so interested in collaborating with our colleagues at Intel in this compression subsystem. All right so let's see how we brought compression back to the edge with only one implementation and without involving either our client or server engineers. So before we go into compression let's look at the request flow using Anboid Mobile. Very similar to Anboid on the server Anboid Mobile proxies request from the client to the edge proxy upstream and then the edge proxies that received that request further proxy that to the upstream service. Similarly on the response path the edge receives a response sent to the client using Anboid Mobile which in turn surfaces that to the application via callbacks in the library. All right so the first step in enabling compression is installing the new compressor filter with a particular compressor library on the edge nodes. At this point nothing changes in the client. The client sends a request which Anboid Mobile proxies to the edge but now the edges compressor filter searches for the accepting coding header which allows the filter to know if it should compress the response for this request. But the header is missing so it simply forwards the request and then when the response comes back it does nothing with it and passes it back to Anboid Mobile. But now we can install the decompressor filter in Anboid Mobile and here I want to highlight how easy that was. We didn't have a filter before and then boom we installed the filter so easy. Well jokes aside this is easy because Anboid Mobile is Anboid so we can install the same filters that we use in Anboid on the server including the decompressor filter. All right so now that we have the decompressor the client sends a request and the decompressor adds the accepting coding header. So when the compressor at the edge searches for that header it finds it so then the decompressor strips the header and lets the Anboid proxy the request. But thanks to the accepting coding header the compressor is now primed to compress the response because it knows that the client is going to be able to handle this compressed response. So when the response comes back the compressor compresses the response and in turn the decompressor filter decompresses it before sending it back to the application. All without the client or the server engineer being involved at all. Nice so we were really excited with this first zero touch implementation and we started thinking what else could we do at the edge with Anboid Mobile in the client and with Anboid in the server. So potentially we could have bi-directional compression so instead of this setup where only the response gets compressed to be sent across the edge we could have a more complicated setup but it is a setup where both the client and the server have both a compressor and a decompressor filter allowing both requests and responses to be sent compressed across the wire. But we could you know do more than that not only are we allowed to do bi-directional compression now but we could also run experiments with different compression schemes and fine tune our compression settings to achieve less degradation at the edge and ultimately higher success with our public APIs. So this is the Anboid Mobile case study and this very very flexible compression system has really enabled us to pursue a lot of avenues that would have been intractable due to the engineering hours required beforehand. So those two were the big use cases that drove compression forward with a cross-company cross-continent collaboration to thank for. However we have many things planned for the compression space in Anboid. First even though the new compressor filter has been around since the 1.15 release the old g-sip filter is still there we plan to remove it from Anboid by the next release 1.17. Second is broadly support which is currently being worked on by Dimitri at Intel. Third is having bi-directional compressor bi-directional decompressor has already been implemented by myself but to have a fully bi-directional end-to-end compression system we're missing request compression this is very easy to add in the compressor filter in case anyone in the audience is interested. Having this pluggable compression libraries enables us to apply compression to other parts of Anboid your ideas are welcome on which parts exactly can benefit from it. Lastly it is worth mentioning that the performance of g-sip compression in Anboid still has room for improvement for example g-sip compression in NGNX works about 25 percent faster than in Anboid when configured similarly. These are just some of the items we have planned in the compression roadmap but we can't wait for seeing what other interests the community has for this new subsystem. That's all we had for today. Thanks for listening and watching us and looking forward to see you in the community.