 Hello, and thank you for the opportunity to talk to you today. Dimitri and I will share our recent work on shielding Linux application containers in an untrusted cloud. Recently, these untrusted clouds have seen a surge in applications computing on sensitive and private data. And when sensitive and private data is involved, the risks of their exposure is even higher. Before I start with the presentation, it's important to note this legal disclaimer. Talk is about the data protection gap that exists in untrusted clouds. Sensitive data is encrypted while it is in transit and in rest, but it is not while it is in use. As a result, sensitive data may be exposed to the cloud's infrastructure system software while it is in use. This creates a strong desire to compute on encrypted data. This new form of computing is called confidential computing. It protects your data against data confidentiality and integrity violations while protecting against replays. It mainly relies on encryption of the main memory. The form of computing relies on trusted execution environments, or TEs. A TE typically is implemented in hardware, in this case shown here in the CPU. The hardware is trusted to isolate the TE from the remaining system. It does so by mapping the TE's memory in a secure portion, hence denying access from any non-TE execution. In addition, the CPU attests the state of the TE to third parties that ask for an attestation. This allows an external verifier to provision secrets to the TE. Today's TE of choice is Intel's SGX software guard extensions. But before looking at SGX, let's consider the attack surface of a regular application. The attack surface typically includes the entire system, software, OS, and hardware. Intel SGX changes this attack surface. It is implemented as an ISA extension that allows an application to be split into an untrusted component and a trusted enclave. These trusted enclaves are shown here as screen squares. After the split, the hardware ensures that neither the untrusted component of the application or any of the system software can access the trusted enclave's memory. In the SGX model, an untrusted application builds the trusted enclave given its initial state, including some well-defined entry points. Before provisioning secrets to such an untrusted enclave, a third party verifies an attestation evidence that is produced by the hardware that typically includes the memory fingerprint, as well as the defined entry points. This allows the third party to now trust the enclave and ship the private keys, for example. Location into an untrusted component and untrusted enclave ensures a very small TCB and strong security guarantees. It increases the effort to build and maintain such a split. In an untrusted cloud environment, though, we're using SGX not necessarily to split the application, but rather to shield against the cloud's system software from gaining access to our valuable, sensitive data of our application. This results in a design where the entire application runs inside a trusted enclave shielded against access from the system software. Unfortunately, the restrictive programming model of Intel SGX denies system calls from an application. To overcome this limitation, either requires modifying the application or using a library OS. A library OS provides a generic Linux system call API to the application, while securely translating these system calls that the application does through the SGX boundary into the host OS. The library OS we've been involved with is called Graphene, as it's been adopted to Intel SGX, providing the interface to run unmodified Linux applications within Intel SGX enclaves. Let me introduce Dimitri, who is one of the core maintainers of Graphene. He will provide the details on Graphene before we apply Graphene to application containers. Thank you, Anya, for the kind introduction. As already mentioned, Graphene is a library operating system that allows to run an unmodified application inside an SGX enclave. Graphene is an open source community supported project, and it is currently in the process of transitioning from a research group of concept to production-ready software. Graphene serves as a minimal boot loader and an emulation layer, seated between the application and the underlying host OS. You can think of Graphene as a minimal re-implementation of a Linux kernel, which strives to resolve most application requests in the form of system calls and open-source files in user space, and occasionally resourcing to host OS provided resources like network IO, file system IO, thread scheduling, process creation, and so on. Graphene additionally implements extensive checks on system call arguments and return values, helping to protect against maliciously crafted values. And by design, Graphene consists of two layers. The stateful library OS layer, that intercepts system call requests from the user application, verifies inputs, processes with requests, verifies outputs, updates the internal state of Graphene, and then gives the results back to the application. And the stateless platform adaptation layer, or PAL for short, which is invoked by the Libos layer only when the host OS must be involved in request processing. As you can see, Graphene currently supports more than 300 system calls, maps them to around 40 PAL ABI functions, and PAL uses around 50 actual system calls to communicate with the host OS. Now, this picture shows the known SGX version of Graphene. Let us move to the SGX version. When Graphene is running in the SGX mode, it loads itself and the application inside of the SGX enclave. The major parts of Graphene move inside the enclave, in particular the Libos layer and most of the PAL. However, PAL is now divided into two separate binaries, the untrusted PAL and the trusted one. The untrusted PAL is used to load and initialize the SGX enclave in the wrapper process and to issue system calls to the host OS on behalf of the enclave. It also serves as glue code to enter enclave mode and then exit it. Similarly to other layers, the interface between the trusted PAL executing inside the enclave and the untrusted PAL executing outside is kept minimal with around 40 so-called SGX outcalls or occalls in total. Graphene takes additional precautions to harden this boundary between the trusted enclave and the untrusted environment. As typical in security critical environments, the application running inside Graphene and an SGX enclave must be accompanied by a security manifest. In fact, it is with security manifest and not the application binary that is given to Graphene as input. The best security manifest is a plain text file in a simple format. Here is a small example. First, the manifest must specify the actual executable to load and run, in this case, the Python free executable. Next, the manifest may overwrite the required environment variables. The manifest should also specify a subset of host OS directories to be mounted so that only they are visible to the enclave application, possibly under a different name. As shown in this example, here the G-Lib-C libraries are silently replaced with Graphene-patched G-Lib-C libraries. Next, the manifest contains some SGX architecture-specific variables like the maximum enclave size and the maximum number of simultaneous enclave threads. These are the limitations of SGX version one. Finally, this example shows how files to be consumed by the enclave must be marked as trusted. In this case, Graphene calculates the secure hashes during build time, appends these hashes to the manifest and then during runtime, Graphene will verify that the files were not modified. There are several more manifest options to Graphene to enable additional functionality. So this manifest is the single most important file when porting applications to Graphene and into SGX. Ultimately, the security and correct functioning of the application depends on how its manifest file is written. A very important aspect of any trusted execution environment is attestation. How can a remote user verify that her application actually runs in a correct up-to-date TE and that the code that runs inside the TE is the actual user application? Moreover, how can the remote user provision any secret inputs to the application remotely and in a secure fashion and then get the results of execution also in a secure fashion back? Into SGX, similarly to other TEs, provides a way for the SGX enclave to attest itself to the remote user. This way, the user gains trust in the SGX enclave running in an untrusted environment, ship the application code and data, and be sure that the results were executed inside a genuine SGX enclave. We will show a complete, most likely simplified flow of remote attestation. Every SGX enabled processor contains a key burnt into its fuses. The processor uses a secret key to encrypt all memory traffic live in the CPU package. So all manufactured SGX processors are listed in the web service maintained by Intel called Intel Provisioning Certification Service. At deployment time, the new SGX machine is supposed to be registered with this service. There is a special SGX enclave called Provisioning Certification Enclave developed by Intel. This enclave is used to connect to the Intel Provisioning Certification Service, send the ID of the SGX processor and other identifying information, and receive the normal X509 certificate for this machine signed with the Intel public key. This certificate is used to prove that the SGX processor is genuine and up to date. Now the user wants to run an enclaveized application with a security sensitive workload. The user possesses a trusted machine, not needed to be SGX enabled. And the first thing the user must do before outsourcing your secure workload is to contact the Intel Provisioning Certification Service to retrieve the required certificates, certificate verification lists and other SGX identifying information. This information can be cached in a local database so that the user doesn't have to establish internet connection to the Provisioning Certification Service every time she runs a workload on a remote untrusted machine. The user also installs the quote verification software that runs all the required checks during the remote attestation. Now the user ships the application binary and the graphene runtime to the remote untrusted machine. Note that no sensitive inputs are shipped to the remote machine just yet. So the remote machine starts graphene which starts the application binary inside of SGX enclave. The user also starts a secret provider application on your own machine. This application will connect to the remote machine, establish a secure communication channel, verify that the remote machine is a genuine SGX enabled machine and that the application and graphene run in a genuine SGX enclave. And after that release secret inputs to this remote machine. So secret inputs are represented as a key ring on this diagram. Note that since they use the shipped graphene and binary to the remote machine, the user knows their corresponding expected measurements. Intel SGX software infrastructure has another special enclave called Quoting Enclave. This enclave with reference implementation written by Intel is used to create a special structure called the SGX quote. This SGX quote contains all the information about the particular enclave including its hardware generate measurements. You can think of SGX quote as a second level certificate that proves the authenticity of the enclave. Graphene contains two libraries that transparently add remote registration to the application. The RATLS library augments normal SSL TLS sessions with an SGX specific handshake callback. The secret provisioning library builds on top of RATLS and runs before the application. First, the secret provisioning library instructs RATLS to collect all SGX related certificates and create a normal X509 certificate chain out of them. Then a secret provisioning library establishes a secure TLS channel with the user's machine. During TLS handshake, RATLS library installs a special verification callback that receives the certificate chain with free certificates, the LEAF RATLS certificate, the second level SGX quote, identifying the enclave and the top level certificate identifying the SGX platform. Since the free certificates in the chain contain all SGX related information, the user compares the retrieved measurements against the expected ones and decides whether to trust the remote enclave or not. If the user side verification finishes successfully, the user then gains trust in the remote enclave and now using the same secret provisioning library sends the secret inputs into the SGX enclave. After this point, the secrets are injected in the application and the application finally starts its execution. The same secure channel may be reused to securely send applications results back to the user. Graphin also implements the feature of protected files. Since the user application runs on an untrusted machine, all input files as well as produced output files must be at least integrity protected. And if user files contain sensitive information, they must also be encrypted. With a protected file system implemented in Graphin, all required files to run the workload are created by the user on her trusted premises first. The files are then encrypted with a symmetric key. After that, the files may be shipped to the untrusted machine together with the application binary and Graphin. After the SGX enclave is created, Graphin needs to obtain the key used to encrypt the files. As explained previously, this key provisioning can be accomplished via secret provisioning with remote attestation. So once the key is securely provisioned inside the enclave, protected affairs subsystem of Graphin uses this key to decrypt the input files and run the workload. After the enclave is finished with a secure computation, the application typically produces an output file. This file is also transparently encrypted by the protected affairs with the same encryption key. Finally, the output file may be moved to the remote user where it will be decrypted. Graphin implements protected files as transparently encrypted and integrity protected files. The specific files or whole directories to be treated as protected must be specified in the manifest. The encryption format used in the protected affairs implementation is borrowed from a similar feature of Intel SGX SDK. We would like to stress that Graphin is an open source project maintained by a dedicated community with contributors from several companies and universities. We successfully enabled several widely used applications in Graphin across several domains and hope to support more use cases as we grow. And at this point, I would like to give the mic back to Anya. Thank you for the introduction of Graphin. We will now switch gears and look at how to use Graphin for application containers. Containers have become the de facto deployment model for the cloud besides virtual machines. They provide a small memory footprint and a fast startup time. We cannot directly apply Graphin to container frameworks. We first have to overcome two obstacles. The first one is that Graphin does not provide an interface to create images or containers. Second, while providing handcrafted manifests as Dimitri explained is possible, it's a huge burden on the application developer or engineer. To encourage the usage of our container integration, we have to simplify the manifest drastically. For this purpose, we have created Graphin shielded containers or GSC. It takes as an input a regular Docker image with an application as it dependencies and optionally a minimal application manifest detailing its memory and parallelism requirements. These two are given to GSC to build a graphonized Docker image. This image includes the application, the Graphin runtime and a generated manifest file based on the user's inputs as well as some configurations of the Docker image. All SGX applications have to be signed by the developer's key. GSC signs applications in the runtime and the manifest in a second step to ensure that the user's signing key is never part of the persistent image. The signed image can then be started via a regular Docker run command which starts the Graphin runtime, the runtime loads the manifest which then starts building the enclave according to the manifest specification and starts running the application. On the previous slide, I simplified the manifest file generation substantially. Let me explain how we semi-automatically generate a manifest file from a Docker image. We base our automation on the observation that a container defines the minimal runtime environment of an application. This means that files included inside the container are dependent libraries, required script files, static inputs or a particular configuration argument. Using this observation, we can generate a majority of the fields that constitute a manifest file. I've listed here only a few examples and we'll go through them one by one. The executable and its arguments are specified typically inside the Docker's image configuration via an entry point or a command. We repurpose these configuration fields to generate the executable and its arguments in the manifest file. In addition, each manifest file includes a description of the file system layout. Since we consider the entire file system of the container to be a dependency of the application that runs, we simply set the file system to the root of the container. Lastly, each application will open files in particular any dependent shared libraries. Without specifying any dependencies as trusted files, Graphene will simply deny access to these files. This is to ensure that any open file has a prerecorded SHA-256 hash and ensuring its integrity and hence that nobody tampered with it. We overestimate this set of files to be all files in the container. Using GSC, we have developed three deployment scenarios. In the first scenario, the client builds the Grapheneized Docker image on-premise and then ships it to the CSP's registry. Afterwards, he can rely on Docker's run command infrastructure to create Grapheneized Docker image images as usual. In the second scenario, the client uploads the application Docker image. Subsequently, she graphinizes the image via a CSP's infrastructure interface. And again, afterwards may use a Docker run command to deploy a Grapheneized Docker image. In this third scenario, a client uploads his image again to the cloud's registry. But instead, the image is not translated via an API called to the CSP. The client directly runs the image specifying a runtime. This runtime uses GSC to translate a regular image into a Grapheneized Docker image on the fly. This approach lowers the required changes in the client's software to a minimal change to run the correct runtime. Before I show you a demo of GSC, I'd like to provide an end-to-end use case that makes use of attestation, protectFS, and Graphene Shielded containers. In this use case, we use GSC to package and deploy Graphene with an application, in particular, a PyTorch object detection. We assume the model, the input, and the output of the PyTorch object detection to be secrets. Therefore, the runtime has to first obtain a provision key to access the model and the input. We do so by generating an attestation and sending it to the verifier. After validating the attestation, the verifier release the provision key to the runtime. We then use the provision keys to access the model and the input from the protected file system. This allows Graphene to transparently decrypt the inputs without a single application change. PyTorch then detects the objects within the input and generates an output or a decision. The decision is then stored securely within the encrypted file system. This use case combined various techniques to protect the confidentiality and integrity of the model, the input data, such as images, and the inferred output from the model. So now I will show a demonstration of Graphene Shielded containers. We are graphonizing the public Dockerop image of Python. For the purpose of that, we're specifying the GSC build command. In this demonstration, we're using insecure arguments, but technically you can specify trusted arguments as well. We're graphonizing the Python image and we're specifying a Python manifest file. In this manifest, we're adding a single line to allow file creations, which is required for PyCache to work. We're then following the GSC Docker build process. It first builds Graphene and then copies over into the application image the required runtime arguments. We are then copying over the Docker run command that specifies the required SGX devices. We're specifying the image, GSC-Python, which is the graphonized version. In addition to a Python arguments, again, those are insecure and we're basically printing HelloLinuxSecurity Summit from an enclave. We then start the container. We specify or we see that the token was generated, the fingerprint of the enclave and the fingerprint of the signer has shown and then Graphene started with a manifest that was produced by GSC and then secure arguments again. You can use trusted arguments if that's required for your production environment. Now we should wait a couple of seconds and see that HelloLinuxSecuritySummit was successfully printed. I'd like to highlight the current and future directions of Graphene. Graphene had its first major release last year and we're currently in the process of finishing our next release. This release includes remote illustration, exit lists, system calls protected for FS, Graphene shielded containers and secure multi-process support using encrypted IPC. In addition, Graphene has been accepted to the Confidential Computing Consortium, which is part of the Linux Foundation. Thank you very much. I would like to invite you to ask questions and I've put up a couple of links in case you want to know more about Graphene. Thank you. Okay. Hello. Well, thank you for listening to the talk. I'm Anjo and beside me is Dimitri. We'll go through your questions. I will read the questions and then we will tag team on answering them. So I think the first elephant we need to address in the room is that, and there was at least one question that I saw immediately, is that what do we do about recent flaws or basically security vulnerabilities that were exposed to Intel SGX? And of course, we and Intel are very concerned about the security issues, but both of us are not the experts either in these attacks and the mitigations. So I would ask you to look at the security advisories that Intel gives out to address those issues. Okay. So let's start with the first question. I think it's very interesting. What are the open steps that still remain to transform it from a research project to a production product? Dimitri, do you want to take this one? Yes, sure, thanks. So Graphene is not yet production ready. It's in the process of moving from a research artifact to an actual product. One important thing to mention is on our GitHub page where we do all our Graphene development. In the issues page, we have some pint issues. In particular, there are production blockers, they list all the things that we need to do prior to going into production. Some of the things that we still need to merge are protected file system. So all of the features that we mentioned, they are almost ready and they will be there in several weeks in the mainline Graphene. Like the protected file system, the Graphene shielded containers, the integration with Docker. And we also need to add more examples and more end-to-end examples for Graphene. And we hope to finish this work in the next couple of months. But you can always check the current status of Graphene productization on our GitHub page. Okay, next question. How does Graphene a certain detect malicious system calls? I think that's also something Dimitri can answer very well. Yes. So first of all, Graphene is a kind of one-way sandbox. And if I understand the question right, it's more about Yago attacks. So first thing to understand is that when you load an application inside Graphene, inside an SGX Enclave, the application can do whatever it wants, technically. So if the application is buggy or somehow malicious and it creates, it issues malicious system call requests to a host operating system, then this is something that Graphene and indeed SGX does not protect at all. So the assumption is that the application that you run inside Graphene and inside of an Enclave is bug-free and correct. And you check it via remote registration. So, and then the reason our problem, what happens if the application issues a correct system call, but the host operating system is malicious and it returns some incorrect return values. And this is what the Graphene library operating system does. It kind of shadows all system resources. It manages its own memory via maze. It verifies all network connections and all files created and used by Graphene. So yes, we have this additional layer of protection against malicious return values from system calls. That's kind of the whole purpose of the library OS layer. Okay, I was looking for more questions. Next question is, can we use SGX or a library as if Intel ME is disabled? And I think the simple answer is yes, but there are a couple of features like the monotonic counters which are supported only through the ME. And if whatever application you're building is using those features, you would not be able to make use of them. As far as I know, Graphene does not rely on any of those, but I'm sure that Dimitri has maybe has another comment. Yeah, this is correct. Graphene does not rely on Intel ME in any way and none of the additional libraries and like SGX infrastructure that Graphene relies on, it does not use Intel ME. Okay, next question would be, can Graphene load executables from the network or more specifically, why are pass specified with file colon slash in the, I think this is related to the manifest. I think Dimitri, do you want to answer this one as well? Yes, sure. So yes, this is a manifest syntax. All paths are prefixed with either file or def at the moment. The idea is that Graphene works, the Graphene manifest specifies URIs. So universal resource like kind of descriptors, identifiers. So technically it can be not only a regular file on your file system, it can also be a URL for example, from the network or a device. So network URLs are not supported yet, but we have files and devices supported and devices, it's another story. It's not a complete support for now, but we're working on this. And I guess I should say that 99% of the time, currently you just need to use this file prefix and use regular files. I'm not sure Dimitri, do you want to say anything else? I guess the answer is just no. Just no. Yeah, this is something we're actually looking for towards the path of production or production readiness. This question we can ignore. What are the most common obstacles when porting applications to Graphene? That's a very interesting question. When I joined the project, Graphene was already in a relatively good state. Since then I personally have not really found big obstacles. I think one maybe to mention is that some runtime languages like Go, they require a different memory layout or more freedomness in the memory layout and have specific addresses in their runtime environment, at which point Enclave sizes just become too big to be able to allocate such an address, which is I guess a current limitation of the implementation of Graphene. Since it does not have this dynamic memory allocation that newer SGX versions allow. I don't know, Dimitri, do you want to make any more specific comments? A couple more comments, yes. First on our GitHub page again, we have the examples directory, where we have a kind of curated set of applications so you can check this list. And it also explains these are good examples to understand how Graphene works and how to port applications to Graphene. And one general observation is that some applications that kind of abuse multi-processing paradigm can be either very slow because there is no such thing as a copy on write in SGX. So whenever you do a fork or a clone, you have to copy the whole SGX Enclave memory state from one Graphene instance to another. So you can see significant performance overheads when you use something like Bash scripts. And another interesting point is that Graphene has to encrypt all communication. For example, on Pypes and FIFOS and Unix domain sockets. So in some applications, for example, in Erlang, they actually use like several processes trying to write on one pipe. And of course, like TLS encryption, which we use to encrypt pipes cannot work on many writers. So this is another interesting observation. Luckily, that was only one Erlang example. We haven't seen any other applications that abuse the kind of multi-processing paradigm. Other than that, it's just a matter of box and some unimplemented functionality in Graphene. But we're slowly fixing them one by one. Next question would be, how does it integrate with Kubernetes? So most of these Docker images, they would integrate nicely with Kubernetes. One of the issues that you have with Kubernetes currently is that it does not provide the SGX devices to the container itself. As you saw in my Docker Run command, I actually specified the devices to be handed to the container. Otherwise, the SGX devices are not available. And that is a requirement that you have to add via an extension, a device extension currently, which is not upstreamed. So if you wanted to run anything like this in a Kubernetes cluster, you would have to get an out-of-tree device extension. I know that there are people working on this to improve upon this situation, but I think that's the situation right now. So we'll have to wait for this to resolve. Next question is, how does Graphene work with limitations that can be used in containers like resource limitations, device limitations, C groups, syscall limitations, we are second or other limitations? So I'm not 100% sure that I fully understand the question. Graphene itself only sees what is available to that container. So if you have external resource limitations like using less CPU cores or memory limitations, then Graphene would basically take that from whatever the container saw. I'm, syscall limitations would apply similarly, right? If you have a second filter for your Docker container, then that applies to Graphene itself. So yeah, since we're running sort of inside the container, we're not bypassing any of these limitations. Do you want to add, I don't know. I think we have not particularly played around with these things, so it's... Yeah, I just want to stress again that Graphene and SGX enclaves in general, they work as a one-way sandbox. So they protect only the enclave that from the underlying host operating system. And whatever the host provides or restricts that will be seen by the application running inside Graphene inside the enclave. So if, for example, the host operating system restricts system call or some arguments of system calls via sec.com, then Graphene will simply see, okay, this system call is not available in the system. Thanks. Do you have any numbers of performance overhead when running Grapheneized containers? Any other restrictions like image size? So I think I don't have any performance overheads out of the top of my head, especially for Grapheneized containers. I am not sure we can cite something for SGX. Restrictions for image size, there is not really one. I mean, the performance highly depends on your working set size, because existing machines today have a limitation on available cached memory, secure cached memory. So the moment you have bigger, let's say machine learning models, of course the performance is going to decrease. Yes. I don't know, Dimitri, do you want to say anything about performance overheads with Graphene? I think it's regular SGX based overheads mainly. Exactly, yeah. The main bottleneck will be the SGX performance and we cannot comment on the performance right now. What mitigations does Graphene provide against side channels? Maybe that's a question we can, and attack sandbox application. Yeah, okay. So I think we're not doing anything specific in Graphene against these side channels. So again, I would refer you to the Intel advisories. Against these side channels. How does Graphene compare to NRX? Dimitri, any comments on comparisons to NRX? To be honest, we didn't look into NRX. We are not aware of its current state. So no comments, I guess. Yes, but so I think one part to mention is that NRX works on wasm binaries, right? So there is a huge difference in legacy compatibility, I guess, because you have to recompile to wasm binaries. Other than that, I'm not aware of their feature set as well. SGX hasn't yet landed in Linux upstream. Is this at all being done using out-of-tree drivers? I think the simple answer is yes. That's currently the only way to use Graphene, either the client driver or the decap driver. Yes, and our hope, of course, is that this driver will eventually be upstreamed. Yes, any more comments from your side, Dimitri? I don't think, okay, I guess. No, no comments, yes. All the work is done with out-of-tree drivers. Yes, is it possible to publish in Docker Hub a secure image created by Graphene? So in its current form, such a secure image, what you're saying would only protect the integrity, not necessarily the confidentiality of the image. So you may or may not want to do that. But you could publish whatever GSC produces on Docker Hub. There is no restriction. I'm not doing so. If your host is compromised in any way, like the colonel is compromised or Docker has a security hole, would Graphene with SGX be able to protect the data? I think that's sort of an interesting question because it relates to the SGX threat model. The basic idea is that the data inside your enclave would be protected against accesses by outside components, right? That's the entire purpose of Intel SGX. Yes, Dimitri, do you want to say additional comments on that? Well, the whole idea of Intel SGX and Graphene is to protect from the untrusted, possibly malicious operating system, the kernel, and any other compromised applications running on the same system. Yes, I'm not sure about time. I think we're probably running out of time. So we will be available on Slack, definitely afterwards, but maybe... There is one minute question left. Yeah, one last question. Oh, maybe what all OSS container software has been tested on Graphene? RedisCache, Nginx, Memcache, .NET Core runtime. So with GSC in particular, we have tested Nginx Python public images. They work out of the box. We have tested the PyTorch example from Graphene. The NumPy example, that's for GSC. The list for Graphene is much, much bigger. And I guess the best thing is to go to the examples folder in the GitHub repo. Those are the ones that are actively tested most of the time. Yes, I don't know, Dimitri, do we want to highlight? I mean, out of the list that you specify, Redis works, Nginx works, Memcache works, .NET Core runtime, I'm not sure we have tried. And I believe that it's only for Windows. I'm not sure. Graphene is currently running only on Linux. Only on Nginx, yeah. Do you want to mention any other application, maybe? One of the advantages of Graphene is that it works with multi-processing applications, most of them. So you can easily run Python scripts, Bash scripts, GCC, for example, which does a lot of multi-processing stuff underneath. So we also tested these kinds of applications, they work. Okay, yes. I'm not sure if we're going to be thrown out. Otherwise we'll just continue answering questions. No, I think we're done. We're done? Oh yeah, no, time to wrap up. Okay, yeah, so well, thanks again. We will answer questions that you have on Slack. I will probably post the remaining questions there as well and a couple of answers. Just ping us on Slack if you have anything else. Yeah, so thanks again. Yeah, thank you.