 Konichiwa, and thanks for joining me. My name is Stefano Cetola, and I work for the Linux Foundation. And I'm gonna be giving you a high level overview of several trusted execution environment technologies. We're going to be talking about Intel SGX, ARM Trust Zone, and RISC-5 Physical Memory Protection. I actually wanna start by mentioning the fact that there's other talks about trusted execution environments going on. In fact, Tsukimoto-san and Suzake-san are going to be giving a talk on TEEP, Trusted Execution Environment Provisioning. This is a protocol which is being worked on by the IETF, and the talk starts in just a few hours, assuming that the schedule stays the same. So please go check that out. Before we get started, I figured I'd added this slide as a brief legal disclaimer that no computer system can be absolutely secure. As we know, it's very easy to make mistakes. So please do consult your platform documentation when creating any software that involves trusted execution environments or other security measures. All right, with that, a bit about myself. My name is Stefano, and I work for the Linux Foundation. There are a couple of different projects that I work on. One of them is the Confidential Computing Consortium. This is a community that brings together hardware vendors, cloud providers, and software developers to accelerate the adoption of trusted execution environment technologies and standards. I encourage you to go check out the website. There you can see a list of the projects that are housed under the CCC and learn more about it. I also work with Risk Five International. This is the home to the Risk Five instructions at Architecture, but it's also a place where we are in charge of growing the ecosystem around Risk Five. And that includes things like compilers, tool chains, and even the meetups that happen regularly in different cities around the globe. So again, please do head over to the website to learn more about Risk Five International. And lastly, I do research work with the folks at Portland State University, specifically concentrating on trusted execution environments and hardware security. So what is a trusted execution environment? At the Confidential Computing Consortium, we came up with this definition and it brings up three properties of a trusted execution environment that must exist. And that's data confidentiality, data integrity, and code integrity. By data confidentiality, we mean unauthorized entities cannot view data while it's in use within the trusted execution environment. Data integrity means unauthorized entities cannot add, remove, or alter data while it is in use within the TE. And code integrity means that unauthorized entities cannot add, remove, or alter code executing inside the TE. Now, as we'll see later in the presentation, there are lots of things that a trusted execution environment might do, but these are the three properties that are core to what a trusted execution environment must do. This is a brief history of trusted execution environments and it's important to call out that many technologies existed before this that accomplished things similar to TEs. However, the open mobile terminal platform was one of the first groups to actually standardize what a TE should be. And it's probably no surprise that it was around handset manufacturers for things like cell phones. The global platform group was the next one to extend that standard. And it wasn't until 2012 that we actually started seeing products on the market, which had trusted execution environments in them. And I found it important to call out here that 2017 risk five ratified the PMP part of its spec. So that's still a relatively new part of the trusted execution environment ecosystem. Let's take a moment to discuss what a TE is not or what we're not going to be discussing during this presentation. First off, data and transit, things like HTTPS, you won't discuss that here, data at rest. So full disk encryption is an example of something that isn't considered a TE. Homomorphic encryption, being able to operate on instructions and data that are encrypted is a fascinating topic, but not something that we're going to cover here. This next Venn diagram is a good example of what a TE is in that it separates itself from privacy preserving computation. And it can include things like a TPM and it can be programmable. But one thing that we'll restrain ourselves to in this presentation is hardware TEs. So a good example is something that is on an FPGA would not be considered a hardware TE. That would still be considered software. So it is still mutable and therefore not considered a hardware TE. This table compares several of the security features that you'll find in various security hardware elements. You'll notice that the TE has highlighted data integrity, data confidentiality, and code integrity as those are integral to what a TE must do. However, things like code confidentiality, authenticated launch, programmability, attestability, recoverability are all optional items that it may or may not do and still be considered a TE. Let's look at each of these properties one at a time just to see what a TE might provide and why it might be valuable. Code confidentiality is an example of something where you might wanna protect an algorithm that is considered intellectual property, maybe a machine learning algorithm that needs to be protected. And that's one property that a TE might have. Authenticated launch is another one. If your TE is compromised, you may not wanna be able to launch the app that should be running inside the TE. An example I like to use is a banking application. If my banking application has become compromised and I can't attest to its security, I might not even wanna launch that application. Programmability is another property where the program may contain code from the manufacturer or may contain code that's loaded afterwards from a source that is known to be secure. Attestability is, in my opinion, integral to the functionality of a TE and that's the ability to measure the origin of the TE and the current state of the TE. This gives evidence that the code has integrity and it hasn't been tampered with in any way. And lastly, recoverability, which is an interesting property in that, let's say the TE becomes compromised in some way, it would be great if it was able to roll back to some previous state where it was known to be secure rather than to simply fail. We mentioned attestation on the last slide and this could certainly be a subject that takes up the entirety of the talk, but just briefly to go over it, attestation is the way in which we measure the TE to assure that it contains the data and the code that we think it contains. It's the way that we guarantee it to be secure and this can be done locally using symmetric keys or remotely using asymmetric keys. And it's really at the heart of how we know that the TE has been measured to be secure. So why might we use a trusted execution environment? Well, I like to use this quote from Bruce Schneier as I think it gives a good example of the type of things that we're keeping secure today. It's no longer just passwords, it's things like facial scans or our thumbprint, things that aren't easily replaced. And so this data needs to be kept in a place where it's known to be secure. This is likely a refresher for many of you, but this is the classic picture of the rings of privilege on most SOCs and processors. Historically speaking, you had a least privilege ring three where user applications were able to run rings two and one where they were more privileged and things like device drivers that had access to hard work had run. And lastly, ring zero where the kernel would run and that was the most privileged level. And a more modern architecture, like an Intel client architecture, you can see that there are additional rings for things like the hypervisor, SMM and the management engine. System management mode, SMM on ring negative two is a good example of what you might consider a precursor to the trusted execution environment where memory was allowed to be secured and only highly privileged operations were allowed to happen. It's important to note that the ARM privilege levels are slightly different than those in the X86 world in that somewhat confusingly, the numbers are reversed. So EL zero is actually the least privilege application level with EL one and two slowly becoming more privileged and EL three containing the most privileged and most secure code. With that, let's begin down the path of learning the similarities and differences between ARM trust zone, Intel SGX and risk five physical memory protection. Starting with ARM trust zone, it's important to make a quick disclaimer that there are many different processors that can run ARM trust zone and we're only gonna consider ourselves with those that follow a certain architecture running in a certain mode. So the ARM V8A architecture running in ARCH 64 state as well as the standard solutions for ARM trust zone implemented by ARM trusted firmware and the open source portable TE or OPTI are the only technologies that will be discussing in this presentation. So have you used ARM trust zone? Well, chances are you have if you've used the popular messaging app line, that's one example of an application that takes advantage of trusted execution environments in order to keep data and code secure. So how does ARM trust zone keep this data and code secure? At its highest level, this is really done via the separation of computation into two separate worlds, the secure world and the normal world. The separation is accomplished by a secure configuration register bit, the non-secure bit or NS bit, which when set to one is set to non-secure and set to zero meaning secure, but it trickles down into more of the architecture than just that one configuration register. It actually has effects throughout three separate parts of the infrastructure, the bus, the SOC core and the debug infrastructure. So let's look briefly at these three hardware components that are involved in any trust zone implementation. First, the bus, which has a rather long name of the advanced microcontroller bus architecture, advanced extensible interface, or AMBA-AXI bus for short. This bus contains two different bits, one for write protection and one for read protection that likewise, when set to zero are secure and when set to one are non-secure. As you might imagine, there are several bits of logic inside the SOC core itself that read the NS bit and to make different decisions based on that bit's state. And lastly, the debug infrastructure, which we won't really get into in this presentation, but it's good to know that if you follow the source on this slide, you can read more about how the debug infrastructure and all of these hardware components contribute to an ARM trust zone system. This diagram is a good high-level overview of one possible implementation of a secure and normal world. Here, you can see the division between the two separate sections. One, the normal world, which contains generic applications and possibly applications with security. Those applications then communicate to a trust zone driver which can make secure monitor calls to the secure monitor. The secure monitor, which stays resonant in memory and is able to bridge the two worlds can then communicate to some secure operating system, some secure kernel, and to different secure standalone applications or services. Here, you can see a simplified ARM trust zone boot process starting from cold reset to some trusted boot ROM. This is usually shipped from the manufacturer, but this is the part of the SOC that is guaranteed to boot securely. That passes off then to some firmware, in this case, ARM trusted firmware, which is usually stored in ROM or possibly in trusted SRAM. And that in turn can hand off to a trusted OS which will then be responsible for the normal world bootloader execution and eventually booting the operating system that the user might interact with. On this slide, you can see a more typical implementation of ARM trust zone. Important to note here is that this is diagramming out reference implementations, including the trusted boot firmware and Opti trusted OS that ARM provides as reference implementations. But here you can see several different stages of boot loaders being run, the trusted ROM passing off to the trusted boot firmware, which in turn passes off to a runtime software and the secure monitor. And finally, over to some non-trusted firmware, perhaps UBoot or EDK2, which would eventually boot kernel, possibly the Linux kernel and run rich applications, where Opti, the trusted OS, would run in the secure world and load secure trusted applications that could run completely separate and secured from the non-secure rich applications while still running in the lower privilege modes. I mentioned that these reference implementations are available and provided by ARM. That's part of the ARM trust zone firmware components. You can go to trustedfirmware.org to learn more about ARM trusted firmware, as well as Opti, the secure OS. In part two, we're gonna briefly cover Intel SGX and see how it's different from ARM trust zone solutions. You might ask yourself, have I used Intel SGX? Well, much like the example with trust zone, there is a messaging application called Signal, which is popular and uses Intel SGX as its means of keeping messaging secure. Now we'll see momentarily that that's dependent upon your system configuration and who built your platform. Unlike ARM trust zone, where two separate sections of execution or two worlds are defined to the secure world and the normal world, Intel SGX provides a mechanism for creating secure enclaves. These secure enclaves are essentially containerized sections of memory. They're a place where data and code can be loaded or executed securely. And they're verified in multiple ways. First of all, by cryptographic attestation keys, much like in trust zone, but also by a hardware root of trust. And these are available on all platforms that support Intel SGX. Looking at the initial enclave setup, it's important to remember that the firmware, much like in trust zone, is the part of code that actually sets up the area where the enclave will reside. This is called the processor reserved memory, the PRM. The CPU then creates blank enclave page caches and stores those caches inside the PRM. It can then load data into the enclave and mark the enclave as initialized once that data is loaded incorrectly. And finally, it will create a cryptographic hash of the enclave's initial state and other states that come after initialization. And that hash will be used during the attestation process. Looking at how the memory is actually allocated in SGX setup, we start with DRAM where the processor reserved memory section is set up by the platform firmware. This is set to a fixed size, usually on the boot of the platform. This PRM area of memory can contain multiple enclave page caches. These caches provide the actual four kilobyte pages, but they also contain metadata. This includes things like the base address and size of the enclave and also the security info. So for example, if the area is readable, readable or executable. Let's walk through some of the instructions that are used to actually create the secure enclave, fill it with code or data and actually execute code inside the enclave. We start with the hardware commands, E create, E add and E extend. These commands are used in the actual creation of the enclave as well as adding pages to the enclave page cache and extending the measurement of that enclave once the pages are added. It's important to note that while not initialized, the enclave is not secure yet because it has not been attested to by the hardware yet. However, as these pages are added to the enclave page cache, the extend command will extend the hash and store it in a measurement called Mr. Enclave, which will be used later for attestation. So we've seen how the system can create, add and extend the enclave page cache and the two commands we're gonna cover next are E init and E remove. These instructions are used to initialize and then remove the enclave. Initialization means that the enclave is considered fully built, the measurement is locked down and user applications can now enter execute code inside the enclave for attestation and all of the things that you would do to interact as a user land application. Once the user land application is finished with the enclave and it's no longer needed, the system will call E remove. This will deallocate the enclave page cache pages. It'll mark that enclave page cache as available and it'll ensure that no processor is executing any code inside the enclave any longer. Important to note here is just that the enclave page cache holds all the metadata about the enclave and it'll only be deallocated once all the pages have been deallocated. Ring zero will use instructions for page management, including E load, E track, E block, and EWB or E write back. SGX ensures that the pages being operated on are done so confidentially and that there is integrity between these pages. If there's two instances of the same enclave, as an example, the pages cannot be swapped between enclaves and the hashes of these pages will be different. Once the ring zero components have built the enclave, the E enter instruction will essentially cause a controlled jump into enclave code and begin execution. The hardware is responsible for saving and restoring the architectural state of execution using E resume. Should any external events like interrupts or exceptions cause execution to leave the enclave using what's known as an asynchronous enclave exit. Once the enclave is initialized and actually in use, there are some instructions that can run in ring three to seal the data based on the key the developer provides and these instructions are E get key and E report. Using these two instructions, SGX applications are able to perform attestation on the enclave, which as discussed is one of the more vital parts of the functioning of any trusted execution environment. During these past few slides, I've been using a diagram that's taken directly from a paper called Intel SGX explained which I've listed below here and I'd highly recommend checking that paper out. It goes into great detail on Intel SGX and is a great resource for learning much more in depth how this technology works. And finally, let's take a look at risk five physical memory protection or PMP. We'll start by looking at the risk five privilege levels which are very similar to those that we've seen in Intel and ARM platforms. Moving from most privilege to least, you have machine mode or M mode, hypervisor extended, supervisor mode, HS mode, supervisor mode, S mode and user mode, which is the least privilege. And I've listed here the different PMP supported configurations or combinations of these different modes that are available. So speaking a little bit about what PMP can do, first off it's required for RVA and RVM implementations. So if you're looking to implement a risk five core, it is required. It gives the machine mode control over permissions for all the other modes. It can grant or restrict, read, write and execute access to separate regions of memory. And it's used in granularity with page level permissions and that granularity is usually configurable based on the system. Regions can also be locked until a reset is performed and those locks can actually lock out machine mode as well until a physical reset is done. So looking a little bit at how we set up PMP, there are some number of configuration registers that need to be set. These configuration registers contain the mode which we'll look at in just a moment. And then also the access rights, read, write and execute that are granted as well as the address range over which the permissions are going to be applied. This slide shows one possible implementation of a set of configurations for PMP. On the left-hand side, you have the same diagram of the configuration registers and the address ranges. And on the right, you have a Keystone enclave example. So this is not a risk five native example. It is one possible implementation that the folks at Keystone use as an example. And I've linked to the paper in which they describe this below. But you can see that those configuration registers essentially lead to a set of some address range where specific read, write and execute permissions are guaranteed. So for example, in the first configuration, which has the highest priority, you see an SM region for a secure monitor and that currently has its read, write, execute bits set to zero so that user space applications are not allowed access to that region. Since risk five is an open source instructions at architecture, there are many risk five cores available on GitHub that you could download and actually run on FPGAs or just simply peruse the code. I chose the Ibex core as a quick example of physical memory protection so that you can actually see how it's laid out. The Ibex core is a small 32-bit in-order risk five core with a two-stage pipeline and it implements physical memory protection and is coded using system verilog. Looking over what's required to enable PMP, there needs to be the actual control status registers. These are the registers we talked about earlier where you actually set whether or not a region is locked, what mode it uses, if it's executable, writable or readable. And these are needed for both reads and for writes. In fact, a little bit more information is needed for the writes as you will need to set the mode as well as the address range. So if you take a look at the four mode options, the three modes when PMP is enabled are top of range, naturally aligned four-byte regions and naturally aligned power of two regions of eight bytes or more. And so these are the different types of address ranges that you can set with physical memory protection. The actual registers themselves are implemented pretty simply in this core. You can see both the CSRs for the configuration and for the address range are created here. And lastly, we have to actually create the PMP hardware here, but probably the most important bit is where the access fault is wired in. So you have several configurations set for read, write and execute. You have regions defined, but there needs to be something that checks to see that the access either should be or should not be granted. That line is then fed into the instruction fetch, whether that be a cache or some kind of prefetch, and to the load store unit. So essentially any time anything is loaded or stored in memory, this line needs to be checked to see if PMP is restricting that region and the load or store should fail. Again, I realized we went through this code very quickly, but it is all available online. Low risk is the name of the company that actually posts this code and the Ibex Core is licensed with an Apache 2 license. So I highly encourage folks to head on over there and just take a look for yourself and browse the code. It's very easy to read and well laid out. Risk 5 physical memory protection is a relatively newcomer to the world of trusted execution environments. And as such is still very simple and straightforward and while easy to understand does lack a lot of features, that being said, a lot of work is currently being done in the risk 5 community to expand the specification. Two features that are currently on the horizon are S mode PMP, which will provide per core supervisor mode control registers and IO PMP, which will expand physical memory to all memory masters in the system. So head on over to risk5.org to check out more and to keep an eye on this technology as it grows and matures. So thank you all very much for watching. I realized that this was a lot of information that we went over very quickly on three rather complicated subjects. So I've provided my contact information and I will be around at the conference all week. So please feel free to reach out on Slack or send me an email and I'm happy to go over any details or questions that you might have. Arigato and thanks for watching.