 All right, thanks very much. Say my name is David Kaplan. I'm on the security architecture R&D team at AMD. And part of the goal of our team at AMD is to define and architect new security solutions for AMD hardware. And I'm here today to talk about a few of those new features that we've developed around memory encryption specifically for our x86 chips. So just to overview what we're going to do this afternoon, I'm going to introduce the features that on the technologies that we've developed talk a little bit about the motivation and the details of how the hardware behaves. And I'll also talk a little bit about how Linux can enable these features. Some of my colleagues have recently, actually Monday this week, submitted some RFC patch series regarding these two features. And I'll talk a little bit about what those patches do. So as far as motivation for this goes, certainly we think that memory encryption is an interesting technology and can be used in a variety of cases. But we chose to focus primarily on cloud cases, especially around public IAS clouds. Certainly security is a top concern in that environment. And one of the top threats that is sometimes seen in CSA assessments and the like is around the hypervisor. In clouds today, the hypervisor is of course responsible for enforcing separation between the guests. And it does this typically using what we call logical isolation features, harbor virtualization, page tables, VM intercepts, et cetera. And of course this can potentially break down. There's been a few high profile bugs related to this. And these bugs can be particularly scary since it can allow one VM to potentially compromise another VM oftentimes without the target even knowing that it's being compromised. And so we wanted to see if that's something that we can help address. The second thing that we found from talking to some of our partners and customers is that the traditional trust model of computing, if you will, this idea of a hierarchical trust model, doesn't really work very well for scenarios like public cloud. That is, when you talk to customers of the cloud, they want a sandbox to run their workload. They don't want anyone else to be looking at their data. You talk to the companies running the cloud and oftentimes they will echo a similar sentiment. That is they want to provide the resources but they don't want to be able to see their customers data. They don't like the legal implications of that access and it would be better if they didn't have that visibility. So what we're specifically trying to protect against with the features I'm gonna go through today are two different classes of attacks. The first, I'm calling these user access attacks, these are kind of rogue administrator type attacks, things like scraping the memory of a guest VM. So like someone who has hypervisor privileges scraping the memory of the guest, injecting unauthorized code into a guest VM and potentially exploiting some sort of hypervisor bug to allow cross VM attacks. Certainly hypervisors, if they're written perfectly, this would not be allowed but we want to see if there's some extra layer that we can add on to here to make it more difficult to conduct such an attack. We're also looking at physical access attacks. Now how important this is varies depending on who you talk to but we are also looking at certain things such as physical probing of the DRAM bus on the motherboard, cold boot attacks and some variations thereof. With more people considering non-volatile memory technology like NVDIMS, we think that this can potentially become more important since cold boot attacks can be a lot simpler if you don't even need to look with nitrogen. So what we've done in this technology is the first thing is we've integrated a hardware AES 128 engine into our memory path. So it's in the memory controller and it is able to transparently encrypt and decrypt traffic as it leaves the SOC. We then went and defined two new features what we call secure memory encryption and secure encrypted virtualization that both utilize this engine to provide different security benefits. The SME feature is primarily designed around the physical attack piece and it uses a single key. The SEV feature is designed for those VM isolation scenarios and allows for different keys for different virtual machines on the system and I'll go through the details of how that works. Some highlights about both of these features. So the first is that we have this hardware engine, it's capable of this inline encryption decryption. Because of that, we expect this to have a relatively minimal performance impact and typical workloads. The data inside of the SOCs is not encrypted and so there's no impact there. This essentially amounts to a little bit of extra latency as we go out to DRAM. Another key feature is that we've defined both of these to be enabled and implemented at the operating system in a hypervisor level. So we're not looking to require application changes with any of these. This is all integrated into the same SOC package. That's right. And then the final thing I wanna mention on this is that the encryption keys themselves are stored in the hardware and they're not visible to any of the software running on the x86 cores. We instead use this entity called the AMD Secure Processor which is a physically isolated security subsystem to manage these keys, load them in. We never then send the keys in the clear outside of the SOC. So for anyone who's not familiar, the AMD Secure Processor is a technology that we introduced I think about three years ago. It's present on some of our recent SOCs. It includes this security subsystem which is centered around a 32-bit ARM Cortex A5. It has its own ROM, RAM, some dedicated crypto hardware, keys, the like. We use this to implement a number of security-related tasks in our SOCs. For instance, something that we call Harbor Validated Boot which you guys are I'm sure at the talk this morning where Matt talked about TPM and Static Root of Trust. We use this kind of similar to the Intel Boot Guard feature to help validate the first BIOS code that executes on a platform. We also utilize this in some of our current client parts like notebook parts to implement what's called a firmware TPM so it can emulate a real TPM without needing to put one on the motherboard. We use this technology with these memory encryption features simply because it is isolated, it runs only AMD signed code in it and it has this dedicated hardware that's not accessible from the outside. So the first feature that we have is secure memory encryption. This is a relatively simple feature. As I mentioned, there's one key involved. That key is generated randomly using a hardware random number generator that we have in the security processor. That key is loaded in the memory controllers and it can be used by the operating system simply by setting a new bit in the x86 page table. So we have a new bit that we call the encrypted bit and if that gets set, excuse me, then the accesses to that page will go through the AES engine as they go in and out of the SSE. So because of this, this feature is really designed just for physical attack protection and we do support devices accessing this encrypted memory as well through DMA so there's not much special work that's needed there. So you would read the ciphertext of that page. If you have a page that's marked encrypted and you've written some data out to it, that data is sitting in the DRAM chips, it's encrypted. If you then clear the bit in your page tables and read it back, you'll see the ciphertext. The encryption key for that feature is generated boot time. So it's different every boot but it stays throughout the lifetime of the while the chip is on. So this is a summary of how this feature works. So there's like with most x86 features, there's a new CPU ID. We actually have a set of functions that provide information about the capabilities and there's a new bit that we call memory encryption mode enable which is set at boot time in the sysconfig which is a model specific register and once that is set, then the operating system is free to go and set what we call a cbit encrypted on these pages. Where this bit lives is dynamic and this is partly because there's really no free bits left in the x86 page table. In the example here that's shown, the cbit is bit 47 of the physical address and this is convenient for implementation purposes as it allows anyone including devices to be able to address encrypted memory simply by setting that upper address bit. It's all right. Right, we do not have support for integrity protection on the memory so there's no replay protection in this architecture right now. Right, so keep in mind that this feature, the SME feature is only designed to protect against physical attacks on the DRAM so for device based attacks we're relying on the operating system to use an IMU for instance to restrict what memory a device can DMA to but this feature by itself is not designed to protect against that type of attack. Yeah, one more question. It supports multi-socket since the system address space is consistent. Yeah, so let me explain that a bit more. So in a multi-socket design, we have a number of different memory controllers in there and we also have a number of different security engines that that AMD secure processor and they will all communicate so there is basically a master one which is kind of like your bootstrap processor that will generate the key and then it will send that out to all of the other memory controllers in the design so they're all consistent. Yes, yeah, we have a protocol for that, yes. So we're using an address tweak based mode it's kind of like XEX where we take the physical address and we run it through a hashing function and we do an XOR and crypt XOR to basically just to, you know, so different blocks of data that have the same plain text don't look the same in the cipher text. It's 16 bytes because it's AES, right? And we allow arbitrary access to any region of memory. That's correct if you copy the cipher text. So the second feature that I want to talk about which is a little bit more complicated is what we call secure encrypted virtualization and this is a feature where we have multiple encryption keys in the design and we can associate one key with a hardware virtual machine on the system and so this is what's getting to this idea of changing the trust model of computing. Our long-term goal would be to have a model where the hypervisor schedules guests and runs them side by side but is not able to, doesn't have this full visibility. You know, we're not getting all the way there with our first generation like many things we have to work incrementally but we're taking a big step with this SEV feature and I'll talk a little bit more about the threat model and how that works in a couple of slides. But the basic idea is that we can associate this encryption key with the guests. We also tag all of the code and data associated with that guest throughout the cache hierarchy. So for every line we know who it belongs to and we can restrict access to only that particular guest. That information is then used when the data leaves the SOC to go and encrypt it with the guest specific key and we call this a form of cryptographic isolation so this is intended to be an extra layer on top of the existing logical isolation features like the page tables and importantly it prevents the hypervisor from being able to directly read and write into guest memory spaces which is a big difference from how the trust model works today. We've integrated this with what we call the AMDV technology that's our hardware virtualization support that's already supported in all the major hypervisors and this is an optional feature in there and so we can run a mix of encrypted and non-encrypted virtual machines and that all works fine. The very simplified high level architecture of this looks something like this. You can see in the red there's the keys that are present in the memory controller just shown one here for simplicity and the AMD secure processor that is the one who actually has access to those keys and also has access to some non-volatile storage as well. The secure processor exposes an API and this is an API that is public. We have a spec on our website for it and includes a number of functions related to provisioning the platform, launching guests, doing migration, the like and these APIs are then exposed through the component we call the SEV driver which is a standard X86 driver that allows the hypervisor to call these API functions to manage the guests without having direct access to the keys or the secrets within the guest. The owner of the guest that's shown on the left here is a key player in this architecture as well since the owner of the guest is responsible for managing the guest secrets, managing policy. Policy contains information about what is allowed as far as moving data, migration, debug, things like that and the guest owner will actually set up a secure communication channel using a Diffie-Humman protocol to the AMD secure processor on the SSC and they're able to authenticate the secure processor and then they can use that secure channel to pass information that's related to setting up the VM and things like that. And then of course the guest OS kernel within the VM is ultimately responsible for protecting its own memory and I'll talk a little bit about how it's able to do that as well. To walk through one example of how this architecture works and we have a number of flows like this and this is just one case but this is around we call launching a guest and so launching is taking a guest that of course is initially unencrypted and turning it into an encrypted guest in a secure way. And so the way this flow starts is it will start with a BIOS and potentially OS image that is unencrypted that will be loaded into memory that's presumably not really secret and that any secret information would reside on a virtual encrypted hard disk. Then we set up the secure communication channel. We do the Diffie-Humman exchange and the hypervisor passes the data back and forth between the guest owner which may be the user of the cloud for instance and the secure processor on the node and there is in addition Diffie-Humman there are some certificates that are exchanged here in order to prevent man in the middle type of tax allow the owner to authenticate not only that this is a real AMD processor but that in fact belongs to the cloud that it's expecting it to. The next step is that the hypervisor will ask the secure processor to encrypt the image and so at this point the secure processor will generate a new random key and it will encrypt the image in DRAM of that initial BIOS and OS setup code and as it's doing that encryption it will do a measurement of that that will be used in a little bit to send back to the guest owner. The hypervisor will then set up the guest we call the VMCB, the virtual machine control block with a key selector so we support a certain number of keys and the hypervisor manages these and it puts the appropriate key number into the guest VMCB and it sends what I call this launch receipt back to the user and so the launch receipt is that measurement of the memory as well as some additional platform authentication information and this allows the guest owner to determine if their initial memory image is correct and if it is then they can choose to release say the disk decryption key to that guest using whatever protocol they're typically using and at this point the virtual machine is running it's encrypted, any new data it creates will be out of encrypted memory and it can use its virtual encrypted hard disk like normal. Question. When you say guest owner? Yeah, in this example I think the expectation might be that it could be a service that's hosted at say your company's site that's managing guests out in the cloud just for example. So getting into some more of the details of the hardware architecture so I mentioned about the ACE at a second ago that determines which key number is used because we can have multiple ones and that is actually the tag that we then keep inside the cache hierarchy in the SOC and that ensures that two different guests will never be able to grab each other's data out of the caches even though the data in the caches is itself unencrypted. The next thing is that we have this mechanism to determine if a page is what we call private or shared and this is done using that same page table bit that I mentioned with the SME feature, the C bit. And in the AMD virtualization architecture we have two levels of page tables. We have we call the guest page tables that are of course managed by the guest and the nested page tables managed by the hypervisor. The guest page tables are used to control if this page is considered private or shared. A private page is encrypted with the guest specific key where all the shared page is accessible to the hypervisor. We have a few hardware rules about how this works so the first thing is that any code that's executed is always considered private no matter what the page table might happen to say simply because we don't want someone to be able to put data in there and get the guest to execute it without knowing the encryption key. Similarly, the guest page tables themselves are always forced to be encrypted so that their address space cannot be easily manipulated. For data pages basically everything else we then use this page table bit and there are some restrictions we have. So for instance, DMA has to occur to shared pages. We don't allow any devices in the system to have access to the guest encrypted memory. And obviously the hypervisor similarly does not have access to the guest encrypted memory. So any memory that's gonna be used for either real or fake device communication is marked shared by the guest and it's understood by the guest that that data can be looked at and should be properly encrypted or whatever it needs to do. One question that we sometimes get asked is say, okay, well the hypervisor is the guy who supplies this key index into the VMCB so why couldn't he just Y and put a different key index into there? And there's nothing in the harbor that restricts the hypervisor from doing so. However, we believe that that would be a difficult thing to exploit simply because of the rules that are outlined here. That is if you have one VM that you're trying to attack, you don't know his key. You may have another VM that you can control but if you try to run that VM with the first one's key the hardware is gonna go and it's gonna try to walk this guest page tables using the wrong key which is probably not gonna be very successful. Even if it can somehow it's then gonna try to decrypt his instruction stream using the wrong key which again is probably not gonna be very successful. So we see that as being a burden towards trying to attack another VM on the system if you don't know the key. So just a quick note on the interaction between these two features. As we talked about with SME there is a single option for encrypted or not encrypted with the SEV feature it's a little bit more complicated because we do have the two sets of page tables involved and so we combine both of their CBITs as shown in the table here. Basically the idea is that a guest virtual machine gets first dibs on whether its data is considered private or not. If the guest marks his data is shared by setting the SEVA to zero then we will consult the next level page tables and decide if the memory should be really unencrypted or if it can be encrypted with the hypervisor key. So we can have a system where basically all of the memory is encrypted is just some with a guest key and some with the hypervisor key. And so this is just a quick example of how this address translation works. On this case we're gonna start with a guest virtual address. We'll translate that through the guest page tables and end up with a guest physical address. And in this example we're assuming that the CBIT is the upper address bit 47 which is set here indicating that this would be a private page of memory. The hardware will then remember that CBIT but take it off and take a host virtual address without that bit and translate that through the nested page tables in order to get a host physical address. And then at the end it will put the CBIT back on when it creates a system physical address and this will ensure that this access goes out to memory as a private guest access. So to compare these two features there are clearly a number of differences. The first thing is on the threat model. The SME features we talked about is designed really just to protect against physical attacks. The SEV feature as it's defined here is so far we like to talk about it as protecting against a benign but vulnerable hypervisor. So not a fully malicious hypervisor at this point. We love to get to that but we understand we're not there yet. But we see benign but vulnerable meaning well-intentioned but there may be bugs. Just like with a number of the security features that were discussed at this morning's talk we want to raise the bar. We want to block certain classes of attacks and I believe that that's what we're doing with this SEV feature and going forward we expect to be reducing the attack surface even more. We think that right now the attack surface is being reduced substantially. We'd like to get it even smaller. There's some other differences between these two features as shown here. The way the DMA works is a little bit different. The default behavior is a little bit different with the SME feature. Things are unencrypted by default and then the operating system has to explicitly mark them encrypted. With the SEV feature we'll actually keep everything encrypted until we get basically in a 64 bit mode or 32 bit PAE at which point we'll then give control to the page tables to decide what's encrypted or not. And then of course there is the interaction with the security processor and the SEV feature uses that API set that I mentioned earlier. So to talk to us a little bit about what we've been doing as far as Linux enablement. So there's a number of software components to this feature and if some of them AMD is developing and we expect to ship and then there's some that we're working with the open source community on. The first thing is that the secure processor firmware as I mentioned it's signed by AMD and it just exposes a set of maybe a dozen or so API calls and the spec for all those API calls is on our website. We're also working on the Linux driver that will basically marshal the data back and forth to the secure processor. The secure processor is basically just a PCI device with an MMIO interface so nothing too crazy and we'll use this to call functions. We're also working on the kernel support so they're say there are two patched RFC series that were sent on Monday to the mailing list one related to the SME feature and one related to SEV and we're also working on the open source hypervisors to add support into there as the hypervisors need to use these new API calls in order to start up the VMs, migrate them and so on. So just an overview of what the Linux support that we have added intends to do for the SME feature. The first thing is that it needs to set up the feature and get the kernel encrypted and so we have code during the early boot phase to get into 64 bit mode and then actually encrypt the kernel in place and so we're able to do that by creating new page shape entries that have the C bit set. We can copy the kernel basically on top of itself and encrypt that image and from that point on we set the C bit on all entries. So the patches that we've developed basically try to keep everything encrypted as much as possible. Certainly there may be scenarios where for performance reasons or something you don't want everything encrypted but the set that we've sent up is focused more on protecting against the snooping cold boot type attacks. There is an interface in there for requesting unencrypted pages for special purposes. There's a few exceptions that are listed there as to some of the places that the current code needs to use unencrypted pages including for just kind of bootstrapping the other APs as well as some structures that BIOS creates back when the system was still running unencrypted and that the kernel needs to access. Now the SEV patch set builds on top of the SME one. The SME patch set of course is intended for when the kernel is running natively on bare metal. The SEV patch set is intended for when the kernel is virtualized when it's inside a guest. And in some ways the SEV boot is actually a bit simpler because as we saw in that launch flow earlier the entire BIOS initial OS image gets encrypted before execution actually starts. There's no need to encrypt the kernel in place anymore and so it actually starts executing encrypted code and just gets the page table set up and again sets the SEV it on most of the pages. The main exception being the DMA pages and so the way that this is implemented in the current patch set is that we make use of the software IOTLB in order to bounce buffer the DMA so we allocate a couple megs of memory space there as shared pages and then DMA gets copied through that in order to be shared with the outside world. Similarly existing para virtualization features use a shared page since they need to communicate with the outside. And so in this initial patch set this setup is relatively static. There's not a lot of dynamic changing of pages from encrypted to unencrypted status. A couple other changes that are included in here so as I mentioned there's the driver that marshals the data back and forth between the hypervisor and the security processor. I believe that's included in the patches that were sent up earlier this week. There's also a whole set of hypervisor modifications and I know the KVM side of this was included this week. There's also of course a QMU side we're working on. There are four main areas that we are modifying in the hypervisor. The first thing is this concept of provisioning so that's around establishing trust in the platform and the ownership of that so you can know that you're not being man in the middle. We have this launch process that I kind of walk through about transitioning from unencrypted to an encrypted state along with the measurement. We also have something that we call the send receive functionalities. The idea behind this is that two secure processors on two different nodes can set up a secure communication channel, authenticate each other and this can be used to migrate a guest VM within a cloud and this involves basically one node taking the image in DRAM, reencrypting it with a transport key, setting it over the network and then the other one decrypting it, creating a new key for it, putting it into memory. And this same protocol can be used for things like snapshotting and also presents some interesting use cases related to starting up basically encrypted VM images that you can create an encrypted image say on your local machine, move that into say a cloud environment and then run it from there. So you said what aid? It's always in Cypher. The secure processor has its own local SRAM and so what it will do is it will, when it gets the transported data, it will decrypt that into its local SRAM temporarily and then reencrypt from there into DRAM with whatever new session key it establishes so it's never visible to the X86 processors in the clear and the SRAM is on dye so it's not easily probable. One other point I wanna mention on this is this idea of key slot management. Because the memory keys, memory encryption keys are in the hardware, there is a fixed number of them and so we have support for basically doing over committing of this and being able to swap keys in and out depending on when you're done running particular guests and the like. I can't say we thought of that. Yeah, I'll look at that. In the API we have a few functions. We call activate, deactivate and then we have flush command because the, okay. Sounds well engineered then. We'll look at that, that's a good feedback. Yeah, I think that we try to come with a simpler API set that was more focused around the VM life cycle and these things like migration and launch. They say the number of APIs we're talking about here are very small and we're not really like sealing keys or anything like that, right? The secure process is just creating them. Yeah, no, I understand that. I think that's something that we could look at more and I would certainly say to please take a look at the API set we do have on the website and send us feedback. It's still, I would say, kinda in beta right now so that would be useful feedback for us. You know, one thing that we would need to think about with the TPM is how that would affect kind of the physical attack threat models to if there's anything, because it's an external party that we can't communicate securely between the secure processor which ultimately needs to write into the memory controller, right? And an external TPM. But we can have a further discussion about that. Okay, I believe you. So this is basically the information I have for today. I did wanna give some pointers to where we have more public documentation. We have a white paper, we have one thing we call the AMD64 architecture programmers manual. That's basically the x86 side of the spec so that talks about the CPU ID functions, the MSRs, et cetera. We also have the secure encrypted virtualization key management document which is what I was just referring to with the API spec and we invite comments on that as well as to the RFC patches that I went to the mailing list this week. I do have two other colleagues that are here with me. There's Tom Lendaki and Burgess Singh. They're giving talks at KVM Forum and Zen Summit respectively regarding this feature and the hypervisor support that they've been working on for that so I'd certainly encourage you to seek them out. There's a few of detail questions around that. And yeah, is there one other question? Yeah, sure, that's not to the guest. In which case it doesn't matter whether it's malicious or vulnerable, it's not trusted and therefore you have to treat it the same way so I'm not quite sure what distinction you're making there. Well, so the distinction that we're making is that there's certain, so there's certain threats that we, there's certain types of attacks we can block with the current architecture. For instance, if there's a hypervisor bug that allows an arbitrary read primitive to a guest, we think that this technology is a pretty good mitigation for that since it could just read ciphertext from other guests. However, a malicious hypervisor that is manipulating, that's a register state for the guests or doing things to the page tables and ways to try to extract information from an encrypted guest, we don't have protections for all of those scenarios right now. We certainly are working on addressing that, but so we don't wanna talk about a fully malicious hypervisor that has basically arbitrary code execution and can do whatever it wants to a guest. We're instead looking at things that say, well, it's pretty good, but it may have these kind of bugs that allow some access but maybe not a complete arbitrary execution of code within the hypervisor. So, yeah. Yes. Right, and you know, as I mentioned, we would really like to get to the point where we can claim that there's nothing that the hypervisor can do other than denial of service, but yeah. Sure. I guess since I'm almost out of time, any other questions? Yes. Right, so I mean, we would consider that right now under kind of the more going down a malicious hypervisor route in terms of setting extra intercepts in order to try to extract information from the registers. I would say that that is our primary, the primary thing that we would like to address in the, you know, hopefully very near future because of those kind of scenarios and certainly there can be very important information in the register state of the guest at certain times. You know, we see this right now saying that we're reducing the attack surface because we're taking the entire memory of the guest and we're saying, well, most of that becomes a lot less interesting except for the register state, which is somewhat volatile, but you know, that is the area that we're focusing on the most right now. Right, so we don't have any side channel protection in this architecture. I mean, even with caches, you can, even with the tagging that we have in the caches, there are still potentials for timing-based cache attacks. That hasn't been something that we've treated as in scope for this future. We do have tagging in TLBs already, in fact, to make sure that VMs can't use other VMs translations and things, but as far as the side channel piece, we don't do anything with that. Unfortunately, I'm not allowed to say right now. Sorry. Any other questions? All right, well, thanks very much.