 Welcome to Mark's and my talk about secure execution and attestation and confidential dump So first I will start with secure execution a short Revisit to secure execution, then I will talk about attestation and mark will continue with confidential dumping So just for secure execution So Normally or without the hypervisors full control of our unsecure guest it can access the guest it can modify the guest state and memory and Sorry missing So and that's also includes any secrets in that guest and you probably don't want your hypervisor to Access those secrets and modify them so Or probably you don't allow to do that So we have the trusted entity we call it ultra visor. It's part of the firmware this ultra visor managed to Confidential state of that VM and the hypervisors to cooperate with the ultra visor to run the secure execution guest And the guest also can share pages with the hypervisor And no one but the guest owner and the ultra visor can access unencrypted confidential state How does it work? The guest owner prepares as a secure execution image and this image includes a public elliptic RFT-Fielman key and This key is used then for establishing a shared secret between the ultra visor and the guest owner From the ultra visor perspective it takes the public key of the guest owner its own private key and derives a shared secret and This means that only the ultra visor can decrypt and run the execution image And the guest owner can also store additional secrets and the secure execution header of the secure execution boot image for example a customer communication key that's later used for dumping and Also keys for the component decryption Such concomit components are the kernel the kernel command line and the init RD These are always encrypted Authenticated and integrity protected and again we can store secrets in those Components for example, we can store a looks key passphrase in the init RD in Kleatex and So now for our station and our goal is to prove that a Given guest runs the expected workload in secure mode in secure execution mode and With IBM secure execution we more or less get that for free and We do not require externalization because our components are encrypted and integrity protected if we inject a Secret a unique secret during the build of that image for example as H pop key. That's unique to this image We can attest that image by simply logging in because no one else has a pop pop the key so I just said We can do if attestation implicitly so why do it explicitly or externally There are some reasons and The first one is We want probably to prove a third party that we're running under secure execution without passing any image secrets Or giving it access to the our image or We want to verify not only that our guest is not only a specific image but also a specific instance of that image because we have it multiple times or In general we need trusted information about our Guest image instance or the execution environment we're running in So the use cases are First of all we go want to get compliant to attest with a third party or The other one is to customize and already prepare generic as secure execution in which we got from another person we more or less trust and We first would have to attest the image externally that it's running under secure execution and more or less the expected workload and Then deploy our own instance the dependent secrets and Then next time we can attest it implicitly while logging in So how does explicit at station work for secure execution? We start the secure execution image at some point via the KVM host It sends a call to the ultra visor and this ultra visor first verifies the hashes of the secure execution image and then starts the image and at this point the Guests will transition into secure execution mode and Then we have the attestor that's running on a trusted system. That's cut off. They're at the top It will generate a request and this request similar to the secure execution image contains a public ECDH key, but this time from the tester and an encrypted measurement key We send this request to the secure execution guest and this guest will do a ultra visor call to the ultra visor and the ultra visor will do an measurement with the measurement key and This measurement contains the hashes from the secure execution image and the configuration unique ID of the secure execution guest image and then we send back the response and At the very trusted system we again verify the measurement. So we check if the measurement are the same We have a new tool to do that it's called PV attest and It consists of three commands Create to create the request on the trusted system perform to do the ultra visor call on our guest and verify to again verify the measurement on our trusted system The current state is that you need an IBM set 16 and the guest has a kernel 5.19 And for a chemo liquid and generating of the secure scheme it just works in no changes required and The PV attest is include in a 390 tool starting version 222 and now we're going to confidential dumping Okay, so now let's talk about confidential dump This is a newly introduced mechanism to securely and reliably dump a secure execution guest by the untrusted hypervisor So let's first compare guest initiated and hypervisor initiated dump to see why it's so important to support hypervisor initiated dumping for secure secure execution guests at all so That's one guest perspective So the big pro for a guest initiated dump is and it needs no Hypervisor interaction at all. So if you are using K-Dump for example, you do not need the hypervisor to interact You do not have to trust them and and the guest knows its data best For example K-Dump allows you to use page filtering and so on But big con is it's not always possible. For example, if you have a buck memory management then the kernel will not boot and for example for early boot problems and so We had already a lot of feedback that many customers or many people Have tried to start and they forgot to use a bounce buffers So the guests just crashed and in that case it would be really useful if we would have a dump of that Then dumping modifies a guest state. That's a problem as well. Sometimes it's even destructive Then needs extra memory for the day for the dump off in case of K-Dump and it must be set up So pros for the hypervisor initiate dump. It's reliable as I've already mentioned. It doesn't modify the guest state and Guest initiated dumping is not always available. For example, if you think about small guests You cannot have the overhead of the memory required for K-Dump But the contra the hypervisor interaction is required and we have to transport the dump and The hypervisor needs the access to the whole guest state and in case of secure execution The hypervisor does not have access to the whole guest state. So how can you do hypervisor initiate the dump in case of secure execution guests? Obviously there was new hardware firmware support required. We have introduced something that Just requires an opt-in. So you have to set in the SE during the SE image preparation You have to allow Confidential dump and you have to see CK the custom communication key. That's as Stefan already mentioned to do so This case is used for the dump encryption and protection. So there were now four new Ultras across introduced. So first initiate configuration dump then second one dump CPU state Third one is dump configuration storage state storage means in S390 memory So just in case you just don't know and then the last Advisor call new address calls complete configuration dump So from a dumping perspective from Kemi KVM. So first we have to stop all these CPUs We do have to do sync to sync up and so on and then we read all the guest pages So read all guest pages is simplified here because of space and for simplicity What is actually happening here is Kemi reads the guest memory in case the page is secure We will get in storage program exception That leads to an export of this page The ultra is a will encrypt the page and then we can just read this encrypted page and this page can be written To dump if it's a shared page. We can just read the page Then the third step is initiate the configuration dump. We have to say hate or ultra is or we want to dump Then we will call for each vCPU the ultra is a call dump CPU state We will get back the encrypted CPU states then for every one megabyte of storage We have to call dump configuration storage state We will get the three components of each page because these pages are in Indempently encrypted by using AS XTS and for each page. We are using a different treat Then you have to call a complete configuration dump and you will get get encrypted AS GCM in this case The key derivation see the I read weak nonce and the storage encryption key So the XTS keys and in the end you have to write this dump data to a VM call file Obviously, we have to this new dump file is different to the normal SV90 dump file So here you can see how this new alpha looks like so it's Pretty much the same as before but now we have some additional information So we still have the node types for the information that the hypervisor knows so in this case for for the vCPU one we have the node type and TPR status and in all that Hypervisor does not have any confidential state or does not know anything confidential about the vCPU So that's just a case if you want to debug the hypervisor and it looks like a normal VM call Dump, but now we have introduced a new node type You can see it here for vCPU one NTS 390 PV CPU data That's just an encrypted blob and we will interpret it decrypted and there's all the confidential CPU data stored Then there's one PT load segment as before but in that case It's just ASXS encrypted and to store all the meter data. We have introduced new sections So we have one PV comp that's just a kind of of the ultra is a call Complete configuration dump and we have PV memory meter. There is the all the tweaks are stored and Because it's an alpha we have to introduce a section header string table as well So If you just someone interested, what's the value of this new node type it is Okay, now we will come to the life cycle So first we have to train away to the secure execution image We have to enable a dump feature and we have to set the cck Then we will copy the secure execution image to the host We will start the image. We will do the verification the high ultra visible Remember the cck for the later dumping then the se guest workload is running if at some moment We want to dump so came you will do all the ultra as a cause and KVM as I've already mentioned It will written to a file the core dump. There's a core dump will be transferred to the guest owner for example and the guest owner can use now a new tool with the name said get them to Decryptus dump and you will get a normal sv9 TV and call them that you can use and Analyze as you are used to by using for example crash so the command lines that are used here are the tool to generate the secure execution images chambered image and there are the new command line options enable dump and com key for specifying this Custom communication key As you can see here the command for creating the dump has this unchanged So you can just use lip word as usual rush dump dash dash memory only So there was no change in the QMU monitor protocol API so far I must say the QMU patches are still under review. So that might be changed okay Then the command for see get them because as I have already mentioned Each page is in them in them in the Indempen sorry for that Encrypted so you can use fuse for example to just create in virtual file system So we you will have on the fly decryption and if you're interested for example in debugging Kernel crash you do not want to decrypt the whole 200 gigabyte of memory for example But just all the two gigabytes in that case you can just use this option you can mount this Dump and then it will create by using fuse a virtual file system and you can use Contrastries crash as we are used as you are used to or you can decrypt the whole file So as you get them my dash dash key then you have to specify the custom communication key The decrypt encrypted file and then that's it and mount and the output file. It depends what you want Okay, so the count state is We need half the support of course so IBM set 16 The KVM changes they have already been upstreamed so it's included in the kernel version 6.0 release candidate one and Kim you as I have already already mentioned. It's still under review. So Look what right now. There are no changes required. So it just works gentle image changes has upstreamed with easy or since version 2.21 and For said get down where it's still working for across Love so let's summarize at station You can use it to verify the sec execution image instance It's useful to implicit as we on S390. We have implicit at the station and And now we have also explicit at the station After after transition into secure execution mode and it can be used to identify a specific image instance And you can attest without revealing any secrets Then let's summarize the confidential dump things first. You have to do an opt-in. So that's important to know here Then it's reliable and a secure way for hyper-wise initiated dumping because the actual guest data is encrypted No chem you monitor protocol API changes right now. So there are no changes in lip thought and See get down will handle the encryption on the flight. You can show us possible and Decrypted dump can be analyzed. For example using crash So thank you for your attention and any questions so far It would be great, but oh Okay, okay, the question was is it possible to you reuse the API for the live migration and Some parts of it sure but For the long run, I'm not sure Yeah Yes, maybe you can answer and it's not about the cck but about the whole ski documents So so the question was We rely on the cck for security of the of the secure execution image, right? We have multiple keys so But the secure execution image relies on the public ecdh key for the guest owner and The public private key bundle of the ultra visor. We only know the public key of the ultra visor, of course So But I think that's a question for Claudio You are you asking What how do you get? How do you get the private key in the hardware? Is that a question? It's already in Okay, so the question was how do we differentiate between different guest owners right each guest owner has his own public ecdh key and private key also but and You you use your own public ecdh key to encrypt the image and if you have a different guest owner you use your own key I'm not sure if that Yeah, okay Let me help here Each machine has a private key baked in and you as a customer just take The public key of the private key and you use it to make your image and then you give that to the hardware and the hardware We just unwrap it and the hardware is the only one that knows that Private key so it's secure all those keys are needed these other keys that I mentioned the customer communication key and what that were not those are keys that are baked into this encrypted blob first of all and Those are needed for these other dumping and Things but those are provided to To the hardware Through this encrypted blob and the blob is encrypted with the public key And the private of the machine the machine has a private key baked in at The root of the trust is the hardware and yes private pride. Yes So, how can you make sure that you're not faking the root of trust of course the public key is signed by IBM so there is a There's a route of certificates. So the private keys that so the Private key and the public key that for the specific machines are signed by IBM So if you look at a public key for an S390 Machine, you can just check if it's been signed by the actual IBM So we with the classic root of trust, you know, there's a private key per machine It's not an IBM white no it's an idea It's a it's a per machine and IBM does not know it anymore once it's baked in I Don't the question is if it's equivalent to a TPM. I'm not an expert on those things. So Me neither I think one more question, right Yeah, you had to question Oh That's true and now I repeat it the customer communication key is created during the image Creation and encrypted using the or a public key one and so AES GCM any more questions. Okay. Thank you. We are here. So more questions arise