 Hi everyone. I'm Naina Jain from Linux Technology Center, IBM. And today I'm going to talk about PowerVM platform TISTO, securing Linux credentials locally. This is a disclaimer which I will flash for a few seconds and I'll just read it as it is. This work represents the view of the authors and does not necessarily represent the view of IBM. All design points disclose here in our subject to finalization and upstream acceptance. The features described may not ultimately exist or take the described form in a product. IBM, the IBM logo and power system are trademarks of IBM Corporation, registered in many jurisdictions worldwide. Linux is the registered trademark of Linux Storewalls in the US and other countries. And other company product and service names may be trademarks or service marks or products. Okay, so let's get started. So as I said, I'm working in Linux Technology Center and I work in Linux security team and we work on enabling various security features on the Linux on Power. And currently, the thing which we are trying is to make use of a platform TISTO, which is a secure storage provided by the firmware. And which can be used for storing various security variables for the like a symmetric keys and symmetric keys, which are then used by various security features across the firmware state. So my original problem was exposed security variables is stored in platform TISTO to use a space for key management. And now I will take you through that how my problem got slightly twisted. And then finally, how did we make our way into the solution. So I have worked on secure boot early for open power. And we had a similar requirement here and then of exposing the variables to the user space and we had used assistive storage. And then that time we have also studied that okay, there are EFI variables and then they also get exposed by assistive space and then they moved to the file system interface. So mostly it is like it is either the surface interface which gets used, or there might be specific cases for file system interface. I just followed the approach and I created the driver for the platform key store and I exposed the variables by assistive interface and I posted the first version of my patches in January 2022. And then I got the feedback from the uppest team. And this one was from Greg Cage, who mentioned that this is like the third or fourth different platform specific proposal. And the second main thing was that to give up on platform specific user or kernel APS on this random Sisyphus or security FS files is scattered around the tree, and come up with a standard place for all of this. The first image I didn't understand here was about third of food different platform specific proposal because yes, there have been few but I wasn't sure which three or four we are talking about here. And then we also got copied on to the another patch series, which was in discussion in parallel in the mailing list. And this was related to confidential computing patches. And it meant for using security FS to read or delete secrets for VM encryption. And Greg Cage had given a similar comment today, referring to our pet series, which said, you all need to work together to come up with a unified place for this, and stop making it platform specific. At that point, my problem statements slightly got modified to expose security variables stored in platform keys to to use a space for key management using a unified interface. Okay. So we are getting into the maze now. And the first question came up is, unify what interfaces code or both. So I spend some time looking around and then hanging. We have to interfaces. And mainly, I mean, that is what mainly I discuss for now, there might be more in current but the context of security here. So one is surface, which is, then we assist firmware and another security FS, which is like underlying the system is but sis kernel security. And, and the point is, the system is mainly used for showing attributes and objects of platform specific firmware within the sub directly inside the system directly. And the security faces a pseudo file system and use for kernel security subsystems. In case of the code which is used to expose the objects onto the firmware, it is the driver code often have to interact with homeware. And then the, the exposed features are categorized by device or firmware features. And in case of the security features which gets exposed by a security of this, the driver code does not interact with formula. In fact, mainly this work on a security features. It's like TPM I'm a lockdown for those. And sis from as I said, it is mainly for the firmware and the thing which we notice is that mainly currently if I variables and open power security variables, which are used for form where security features. They are both exposed by a system. Even the EFI manifest, which is used by if a variables is mounted on onto the sis firmware mount part on. And there might be probably some of the platforms as well, but these are the two mainly which I have looked at. So here I'm not saying one is better than the other one like that. What I'm saying is they are just for two different purposes. And now, if we observe the kernel code structure. Okay, we all know that, okay, there are different directories and each of them have its own purpose like a physical storing file system code. I can swap that from specific code. There's what and drivers what I do not know much about them but I got to to look at them because of Coco patches and it seems there for virtualization. And I do not understand the exact difference between the two, the two directors but yes, they both seem to be for virtualization. So some of the observations I have from here are if I reference is our specific, but it is part of file system. I'm not sure why the security FS is a generic file system for Linux kernel security modules, but it's part of security and not of FS. Here probably the reason was because it was meant only for security modules, and we have all modules say security subsystems into the security directory so probably that but again I'm not very sure. Then what sounds like to have a virtualization code but KVM is part of this was then is part of drivers. Again, I'm not sure. And how drivers or what is different from what as I already said, I don't know the difference. So I feel there is already some mismatch here they are in the kernel code. There is a third thing which is there are different environments and they differ not only syntactically but also cementically. So let's see. Let's consider the platforms like if I open power power VM platform to says will be the new one and then the cocoa and they have a different requirement for the users. Some of them want to allow the right to create and delete of the files, but some of them want to only do the read and remove like cocoa. And there might be others which only do the read write like open power to get it an unauthenticated variables. It does allow live updates. However, it has its own concept of policies and the consumers I already existing code, but one of the comment. What we received from my to get it was that these are cementically different and if we tried to use that for non EFI platforms. It can become confusing for user and it is also maintained difficult to maintain the consistency of over the time and which we fully agree with. And so yeah, we had done our own in the platform specific surface interface for open power then. EFI variables open power are already in upstream and cocoa code was recently upstreamed and power via platform key so code is what we are trying to upstream now. So what do we do next now. And then there was taught that don't we need something similar to sis kernel security for former security features. And in the similar lines, like security FS should we develop a similar file system for former security features. And that is where I want to propose a new former security file system called former security FS. So two parts here. First is at the code level. The proposal is to introduce a generic layer from a security FS which provides API, which can be used by platform specific code to implement or support file system or I know the operations based on their requirements. In the platform specific code itself they can handle the underlying semantics requirements or the semantics implementation. So we are talking here two layers of the pool. Then for the user space, we define a well known mount point, which would be sis from the security. And this would be so it looks similar to security FS with the differences. What is its purpose and it is mainly for former security first features exposed onto the part of system there. So, how do we differentiate the security FS with former security FS security FS is exposed by a sis kernel part by the former security FS is exposed by a sis firmware part. The main purpose of security features is for Linux kernel security features, which do not have to interact with firmware. The changes are used by Linux kernel at runtime and are not persisted. For the firmware security FS it is to support firmware security feature features kernel access mediator between user space and firmware. Objects are managed by firmware persistent and then access across from this day. The part the reason the security FS came into was so that all LSM scan use this instead of creating their own. And exactly that is the problem now we are trying to solve for the platforms and now that is the reason I'm saying for firmware security FS so that all platforms can use this instead of creating their own. Both of these systems allows colors to define their own I know the file operations. And security FS may not have more need of clear letting user to create because they do not get purses, but firmware security FS does have a need to let user to create their own files. Where they want, where the platform specific requirements might want to get them persistent on to the security storage. Security FS may not have to deal with complex firmware semantics, but from the security FS may have to do. One of the question which people might ask is, if they are so much similar with only three difference for kernel and firmware, should there be be should it be should security FS be extended. Well, says kernel and says firmware are meant for kernel and firmware features respectively. And so we would prefer to expose firmware managed objects in this format, because that is the key differentiation here. After coming up with this point of the generic firmware security file system. Now I will tell you how we are trying to use it for our platform specific requirement. First I'll give a brief overview of platform key store, which is a local small protected keystone managed by PowerVM access via HQALS. It is an instance available for each LPAR. So each LPAR has their own instance of platform keystone and where they can keep and store their options. Data is encrypted with unique key for each VM and stored in SP flash. It's support for active and active live partition mobility. And there's a redundant copy and kept it and stored on hardware management console for any hardware failure. I do want to thank Joel Wolfrit from IBM firmware team who shared his this slide and who is mainly the owner for this work. Of the platform keystone for the firmware. And this is the use case for which we want to expose the variables using the CIS firmware security part and with the generic files from a security file system. So I'll give a brief on to the PowerVM guest secure boot flow here. And so this is how we have the hardware system and which is a stack of host boot and hypervisor which which makes a part of the firmware. And that is the root of trust and this is also the firmware secure boot supported so there's always the firmware secure boot enabled here and all are verified so this forms the root of trust for all the guests. Each guest has its own stack where the first list partition firmware pfw which loads grub and then grub loads kernel. The partition firmware is verified by hypervisor which verifies grub and then verifies the kernel. Now when we are talking about verification for sure we need the keys. And where do we keep these keys so these are the keys which gets stored on to the platform key so as I said each guest has its own instance of platform key so so yes there's those keys are separate for each of the guests. The kernel provides an interface for exposing these these keys or key objects on stored in pks to the user space so that user can modify these keys as required. For example in case of key rotations or if the signing keys or the grub signing keys are leaked they need to be updated and some of them because of the vulnerability there might have to be the revocation of the kernel binaries or the grub binaries. And so all those which gets stored in pks might have to be updated dynamically and that interfaces what kernel is trying to provide. The kernel talks to the pks via hypervisor with the use of edge call which is called hypervisor call. The grub and pfw are mainly the user of these keys and they also use the edge call or the OF client interface and request the keys from hypervisor and further from the pks. So now we understand what we are trying to expose where are we trying to use these keys. Next I would give a brief of the various objects which we are storing or maintaining in platform key store for the purpose of secret booth and we call them authenticated variables and that is because these variables are signature verified before they are. updated. We have a platform key and KKK which is mainly used by PowerVM to authenticate updates to other variables. Then there is db key which is for kernel verification key and grub db which is for grub verification key. So the, yeah, we are keeping the grub keys separate from kernel keys that way if there's a leak or leak get exposed because of the losing grub privacy or anything that doesn't expose the kernels or that doesn't get wrongly used by the exploiter. So, yeah, we prefer to maintain the grub keys and the kernel signature separately. Then similarly there is a dbx and asbed which is for the revocation list and dbx is for kernel and asbed is for revocation list of grub. There is a discussion to use asbed for kernel also. However, it's still in discussion and here we are considering that and which might be placed dbx overall so yes that is in consideration. We are also proposing two more variables which is trusted CA and module db. These are mainly used by kernel and for third party module verification keys. The trusted CA stores the CA certificates which is used for signing code signing keys for the modules and module db will show the code signing keys which is used for signing the third party keys. And these are like CA keys are loaded in machine key ring and then the code signing keys are loaded in secondary key ring and this is as per the current work going on into the kernel mailing list. I would like to emphasize here that PowerVM variables may sound from the names and all like EFI variables but these are not EFI variables. And apart from this main use case which we are trying to satisfy now, the platform key store can have other use cases. It might be used for storing symmetric keys for like boot device encryption, self-encrypting drives or for like unlocking required logical volumes without requiring a pass phase or for other public key and certificate protection. And they may have their own interface for user or they may use the file system. Okay, so after having a brief on to the what is where and why we are having this requirement of exposing user variables and what is pks. Let's look at the design. So the first layer is the driver support, which is very much like a specific code which is required to interface to firmware that does the edge calls for talking to the pks. And then there's a generic firmware security file system there. This will be used by by platform specific code to expose its variables. So the firmware security fs. It defines up from the security need function, which can be used by platforms to implement your own for filling the super blog. And then the third is for exposing a secure variables so power PC specific code to expose variables are using fs from the security fs. And here we implement art from the security need function, which again goes into the specific code. The file operations and I know the operations together we support is read write create. This is how the interface would look like. When for now I'm mounting firmware security fs and this is how it can be, but it can also be done similar to what security fs gets done by having it part of system D or the other options. Yes, it gets mounted on says firmware security. The sectors is what created by platform specific code. And the variables are an example of the list of variables I'm showing here and how you can write to it and then that we do not allow removal and only root is allowed to do the read write. So you can see how we have gone through this maze and we have come up with the solution which solves our problem while including community feedback for unified path for users. So, I would come back to this question that does this satisfy unification requirements for new firmware features. Well, we do think yes. And we have not come up with a common interface, which can be seen for Colonel security features assist Colonel security and the firmware security features assist from the security. We also have done come up with a generic code, which so that there's not need of platform specific file systems or surface code, they can use from the security fs is generally interface to expose variables. Yes, they can define their own I know then file operations. Based on their requirements and their underlying semantics, and it can be used by what will I say non virtualized firmware. And they are not to be differentiated. Existing features may or may not use it, however, the main intention for this is for new interfaces, so that we do not continue the scattering of the files, which was the original requirement from the great kids. And so what are we unifying, we are unifying both. We have unified the interface, and we are also unifying part of the code by providing the generic file system. I do want to thank my team where we had a lot of discussions and inputs for the requirements and design of platform to store and its interface. I also want to thank Alan Palmer, George Wilson and Mimi Zohar for providing reviews and inputs on to my presentation. And I want to thanks Linux Colonel community for the review. I request people to give their feedback on my patch set and also on to this proposal. And I hope this is an acceptable one. Thank you.