 All right, thank you So welcome everybody to our presentation now on the namespacing of the linux integrity measurement architecture My name is Stefan Berger. I work for IBM research and this is my co-presenter I'm Christian. I work at Microsoft. Yes So just for those of you who don't know the integrity measurement architectures Just want to say quickly what it does basically it it's allowed allows you to configure it We are policy and it extends the trusted boot that the bios for example starts and allows you to measure applications and executables and libraries That you start in under linux and then also it extends secure boot where it allows you to Enforced the signatures on on executables So that you can ensure that only sanctions executables are being started on the on the linux So that works today on the linux host, but we want to make that available to Containers this is not the first time. We I have actually presented. I'm a namespacing a couple of years ago, I Also already presented it at plumbers We had a different slightly different architecture, but I think due to the help of Christian and James bottom Lee we have put the train on the right track or better track this time and Hope to be able to upstream this work the first path series of the patches is out on the mailing list and Well, we hope for collaboration to get the the next part upstream Right I'm one of the I would say main reviewers. So I'm not necessarily involved in all of the architectural stuff and so on So I'm the running commentary during this presentation more or less Yeah, so the motivation is probably something that Stefan can cover a lot better better than I can but you can see the points like the configurable I'm our policy Allowing to what do you want to say something about this yourself? but yeah, basically what we want to be able to do is Allow I'm a to be able I'm a to be configurable inside the namespace With a policy so that for example when you have a when you said auditing rules inside the namespace that the The namespace is going to emit audit log entries on the host and ideally those audit log entries on the host would be Distinguishable from those That you're setting from the on the host, right? So that you have distinguishable audit rule audit messages that originate from the container with the stores from the host And then we want to be able to use the I'm a policy for Enabling I'm a measurement. So we want to have a log per I'm a namespace so that you can see what exactly was executed inside a nightmare namespace and then I'm Logging and measurements often go hand-in-hand with attestation, right? And in that case we want to be able to attach a virtual TPM to the I'm a namespace and the virtual TPM Then would vouch via a PCR quote for the validity of the log right and then another major use case as I already mentioned before is That we want to be able to lock down containers That means that we want to make sure that only sanctioned software from a well-known source can be started within the container So unauthorized software that some intruder may bring into the container should not execute So also that should be configurable with a policy with the I'm a policy and for that we have what's called I'm a appraisal here We want to be able to Have per container key rings That means each container has its own set of key rings that are independent from those of the host so that you can run your Distribution a different distribution on the host where there's a different distribution inside containers that brings along different keys So we would implement a bring your own Key model and then for visibility what you want to be able to do is you want to be able to run key lime inside the containers Key lime gives you allows you to set allow lists For example, where you can verify or reconcile What has been started inside the container and make sure that whatever is running inside the container? Is a part of that allow list the allow list basically is to define three hashes of these executables that are going to be started Or it allows you to detect if something I'm even a Non-signed executable was attempted to be started inside a container And of course Namespacing is a type of virtualization and we would expect that the I'm a namespace inside the Insider container behaves just like I'm on the host so that gives you the least surprises Also, it accommodates software that has some kind of assumptions on how I'm a works on the host and now can transfer these assumptions inside the container right and so Okay, we have a Lot of namespaces in the next nowadays As you can see we have C group IPC. So network mount pit time user user namespace and UTS namespace and so One of the starting questions is how the I'm a namespacing story should look like What do you want to have a completely separate namespace? Which would mean all of the infrastructure that you see right here? So if you create a new namespace via clone or on share, you'd need a new flag For example for user namespaces, it's clone new user. So we would need a flag clone new I'm a and Then if you want to move into a separate I'm a namespace you use the set on a system call which allows you to switch namespaces and Then you also need to clarify things like If you create a new I'm a namespace, is it available to unprivileged users? Is it something that should be guarded behind capsis admin? So it's it's kind of difficult from an architectural perspective at least for me initially it was where and how to fit in the I'm a namespace So the first time I encountered this The proposal was clone new I'm a Which would be passed to clone but then and I mentioned this to Stefan earlier you get into issues if if you need to specify a lot of different parameters that need to be set up at namespace creation time Then you need to pass all of that info through for example the clone system call so you end up with a lot of I'm a specific stuff inside of core kernel process creation functions, which is something that I found kind of not very tasteful so The solution here is I think that Stefan has chosen is a bit different which makes it easier To implement in my opinion another difficulty is that there is no concept of containers in the Linux kernel Which makes it's also kind of If I'm a namespacing is mainly useful for the container use case Which is that's what I understood it How it's supposed to be then it becomes kind of difficult To square this with the fact that we don't really have a notion of what a container is inside of inside of the limits Colonel this is something a user space concept as search Helen once put it a user space fiction And that's mostly true. So we need to figure out If we don't make a separate Ima namespace What other namespace can we piggyback on or use for Ima such that it still becomes Meaningful and useful to container workloads or to most container workloads on In user space and that has quite some consequences if you think that's on the next slide. Thank you So the approach that Stefan has chosen Was to piggyback on the user namespace. We it makes a lot of sense in various ways And there is also a social engineering argument I think that is quite nice in the sense that We want users to use unproledged containers and by piggybacking the Ima namespace on to User namespaces you're also forcing users that want Ima that want Ima namespacing or The integrity guarantees that Ima gives you when used in a container to also use user namespaces, which I think is quite nice Side effect. It also has per user namespace isolated key rings. I think this is something that you were quite excited about because you need this and Normal users can also spawn an Ima NS that probably means unproledged users. That's what you're getting at Right. So that's that's something that I'm kind of on the fence about in not necessarily because of Ima But the user namespace is a big and important security isolation mechanism for sure but it also lets you get at a large attack surface in the kernel quite easily so by having unproledged users creating user namespaces it's always kind of It's not always a good idea Like if you have a user after free in a subsystem that you can control from a username space and you already have a privilege escalation so which makes it kind of Difficult, but the Ima namespace putting it on the username space also makes in my in my opinion makes the The ownership and delegation model a lot a lot clearer So all of the privileges that you require would be tied to the username space So there is no separate Privilege levels that you need for the Ima namespace per se you can all do this within the username space, which is nice one of the really big requirements that I had and that I talked about quite a bit on the mating list is Putting Ima namespacing in some shape or form into the username space cannot have any impact on the performance of username spaces or on the creation of username spaces when Ima is not used and I think we've we've now gotten to the point where this is actually the case and Yeah, there's some a lot of technical details on how to make this work nicely because so that you don't up end up with We had ref count issues So Ima namespace taking a ref count and username space user namespace taking a ref count on Ima namespace and all All kinds of weird lifetimes issues. I think we solved all of that And the overall design is you create a user namespace and the Ima namespace is more or less created as soon as you mount a Security FS instance. So as part of the work that we did I Basically I made it possible to namespace security FS Or it's at least that the design that I kind of gave you Which I think is the right thing to do in any case because the way security is after Don't want to attack anyone but the way it's security FS is currently implemented is not very nice and so in order to make this work for the Ima NS use case we had to kind of namespaces and I think if we do it Right, then security FS itself can actually benefit from this by making it a File system that works way nicer in namespaces. Anyway, why the security FS approach? in my opinion The fact that there will be a lot of knobs inside of an Ima namespace that you will need to able to twist and turn and so on Makes it makes it well suited to tie it to the security FS File system because other LSMs like app armor, I don't know about as a Linux, but App armor for example, and I think a few others provide knobs inside of security FS already so there is precedent for doing that and You don't end up with a lot of unnecessary I octals and so on or dedicated Ima system calls or whatever or you have to put it into struct clones So by making it part of a security FS instance You actually can put in all of the knobs that you want in there one slide one thing that I found a bit unpleasant Is but that we have to do this way is you mount security FS Which creates internally for the kernel and you Ima namespace, but the Ima namespace is not really active You can then configure it and set it up and then you can activate the Ima namespace at which time It actually takes effect But this is some I think there's some internal Ima Requirements that kind of make this necessary. Yeah, I mean, it's basically what what you find in that configuration stages You find those Linux kernel parameters in SysFS again You write your values that you typically would put on the boot command line You write them into SysFS and it becomes the configuration for that namespace So you can run your host for example the shot to 256 for measuring and you can run your containers with a Different hash shot 384 if you wanted to for example or what makes a lot of sense is that you have the template name inside the Container a different template for example running on the host So you find template name configuration inside the container and that's basically yeah That's where a security FS actually comes in very handy to be able to configure your container and yeah You're activate you go from a from an inactive namespace to a configuration stage To the activation of the Ima namespace just providing again into security FS and then you have an an active Ima namespace Once you have activated the configuration is locked You cannot change your template anymore, or you cannot change your your hash anymore So that should give you a lot of Should give a lot of flexibility also to extend this in the future and provide more configuration knobs for containers So here now I show you a short command line that I for example use for testing things to create a Directory structure very simple. There's gonna be a very simple container. I use busybox a lot because it's don't have to Copy a lot of dependencies in terms of libraries into this simple container And then you run this unshare command line that creates basically a user namespace mounts a proc file system in it creates a pit namespace also in this case because I believe you have to actually do this and then puts the root of FS to where we copy the executable in and that's our mini container just to Test out Ima namespacing and we do the mounting here Security FS is mounted here in this case on on the mount Directory because sister fest is not easy to come by in this case and be activated. We don't go here through Configuration stage and then we run an audit policy for example inside that container Now the thing with the audit policy here is You have we decided basically that you have to be host route to be able to set that audit policy in the container the concern here is Primarily that you can cause flooding of audit logs inside a container if you wanted to I mean we opened up the Creating Ima namespaces for normal users now you and also users gonna come use containers where They hopefully have Ima namespacing enabled so not everybody should be able to set auditing rules and cause lots of audit messages on the Host so that it drives audit Subsystem into dropping messages and audit messages get lost so that is a restriction on the auditing rules However, normal users can so Unprivileged users can set measure in a praise rules inside of containers So that's there's no there's no restriction restriction here on that So there are lots of challenges when it comes to namespacing Ima and I just want to throw a couple of you and we go more and more detail later on so Basically grouped and into different Groups here on the conceptual level on the implementation level and performance level so on the conceptual level We basically don't want route to abuse user namespace spawning and an Ima namespace configuration and so on To hide its activity on the host Typically what you can do is you can write a policy so that whatever route does in the TCB Is going to get locked in the Ima lock things get measured and locked in the Ima lock So we don't want to open up here a security hole that route just spawns username spaces configures possibly Ima namespaces and tries to hide its activity in That he does basically on the host when he shares all the other Username space if you see that in the subsequent slide then I already talked about this audit messages that are emitted from Ima namespaces They should be distinguishable from those emitted on the host There's a patch series out there and has been out there for a while actually that allows The configuration of containers with an ID that then allows you to distinguish these audit messages I admit from the audit subsystem from those on the host by setting a container ID And then I already again also talked about this We want to avoid the flooding of the audit subsystem with messages so we don't allow route to set these audit messages for Inside the Ima policy will only allow route to set these audit messages prevent Unprivileged users from doing this so they don't just create Containers and then flood the audit system. We also want to allow this allowed indirect control to Unprivileged users to cut audit messages and what this means is basically that you can the users would be able to set Policies which then audit which then are audited line by line So again, this could be abused by just creating lots of Containers just quickly setting a couple of policies flooding the subsystem tearing down the container and doing the whole thing over So and then what is important is we can distinguish Do you want to be able to distinguish where the audit messages are coming from from which container? We also need to be able to distinguish where the log entries are coming from from which container We'll see why the while that is important So on the implementation level we have a particular issue here with having to track state of Files for each individual namespace so that we can track which particular files have already been measured by containers and appraised and then particular shared files between containers are an issue Where we where it can happen that one container modifies a file that then has to be again Re-measured in all the other containers that sharing that file So we have to organize our data structures accordingly, which then leads to some some particular locking issues when we as we will see Later on in the presentation So each user namespace needs a distinguishable UUID so that we can distinguish where the Measurements are coming from of course testing as always is a big issue. We don't want to break anything The size of the undertaking here is that there are more I have more than a hundred patches now that make up I'm a namespacing all the different Parts of I'm I'm auditing. I'm a name. I'm a measurement and I'm a appraisal So testing of this is important that nothing breaks and nothing breaks on the host And something that I have not looked at at all, but that may be a concern also in the sometime in the future is Crew this is container and or check some checkpoint and restore in user space And what that actually means to I'm at the implementation But also on the conceptual level for example if you were to checkpoint a container Reboot the system and restore the container what that means about the logs that you have for example on the host because they're now All erased or they don't look as they did before you did the checkpoint and restore All right, so on the performance level we have something that we need to introduce primarily to to the Wanting to not allow user the root users to hide its activity what we call hierarchical evaluation of the activity in the I'm a namespace Again, I have some slides on that so for now. It's just photos at you Yeah, and then there are some Performance issues to the locking that is needed for the tracking of the state of the of the shared files So one of the challenges again going back to the first item in the previous slide you are what don't want to allow root to hide its activity in Parent namespaces so for example Share unsharing a user namespace that creates a new user namespace And then the root user runs some program in that user namespace That should be subject to Evaluation against the policy on the host. He should not be able to hide its activity because with this unshare Command line here. You basically have just unshare the greater the new user namespace But you're still sharing the same pit mount and IPC and so on namespaces of the host So that's pretty much like running Executed on the host but here you run it in a different user namespace or you have that example here of Creating nested user namespaces where you can go to a 16 or 32 levels deep and try to run your programs there And all of that should still be visible in the parent namespaces So this hierarchical requirement actually makes it kind of nice because there are two namespaces pit namespace and user namespace Which are properly hierarchical so they're already walked anyway and implemented in a way that you always know what's with the relationship between the parent and the child namespaces So I'm on namespaces. Therefore are not completely isolated, but for example if you look at For example the network namespace. It's not completely isolated either Otherwise you wouldn't be able to send any network packets out. So you have to be able to connect them between Container network to the to the host network. So there are ways to do this similarly. I'm a namespaces are not completely isolated either So what we basically want to do is here as you see in this green nested child I'm a namespaces that if there is a policy here that's a lot that where you're supposed to be able to whether the container is supposed to Take measurements of the executable that you start in there It's going to start the executable and take a Measurement of the executable and log it in in the log inside that I'm a namespace, but then due to our requirement that root actually also regular users are not supposed to hide activity in In child namespaces we step back to the parent namespace Evaluate the file activity that you have done in the child namespace against the policy of the of that child namespace and then for example log again here and we do this all the way back to the root I'm a namespace so that All the activity that happens in child I'm a namespaces is again visible in the root I'm a namespace So but that also means that for example when you when root is Let's say running an I'm a an executable. They're supposed to be appraised Inside a child namespace it may not execute because it's the child I'm a namespace doesn't have a key that a lot would allow it to To execute so that on the other hand would mean that root cannot just spawn an I'm a namespace Because he doesn't have the key to execute this executable Escape into that I'm a namespace or the username space and try to execute that file there and there's no There's no policy for example set. So he's gonna be subject to every evaluation against the policy on the host so we call this Hierarchical evaluation of the I'm a policy and that has some effect also on on the performance of the system and so on as You can imagine you have to do more more work obviously However, you can write your I'm a policy also in such a way that all I'm a namespace activity is being ignored and that's You can do this through you ID selector and how How you map for example to user IDs in these child I'm a namespaces. So every Every spawned I'm a namespace may then cause logging on the host and What we do here is we basically create a new your ID for each user namespace user namespace already exists today They don't have a configuration stage So what the what we resorted to is basically slap a you ID on this user namespace So that it becomes this Distinguishable from the activity of all the other user namespaces that are being created What that however means is that you have possibly lots of different measurements on the host each with a different UU ID. So for example, if you were to execute the above line multiple times in Yeah, and run it in obviously in in child namespaces. You may get different Give different log entries depending on whether we decide to Relog the sum program here in this case are in the in the parent namespaces Also, then for that we introduce new I'm I'm a template that allows you to make these UU IDs visible in a column so you have the log and you see an additional column there where you the UU ID of the of the username space where the executable was run was How that so that you see where it was one So then coming back to the the organization of the data structures Where we need to make sure that we can track for the files Between the file the state of files that are shared between I'm a namespaces. So there's currently what's called the I'm integrity Cash the I end cash where I'm a tracks each individual I note whether it has Already measured the file appraised the file and resets the flags if the file for example was changed or needs to be Reappraised so what we will introduce is a what we introduce here is and per I'm a namespace integrity I end cash That is mostly to avoid that for individual namespaces start locking each other out too much And here what they're striving to do there is is to optimally organize our data structures. So we want to use lists that have access complexity in order of O of N and again the trees of log of N and At the same time I work avoid searches across namespaces so that for example if you have a shared file You don't have to search across all the namespaces and see where has this to with which namespace has this file been Been shared so that it can for example It may have to be remeasured or when you reset flags that you find them pretty quickly So what we have here is what's out there right now is what what is a global? I'm integrity cash. It's a basically red black tree where State represent state information for each I note that was accessed per the policy of I'm a is Is maintained them so fires that are subject to the I'm a policy that are Maintained in this RB tree And then what we now do is we introduce an RB tree per I'm a namespace the goal here is to Spread out the locking between these individual namespaces so that they can have better performance Right can access their file files faster because they don't lock each other out when the when the RB tree Needs to be modified so we track individual I notes that are Executed in in these namespaces and In an RB tree in individual RB trees and then what we do is when these shire files are shared between Namespaces we also put them on a linked list so you if you for example want to reset flags on I know it number two You walk across this list here, and you reset the flags for example Causes a re-measurement of a file when the file was Modified in one of those namespaces so and in this case So we have done the problem that our files are now on on two lists one time. It's on the list to For each I note and then on the other hand it's located on on an RB tree So that leads to what I would describe as a mesh problem And the situation that you're going to run into is that at some point when you tear down and I'm a namespace You have to basically delete the RB tree But if what can happen at the same time as you tear down the RB tree is that you Delete files in the file system, right? So that actually happens and you're running in that problem that these The state information that you have stored and that this look it linked onto this linked list And it's also located on the RB tree now has to be Deleted from both at the same time now the solution is here And that's a lot becomes a locking problem and the solution is basically what I call group locking Where we only allow Concurrent namespace deletion or concurrent file deletions. So either you delete you're accessing this All linked lists that this you're coming from from the left side or you're coming only from from the top side So that's the shared accesses to these notes here that are shown as black dots can be avoided and that you don't run into this linking problem that This access problem on on on items that are Located on to on on the RB tree and on on a linked list So now to the into the area of performance and I've done some performance measurements For that I have compiled the Linux kernel and I'm using this policy here that shown the file check rule that's that's in this Policy because there are a lot of measurements to be taken when you compile the Linux kernel Basically every file that's opened every Dot c or dot h file that's opened is going to be measured. So that puts a lot of load on on Ima to see that Basically, it's a good use case for for measuring the performance of Ima the other two rules basically measure Libraries and executables that are being started during the compilation process all test cases that we have done are Compiling the same Linux kernel with the same config of course so that every every time the same kernel is being compiled and What I did is I derived from a Fedora 36 kernel The the configuration that I'm using for the Ima name spacing the Ima name space patch kernel It makes a big difference which one you're using here as a baseline to derive the configuration from because there are Differences between 5.17 kernels versus 5.18 if you use them if you compare the performance of stock 5.17 versus stock 5.18 Have have performance differences. So I'm using one that is as close as possible to the one that I I'm currently working on for for development purposes So all patches of all stages are applied in these tests. It's more than a hundred patches The tests inside the Ima name space use the same policy as on the host But I'm wearing this sometimes I'm running The host with a policy sometimes only inside the Ima name space and sometimes in both So that causes as much load as possible. I do a make clean before a reboot so that cleans all the files nothing Nothing gets that every every time you report the system. It starts with a clean kernel already and then we do the build I'll do the test the three test runs each and Take the average basically. So this is how this this looks like The the baseline here was a 5.18.5 kernel from Fedora I applied the policy on the host built Linux kernel on the host took 358 seconds then I Used the 5.19 kernel that I patch with Ima name spacing support again built on the host use the same policy on the host So it's pretty much just a different kernel, but due to the locking that we are doing here this group locking that I had to introduce there is a little bit of a performance degradation So you have it took 360 set two seconds. That's one percent performance loss basically when I started compiling this from inside the host from from the host so the next the green one here is basically I'm building now inside the Ima name space on the pot There's only a policy on the host the Ima name space doesn't have a policy doesn't do any measurements So this is pretty quick the the measurements The the results are pretty much the same here again 362 seconds because we can skip over the measurements taken inside the inside Inside the Ima name space and going immediately to the host. So there's no There's no performance degradation expected then I did a policy Put a policy inside the Ima name space only for some reason. I got a little bit different Performance numbers here is 366 seconds that actually shouldn't be the case because we're on the host There's no policy. It should skip over that pretty quickly When when it does the hierarchical evaluation and then Let's currently work in progress is when I have a policy on the host and in the Ima name space In this case, we have a lot of performance loss at the moment because Every file is being measured twice That's measured inside the Ima name space and then measured again on the host instead of recycling for example or trying to find When you go on to the host trying to find a measurement that was already taken in an Ima name space When you do the hierarchical evaluation So now to a test suite I build a test suite Based on common available tools the tools the unshared tool that I have showed to you earlier I'm using NS Enter busy box because it's easy when it's statically linked just need to use one executable Don't need any dependencies And key control in EVM control are commonly used tools when you do Ima appraisal and when you deal with key rings and so on it provides coverage for nearly as every aspect of the Ima Enablement in the Ima name spaces. I'm using different policies in these containers Auditing policies measurement policies appraisal policies or combination of all of them It touches most of the aspects of the Ima policy language that is relevant to containers. So all of the language pretty much is covered with that I'm connecting a virtual TPM to Ima name spaces to test that the Ima added virtual TPM takes gets all the measurements or that gets all the PCR extensions and the number of PCR extension is correct and Well, I can run many hundred of these containers concurrently the intention here is of course to make sure that locking holds up and We're testing also with nested use on Ima name spaces to make sure that the hierarchical evaluation works correctly that's executables that are Not signed on the host don't execute because you're not trying to run them on inside the container Yeah, so you can run multiple instances of this test suite concurrently again to make sure that the locking holds up and In that case you may get some errors Some timeouts that you can trigger obviously you're reducing the number of cycles that the test suite has to do its work So you might run into timeouts or what can also happen is log rotation that occurs Because we're graphing through the number of results that have been put into the audit log log rotation can come into the way and And the number gets good So and in terms of the upstreaming that we want to do we put this into Basically four different stages the first stage is out there That's where we virtualize the security file system That allows us to run multiple instances or create multiple instances of security FS Then we have support for spawning an Ima name space That is using mounting of the security FS and Activating it. Yeah, there's an Ima policy support in this First series that only allows you to enable auditing message because we only enable auditing because it's the least amount of work And so the least amount of patches That need to be upspent and then there's Linux capability support for being able to The user to for example set poly read policies in the Ima name space that he otherwise Regular users wouldn't be able to do so in that in that part of this staging We basically kind of looked for all the static variables and global various and put them into an Ima name space and Yeah, but it's and in the second stage you want to enable Ima measurements. That's a couple more Patches to it. There's gonna be an Ima names per Ima name space log That's behave just like the log of the host and in this stage We will add the configuration stage To Ima name spacing that system be a security FS. It is not necessary to do this at the earlier stage So the goal being I hope that after we have finished stage one most of the stuff is basically Things that you need to do in Ima Not in generic infrastructure. Yes I mean there are a lot of things are being put in in place in in the first in the first stage And there are some things with the locking that are maybe a harder to do to the chest But once we have that I think that the rest should become a lot more easy and then in the third stage we want to enable Ima appraisal with Isolated key rings for each Ima name space again need to introduce Linux capability support here So that regular users can also write security dot Ima Extended attributes which they are not able to do today for example. It's that's a privileged operation So I'm being told two minutes And then also in this case we have to touch EVM The extended verification mod extended verification module that is pulled in due to Ima appraisal in the first stage We will then add the virtual TPM support So the prototype I build a prototype with a run C that is a pretty commonly used Golang library uses several Container projects and you'll find all the things pretty much automated That have been done before here You have to write a configuration file where you put the configuration information into this configuration file to set the hash of the That is the namespace is to run it with the template that the namespace is to run with you put this in the configuration file and Run C sets it up for you It also allows you to attach a virtual TPM actually you can attach Merch multiple virtual TPMs and you can say for example if it's the virtual TPM Support is to be run software TPM setup, which simulates the manufacturing of a TPM It also simulates what's typically done on the horse via drake cut scripts and that is It tries to find in the directory structure the keys That it needs to put into the key rings so that Ima appraisal has the keys needed for an evaluation And also it loads the Ima policy in the container so that and that Nothing of what you have put in there in terms of keys or so or signatures Gets modified while after you build your container. It's a good idea to sign your containers So that this container is basically protected and the keys inside that container protected in the signatures inside the container Protected so now to the summary you should think of namespaces as instances of objects Basically, you want to create new objects put put your variables in a in a in a data structure so you can create those new data structures and Basically, what you have to do is you have to pass around In our case the Ima namespace data structure between the between the function then I already talked about this Ima has some unique issues That we have to cover. There's the hierarchical evaluation of the policy. There's this particular Need for tracking the state of files between the shared between Ima namespaces And I would suggest if your namespace a subsystem make it work like the system works on the host so that it causes the leaf the least of the surprises and At least in our case test driven development has helped a lot detect errors of Yeah, no, it testing is always a good idea So some acknowledgements here at the end. So I want to Thank James bottom Lee and Christian browner for putting the train on the right track and for the Multi-instance capable security file systems and for the patch reviews Dennis smacking for the work on the Ima namespace capability Support and Mimi and Sir Charlene for feedback and the patches. So I think we're Actually over time Thank you very much We have a Virtual question from an attendee mark Novak. How are vt. PMS endorsed and how is their internal state protected? Yeah the internal state of the virtual TPM is primarily protected through encryption if you wanted to but then you have to have trust into the The root user on the host right so that the state is being written into files So you have to trust it and the endorsement. Yeah, you have to I guess what he's alluding to How Where the certificates are coming from who's creating the certificates in this case you have to set up your own certificate infrastructure so that these Certificates that you create for the TPM. They have to be files basically that are put on the on the file system. I Hope that answers. I had hope that answers the question Anymore questions. Otherwise, I guess we all want to go to lunch And then if you want to collaborate with me, please I'm open to collaboration. It's a lot of work to get that stuff upstream. I think Thank you