 Good afternoon. I'm Crispin. I'm here to talk to you about App Armor. I'm going to make sure it ends on time so you all can get to the awards ceremony. So what is this App Armor thing? This is my talk overview. I'll zoom in a little and iteratively describe App Armor in more and more detail. Some deployment scenarios, a demo, and how it compares to other products. So the security problem that App Armor is attacking is the problem is imperfect software. Reliable software does what it's supposed to do. Secure software does what it's supposed to do and nothing else. And it's the funny something else that allows the attackers to come and tickle your software and make it do something peculiar and that's where they get you. So the solution is only use perfect software. Yeah, slight supply problem. The App Armor solution is enforce that applications only get to do what they're supposed to do and something else is not permitted. So what does do mean? At the ultimate detail of what this program should be doing is the code itself, which specifies it in every last little bit of detail. Unfortunately, we already know we can't get that right. It's too detailed and therefore wrong. To make it correct most of the time it must be simple. What App Armor attacks is operating system resources. It's an OS based host security system so it can mediate what the application can do with the operating system resources. So that's what we're doing. So what would you do with that? You could make machine be network secure. You can find all of the programs that are listening to network ports. And if all the ports lead to App Armor confinement policies then there's no way for anybody on the network to get at the machine unless it's going through a policy somehow. So this reduces the scope of securing your machine a lot. It's down from every program on your machine to only these ones that you have confined by policy and only within the range of what the policy permits. So it's still your fault if you get the policy wrong but it's much easier. So is that really secure? Well, that's hard to say. Security is what you call semi-decidable. When it's insecure you can demonstrate that. Hi, Rochelle. But when it's not to show that it is secure you have to show the absence of hacks and that is like well show it to lots of smart people and wait and when they stop coming then it's probably pretty okay. Hence all the DEF CON talks about breaking things and relatively few about securing things because showing that it's secure is hard. So we put it to a practical test. Let's show it to some smart people. DEF CON 2002, we took App Armor into the CTF games. There's going to be a few slides here describing CTF. Sorry if you already know this stuff. The idea was to put an App Armor machine in the line of fire and let attackers hack on it. This is how CTF works. That every team has a server and in the center of the network is a score machine that comes along and it has network sex with your server. And if the sex is satisfying then it awards a point to whoever's flag is on that machine. And of course the attackers are trying to change that flag. They're also trying to mess with your services so that they don't work and so forth. A little more on CTF in a minute. So zooming in a bit on App Armor, this is the requisite boxes and arrows diagram. The most important feature here is Linux 2.6 which has a feature in it called Linux Security Modules that allows you to create mandatory access control policy engines and insert them into the kernel so that it has kernels authority but you don't have to ask Linus or ask your user to patch the kernel every time you want to use it. You can just insert it. And above that is the App Armor user level management software. Over here would be your stuff, your applications, each one of which would be wrapped in an App Armor policy describing what this thing is allowed to do. So some critical issues of doing this kind of design. Number one is complete mediation. If you build a fence three quarters of the way around your house your dog will run away. Three quarters done is not done in the security context because the attackers are smart people. They're what's called intelligent adversaries. They look for the hole in the fence and go for it. So even if you're 99% done you're not done. And the only way to do that is to be in the kernel. There were some products that tried to do security mediation at the library layer and that has the appearance of looking secure. The application apparently can't do anything you told it not to but the attacker who injects malcode into the application can bypass the library and head for the kernel. That's not the case if you put your security in the kernel. The other critical issue is the security model. You have to live with this thing so it has to be tolerable. At a high level there's two kinds of security policies. Misuse prevention and anomaly prevention. Misuse prevention is what you're accustomed to in antivirus products. Can't run this program or this one or this one or this one. You make up a blacklist of bad things that are not permitted. This one, this one, this one. 75,000 signatures later they're not done because there's always another way to write a virus or malicious code or whatever. This is the signature based business. It ends up being an arms race of continuous updates because there's always another one. The other way is anomaly prevention where you specify a white list of what's permitted and everything else is denied. And that's highly secure because the bad guys can't just go and bench another one to bypass you. But it's hard to live with because if you don't specify every last thing that your application needs then you're going to trip over your own shoelaces. The security system causes your production application to die in the middle of running payroll and your manager wants to know what the fuck you're doing. So sort of rock hard place. AppArmor tries to get in between those places with a hybrid model. It's a hybrid in that within a policy for a given application such as Apache we're using a white list. We specify everything that Apache is allowed to do and everything else is denied. It can only do what we say. Systemwide it's a black list. We don't specify, we don't control any program that isn't explicitly told to confine. So you can profile the applications that you care about like the ones that have network ports and nothing else and be secure because those things that the network can connect to are confined. What those applications that are confined can do is confined and your administrative stuff, your CP command, your personal shell, your window manager, they are not confined. So you don't have to live with it. You can only confine only the things that you care about. It's also focusing on applications instead of users. Back in the day, mandatory access control systems were designed for the main frame where all 400 of us shared a computer. The security system was trying to solve the problem of allow the marketing department, the engineering department, the accounting department to sort of cooperate but not really get in each other's shorts because it's really bad when the marketing people get a hold of early engineering notes and start marketing it before it works. The problem with that is that users are very big hairy creatures with opinions and they change their opinions and so they want to know why they can't do what this thing that they want to do even though policy says they shouldn't. And so access control systems that can deal with that are very complicated. Applications are much more behaved. The web server mostly serves web pages and the mail server mostly serves mail spools. They tend not to do each other's thing. So it's much easier to create a policy that says the web server can serve the web pages and the mail server can serve the mail spools and they don't get to touch each other. So this allows App Armor to be a great deal simpler than legacy access control systems that were designed to solve a harder problem. Zooming in some. This is an actual App Armor policy. The example I'm using is NTP, the network time protocol daemon that keeps your clock synchronized with whatever clock you want. Either your enterprise's clock or the NIST clock at NTPpool.org. And it has two interesting properties. One is that it needs root's privilege because it's going to mess with your system time. And the other is it needs an open network port so that it can ask the time server what time is it. This is exactly what all of the security guidelines and trainings tell you not to do. Don't have privileged open network ports because if there's a bug in it someone can pop that daemon and they own your box. They trojan your kernel. They do evil things to you. With this App Armor profile in place these here, these rules specify the five posix capabilities that NTP can have instead of all 32 that Linux has. Things like sys time. Oh yeah it's going to need that. And these rules specify file access. These are absolute path names with embedded shell wild cards. So we've got a rule that says slash TMP slash NTP star and read and write. So the net impact of this is that never mind that it's root, it can only have these five posix capabilities. And even though root nominally has access to the entire file system to do whatever it wants this NTP instance can only access these files. And so what could a bad guy do if he popped my NTP daemon? Well basically he can change my time. That's what it's supposed to do. That's not a great thing. You don't want bad guys changing your time on you. But that's a heck of a lot better than having a trojan kernel. Much easier to clean up. So how would you create such policies? This is the workflow. It starts with the machine analyzer. What it does is scan your machine for open network ports find the programs listening to the open network ports and report the app armor profiles wrapped around those programs. Your goal is to get the machine to the point where all of the open network ports lead to profiles and that keeps you safe from network attack. So that tells you where you should start. Pick one of these unconfined programs and get going. There's a template generator. It's a crappy static analyzer that takes a look at your program decides whether it's an x86 or a shell script or whatever and spits out a half-assed guess about what the profile should look like. This is just to get you going. It then puts it into learning mode where the rules are not enforced but violations are logged. You then go and run your program and you accumulate a log full of events of what your program should be doing and then you talk to the optimizer that scans the log turns thousands of log events into dozens of rules by asking you questions about, well, oh look, he asked slash temp slash something. What should I do with that? Oh, let's generalize it through a wild card so that he can just have slash temp. And finally there's a colorized VIM syntax and what it's doing is highlighting in different colors the significant elements of the policy. So green is pointing to other profiles that says that this program gets to execute some other programs and they may have a policy of their own. Yellow means kind of dangerous. So all the writable files are in yellow. So if you want to know what the hacker left on your machine, you look at the yellow lines, that's what they could touch. And red means very dangerous and so it's highlighting in red any positive capability and so forth. I just did that. There we go. Stuff to notice about the language. The first is that it's an overlay on top of Unix security. You have to pass all of the usual Chimad grep access mode bits before LSM will even ask App Armor the question of what's going on. Is this allowed? The second thing is no new jargon. The access rules, the path names are exactly shell syntax wild cards for absolute path names. So star means what you think it means. Square bracket means what you think it means. Squiggly bracket means what you think it means. The only slight variation is double star. A single star matches up to a slash character just like the shell does. Double star will eat the slash character so if you say double star it means recursively give me the whole tree. And you can compose that with dot suffixes so you can say give me public HTML star star dot HTML if you only want to export the HTML pages and please do not be exporting the Perl source code. Thank you. So if you know how to use bash, Chimad and grep you already understand App Armor and you can probably reverse engineer the policy by yourself. It includes a pound include which means exactly what you think it means. We use that to make abstraction so there's like a pound include for name service that gives you all the rules necessary for doing DNS and UID lookups. There's another one for console which gives you all the TTY stuff for talking to the keyboard and so forth. And these aren't special. You can make up your own if you are running an oil company and you want to have pound include seismic for all of your seismic modeling apps. App Armor will just treat them the same as it treats are the ones that we pre-define. We ship a bunch of policies. Some of them are on by default and some are not. Netstat, Ping, Traceroute, those are pretty well behaved programs. We're sure that the profile we provided is good enough. On the other hand, things like Apache, we don't enable that profile by default because I've never met a web admin who has the decency to put the web documents in the same place as anybody else. It's always on nfs2 slash that machine over there that belong to Bob but Bob doesn't work here anymore. App Armor had no chance of knowing where Bob's machine is so we disable it by default and if you want the web server to work then you put it in learning mode, let it serve some pages and it will very quickly figure out that that's where the web docs are and should just allow that. We could have shipped it in learning mode but then it would appear to be confined but isn't and that would be dangerous. We could have shipped it on in force mode by default and then your first user experience would be App Armor broke my web server. That's not too cool either. So it's something that you can go and get and turn on if you want. The demo, I'm going to hack myself. So here I am serving a website it works a lot better when I start the web server. There we go. And it has a web app on it. Anyone remember PHF? Ancient Vulnerable Web App. And to this day we're still vulnerable to this kind of attack. It's called Command Injection where the stupid web app will run shell commands if you put them on the URL with unfortunate consequences. So now I'm putting it back with the soon to be patented unhack button that uses the same vulnerability to publish a web fix. I think it's sad that that's the easiest way to publish a web fix. So now let's fix this problem with App Armor. We're going to build a policy for Apache and that web app while you wait. GenProf is the wizard for please build me a profile for this thing. And then you just give it a path to a program. That's Apache. Okay, and it says great. I've already run the static analyzer. I've put it in learning mode. Go run your program and I'll figure out what the hell. I do a restart on Apache. And down here you see the events of this is what Apache has to do when it starts up. You visit the home page and you see the events for showing pages. You run the web app and you see the events for running the web app. Don't run the hack attack and don't smoke in front of your kids for the same reason. Do another restart so that App Armor gets to see what a stop looks like. We need that. So now we have a log file full of events and if you had too much unstructured time you could manually turn that into policy but the software is much better at it. So let's scan the log, turn it into policy. The first question is App Armor saw Apache execute this PHF child and it wants to know whether the child should be either included as part of Apache's profile or out there with its own discrete profile. Giving it its own profile is the secure answer. Saying inherit for be within Apache is the easy answer. So if you go home and start using App Armor, choose inherit mostly until you know what you're doing. I'm going to choose profile because I can. It wants to know whether to cleanse the environment variables for the same reason that set you it does. Then it asks about the POSIX capabilities that Apache uses and just give them all because there's no reason not to. Your app needs this and denying it caused bad things. App Armor is putting square brackets around the default answer so if you don't feel really alert today just hit enter and it'll do the default thing. Now the file access. Apache wants to access Etsy to Apache Conf D. Okay, right. That's reasonable. And it wants deep into Apache Conf D. No way have I covered Apache. I've not got anywhere near touching all the configurations for Apache so I hit the glob button a couple of times. And now this proposed rule is saying that Apache gets to access slash Etsy slash Apache two star star. It can read the whole tree which is what you want anyway and you don't have to specify all those niggly rules. Some mime types some PHP stuff. Oh look, a pound include for be a PHP server so I don't have to think about it. There's a pound include for serve web data if you put the web pages in the SUSE place for putting the web pages. Or you could explicitly learn them yourself but I'm going to go with the pound include. If you hit glob with extension it wildcards the mod CGI here and leaves the .so so this is an easy way to say give me all the .so files. Let Apache have some logging capabilities. It's pit file, name service, done. So you don't even need to restart Apache because you can change an app armor policy out from under a running process. Here's the website. Here's the web app. Here's the hack attack. And nothing happened. There's your intrusion rejection message that if you're not in the front row this says that that PHF application does not get to execute bash. If it had been a bash script it would be executing bash now wouldn't it. But it wouldn't have right permission to index.html. That's the security value of the white list. I don't care what the bug is in PHF. I will tell it what it's allowed to do and it doesn't include corrupting my home pages. So the simple point here is that I stopped the attack that I knew about ahead of time. Well boring. The powerful thing here is that I was able to build a policy a workable policy for something as large and sophisticated as Apache while you wait. And the value to you is that you can use app armor to secure whatever software, custom, local, obscure, that you have to be hosting very quickly. It's not about how secure does SUSE ship an operating system. It's about how you can secure the stuff that you have to run that we didn't ship you. This is cartoons for the demo I just gave. So if you get copy of the slides that's what all that cartoony stuff is, is a lame version of the demo I just did. App Armor on SUSE Linux has this cool yask interface for clicky buttons so if you don't like shells you can use them. App Armor has this feature for sub-process confinement. Which sounds complicated but if you think about mod-pearl way back in the day if you wanted to run your web applications as pearl scripts then the web server would exact pearl and run your script. Well that's cute but pearl is a multi-megabyte executable image and that's not happening transactions per second. So Apache came up with this module facility so that you could load interpreters right into the web server. So you put on mod-pearl or mod-php. Now when you want to run a pearl or a php script it just opens the 100 line script, sucks it in and runs it in place. So now that can happen transactions per second. That's great for performance and crappy for security because now you're running all of your web apps in the same security domain in the same address space. So if there's a bug in any web app the attacker owns your website. Wonderful. App Armor has an API called change hat. Fireman's hat, policeman's hat, mayor's hat. So that software can declare to app armor, hey I'm going to change what I'm doing right now to some other domain. And then we added a module to Apache, mod-app armor that throws a change hat call every time you try and run a script. And it makes up the name by looking at the name of the script. So the net net of this is that you can have a policy that looks a lot like an app armor policy for a program for something as little as a php page that never ever appeared in the kernel process table. Yes, shiny buttons but it's all fundamentally command driven for those of us who are allergic to mice or more practically for those of us who are trying to administer a machine on top of a telephone pole on the other end of a 2400 bot modem to Malaysia. It's not just about servers. You can profile desktop apps too. I love game or pigeon as it's now called. It lets me do yahoo I am with my wife and IRC with my development engineers and Novel group wise with my corporate peers all in the same app. But this is undergoing what you might call rapid internet development. It gets frequent bugs. And if some stranger you're talking to knows a buffer overflow they own your I am client then they own you. With an app armor profile around it then an attacker who owned my game client would discover mostly my game files and they just might learn that I love my wife and respect my peers and not the other way around. But they don't own my desktop. They don't own my mailbox. They don't own my SSH or GPG keys. This is a view of the shell command and network secure system scanning my laptop. This was my laptop a month ago. And this is my laptop now. I did that Thursday night because it occurred to me that hey I'm taking my machine into DEF CON. I better secure it. It took me about an hour. So to network secure a system you use that list, pick a program, secure it, move on, run the unconfined command again until you have all the network ports that are open secured with a policy. Best uses for app armor. The corporate speak is companies whose network servers are running mission critical applications. You have a high cost associated with compromise regulatory compliance all that Wall Street stuff. My point of view is if it's a Linux application and exposed to attack then app armor is good for it. But if it's not on Linux you're going to need a different solution and if it's not important then I don't care either. Really that means the obvious application is network servers, web servers, mail servers, DNS servers. Business applications, those are just big bulky versions of network services like your CRM system or whatever. Those things should be app armored because they're exposed. Corporate desktop. Like I said with game it's nice to have profiles around your critical network clients so that if someone sends your workers malware it doesn't own the client. Firefox and Thunderbird are wonderful but they are not perfect. Put profiles around them so that the next time there's a buffer overflow in a JPEG handler that a malicious website doesn't own your users. Point of sale terminal. If you wanted to build a kiosk that you're going to lay out in public and let fun people like you play with then you want to treat the keyboard and the mouse as being just as malicious as the network so you profile everything that touches the keyboard and the mouse which is not that difficult to do. What you do is you tell app armor to profile X and then start it and app armor will create profiles if you tell it to for every single application that X starts. Just hit that P button and you'll get profiles for all of them. Now for an engineering workstation that would be a lot of programs. That would be an annoyingly large number of profiles but for an airport kiosk, no it's not it's like dozens of programs at most so that the web server has helpers that can play flash videos. So this is the range of applicability for app armor. So what happened at CTF? 2002, the ghetto hackers handed out a reference system that was a deliberately vulnerable and stale version of Red Hat Linux. Our team, I was the captain this was my mistake. We were too focused on demoing the technology and not enough on winning the game so instead of launching the vulnerable server and trying to score some points we kept it down until we had it all ported on to Immunix which took the first 12 hours of play and that cost us enough points to lose so we only placed second. The next year the target system was open BSD. Oh well we learned something and we launched the OBSD system anyway and we eventually got everything ported to app armor and that was nice but there was a lot of work importing. 2004, the target is Windows. Oh yuck. Now they were trying to be nice because all the apps that they demanded that you host were largely Perl or PHP or Python or something but they had a whole bunch of nasty dependencies and library issues and it turns out that it takes more than a weekend to port five applications over it from Windows to Linux. This trend isn't good. 2005, Kenshoto takes over and Kenshoto changes the focus of the game where now the server to be defended is a virtual machine that they own and they hand you a network cable saying this is your machine to defend it. You don't get to change the operating system, you only get to change the applications and so this is effectively a new game. It changed the focus from both attack and defense to mostly just attack. It's tightly focused on reverse engineering and code auditing applications and defenses are not really permitted. I'm told this year's game they've enhanced the ability to defend because they're looping the network path from the other guys to your server through your table so now you can deploy network defenses. You still can't deploy host defenses because Kenshoto still owns the machine. So it's a game, it's a valid game, it's not a game that App Armor can play. Comparisons the other major system that you'll run into in the Linux space is SE Linux and it uses a system called Type Enforcement where you assign users and programs to what's called a domain and you assign files to what's called a type and then you write a policy that say which domains can access which types. App Armor in contrast is using explicit path names for everything. The name of a program to be confined has a path name, the names of the resources it can access have path names and you generalize it with wild cards. You can think of SE Linux as the post-it note security model. You put post-it notes on everything and then you write a policy that says that the blue ones can access the green ones and the yellow ones but not the purple ones. The label splitting problem is that a single label you know the purple sticker describes an equivalence class of all the files that have purple stickers on them and if you were correct when you made up that abstract class that says that purple should all be treated the same that's great but if you ever discover that some of the files in the purple class need to be treated different than others you need to split that label, relabel all of those files rewrite all the policy that uses those files possibly with broad impact and so you end up rewriting your security policy. App Armor does ad hoc generalization. If you want to generalize to say treat all these files the same you write it in a shell regapps so you say slash s-r-v-dub-dub-ht-duck-star-star.html so my profile can treat this bunch of files as an equivalence class while yours is treating that bunch they may overlap but they don't have to be the same. Network Storage, NFS doesn't carry labels so you can't have SE Linux nuance labels on anything that's mounted over NFS. and all or nothing policy that says you can either have that entire NFS mount or you can't. App Armor doesn't care. You can have all this regapp stuff digging down into NFS mounts all you want. This is a summary of the Red Hat web page on how to create an SE Linux profile. Shorten for your comfort because it's really seven pages. It's 16 steps and the equivalent App Armor steps are five steps and that would be a cheap shot if the App Armor steps were big but they're not. They're actually smaller than these steps because the SE Linux steps are things like edit the configuration file debug it, allow the rinse, repeat. And the App Armor steps are run the programs and answer the questions. Here's resulting policy. Thing one, they're similar policies for WooFTP. Thing one to notice is that the App Armor profile is a quarter of the size. And thing two is when you zoom in and actually look at the policy the SE Linux policy is a programming language. It's an M4 scripting language. It's scary stuff in it. And the App Armor policy is absolute path names, shell syntax, wild cards and RWX. There's an O'Reilly book on how to read SE Linux policy and I don't think you need a manual on how to read App Armor. SE Linux has added new GUI tools and they are very nice GUI tools and they're very powerful and what they do is they enable you to very conveniently turn on and off already pre-authored security policy. They don't help with creating new policy so if you're running any software that didn't come out of the box from your distro vendor, too bad. You're going to have to learn to deal with SE Linux the hard way. App Armor has GUI tools. They're not actually all that great. They're pretty primitive tools. It's the security model that makes it powerful. Near-term development. We're going to add network access controls. It's something that SE Linux has and we don't. We will have it in the next release. Profile merge tool is an interesting thing that two App Armor profiles can be merged so that the result satisfies all of the requirements of the two antecedents. This is a theorem that you can always do that. So if you have two concurrent people doing development on a profile and they conflict well no they don't conflict. You can just merge them and ship the result. There is a profile sharing portal that will be revealed for the first time in September in the UK that will automatically share your profiles with profiles that other people on the internet are using optionally. You can ask to you if you want to use the shareable profile and it asks you if you want to publish yours. Or you can have your own internal profile sharing portal that is behind your firewall. In addition to ModPearl and ModPHP, we now have Change Hat for Tomcat so you can confine your Java servlets with little App Armor profiles. There's a PAM module that lets you use Change Hat for PAM so that you can do user oriented confinement. There's a growing collection of CIM providers if you're doing that enterprise management stuff. Yes, sir. Tomcat, PAM and CIM are available. There's Further Term Development, Availability, App Armor is included in every SUSE 10. Yeah, that's an interesting idea and we should talk about it offline. The general theme, sorry for the audience the question was were there any plans to take the Change Hat to other web authentication schemes? The general notion of Change Hat is that you can embed it in an application but App Armor's design goal is to never have to change applications. You want to embed Change Hat into application servers. So that's why Change Hat is hacked into Tomcat and Apache being an application server for ModPearl and ModPHP. PAM is an application server for login or for authentication. So if you have some kind of application server class where stuff with different security domains is appearing but it's not appearing with unique PIDs, then that's where you need Change Hat to distinguish them. So App Armor ships with every SUSE 10. I have a small number of Open SUSE 10 2 disks here if someone wants one. App Armor is fully GPL'd. That's the community website if you want to get involved. We have all the usual mailing lists and an IRC channel IRC.oftc.net pound App Armor. App Armor is going into Ubuntu for Feisty Fawn. It is in Universe and for Gutsy Gibbon it will be in Maine. App Armor for everyone. Matthew Snellum ported App Armor to Gen 2 but I haven't heard from him lately. Debian. The Ubuntu devs should work pretty well on Debian but there is no Debian maintainer. If someone is a Debian Committer and would like to maintain App Armor, please talk to me. I just said all that. App Armor for Red Hat. I had some of my engineers actually port App Armor to Red Hat but again there's no maintainer. The gotcha here is that App Armor requires a kernel patch and that invalidates your Red Hat Enterprise support. So if someone say a CentOS maintainer and would like to maintain App Armor, that would be great. Please talk to me. Questions? Sir, the question was whether you can use App Armor on a non-fresh install. Yes, you can. You could install App Armor on a machine that's been up for years. You would then have to train App Armor on what your apps do and it will just learn it from your training cycles. Other things that you would want to know about is that you don't have to do the profile generation on the production machine. You can put the application in learning mode and then ship the profile collection that you started with and the log to a security analyst box on the other side of the country where he runs the learning tool to generate the policy. It doesn't even have to be on the same machine. You can put a program in learning mode and leave it there for a month, collect all the logs, feed them to the learning tool and create a policy based on a great deal of experience. We took App Armor into a large dot com vendor where their QA cycle was run 30,000 real transactions from history through their website and in an afternoon it generated a quarter million log events and in about an hour App Armor reduced that to 400 lines of rules. The question is whether you can use Change Hat on user space threads. The significant feature is that you have to call Change Hat every time you change threads. So if you can put Change Hat in the scheduler, then yes. Otherwise, not so much. The question is whether App Armor can stack with other LSM modules. Complex mandatory access control systems do not stack well with each other anyway. They can be stacked with SE Linux? Which module can be stacked with SE Linux? Which module can be stacked with SE Linux? The LSM stacking model is not comfy. To stack any two modules together at least one of them has to know about and cooperate with the stacking. I'm unaware of SE Linux facilitating any stacking so if some other module can stack with SE Linux it's because the other module was designed to do so. I'm unaware of any module that tries to stack with App Armor. App Armor incorporates the standard Linux capabilities module into it so you throw out the original capabilities module and just use App Armor. We are looking at other modules. DAZUKO comes to mind and I'm also interested in building an LSM module that incorporates some of the features from OpenWall Linux from Solar Designer. But as far as I know none of that exists. The resource hit, the question is what resources does App Armor consume? When we benchmark it it shows a performance slow down in the 2% range at most, usually a lot less than that. For memory it's dependent on how many profiles you have. There's a small cost per process instance but mostly they point to a reference profile so the profiles aren't duplicated per process. So if you had a very large number of profiles or a very large number of rules that would start consuming a bunch of kernel memory. The largest deployment I was aware of they had over 100,000 profiles loaded. What we in recent work we translated App Armor from loading the text of the profiles into the kernel and doing reggaps matching on access requests, turned that into a DFA. The policies are now compiled into a state machine that is loaded into the kernel and that is a pretty hot optimization for matching the rules. The question is can you use the learning tool for monitoring exploration? Absolutely. App Armor has been known to generate more than one what the fuck moment. My favorite was when I discovered that Game was running WGED on startup. So yeah it's very good for understanding what your software is doing. The performance hit in learning mode is somewhat higher that can be like 20% slow down and that's mostly because it's doing a lot of logging. The performance penalty for running in learning mode is a lot less if you give it a mostly done profile and so you're just looking for the what the fuck moments and not the usual. We're working on an enhancement that we call log once where instead of logging the same event over and over again it'll just log it once for some value of recently because it would require infinite state to recall all the events that we've logged before. Well thank you very much. I'm going to call it good. We have about 15 minutes for additional up close questions here and then it's time for DT.