 This talk today is about how we upset someone. And that's never a good thing to have to talk about. Because when you upset someone, it means that you've done something wrong. It means that you've changed something, that they liked. It means that you've broken something, they expected to keep working. And so this is where we start from. Poor Skalsak is very disappointed. Skalsak is probably not someone you've come across before. We certainly hadn't. So we dug into what he works on and checked out his GitHub. One of his projects is PyPyCats, which those familiar with information security will be aware of Mimikats, which is one of the most popular hacking tools that gets used. He owns the Pure Python implementation of that. He's also got proof of concepts for a Cobra's attack in Pure Python and also a couple of CVEs in Pure Python. Maybe upsetting this guy by breaking stuff is not actually that bad a thing. He doesn't seem to be doing nice stuff with Python. The way that we broke him is by implementing PEP578 for Python, which is auditing hooks and can be used for security transparency. So that's the subject of today's talk. We're gonna go into details about what was changed, what's been added, and how you can use it to make people like Skalsak hackers using Python very unhappy. So my name's Steve Dower. I'm a C-Python core developer and the original author of PEP578. Also worked with a few other people on it, especially one who acknowledged James Powell did a lot of the implementation work for that and was a big help. You can find me on Twitter at Zuba. I also work at Microsoft. Hi, I'm Christian Heimus. I'm author C-Python core developer. I work with the BDFL delegate, so the person that worked with Steve to get the PEP landed. Also, I work for Red Hat and you can find me at Christian Heimus on Twitter too. So today's agenda. We're going to first explain what are actually audit hawks and why should you use them, how you can use audit hawks to improve security of your programs. And finally, Steve will go through some Windows-based example. I'm going to go through some Linux-unix-based examples and we're closing with a summary. Oh, you got the clicker. Thank you. So a runtime, oh yeah, that's pretty easy. So the runtime hooks. So it's only a small fraction of things you actually do if you want to create a secure environment for your processes and for your services. It's not a whole solution, it's a small part of the solution. With the runtime hooks, the auditing hooks we have, you can see what the interpreter is doing internally, hook into different parts with the interpreter from which files are my process opening, which sockets I'm connecting and compiling, executing bytecode, importing models, et cetera, et cetera, et cetera. By default, these hooks don't do anything, so they're just empty stops. If you don't hook any of these auditing hooks, use them, they also don't impact the performance of the interpreter, which is probably something that several of you care about. And these hooks are designed for security engineers to inspect what's going on. It's something not like ready to use solution, but you have to build your own solution, the foundation to create them. And as a security engineer, if you want to take care about creating a secure environment for your services, there are multiple things you should do. First, of course, please install security updates. And then you want to run your services as a limited user account, don't run them like as routers admin. And then please install security updates. Well, then do something like use a firewall, restrict your processes, and yeah, install security updates. And then don't install random stuff from the internet, actually control what you deploy on your machines, inspect what you control on your machines, don't install like a new version or don't fall trapped for like typosquatting, where somebody has a high PI picture that sounds very similar, but has actually a typo. And yeah, install security updates, please. And maybe at the end, if you have done all that and even more things, then you may think about adding all these security hooks. So let's have a look at what hooking this actually involves. We have two sets of APIs for this. It is meant to be used as a security feature. If you're that committed to doing this, you should really be thinking about compiling Python from source, controlling those sources so you know where it came from, and possibly modifying the runtime itself. The aim of the hooks is to provide a C API to make that safe and reliable, so you're not constantly modifying the full source code of C Python. There's also a Python API for hooking into these events as well, which is very helpful for testing, and there's a few scenarios where that makes sense. But these are the general APIs. So you're gonna have some kind of callback, whether in C or in Python. And then it's a simple call, it's either PySys, add audit hook, or it's sys.add audit hook from Python. That's gonna give you a callback that is called on each of these events. These events can be called from anywhere. So they will get called on a variety of operations, variety of contexts that Python may be running in, certain locks may be held at very low level, and you do need to be careful about what you're doing it, but there are, for the most part, they're not that complicated to implement now that you actually get a callback for it. The pros and cons of each of these approaches are important to consider. Obviously implementing in C is more complex than implementing in Python, but the upside is it's gonna run faster, it's going to be harder to bypass for an attacker who may be executing code in your Python that you don't expect. If you've added your hook in C, you're still gonna get those events. There's very little that an attacker can do running pure Python code to stop you from receiving these events if you've written a C hook. But it does also require compiling and deploying your own copy of Python. Now, Linux users probably not at all concerned by that because you're almost certainly doing it anyway. People running on Windows Mac OS are less likely to be doing that, so that's a bit more of a step up in complexity, but it is worth it if you're looking to enforce stronger security boundaries. Doing it from Python, of course, is very easy, very convenient. You can do sys.add-ordered-hook-print, and that will give you a printout of every single event that's going on very simply. The downside is it's per-sub-interpreter. Again, if you're not familiar with sub-interpreters, have a look at those. They're a very complex area, but if you are using a Python hook, it's only per-sub-interpreter. You'll need to keep adding it to each one, and it does run slower. You may notice more of a performance impact if you have Python-based hooks than C-based hooks. What kind of events can you expect to appear while you're running? This is a small sample. That link at the bottom has the full table of almost the full table of events that C Python currently raises. Other implementations of Python may add different events, and certain libraries that you use can also raise events, or come back to that, but here's a handful of a selection. The built-in's input function is going to let you know anytime input is called. It's also going to send a separate message with whatever is typed in. Your production server is probably not using input, and if for some reason input starts happening on your production machine, you probably want to know about that. Things like exec-import-compile, very useful. You want to know what code's running, and you especially want to know if code that you didn't write is running. We have more on that, more on being able to prevent that in the first place later on, but this is the way that you can at least be notified that someone is dynamically compiling code on your production server. Socket new, and there's a range of socket events that will be raised glob-glob, just in case. That's actually a very common thing that attackers are going to do if they manage to execute code on your machine. They want to find out what files you've got there. They want to see what tools they have access to. They want to know what user accounts are set up. If you start getting events for glob, again, if you know that your app uses glob, you can probably ignore them. If your app never uses glob and you start seeing it, that's a good warning sign. So what should you do with an event? You get the callback, what are you going to do? The first option you always have is to do nothing at all. Depending on your risk profile or your threat model, certain events, it's very useful to do nothing with them at all, just ignore them. You can log it. You can write out the details somewhere and keep a semi-permanent or a permanent record of all the things that your app has been doing. You can abort the operation. From inside your callback, if you raise an exception, most of the time it's going to interrupt the action. So if a socket raises an event saying I'm about to connect to this address and you throw an exception, it's just going to abort it. It's going to be canceled. Your code will get an exception instead and that socket will never be connected to. So there's that option. And you can abort everything. If there's an event that you recognize that you really, really don't want to happen, you can just call exit and tear the whole thing down. But the correct answer is to log it. And this comes out of an approach to security that's known as assumed breach where a lot of people and kind of the traditional approach to security is you set up a really strong boundary so nobody can get in and then you're fine. And as long as nobody gets in, you're fine. And it turns out that the attackers are better at getting in than we are at keeping them out. And you only need one gap in that wall for someone to get through. And once they're in, if all your defenses are in that wall, you have nothing. Think about an office building. Your office building has locks on the doors. That'll keep people out. But there's also motion detectors and security cameras on the inside which are completely useless if your doors are perfect. Why are they there? Your doors aren't perfect. People all get in, want to know about it. Think of these auditing hooks as security cameras or motion detectors inside your app so that you know when someone's moving about in there. You will see legitimate people moving about in there. You need to be able to filter those out but if you're not logging it, you have no chance. Logging everything is very important. And your first instinct is probably to filter out events that aren't that interesting, that aren't that relevant and say we don't actually need to keep this. We can save disk space, we can save network traffic by not logging certain events. That's actually a really bad idea. If you have complete logs, maybe you're not checking everything. Maybe you're not constantly reading them but when it comes to a retrospective analysis, when you discover your emails being published on torrent sites or wherever they get published these days and you realize that someone's in your system, how are you gonna find them? If you have all of the logs, you've got a chance of working backwards and locating what those attackers have been doing in your system. Anomaly detection is a growing field especially using machine learning to keep kind of a profile of what events happen during normal operation, recognize when that changes, even without someone having to review every single thing that's going on. And of course, incident response. If you have a live persistent threat inside your network and you wanna find out what that's doing, what services it's approaching, what IPs it's pinging, which Twitter accounts it's looking at for its next set of instructions, having all of these in your logs already will give you that information, will help you expel them nice and quickly. Premature or log filtering is going to cripple your defense. So log everything. And as I said, there's a way to create auditing events as well. For the most part, the intent is to listen to them. But if you're developing a library, if you're developing various extensions, then it can be very helpful to create your own events so that you know when these things are going on. So there's a C API, which is based on the Pi build value API. If you're familiar with that one, takes a format string for the arguments that are gonna be passed in and the name of the event. If you have the option to use the C API, use the C API. There is a Python API, sys.audit, which similarly takes a list of arguments. It's very easy to bypass that one. You can reassign sys.audit to another function and then those events go away. So it's very useful for testing, but if you can use the native API, that's strongly recommended. We recommend in the PEP that third party events should include the module name as part of the event name, helps namespace things. If you have two modules raising the same event for different purposes, that's probably gonna cause issues. So putting the name of the module that you're raising it from as part of the event name helps keep things separated so there are no collisions. And just as a recommendation, validate the parameters you're gonna pass in first, then raise the event and then do the operation. That means that hooks can assume that the arguments are gonna be valid on the way in and they're not gonna end up logging things that are just gonna raise an exception and not run anyway. But it gives the hooks a chance to abort the operation. So if you notice something going on, even if you wanna quickly deploy to expel a current persistent threat, then you may have a very specific case where you want to start raising an exception for a certain IP. And if you've implemented these events in a way that the operation is already going, you can't actually interrupt that with a hook that's going to raise an exception to abort it. So you wanna try and put the auditing events after parameters are known to be valid, known to be the correct types, the correct within range, but before you actually do anything with them. And next, Christian is going to tell us about how you can just stop code running in the first place. Yeah, so a rather special case, they should not hook but a new piece of code we added the IO open code function is, if you look at how usually binaries executed with shared libraries and native fixed code, the kernel on the operating system knows the difference between yet actually code that runs on your CPU and that's data for the code. But with Python, we have like this rather bad case that PyC files are considered data for the CPU in the operating system. And this IO open code function on the first step to teach the operating system in the 10 that we're going to open something that's going to be executable code. So this is a simple function that basically bolts down to just doing opening the path we path through for binary read only and returns a file like object. So it doesn't have to be actually a file object. We have some examples later on that uses bytes IO that looks like a file, but it's not taking a file. If you hook into the C API, you can override that, but you should only do that in the beginning before you actually start the rest of the interpreter. You pass in the callback and the callback gets called with the file name and additional user data. You can add custom information to your callback. And what can you do with the IO open code thing? For example, you can verify different attributes on the file. You can check properties of the file. It's actually a regular file you're opening. You're maybe opening a pipe or a socket or some kind of special file or file on a file system that you don't expect to load any code from. You can validate the content of the file. So you can compare it to checksum. You can do this like code signing. You can make sure that nobody, while you're working on the file, does something with the file. You can lock the file content before you load it. And instead of what I mentioned before turning the actual file, you first load the content into a memory buffer into a byte's IO and then do all the operation. Because if you read it for like two times, first time to do like a checksumming and the second time to actually turn it to Python, then there's a possibility that an attacker can use a time of check, time of use attack. So if an attacker can just intercept the second reading of the file and replace the content with some notice code, that's bad. So always first do something, read the whole content into a buffer then validate the buffer and pass down the buffer down. And there's some caveats. One thing is if you implement that in C, it's executed while the import hook is hold. So you can't do another import while doing this hooking. And some calls may assume that's a actual regular file that's backed up by a file descriptor or physical file on the file system. But all the Python standard import system doesn't assume that just requires a file-like object. Some additional things you have to do if you wanna validate the files and deal with the IO open hook is also, you don't wanna have any additional code in your project that like bypasses the whole IO open infrastructure. So in Python in 3.8, we replaced all code for importing and several other things like sip import and we plan to do that for pickle with the IO open code thing. But if you bypass that in your own application using like compile exec or exec file in Python 2, well. Other parts, you wanna make sure that you load file that come actually from the file systems. You can use introspection and other file system and operating system tool to verify what kind of files you load. So if you allow like dash C, where you can just pass an arbitrary Python code or do like curl some evil side, shell to Python 3, shell and data in, then the admin has no chance to see which code you're actually executing. And also restrict which kind of environment you have like the variables can play 20 tricks with your Python program. And also for which places you allow to load code. So maybe you don't wanna allow it to read code from like temp tier when temp you're the only place where an unrestricted user can store any files on disk or a home directory. Maybe you wanna restrict that. So now we come into Windows section. Where Steve will tell you how you can hook in all the hooks on Windows. Right, here is where we get to real kind of applications and a few samples of things. So I'm gonna go through three points of integration that exist in Windows that this enables which were previously unavailable. So these are operating system features that are really powerful security features. I suspect there's a very low number of Windows developers in the room because that's fairly typical for these conferences, unfortunately. But these are features that get cheers from the Windows security-focused conferences because they're really powerful at locking things down. With these hooks, with the open code function, these become available to Python developers. And as a security engineer in those contexts you can integrate Python code in your Python apps into the rest of the security infrastructure. All of these code samples are available at this GitHub repo and so you can go and grab those. You can get a copy of Python 3.8.beta.2 and compile them and try them out for yourself. So the first one is the Windows event log. The kind of canonical example of code that you don't want running on your production servers is something like this. And in fact, this is a, well, no, this is a great example. So you'll see that it's called S-Python. S-Python is kind of the code name that we've been using for a Python that has more things enabled. There's a lot of good reasons for a development cycle to have a Python binary that doesn't have these things enabled and use an S-Python binary in production that does have an enabled. That lets your developers use a whole lot of things that might otherwise be restricted. And then when you deploy, if you don't allow that Python binary to exist it also, it takes away an entry point. There's also some interesting kind of semi-research which I haven't linked to but I probably should have, where some would say this is security by obscurity where simply renaming your Python executable hides the fact that attackers can run Python when they get on your machine and they're just gonna figure it out and do it anyway. Someone did a study where they changed their SSH port by one and saw like a 99.9% reduction in attacks. And if you think that's a bad thing then I'm gonna disagree. If you can reduce 99.9% of attacks by changing the name of something then you should just do that. So anyway, this is kind of the canonical example. If you get onto a machine and you have the opportunity to run one command that command is very often, surprisingly often for most people going to be something like Python-C decrypt this base 64 string and then execute it. And quite often that string involves opening up URL lib and downloading more files and decrypting those and executing those. In this case it just prints hello, EuroPython which is a much better option than all the rest of them. But this is fairly common. Looking at that command you can't see what that's going to do. You don't know what that's going to do. And typically once an attacker is inside Python you have no idea what it's doing until you see the results being published on, what is it? What's the site with all the passwords? Have I been pooned? Have I been pooned? That's when you find out that someone's been doing this. But we want to find out sooner. So the Windows Event Log is kind of the central event stream service on Windows. It has a handful of integrations with other more advanced features that are really helpful. There's an Event Log viewer for starters which you need to be able to actually look at the events that are in there. The valuable ones from a security perspective, event forwarding, you can configure it to automatically send all logged events to another machine and that completely prevents anyone on that machine from clearing the logs because the messages have already been taken somewhere else. You have protected event logging where all events can be encrypted immediately with asymmetrical encryption which means you can't actually read back the log on that machine. You have to take it to a machine that's enabled for it. Clearing and modifying logs automatically adds in your event. So you will be notified if someone clears a log even if it's you. Because one of the best ways to find out that someone is doing stuff on your machine that they shouldn't be is your event logs get cleared. So knowing about that is helpful but it also gives a very simple API for logging the events in the first place. Which in this case, so this is my hook for when code gets compiled, these two lines and a little bit of boilerplate that's being generated elsewhere is gonna record the event. It's gonna record the code that's being compiled and the alleged file name it's coming from. The compile function takes a parameter that says the file name. We can't necessarily trust that but the code is legitimate. We know that that's what's being compiled. This is gonna lead to the event viewer where you can see at the bottom, the event has been logged. We're compiling from a string and there is the actual code that's been decrypted from that base 64 pass into exec. And so now because that code has been run through as Python, we actually have a record of what that was. And if that's been sent to a central machine, maybe we're monitoring a thousand machines but this gets red flagged on our central server and we can see that 10 of those machines are executing arbitrary code that they shouldn't be. Now we have something to start with. We know we're under attack, we can start investigating, see what it's doing, trace it down, kick it out. From the point of view of blacklisting or whitelisting code or allow safe listing or deny listing code, the typical approach on Windows is code signing. So in short, this is attaching a signed hash of the file so we'll make a hash of the file, we'll sign that hash with a certificate where the public key is on the machine to verify against but we've signed with a private key on a trusted machine. And then we can verify that on every use. And depending on your configuration, the kernel will verify that whenever you start running a binary file. Python now with the open code function can also verify its code files against cryptographically signed hashes. Unfortunately, Python files don't have a standard for embedding a signature in them, which is the typical approach on Windows. But we can use catalog signing. So a catalog file is essentially a list of hashes and file names for a set of files. The entire catalog is signed and we can refer to that to see whether the file we're looking at has been signed and approved. The standard Python installers on Windows for a couple of versions now have included a signed catalog file for every non-binary file in the package. So if you've installed Python on Windows recently, you already have one of these catalog files that you can verify every file from the distribution against to see if it's been modified or to see if something extra has been added or disallowed. And that file looks like this. So it has a handful of various properties. This one is signed by the Python Software Foundation as with all the files in the standard distribution. And that security catalog tab, which I didn't show because it looks like a random number generator output is a list of all the hashes. So this is the code for doing it. This is literally all boilerplate. There is nothing interesting here at all. You copy paste this from somewhere just like I did. This is the interesting function call where we go, hey, Windows, can you verify whether we trust this file? And it will come back with a yes or no. And if it says no, then you don't trust that file, you abort and you get out. And what this means is we can run our build of S-Python here that has this enabled and we can import URL. We can import async.io. Those are part of the standard library. They include pre-compiled binary files that have been signed. They include Python files that are included in the catalog. And because they're all there and they match, we're allowed to import them. And then if we try and import some unsigned file, we get an error message. The error in this case has come straight from the operating system. No signature was present in the subject. You can totally replace that with a different error message. Of course, this is not a built-in feature of Python at this point. You have to add this in based on our code samples if you like. You can return any error message you like at all. And bringing this all together. In the latest updates to Windows, to Windows 10 and Windows Server 2016-2019, we now have a feature called Windows Defender Application Control, previously known as Device Guard, if you've heard that name. This allows a kernel-enforced configurable policy for allowing denying applications from running. Which is actually just a short way of saying it's a massive XML file. It allows you to use either signatures or catalog files or file names or paths or other attributes to determine whether executable files are allowed to be run or not. It's already integrated with event logging and detectors. It provides good feedback for users. So you don't just get random error messages, you get helpful messages. One thing I didn't put on that list is your configuration is signed and includes the list of people that are allowed to sign replacements for that configuration. Which is a very nice feature. It means that you can't actually break into someone's machine and replace their configuration with one that allows you to do stuff, unless you're signing it with a certificate that they previously said was allowed to sign it. So you end up with this chain of allowed configurations that keeps things really nice and secure. So how this looks, on this machine, here's my Python install. You can see S Python towards the bottom there, but I'm on the regular Python. What I've done is I've explicitly banned Python.exe from running on this machine. Everything here except for S Python is signed with the Python Software Foundation certificate, but I've explicitly banned Python.exe. So if I try and run that, then I get this big pop-up message, which admittedly is not the most helpful message, but it's better than simply saying access denied, which is what you'd probably expect. And if I try and run it from PowerShell, then I get a similar message. Program Python.exe has failed to run. Contact your support person for more info. So this is an IT configuration set up for your machine that will prevent applications from running. Like I said, big XML file, this is about half of my part of the XML file that gets merged into one that's about 20 times longer that lets the rest of Windows run, because by default you can block all of Windows from running with one of these things. This is the interesting part. We explicitly allow the S Python executable because it's not signed. We allow anything signed by the PSF, and then we deny Python.exe. And just for fun, I've denied SQLite, Ctypes, and LibSSL because that was part of my example that I was demoing with. If you grab the sample code for this and try it, for starters, I recommend doing it on a virtual machine, not your main one. But then after that you'll find that you can't import SQLite or Ctypes or LibSSL either. And so in this case, if I run S Python, it will import a whole lot of modules. There's a lot of built-in modules that are signed by the catalog file that are approved here, otherwise we wouldn't have gotten as far as the rebel prompt. But then when I import my unsigned file, it tells me it's blocked by policy. Again, that's a customized error message. You can do anything you like there. You can shut down the entire machine at that point if that's what you wanna do. It's completely custom code up to you what happens when something fails validation. As an extra bonus, I can tell from within the executable whether policy is being enforced on that machine or not. So I'm actually using that in this sample to disable the dash C command. So this is the code from earlier that was gonna say a nice hello to everyone, but I've blocked it. I've said you're enforcing code integrity policy on this, so I'm not gonna let anyone use dash C. There is an option to go into what's known as audit mode, which doesn't actually block anything, but every time you hit something that would be blocked, it's gonna log a message. So you get a nice record of everything that people are running and you can go back before you start enforcing it and whitelist all of those things that should be allowed to run. In this case, I print a different message and do let you use the dash C. This is just proof of concept that you can tell the difference from within the program. You can behave differently based on how it's configured, which gives you a lot of flexibility to integrate Python with the operating system level protections that are available. As I said, all of these examples and in fact, all of Christian's examples as of this morning are also in this repository. Feel free to go and check those out and now we're gonna jump over to some of the Linux options. Thank you, Steve. So this is mostly on Linux. Some of the examples may work on MacOS or a BSC, but I haven't verified that yet. So, one of the points I wanna talk about is first of all, quick advertising from our talk tomorrow, de-tracing system tap. So I'm giving you a talk about tracing and profiling tomorrow. And we're more about sys-logging, just a quick intro. Steve already covered most of the logging for Windows. Sys-logging is kind of similar with some more feature set, but yeah. And how to do IO open code on Linux with kind of code signing. But so, some of the prerequisites to doing that is of course, place install security updates. If you take anything from this talk, it's the first point, always update your machines. So you don't wanna run your Python interpreter or your application as a privileged admin user as root because root usually can replace files and do other stuff. And lots of assumptions are you run a way that you can't modify like your binaries easily. You also want to restrict where you can write to, what things you can write and modifying your system. Again, with unprivileged users. You want, especially running containers, run some kind of kernel security policy. So there's app armor, there's SELinux, there's Tomojo, there are different kinds to further restrict what different kinds of application a special context can execute and modify and do on the system. And finally, you wanna configure a central logging like there's SysLog, there's R-SysLog, or there's John L.D., if you're running system D enabled systems and forward your events to some remote machine so that it can't override and destroy your log files. And just, so first point, system that in D-trace, it's a way to tall the kernel to log what's going on. Again, tomorrow I will show more details how to use these kind of features more elaborately, but as part of the PEP578, I added markers so you can actually audit. And if you run this script here that attaches to Python and use the audit probe and just prints out the string, the first argument of the string, and if around like Python, 3-H, C-pass, you see like, okay, in the end, surrounding a command here, it compiles something and then exit the actual pass. So that's the one to integrate with system D-trace. Logging, so SysLogging is very easy on most Unix systems, you have to first open the log file which is not necessarily required, but you can configure what you do. And this is probably giving some option that in case logging doesn't work at all, it's just like maybe down it still prints something on your standard console and on SDDR. And also it locks a pitch so you want to know which process identifier does something funky on your system. And then you just, if you have some kind of events you call log, just log with the severity, like critical, and there's a format operator and something you want to probably do if you run in something, you want to app or process, you don't want to run like exit or a shutdown, the Python operator like manually, you want to use underscore exit which just stops the process immediately without doing much of cleanup. Because if an attacker was able to modify you at the pre-turn away, then the cleanup, the shutdown and the exit hooks may do execute additional code, but you would just want to kill that one. And if you're running a container platform like Docker, Kubernetes, Potman, whatever, you have to set up your container environment to have a syslog endpoint in your container. By default, you don't get that on, as far as I know, most container platforms. So that's something you should keep in mind. So how could you implement IO open code on Linux because we don't have this fancy catalog file like on Windows, which is really cool? So I came up with a rather simple proof of concept that does some verifications. For example, one thing I want to verify is that my file is actually a regular file, not some kind of socket or a pipe or something special on a special file system. And I also want to deny any kind of non-executable file system. So don't execute something that's maybe stored on the proc file system. If you have a hardened system, you often mark your temp directory as non-executable. So you can't copy a binary to that execute that the kernel will disallow that and you can do the same kind of checking also manually. I do the usual rounds, I load my file into a byte saio and then use the OpenSSL libraries, which I'll use by Python anyway, to hash the file content and then very far the file content against a special property contained in an extended file attribute. The example is also on Steve's getup repository. So what's an extended file attribute? So extended file is a feature on Linux and also Windows has something similar with things called streams. It's kind of similar. So you just have key value pairs attached to your files or directories. There are some additional properties you can store mostly arbitrary data. There's a namespace, you have usually four namespaces, you have user, you have trusted, you have system and security. The three last ones are restricted to you for something, even a kernel policy to do something with that. So I used the user one and the user attributes are controlled by something called DAC, discrete access control, better known as the standard user group, other people read what execute bits. So if you can read a file, you can also read the user attributes, you can write to a file, you can modify or create or delete extended file attributes. And the whole concept is inspired by something called IMA, integrated measurement architecture, which is something like code signing and verification, which is currently under development on the Linux channel. And just to look at that, so it's how it looks on the shell. So I created a bunch of different files using extended attribute with a hash. This is one file like the OSPI and this is my user.org.python. So the standard suggests you should use some kind of an identifier based on your domain name you own. This is the hash. That you actually get F attribute, it's file attribute to the internal API, etc, etc. So it's a bit confusing, but not that part. So example, so if I have a like modified OSPI file that doesn't match, my example will just crash and tell me, there's a mismatch during the import. And then I have a script that can be used both to generate and update the hashes. Like after updating Python to a new version, you need to regenerate these. It's a very simple Python script that just creates a new one and then I can run my example again and just it just works. And the script is fairly easy. I just have a list of file names, Python file names. So I use the hashlet model to hash the content here with SHA-566 and then use setxutro to update the hash. Yeah, that's it. So there are some caveats. You need to, if you would use that in production, you need to actually protect the ones because a user or a attacker that goes control could in theory create its own Python file and add its own hashes. So other you have to use one of the protected namespaces which I can't use easily without using special powers or kernel policy. You can also do something like a signed hash but it's going to be slow if you do the signing for every file. So the catalog files, you have like a list of hashes. The whole catalog is signed one time you do every time getting big slow or if you run a container as a application, you can also do something like blocking the syscall. So there's a tool set on a Linux kernel called seccomp which blocks and disallows to execute syscalls. And if you have to block three different syscalls here, again, tomorrow morning at 10.30, I will explain that during my talk by the Q3. And there are also some open issues with that. So you can do something with lepreload or writing like parts of your program. If you want to test that with a container, so initially I wanted to offer a container, you can play around with that, but there's some problems with how you can store the user attribute in a container image. It doesn't require that. There's an issue of that. You can also be allowing user or attacker to write to procs-solve-mem which is just a file that is actually the whole process memory of your process. Let's get broke around that. De-lo open is something that we both facing issues on Windows and on Unix platforms is that's the call you use to open an extension binary. So if you have a C extension Python model or like a C type, CFFI, library U load, that takes a file name. And there's no easy way to verify the content of these binaries without being subject to a time of use, time of check operation. And there's fun attack called snake ito which abuses memory file descriptors and deal open and the proc file system to actually download the binary and inject some stuff. And there are probably many, many more things that can go wrong. Again, there's just a small list of things that can happen. There's currently an effort to implement a new feature on Linux kernel called omae exec. So that's a flag for the open sys call. It's a hint for the Linux kernel to tell them, okay, I'm opening a file, but this file actually may be something I'm planning to execute. It contains code. It comes from GNU Linux ClipOS 4, security distribution for Linux. And there are a couple of videos and talks about that topic. And with omae exit, the kernel knows, okay, this is a binary file that maybe contain code or a text file that contains code and you have to do extra checks on that. And then you can have the kernel security policy perform additional checks like requiring to have the expert, the exude a bit on the file or deny file opening on the actual file systems, et cetera, et cetera, et cetera. So closing summary, Steve. Sure, take it. First point. Yeah. So the whole idea here is that when your security is good, ordered hooks can make it better. There's a long list of things that as a security engineer you need to do to lock down your production systems. If you miss anything on that list, this point is not gonna save you. It's probably not even gonna help you. It's just gonna end up wasting your time if you missed anything earlier. These hooks are intended to provide transparency. Security is your job. Python is going to help you with that by making sure you're aware. You can see what's going on. And along with everything else on your operating system that's helping with that, it gives you that added information to make smart decisions and to make fast decisions when the time needs it. These hooks enable the use of operating system technologies that have often been around for decades at this point, but were previously unavailable to Python. They do require custom implementation. There is some work involved. We hope this whirlwind tour has been some inspiration and hopefully extra information for that. Python now gets to play with the rest of the operating system world as far as securing and hardening things. And of course, the most important point. Well, yeah, please install security updates, please. So thank you very much for coming. Here are some helpful resources. I believe we are out of time for questions, so feel free to come and chat with us outside. We'll be hanging around just out there. Contact us on Twitter. More would love to hear what you'd like to do with this, how you'd like to be able to use it, what you want to integrate it with. So please come and chat with us. Thank you. Thank you.