 All right, we're back live. Thank you, Leeran, for your support and supporting DEF CON and the Red Team Village. It's a pleasure to have you here. So without further ado, take it away. All right, thank you. So thank you all for coming to this talk. It's my first time presenting at DEF CON. I think last year I got caught up in the DEF CON Blue Team Village trying to win that CTF from Open Sock. It was pretty dope. But I think this time today, especially, we're going to talk about something more red team-ish. Why? Because I'm on a red team and it just seems appropriate. And so this is the talk. Y'all trying to bypass Python 348 autohooks or nah? So who am I? My name is Leeran Gray. I go by Daddy Coco Man in most areas. You can check on my website, daddycocoman.dev. I haven't updated it in a while, but there's some cool stuff there, I think. You can also check on my GitHub. I have a couple of projects I've been working on. And hopefully I might have some more interesting projects up there very shortly. I was in the Navy for about 10 years. I used to do electronics. I used to work on radar and communication systems. And I moved over into the cyber field. And I'm on the Microsoft Azure Red Team now. So you may hear some Azure-related things if you follow me on Twitter or wherever you may follow me. Hopefully it's only on Twitter and not anywhere else like real life. I also love Python and Pythonic things. So we'll talk about some other Pythonic things later in this talk that I really got into at a certain point in time because it's pretty cool. I like the way Python is written as a language, more so than statically compiled languages. And I'm also a nerdcore rapper. I go by Omai. It's an I, not a one. You can find my music at MCOMI.com. I'm actually on the Def Con 28 tape. I got that hot track. I got that heat, X's and RAM. If you wanna listen to that, please do. And hopefully you enjoy all the music. So let's talk about the agenda. So first let's talk about why we're using Python 3. Then we'll talk about the two PEPs that have influenced audit hooks for Python 3.8. We'll talk about how to implement them, how to bypass them for Windows, how to bypass them for Linux and how to bypass Python. And bypass Python is very, it's in quotes for a reason, but we'll talk about some interesting concepts that came up over the last year. So quick question. Why are you still using Python 2? I want you to look every single one of these people in the face and convince them that they have the time to listen to you explain why you're still using Python 2. I'll wait. All right, so you might have a legitimate reason, right? There are sometimes there are tools out there that haven't been updated in the last 10 years or whatever. And if you are using something that is written in Python 2, I just suggest making a pull request, helping them out, helping them move up to Python 3 and try not to break that backward compatibility if you really need it. So here's the major Python 3 changes, right? That happened in the last couple of versions. Python 3.5, we had the async, a-wait keywords that came out. Python 3.6, we had format strings. So you could say things like, hey, my name is Daddy Coco Man. And then you can print my name is in Curly Brace's name and you would get my name is Daddy Coco Man. We also had order dictionaries. So now Python recognizes the insertion order for dictionaries and you can always keep track of it. Previously, dictionaries would just be a random result every time you would try to print them or call them. But now they're ordered. It's great. And Python 3.7, async and a-wait actually became reserved keywords. We had the breakpoint functions so that we could answer debuggers in the middle of our scripts. And we had the data class decorators so we can make like these struct-like classes. So you can say, here are these fields that a class is supposed to have. And here are some functions that may want to implement just like classes, but a little bit better in my opinion. Now they serve different purposes. So for Python 3.8, we have this controversial thing called the Walrus operator where you can assign values while evaluating an expression. So on something like this, we can say it reads if the length of A is greater than 10, then we can assign it to N. And then we moving to the loop logic for print list is too long, blah, blah, blah. But if length of A was not equal to 10, then we wouldn't assign it to N and we would just move on. So it's nice, it saves you a line of code. A lot of Python purists really don't like it and that's okay. People generally don't like changes to languages that they've been using for so long, but oh well. So next we have the async.io enabled REPL. So you can do python.dashm async.io and you can start using those await and async keywords inside of your REPL. And what we're here to talk about primarily are these audit hooks. So audit hooks are runtime events or hooking those runtime events to perform some action, some arbitrary action, whatever you want to do. So the first pet we'll talk about explains how the audit hooks are supposed to be implemented or how they're supposed to work, why you should have them. So the proposal was that sysadmins could use audit hooks in order to add some feature to their Python executable. So some suggestions that came in were integrating AMSI, a script block for Windows and then for Linux you could send events to syslog or auditD. And I meant to link it in here and I guess I didn't, I must have removed it by accident. There is a github, github.com slash Zuba, Z-O-O-B-A from Steve Dower who has a bunch of examples of what these audit hooks can look like in different implementations. So I would definitely check that out. Another thing you can do is restrict the importable modules. So you can say, well, maybe I need my people who are using Python in my environment to use the request module, but maybe for whatever reason they don't need to use the M-map module, right? So you can say, well, I want them to use specific modules that are only for my environment and I don't want them to use anything else. You can also do things like remove command line arguments. So before I mentioned with async.io or just a lot of other command line Python modules like HTTP server, you can actually remove that dash M or the dash C to prevent people from using those switches. But the one thing to keep in mind is that the PEP actually reads that you should detect the code over preventing the code. And the reason for that is because sometimes you may use a, the Python dependency tree graph is out of control sometimes, right? You may implement, you may try to use the NumPy module and underneath it, it uses C types. So if you try to block C types completely because you didn't want someone importing it, you may actually mess up using NumPy. So it's important to just detect what's occurring rather than prevent what's occurring because you might mess up something that you had no idea was going to go wrong. Another thing that the PEP recommends you do is implement OS security level feature to protect your modified Python. So that means device guard for windows, maybe do some EXE signing, use catalog files, that kind of stuff and extended attributes for UNIX. So maybe you want to add file attributes that have the hashes of the permitted Python files that are allowed to be used. PEP 578 says, here's how we're going to implement or how you can implement these audit hooks. And they can be implemented inside the executable or inside of the script. So if you want to do it in C, you can actually recompile a new executable, a new Python.exe, and all the hooks that you applied to that code will be run by this new EXE. Or if you wanted to just do it in Python inside of your script, you can, without compiling it, you can use the Sys module. So import Sys and then Sys that add audit hook. And then the audit hook will be a function where you can handle all of your audit hook logic the same way that you can do it in C, except it will only apply to the script where it's implemented. So here's two examples. On the left side, we have the Python implementation of network prompt hook, right? We have, this is some bad Python, so ignore this, but we have an event that gets passed into the function. And then we're looking at this event and we're saying, does it equal socket that get address info? If so, I want to take some action. I want to warn the user that we're attempting to resolve some URL. If it's socket.connect, we're saying we're attempting to connect to this URL. Do you want to continue? On the right side, we have the C version, which you can use to compile a new Python executable. Notice it's considerably longer. And personally, I never really learned C and I don't really plan on writing C anytime soon. So good luck to you if that's your thing. So here's a list of audit events that are currently implemented in Python 3.8. You can actually pull this from the documentation itself, but what I have highlighted are some of the things that you may really be interested in if you see these events occurring. Now, keep in mind that you have to modify this for your particular environment, but some things that you may find interesting are when if a script uses the compile function or the exec function, or maybe some of these win-redge functions on the right. Things that you may not normally see in your environment, you have to be able to adjust your audit hooks appropriately. Pty.spawn, I don't know a single case of anyone using Pty.spawn outside of trying to get a shell after they've already compromised your system. So things like that may be of interest to you and you may wanna capture those events as they occur. But again, the focus is on detection and not prevention. So we wanted to make sure that we are aware of these things are happening, but we don't wanna actually prevent them from happening unless you have a really good use case or use scenario to not let a particular event happen. Oh, gee, I mean, you can totally do it. You can block all events if you want to, just not have like a fake Python just running, sitting there. I don't know, honeypot Python, I don't know, honey Python, whatever. Anyway, so bypassing audit hooks for Windows. So first we'll talk about how auditing works. So when an action is, or when an audit hook is, is signaled, Python determines whether the action matches an audit rule. And that's kind of, maybe this is not the best explanation. What happens is it will run through every single audit event or audit hook that you have, but you have to write your audit hooks to only care about the events that you want. So for example, in this network prompt hook, we only care about socket events. If it doesn't equal socket, then we're gonna return because we don't want the rest of the hook to go through all that logic. So, and when you write your audit hook, when you write your audit hooks, you have to make sure that you only process the events that you care about. Also, you probably shouldn't do anything like, like a while loop, that something, some unbreakable condition, that's just not a good idea. And so the other thing to notice that all audit hooks are processed for all events. So even if I open a file, right, which is an audit hook event, open, the open function, it's still gonna get passed to this network prompt hook, but it won't do anything because I set a condition to only care about socket. So once of the source code to figure out how these things work. So on the left side, we have the pysys add audit hook function and what it does when it adds audit hook, it literally just adds the hook to the end of a linked list. When the hook is called in the middle or the upper right, it will actually just go through each hook that has been defined and just goes through all those functions and determines whether or not it needs to do anything. Or based on, it will, you decide whether or not it needs to do anything, but it will just run all of those functions at once in order, as you say. On the right side, we also see that tracing is disallowed. Now, what I originally thought this meant was that I could not see into what an audit hook looks like, but it turns out that that tracing set to false is only for the Python debugger. So if you're using the Python debugger, for every reason, maybe in your IDE or maybe on the command line, you actually can't trace into the audit hooks. But if you're using an external debugger, so let's say maybe you grabbed a copy of a modified Python off of a system and you took it back to your station and you opened up a debugger, you can actually see what's going on. So here's an example, right? The same example will be had the socket that get address info. I can actually look into this audit hook to determine what it's doing. I'm not the greatest at assembly, and so I know what this says, simply because I know what the code says, but you can still get a general idea. You can see the comparison strings, the S-Python that string compare, and you can see that it's looking for certain audit hooks to occur before moving into some other logic. And you can also see the strings there in our attempt to resolve the URL that it tried to resolve. So here's some ideas for bypassing. By implementation, audit hooks can't be cleared once they're set. So there is a function, I think, to like to clear audit hooks, I think, but it doesn't actually work. You also can't overwrite, you can't like reload the Sys module in order to clear all the audit hooks either. So one thing we can do is maybe overwrite the head of the link list, the point to null, but it might be possible and attempting to locate the start of a linked list in memory might be a little bit difficult, but I'm actually really new to exploit development. So it might be possible. So if someone knows something I don't, please tell me, and maybe we'll figure that out. But in the meantime, maybe we can just overwrite the bytes in memory that calls the hook functions. And that smells like PowerShell, right? Everybody knows PowerShell and AMSI bypassing, and we can actually apply the same concept here. So there's plenty of resources to draw inspiration from. Adam Chester over at MDSEC has plenty of articles about bypassing AMSI, so does Rasta Mouse, and there's a lot of inspiration just to draw from just by reading their blogs. And it's also significantly easier to determine the address that you need to overwrite with these new bytes that you want to overwrite with. So let's go that way. So if I take the Python 3.8 DLL and I open it up, or if I run Python rather, this modified Python, and I go check out the Python 3.8 DLL, I can go find the address for PySys audit. And we can see, this is the 32-bit version. We can see the address and set a breakpoint on it. If I scroll further down, I'll get to a point where a function is called and under call EX, it has the name of the hook that is being called. So if I were to step into that call, I would actually see the logic that I showed you earlier. So the offending, I say offending, but the interesting call that we want to overwrite is where EIP is currently pointing. And that is called EX for 32-bit and call keyword pointer for 64-bit. So again, it's very similar to PowerShell. We need to load the Python 3.8 DLL. We need to find the git address for PySys audit. We need to change the memory permissions and we need to just overwrite those bytes. And I learned this week a couple of days ago that I really need to go through all these versions of Python and grab their offsets. So I went through the 32 and 64-bit versions of Python from 38 to 3.85 and even just for poops and laughter, I went to 3.9.0, beta five, just to see if it still works and it does. So we just have to set these offsets and we can change the memory permissions and overwrite those offsets. Let's talk about Linux real quick. So I have never done any sort of major Linux exploitation other than like a buffer overflow, right, that you find in like CTFs. I've never done any patching. So I didn't fully understand at the time how the Python packages worked on Linux. So there are no DLLs, right, because it's not Windows and there was no sysmodule.so library that I thought was gonna be there and it just wasn't. So it's like the same though, right? You can find the PySys audit address by searching the Python executable itself. So I guess it's statically compiled into the binary and you can just pull it out. So if you use the read-elf, we can use read-elf to figure out what the address of PySys audit is and we're gonna cheat a bit because I actually don't know how to parse L files. So in my code, I just sub-process read-elf and grab the output because I actually have no idea how to do it otherwise. And then we have to locate this address in memory. So we, once we grab that address here, it's five, three, bravo, seven, foxtrot, zero. We need to figure out where it's located in what region of memory that it's mapped to. And in this picture, we can see that it's under, it falls in the range of F230 and 6AE. So we need to modify those permissions. And we can do that, of course, with libc and nProtect. So we need the write execute permissions to change and to read write execute permissions. And sometimes, yes, sometimes it just be like that. I am watching the discord, by the way. So what this looks like is we load the DLL for libc or we load libc rather to say and then we find the read-elf, we find the address of PySys audit, we find the location of the memory mapped files and then we determine in what region it falls under. And then we change the permissions and then we just overwrite the bytes in memory with these knobs. So one thing I didn't go too far in depth in Linux is because every build for Python in Linux is different. So I don't have the time of the day, like I'm stuck at home, I only go out to the supermarket if I need to, but I still don't have the time in the day to go through all of these Linux versions and try to find these offsets. So later today, I'll probably put this code up on GitHub. And if you have a version of Linux where you want to bypass audit hooks for 3.8, then you can send a pull request and I will happily accept once it's been tested. Let's talk about detection. While everything that I've mentioned works, everything up until you patch the binaries, the patch the bytes rather can be detected. So importing C types can be detected, importing platform, reporting Sys, every single function call that you make can be detected. Until you hit that men move and you overwrite the bytes. The way to defeat this is ironically better audit hook logging because if you can detect that these things are occurring and you send them off and you ship them to your Seam or however you, your windows event logs or however you collect your logs and you see that these are occurring, well, then you have a better chance of understanding what's going on with your Python executable on a random given station. So yes, it's sort of like, it's like running AMSI bypass in PowerShell where you might get caught, right? It's the same thing. So that is something you should, you should probably implement better audit hooks. And it might be harder for someone who doesn't program and see to implement, right? I don't think most Sys admins have a lot of C knowledge. I could be wrong, I could be naive. I've never been a Sys admin, I have no idea. But it's probably not something that you come across in the general resume where you need 10 years of Adobe XD that came out last year. So it would be great if somebody came up with a template of audit options and in action to help Sys admins compile those custom Python executables. But as I mentioned, I don't have the time so it's not me. So I'm gonna give a demo real quick. Hopefully we can all still see the screen that I have up. I think it should be working. A Discord confirmation would be great if you can see this terminal. We can see it. Or a confirmation from Omar. Yeah, we can see it. Either one would be great. You should believe it. Cool, thank you. Thank you, you can see it. Cool, great. I have this executable here, this S-Python that's been compiled, right? And I'm just gonna enter it directly. And I'm gonna try to run this code. And all I wanna do is get the status code of the redteamvillage.io. I just wanna make sure it's up because redteamvillage has been doing a great job and I wanna make sure that their website is up. But if I do that, I get this warning saying, you're attempting to resolve redteamvillage.io. Do you wanna continue? Yes, I do wanna continue. Yes, I do wanna handle the socket event. Yes, I do wanna connect to this. And finally, I can get the HTTP status code. That's great. But I don't wanna have to go through all that again. So maybe I'll just do this audit bypass and we'll see if we're getting the handle. I'm pretty printing all this arbitrary stuff so it doesn't really need to be printed out. We get the audit address, we find the protections. Using the offsets, we find the region of memory that we need to overwrite and we successfully change the memory. And if you ever played NBA Jam, like the video game of your life, this is the point in time where I say boom shaka laka and we bypass the audit hooks. So now when I run the same code, again, we just get status code 200 and yay, right, we win. We don't have to worry about the audit hooks and going through all that stuff again. So that's great. Let's get back to these slides. Oops, I lost my cursor. There we go. So let's talk about bypassing Python because as I mentioned, everything up until that point where you overwrite those bytes can be detected in memory or detected by an audit hook and that could be bad, right? So let's take this a step further. So since the bypass is easy to detect, are there any other methods that we can use to overwrite or to bypass Python in a sense, right? So there are other ways to load and patch GLLs. I've had some conversations with some pretty smart people about Python internals and they've suggested other ways to do it. So just loading virtual, loading GLLs to kernel 32 and virtual protect and all that stuff is just one way to do it, but there are other ways to do it. But everything that you do, that way has to be run in the context of the Python interpreter. So no matter what I do or no matter what you do, rather, you can be detected no matter how you do it. So the question is, can we run Python without running the Python executable? And obviously the answer is yes because I wouldn't be asking you this question and wasting your time with that bit of the slide. So there's a package called PythonNet, which is a legitimate package, it's great. And it allows you to access the .NET CLR from normal Python. It's not iron Python, right? It's not iron Python at all. Iron Python is an implementation of Python that is, as far as I'm aware, not even MSIL compliant, but the syntax is still similar to iron Python. And as of May 2020, it supports Python 3.8. That's great. And my camera just went out. So I'm going to turn my camera off if that's okay. So we have the, here's some text, right? We can import CLR from collections. We import counter and then we can access the CLR the same way that we do it and sort of iron Python-ish ways, right? Where from system.diagnostics we import process and we can actually list all the processes in a CLR manner. CLR like syntax or .NET like syntax, I should say. You can actually also embed Python in your .NET executable. So PythonNet comes with a .NET assembly called Python.Runtime.dll and that can be used with .NET languages to run Python. The Python types for the basic primitive types are actually written in C-sharp and compiled as part of this assembly. The only requirement is that your path variable must include a directory with the Python runtime installed. So in this case, we're looking for Python 3.8.dll. So here's an example of what it looks like in C-sharp. We have this main function and then we are using the Python gil and we can import modules that Python knows about such as numpy. And when we run this function, we get this output to the right and it's pretty cool that we can actually run Python inside of .NET. Now we can use C-sharp to write this, but I like Boo Lang. The reason I like Boo Lang is because a byte bleeder and the Silent Trinity project that C2 framework that he has, it's pretty dope. I've contributed to it in the past and I really like Boo Lang because it's very Pythonic, right? We're looking at the, if you look at the code that's on the screen right now, it almost seems like it's Python, but it's not. It's Boo Lang. Also support byte bleeder in his open source sponsorship program that he has, it's definitely worth investing in people who have helped you in your corporate endeavors for pen testing and retelling. So what we need to do, as I mentioned before, is point the path variable to a Python installation folder, which gives me an idea, right? Because the path variable can be set to any folder on the disk or on a network share. So you can already see where I'm going with this, hopefully. I tried to find a black one, but there are no black emojis. So we can do Python across a network share by simply by pointing the Python path to it. So in this case, I had a share called winwork slash Python 37. And this was 37 because at the time I was testing with 37 before 38 came out. So it still works with 38. So ignore that. And that's what it looks like, right? So the path variable can be set to the server slash share just as long as that share points to a Python 3x installation. And this is what it looks like. So you can see all this SMB traffic and you can clearly see that I'm loading Python 37 across a share. Now, this isn't like an abuse of anything, right? You can actually install Python when you install Python normally on a network share and everybody acts as it's slower, right? Because you have to load those resources remotely, but you can actually do that. But if we can use SMB, that probably means we can also use a webdav. And sure enough, we can actually load Python across webdav by sending the server and the port. So back, whack, whack server at port and SSL is optional. If you, there is a registry setting in Windows that determines how you can authenticate with webdav and you may have to set that in some environments. I was having issues with this earlier today or earlier, I guess it was today or last night or whatever. And something broke and I couldn't figure out what it was. So I can't show you what it looks like going across a webdav server. But it works, here are the packets. It worked at some point in time, I just need to figure out what happened. So we can also host our own repo to remotely load packages, right? We don't need to pip install anything on the target box assuming that we need extra modules that aren't already there on the Python 3.8 installation. Empire did this by implementing a custom import hook that allows you to modify the logic in how modules are loaded. So it allows you to load things remotely. And if you want more information on that, you can check out Chris Ross's blog on in-memory Python imports or you can check out the Soolinks GitHub repo for remote importer and you can check that out. It's really interesting stuff. There's also like another module out there called HTTP importer or something like that that allows you to import modules over HTTP directly. And of course, Empire was written a long time ago. I know a new company has picking up Empire, I think it was BC Security. And I think they recently released a tool called Starkiller that seems really interesting. You should probably go check that out. If you wanna load a package remotely, you need to just have it in the standard format. So here's the request module and I want to just import this. This is what it needs to look like. It needs to be the zip file. Inside it, you need to have the name of the package and then a folder with the name of the package. And inside of that, you need to have all the files that you need for the package. Let's do a demo of that real quick. All right, so in this folder, I have some code called scarysnake.boo. Everybody who writes C sharp tools likes to call their tools sharp something, but no, this is boo, it's about ghosts. So we're doing scary snake. And so we can actually compile this. I wrote a little thing using the compiler that comes with boo and we can just compile it real quick. We'll also use IO repack to package the additional assemblies that we need that includes the boo.lang assembly and some other additional things. Most importantly, the python runtime.dll that you see here. That's most important one, right? And now we have this tool called scary snake. And it tells me that the python path must be set, right? So I'm going to set this python path by just passing it as an argument. Sorry, I had to go find it. And then we can do something like maybe enter interactive mode. And this is the output that we get. It's basically, I tried to imitate the same output that python gives you when you start the interpreter. It shows us where the path is. And again, if you were doing this remotely it would be over web dev. I think I had set up a, in the past I had set up a server in Azure and the load time was about 30 seconds just to run python. But if it takes that much time to further escalate or do whatever you need to do with the python to not get caught, I think it's worth the investment. So we can do things like, here we go again. And we're going to print out the status code of course from redseamvillage.io. And without even dealing without changing the normal python executable we basically brought our own interpreter to the system and we were able to run python using the python that's on the system be able to run python without worrying about audit hooks. So once again, boom shaka laka, I'm on fire. I'm not literally on fire. Don't call the fire department. So some of the other options that are here include the local file. So you can run a local file. You can also run a remote file. I forgot to actually test this this morning. So I don't know if it works. So I'm not going to actually do it because that would be embarrassing. But it has worked in the past. I'm trying to run a remote file. So you basically download the python file from the GitHub or wherever you have it hosted and you just sort of run it in memory. So let's try something real quick, right? I forget which system I'm on sometimes. There we go. Window, terminal, split pane is great. So I'm going to try doing this. I want to import the request module, but I don't have the request module. Just to show you that it works. That I'm actually using the actual Python that's hosted on the system. I'm going to Python dash mpip, which by the way, you should always do Python dash mpip because if you have several different Python versions, you don't want to like confuse yourself. So you should always specify the version of Python that you're using when you run things like pip. It's just good practice. And we're going to just pip install requests. And if I've installed request and now I can import it with no problem. So again, you can definitely use the Python libraries that are on the system to do it or you can host your own remotely and sort of just pull from a remote repo without actually touching anything on the disk. It's really up to you and what's in the best case scenario for your immediate situation. But I think it's great that we can actually just do this in the first place. So far, just to reiterate, we've talked about how to bypass auto hooks for Windows, how to bypass auto hooks for Linux and how to not run Python on any system without running the Python executable whatsoever. For detections, it's a little bit difficult because everything is in .NET. So you have the same problem of .NET detections that you would find in any other .NET binary. If you're collecting ECW logs somehow and you have the space for that, wow, that's impressive. But if you have that space, you'll actually see that the Python runtime is loaded. So if you're in an environment where you don't normally see that, right, that could be something you detect. In this case, because I wrote it in Boo, it actually also detects the Boo.lang binary. But you could write this in C-sharp if you really wanted to, if you wanted to be a conformist and write all your tools in C-sharp, you could do that. And obviously, you wouldn't have the Boo.lang. So in conclusion, Python auto hooks are great. When they're implemented correctly, you should protect your custom Python executables with some OS level protections. You need to do something, right? The same way you do any protections of any other binaries on your device, you need to make sure that your Python executables are protected. Because if I can pull your executable off of a system and you have auto hooks enabled, I can debug it and determine and get a pretty good idea of what your system, what auto hooks you've implemented just by looking at the code or looking at the assembly. And of course, that ties in with, don't rely solely on Python to protect your Python. So even if you had all of these auto hooks, again, I can pull your executable off and do some other nonsense with it and then figure out how to bypass your auto hooks. So thanks to everyone who has had some influence on this talk. Steve Dower, who works on Microsoft, he's a Python core dev and he's actually the one who's done most of the implementation work for auto hooks. Byte Bleeder, who's got me into Boo Lang and in the embedded scripting, go check him out. Chris Ross over at Specter Ops worked on Empire and Adam Chester.