 are not rock stars. Hackers are not rock stars. They are rock stars. We got nothing on them. Alright, so before I say anything, I want to thank you by new. Only FConk could go to China, and I think only you could make that happen. So thanks guys, and thanks to all the other sponsors Rock Band is my first time in China for some of Beijing. I've been to Asia before, but never out here. It's really exciting. So what are we here to do today? Well, as you think you know talk, so I'm supposed to inspire you all to build crazy, interesting things. This is a technical talk because they say you stop writing code in your 30s. That is a dirty lie. I've written more code in the last few years than ever in my life. The goal of this talk is to connect a series of concepts you may never have thought were linked. The quote unquote hacker mindset really is just saying you know those things might be connected. Hamburgers and cheese are great. Hamburgers and clouds are sharp. Anything can be linked. I kind of want to like expand your horizons there. So let's consider this sort of a sky dive. We're going to start with a bird's eye view, and then we're going to dive kind of straight to the weeds and get ourselves into a bug's eye view. So I wanted to start this discussion. There's a problem when we talk about hacking in terms of only software. So like if you already understand software, maybe you'll understand what I'm talking about. But what does hacking look like when I have nothing to do with software? And I figured you know, most of us can see I do have a hobby or human perception asking about colorblindness sometimes. There is a myth in human vision that we see at 60 frames per second. And he says it's a myth because you can run pretty much any experiment you want and it's like no, we don't see it 60 frames per second because we don't see it frames per second. Your eye just kind of like wanders around and collects a bunch of gunk and your brain's like dreaming works. This is why you're able to see things when your eyes are closed and you're unconscious and your brain is mixed up. It's kind of cool. But why is the number 60? Because this shows up all over the place. Now my traditional answer to why 60 frames per second is, oh, they say it has to do with human vision, but really it's 1940s television technology. That's just how fast we ran TVs back then. It's true it was how fast we ran TVs back then. But why did they choose 60? Could have chosen any number numbers, but why 60? I didn't realize my answer was incomplete until some news came out of Eastern Europe a few months ago that everyone's clocks were running slow. I actually literally mean like the clock radios on there like, you know, cabinets. European clocks do six minutes under this beauty-saps power from this electricity grid. Turns out most of us survey it don't like each other very much under statement. And there was some stealing of electricity and it was slowing down the frequency of the power. Just like a water out of the tap has a certain temperature, power out of the wall has a certain frequency. That's supposed to be around 60 frames per second or 60 hertz and it wasn't. It was going a little slow and so everyone's clock was going slow. Now you could just have like a clock in your clock radio, a little coarse crystal that vibrates, but you could keep it cheap and use whatever's coming out of the wall and assume it'll never change. Just because you can doesn't mean you should. I'm looking to tell you how often this happens in security. Oh and that thing, we'll just get it from the upstream. They'll take care of it for us. Well they weren't taking care of it and all the clocks ran slow. So we didn't make television 60 frames per second for human vision. We made television 60 frames per second because there was a 60 frame per second 60 hertz clock handy. Okay well, why was that clock 60? Why did the power lines go to 60? So now you just get to keep digging. Power was 60 hertz because in 1890 that was what we were good at making technology do the porting Wikipedia. The induction motor was found to work well on frequencies around 50 to 60 hertz, but with the materials available in the 1890s it would not work well on a frequency to say 133 hertz. And it was too small to see up there. You actually see like all the various frequencies that people picked for power lines. So now you get this 60 frame per second thing which supposedly did do with vision. Why did they pick these numbers in the 1890s? What the materials did? Well quote, there's a big relationship between the number of magnetic poles in the inductive field, the frequency of the alternating current and the rotation speed. Technology now is physics. Actual physics encourages 60 frames per second. And this is what blew my mind. And why did you bring this up because I did not expect to find anything close to this when I started digging into the number 60. You know our brains are physics too. The brain circulates electromagnetic signals. We've been in a series of waves called gamma waves that went 25 to 100 hertz. Pure speculation, you might have 60 hertz and 60 frames per second and all these things because physics is a shared element between power generation in the 1890s, between television and our brain. Correct? This might not be. And that is part of hacking too. You might be thinking why am I going to this plane in a talk that's supposedly about bugs? And what I really want to get through here is we don't necessarily know why things are the way they are. Usually as a society, as a people we do things because we've been doing them. And they mostly worked. What hasn't been a problem what I want to get across is there's a universe of reasons why things are the way they are. Sometimes they don't need to be that way anymore. Sometimes they really do need to stay that way and we just don't know why. So stay intellectually honest as you go through these deep dives. Understand you really are operating an ignorance that's actually your strong point. You don't know why the thing is doing what it's doing. That might mean you find something new. It might mean you aren't. Just have such humility as you explore. But also explore. I do it. It's okay. Just know that you're speculating. So speaking of speculation let's talk about what is actually going to actually become one of the ugliest bugs we've had to deal with in the last time. Speculative execution bugs. So there's a couple that people have picked me and talked about. I assure you there's a hundred more. And I'm not exaggerating number 98, number 99, number 100. There's a giant pile of flaws in this arena. What are specter not that? Best explanation that I've had that I've been able to explain to people is you ask your friend, oh hey, go to the coffee shop. They're like, no! Hey, do you go to the bar? No! Hey, you go to the club? Sometimes you're not saying the same thing. These bugs are timing bugs. These bugs involve a security boundary that's attempted to be maintained by a microprocessor where it either is or is not supposed to give you some bit of information. And in fact, it finds out that it's not supposed to, it rejects, it gives you an exception, it gives you an error. But when it gives you the error, it's different based on some information that you don't already have. Generally it's the sort of thing where, hey, the first character is a password in A. It's the first character in this password in E. It's the first character in the password in C. And when you say C, that is the correct answer. I don't think these go too fast. One thing is go too slow. In the case of specter, it's where you're trying to read data that you're not allowed to. You're told no, but it's the wrong time. Actually that was meltdown. With specter, you're trying to run a code you're not allowed to. And again, things go faster and slower based on what you were or were not allowed to run. What went wrong? Now, commonly people say when things go wrong, oh, you know, Intel, they were lazy, they were stupid, they were bad people. I have to tell you, in 20 years I've never seen stupid moralization fix anything. Like we're engineers, sometimes things are going to fail. We can either figure out why they fail and do something about it, or we can point and laugh. And don't get me wrong, a lot of people like pointy laughing and they can go have fun doing that. But as engineers, and because we're sort of an engineer, we have better things to do. We can point and study. So digging in, the assumption in a lot of vulnerability analysis for bugs like specter meltdown, for the specter to do execution flaws, the assumption was that you could find out something was in memory, but not what it was. So here's the deal, maybe a little less in how microprocessors work. It doesn't matter how fast your chip is, doesn't matter how powerful it is, doesn't matter what crazy 3D instructions it can do, it doesn't have the data in hand to work on. So a lot of the real world work of changing a fast chip is in making sure the right data is available at the right time. So your system might have you gigabytes of memory. It only needs to work on a small subset of data at any given moment, but that data really needs to be available, it's just going to sit around. I guess I've got to wait like 20,000 cycles for the data that I need to work on. That means it's running at 120,000 times the speed or so, for certain instructions anyway. So your chips have caches. They have local stores where it's like, okay, so I have a copy of what's in memory over here. If I need to run it, I don't need to go all the way out to main memory. I've got someone, what's called my L1 cache. The assumption was that you could know that something was in L1, but not what was in there. So that was a nice theory. Unfortunately, we have very rich things that we can ask the chip to do. We can tell the chip, hey chip, I want you to go get me this memory plus the value of that memory over there. Like the address, it's like, hey, there's a house at 55, go to 50, and by the way, take that 5 that's over there, and add that to the address too. In fact, I don't even know what's the 5 over there. Whatever is over there, add it to the address. Maybe you end up with 55. So without any care for the security model that the chip is trying to maintain, it will simply go ahead and be like, okay well, I've got this base address to 50, that value over there is 5, I guess I've got to go retrieve 55. If you're an attacker, you're like, hey, do you know about the address of 51, 52, 53, 54, 55, and 55 actually is already there, because that's what was forced in. And so now the attacker knows, okay that value was 5. There's only 256 possible values of a byte, so you do this 256 times and you get the byte. You might say, oh, but the difference is going to be really small, but it's all really fast anyway. Yeah, it's really fast. That means you can do it like a million times in a second. Any small difference multiplied by a million is no longer a small difference, especially when you can do what's called a CL flush, a cache flush, make the difference as large as possible. So that's kind of the that was kind of the transition that has happened, the gift in attack methodology. But there's something underlying all of this which I really want people to understand. Spectre and Meltdown, this entire bug class exists because we assumed we could make computers faster. A lot of people are like, why do we have these bugs anyway? Isn't this just mad? Can we like think things are correct? And we can. Primary consumers of mathematical proving software in the world are microprocessor designers. You know why? Because when a chip screws up, you can easily like, ship a patch. Like, we try, but like, that stuff is, those bugs are literally etched in stone. That is, in fact, what they are in. Do not like having microprocessor flaws. And so, we do a tremendous amount of work with what are called theorem proveers, what are called stat solvers, and we make sure the right bits, when the right bits go come in, the right bits come out. It has not been part of the equation. That is why your computers are allowed to get faster, your phones are allowed to get faster. You know, if you really think about it, a faster computer is doing different things. It is, before it took this long, now it takes that long, that computers act pretty differently. But we define it as operating the same way. So, all of these bugs happen because security has been made to depend on an undefined element. So, context matters. If you'll excuse me hopping to the 30,000 foot view, are two things the same or are two things different? Depends on the context of what do you mean by same and what do you mean by different. Same thing might be predictable or random based on context. Corporating means relatively predictable. You might even call them plotting. But an executive at a corporation, that guy could like quit tomorrow and like to have a nervous breakdown. An executive can be erratic. But the actual heart in that executive community that is actually beating, erratic by, you know, a behavioral input. A heartbeat is pretty steady. But a single cell in that otherwise beating heart, a single cell might be erratic. We're talking about the same point in space, the same point in time. Is it predictable? Is it random? It depends on the context in which the question is asked. Are two computers doing the same thing? Are they doing different things? Well, it depends. Does time count or does time not count? And there's not a right answer to that. There is no one context. A huge amount of what we do in hacking and what we do in security is we play context off one another. Or like, that context thinks this is stable. That context thinks it's not. They are going to behave in a different way. We said something right down in the middle. Tremendously reliable source of vulnerability. So, the state improvers didn't fail when they showed no leakage of information between contexts. Because the right bits went to the right places. They just weren't being asked to prove these particular elements. The last thing that we did that caused Spectre to melt down to a curve is what I call the great re-purposing. We turned a stability boundary into a security boundary and hoped that it would work. Spoiler alert, it did not work. Historically I know this is going to surprise you guys. Most software is pretty bad. It kind of barely works. Historically, most code would crash all the time. The game was making it so it would only crash itself. How do we get it? When this particular, the calculator does, it doesn't take down the server backup. Because I can live without the calculator. We did this stuff where all the software was specifically off into the resources that are required. This is my RAM, this is my network connection, this is my stuff. So, when the software blew up and visibly blew up, you can clean up just those parts. So, the theory was that hackers were just kind of a news source of misbehavior. Instead of the software crashing randomly, it was crashing because someone intended it to. But otherwise we could still see it doing bad stuff and we could still cut just that part off. It kind of worked. The thing about hackers misbehavior, hackers are actually pretty well behaved. When hackers crash code, it doesn't actually crash, it actually does like re-control things. So, what hackers are doing is changing smaller things from a computer's perspective that are bigger things from a human's perspective. So, you don't get these, these random noisy data structures that immediately cause all these exceptions to fire. Instead you get nicely well-defined data structures that cause the calculator to pop up. In the context of Spectre and Meltdown, attackers change time. Which if this context is not defined to exist, these operations that might happen fast might happen slow. Fast and slow don't exist in the context of the microprocessor running these instructions. They're allowed to take as long time or as little time to get the memory and get data out of memory. So, it means nothing yet, but it means everything to the users, to the administrators, to the security models, but nothing in this context. That's what's going on. It's worth noting, by the way, the exploits of Spectre and Meltdown involve attacking the system timers, which often operate once every, the fire was 15.6 milliseconds. Kind of a specific number that 15.6 you take the reciprocal and you end up at 64 frames second, because the only thing computers like more in repurposing 60 FPS is powers 2. So, this is the thing Spectre and Meltdown, we bits that we would prefer they not. Can't we bits you do not have? There is a hidden architectural choice in all of these bugs. That architectural choice decision is context switching. What we'll do with computers generally is, we have a tremendous number of context in which they're operating. One moment it's you, the user, another moment it's the kernel running some device driver, another moment it's the web browser as a whole calling something, another moment the browser from an individual page on that web browser it's constantly flipping around and flitting around and some of those contexts have special access to data and some of them don't and we hope the design of our interesting architectures mean when we switch from one context to another nothing is left over. You can do that context. And it's kind of close back to quote security domains, get to computers. You can do that. Computers are small now. Like, I don't know if you guys have seen this stuff coming out of Shenzhen. This is beautiful. Look at this thing. This is 20 bucks. 20 bucks. It's got four cores. It's got a half gig. I actually have one here. They want to know if I bring a camera. I'm like, no, I want you guys to like strain your eyes as I see this. Hey Raspberry, you're buying a nice job, but here's a real computer. It's got an Intel. It's got a Apollo 4.4. It's got PCI Express. It's got 5 gigabits, 10 gigabits, USB 3. It's got a PCI port on the back. You know, yes, we can totally go ahead by switching work, trying to pretend like we have 100 computers. Maybe we should just have 100 computers. We could potentially do it. So present approaches to dealing with specter and meltdown has been there's a lot of work that we put into making context switching able to be aware that there is a security boundary and that we should not just protect what is visible at the user level or even at the kernel level, but actually it was called a microcode level. So let me tell you something Intel chips have been running UX86 for a long time. There's another operating system underneath every operating system that you run. And that is the operating system that's managing all sorts of this prediction what memory to grab when. That's the thing that we're interacting with with all the specter and meltdown patches. It's incredibly painful and honestly it's pretty slow. There's a great article from I think there's a great proof by the guys. From this company Research in Motion and they made the Blackberry, they basically made a really compelling argument that the iPhone was totally impossible. And their argument was incredibly compelling. Right up until the moment the C-drop dropped an iPhone on the table and said, well, what do you think of that? The iPhone was impossible because well, we just sort of assumed a certain amount of interactivity required a certain amount of power and electricity and resources and here's what we were going to get from batteries and here's how fast batteries have been getting better. And you can intersect all your curves and you end up with the iPhone will never have a battery life more than 20 minutes. And if all of those assumptions were true, that would have been correct. All of those assumptions were not true. The reason computers got fast is because this microcode layer, these layers that do all of this prediction about what resources are required when, what work can be done in advance so then you don't need to wait. You find out in real time like, okay, if someone asks for this, they're gonna have to do a bunch of this other work first. The work to be able to be long. It's an interesting thing. One thing that every class developer is mad at, but the microprocessor guys are masters of is this will probably work. The microprocessor guys have mostly figured out how to do. We're gonna try this because 99.9% of the time is exactly what we're supposed to do. 0.1% of the time it was the exact opposite and we need to hit the brakes and pull everything back and make it like it never happened in the first place. Pretty much only the chip guys have this working. But man, you ever wonder why like an Intel chip's really fast and like chips have the exact same specs. They're just as good memory, they're just as good instruction set, but for some reason everything is slow. That's why. Rage prediction. Because we didn't have the worst machine learning around yet. Oh yes, prediction and learning of course they're linked. Kind of obvious when you think about it in retrospect. Tangents aside, I like going on Tangents when I said we have patched everything in case there's a security boundary. So we mean there's always a security boundary, there's a ton of machines where the only user on the device is the administrator. There is no non-route or sometimes the administrator is only not administrator when they're running a web browser. And some portion of the administrator's code is a non-route actual non-user. They're still the user, but they're not like the user who should run code. They're the user who should run JavaScript under a particular domain name. It's called an origin under the same origin policy. Out of what you might call an electrical domain impedance mismatch between the security models that are architectures exposed and what we actually do in the real world. Let me tell you, there's nothing about the most popular security model in the world is the same origin policy in your web browser. It's the thing that makes it so you can have one tab open for email and one tab open for news and the news side can't read your email. Like someone had to build that. That security model is nowhere to be found in how it ships work or how operating systems work. But it's like the most popular thing we do. So to deal with the horde of bugs that are coming along with Spectre and Meltdown. I'm not joking. I think it got announced today. We will get a more explicit type of security domain to be declared. It will look like users will look like processes. It probably won't even be constrained as single machines because guess what? As you look at the cloud from the larger scale or as you look at if you want two computers, two security domains, get two computers at the small scale, all sorts of resources get lashed together. And the amount of flexibility that we're going to need to declare what is happening is not able to manipulate each other. It's going to get complicated. It's going to be interesting to watch it grow though. There's a shocking amount of operating system design work going on up there. I thought I was being all clever and not going to lie. Oh man, I really thought in the future and then it's like the HPC guys are like get in line and we have figured this out long before you. And why hasn't HPC high performance computing figured this out? Well it turns out huge amounts of just how operating systems normally work involve a user kernel boundary where the user is untrusted and asks a kernel to go do something. This turns out to be ungodly slow. I'm not sure how much of our computer performance we're giving up to this, but it's pretty clearly double digit. Everything fast goes ahead and just for performance reasons. Just for purpose, says user kernel needs to go. You'll see like the Intel's networking stack DPDK will run entirely in user space. There was a Linux web server called tux. It should be obsessed in Windows. It will run entirely in the kernel. There's a cute little thing. You will know somebody is an old school kernel hacker. Have you ever heard the phrase kernel both Linux? And you'll know they're a security person because their head just exploded. There's a funny thing called run kernels. They claim to not have kernels in them. This is not actually true. They totally have all the kernel code. They just don't run the kernel code with the kernel boundary. They run in kernel as a library. There's even a thing called LKL that is Linux not as a library. It's really cool. You can run a command and watch Linux boot here in Python. Like load the library but it's not as handy. There's a giant amount of work going on here because when you look at the HPC space, when you look at all the people that are looking for performance, what they are really saying is there's a security boundary but not here. The computers all trust each other. It's not like 500 of the computers over there are run by bad guys but we want to hopefully still get useful compute out of them. No, they're all mutually trusting. So, there's a thing that security forgets which is that you can have a place where a wall could be but you don't always want to put a wall there. Not all walls are a good thing. Like build your boundaries to what the actual security model is. Otherwise you'll have fake boundaries or speed bumps for people to jump over. You just stopped useful work from happening. So, why am I telling you this? Well, me being kind of real here, security that doesn't care about the rest of IT is security that grows increasingly irrelevant. Computing in 2023 is not going to look like computing in 2018. Too much is changing. That year that I showed you over there was not around 5 or 10 years ago. That matters. We build things differently now. In fact, computing in 2018 doesn't look like what most people think computing in 2018 looks like. There's a lot of mental models even among hackers. People are really, really stuck on what they have, you know, on their desk. Let me tell you I go ahead and I rent a machine for $13 and it's got a terabyte of RAM. I solve problems differently than you do. Also means I have vulnerabilities different than you expect. So, that's the flip side. You're just looking for bugs. Look for things people think don't matter. And for time, look for the flawed assumptions that developers have, that operations has, that, you know, between how people think this system works and how it actually does. Bugs aren't random because their sources aren't random. And mismatch in assumptions starts in the developers thinking your system's working one way and it's really working another. And then going on is what as a hacker, a hacker thinks, hackers have to model, hackers are just modeling code. They're not good ones. Hackers are modeling the developers and say, what did that guy screw up? I tell you. I can't tell you how many okay, this is my own process. I used to run a hacking group out of Seattle. We spent a couple years at Microsoft. Every time I would sit down with a team, I'd just be like tell me your story. Tell me what you're worried about. Tell me how you think your system works. And in order to do that, I shut up and listen. And listen for all the things they did talk about. Because that was always where my first bugs came from. Oh, you didn't think of the off layer. Oh, you didn't think of the storage layer. Oh, you didn't think of the directory layer. You just looked for what you see in the code and what the developers are thinking about. Like, my favorite one of my odds was going off on a complete tangent. I'm like, reading through the docs. Like, verb here. Repair. Like, hey Bob, I'll get this look on his face. That's still in there. Yes. Hey, Bob. People think bug finding is purely a technical task. And it's not because you're playing with people's assumptions. It's so much more, or at least there's such a strong angle of understanding the source and you'll find the destination. So right now's about a good time to introduce what I might say is the Archie only made the correct catchphrase designed to spark interest. There's no such thing as reverse engineering. It's kind of a cute little combination. Let me tell you what I mean. Our team, we only make the car drive left. Right? That's like, that's some other guys. It's just my job to get my plane in the air. Ooh, okay. There are different teams. There can be a different team that handles takeoff and landing. It's just if you don't care if your work affects other guys, it's going to crash. So my thesis is there's no reverse engineering. There's forward engineering. There's engineering, okay? But we do have cultural elements in engineering that block the integration of forward and reverse. And the primary thing that we seem to do wrong is I think we have aggressively separated development and testing and it's biting us. It's biting us really. Hackers like, you know, I was the director hacking. I was the director of penetration testing. You know, hackers like the penetrations. That was cool. But no, but testing is the important part of that phrase. We are a specific branch of testers that I don't know gets on cooler stages or something. Testing shouldn't be split off, but it kind of has been. And it's kind of had to have been because people, when they write code, tend to see that code for what it's supposed to be. And as a tester, you're trying to see it for what it really is. These are stupid things. What this means in practice is that large amounts of tooling in software that tell you what software is going on are isolated to a different group of people. So the developers who already have a problem psychologically only seeing what their code is supposed to do are also isolated from all the software that would tell them, no, actually, you're doing five of the techniques for what you really shouldn't be. So the crease is enormous and bias in developer knowledge. Anything that's too testy goes to the test people. They'll figure it out for us. And what it does ends up biasing the generation of a lot of code. And let me actually show you in concrete terms what I mean by this. So fortunately, ancient language pretty fast. Let's do a lot of interesting math. Python, convenient language, not so fast, but fast enough. Useful. And last, there's a package called Numba. We can't just stick to hacking. There are some tools out there that do some freaking magic. Numba is, as far as I know, the first practical environment for taking what is Python as a nickname, executable pseudocode, it lets you write Python and then it does a bunch of magic and suddenly you have code that's approximately as fast as Fortran. I'm not even joking. It is a LVM CPU GPU optimizing platform. It is an optimizer, which means it works like most optimizers do. It constrains what it thinks can come in and, given those constraints, throws out a bunch of optional work and run time and ends up delivering very fast execution. You think about all of these what I just said in the context of hacking. What happens when you define your constraints incorrectly? There's a simple answer. Vulnerability. Stuff blows up. At best, you get the wrong answer. Most commonly, you get undefined behavior, which in the presence of hacking becomes redefineable behavior. Problem that you think Numba has to deal with is that Python is dynamically typed language. When you have a variable you don't really know what's going to be in there. There is some dependency on the developer telling you this is going to be an integer, this is going to be a number that can only be between 0 and 2 to 63 or 2 to 64 and so on. There's some developer pain to go ahead and declare up front what data types are going to be in one place. The whole reason they used Python was they didn't have to declare that. They could just say there will be something there, figure it out or run time. Well, really cute code came out recently. Dropbox wrote something called pie annotate. And what pie annotate does is it looks at so you write your code, you write it, your normal way to develop and you run through the test kit or you run through production data and it looks to see what's going through and it says, ah, this is dynamic but man, all we get in production is a bunch of flows. This is probably a flow type. And here's what's key. It goes ahead and it updates the code. There's actually optional typing information in Python now. That's one of their pet extensions. This is the thing that you don't do. You never have run time, ultra compile time. Only the developer is allowed to actually touch the code. That is a unnecessary constraint. Now, this work has only been come up for correctness. It's not come up as a mean to accelerating number. I'm just getting in front of you guys here saying, oh man, we can run our code real fast by using pie annotate for performance instead of just security. Even the authors of this code consider it appropriate only for legacy. They're like, oh yeah, but the developer should do the right thing. Okay, yeah, it's not vitamins. Is that not eating your weedies or eating your cereal? Does it job to do, which is to secure like millions of lines of code? Let's figure out what we've got to do and if we can discover stuff at run time and what needs to be done to keep things secure, yes, do that. Why shouldn't run time influence code? The approach does seem weird by standard models. It's a little similar to what's called profile guided optimization. It's where you run a bunch of test code, you spy on what happens, but a lot of profile guided optimization only lives in the compiler. It doesn't actually touch the source code. What I'm telling you is no, we should actually update the source, partly because the developer sometimes they say, no, it's just all floats today, sometimes they're big floats, sometimes they're it injures, sometimes they're this, sometimes they're that. Developers don't know nothing. It is a little bit of pair programming with the machine, but if anyone has ever coded for Android and you add that tab key a lot it's not like that's that new. It's important to realize that we are loosening the assumption that the developer knows what the system is supposed to do. See, we think we're the only ones as hackers that are ignorant, but the truth is everyone who touches the computer is a little bit ignorant. Developer tools are weird though because they are the things that assume the developer is right because that's what you do with optimization. Optimization, you assume okay today I've told you exactly what I need, I can throw out every handler, I can throw out every other possible path, let me just do the thing that my ground source correctness and truth and loveliness the developer has said. And any day a hacker tools don't do that, a hacker tools are all like let's figure out how the hacker scooped things up, the developer scooped things up. It's how times it works. All of our tools are incompetent. All of our tools are blind. The one concrete difference between reverse engineering and normal engineering is do you have the source code? I know this one guy, he's gonna laugh when he finally sees these slides he doesn't even bother with source if it's a C++ code on it. It's like man, C++ lives your face forget it. What is the computer executing? Binary. What am I gonna read? Binary. That way whatever I happen to get out of it, at least me and the CPU are on the same page. Forget what the developer thought, he doesn't know what's happening anyway. You know what, that guy rexed code, he really quite does. So, what are the directions that I'm moving in in building tooling that really unifies the building and breaking mindset? I'm a little tired if I'm using an open source platform. Why can't I see the source every time something crashes? I mean yes, I can compile my particular app, but you know it's got a library and it hoops it didn't have the source. Or it's calling into glimpy and it hoops it didn't have the source. Or it's calling into the kernel and it hoops that didn't have the source. This is ridiculous. Like yes, I understand the tools did not want to take a hard dependency on source being available. I once went into a SSD manufacturing plant and I'm like I want to audit your SSD from top to bottom and they just looked at me with these sad eyes. And I think they basically finally got out. There probably is not a company in the world that has all the source code required to make a solid state disk. Each of them passes a binary blob on to the next so that they won't be like factored out of the manufacturing equation. We have a little bit of that going on in software, but it's unnecessary because in software we have environments like Gen2 where you can say I need you to compile everything and I need you to put all the source code right there. So I've been playing with what's it like having source code access to absolutely everything in a system. Oh my god it's awesome. Like error messages are terrible. Like developers pretend that they know what they're telling me oh it's because you forgot to like set this flag on this thing and it's like they're making random noises. Show me the source code that's crashing. And so this is you know the middle of SSH and this is the middle of glipc. And I just have a nice consistent if it blows up I get to see why. It's lovely. It's lovely for hacking. It's lovely for development to be honest. And I mean in this scale whatever is on that machine. So like it's running on SSH a little tiny thing and it's running on Chromium frankly an operating system larger than Gen2 in the first place. So you know I like the joke APD is all the busted. APD is the new hotness. Always the debugging. It's a legitimate question. If you're always running a debugger sometimes that code is running in the kernel. Sometimes that code requires permission to look at. You know you can be typing to go all the time Are we getting root? Okay. You never get this feeling it's easier to be root on someone else's machine because like you have root for like a year somewhere else but you are typing to pseudo you're getting root like one line at a time. Like I need to do this thing as an administrator. I need to do this thing as an administrator again. I'm still doing something as an administrator. It's an awful user interface when your work as a hacker involves constantly having to do things that alter the machine state and therefore constantly requiring permissions. And by the way you get such a wide variety of broken software behaviors. So like here's Wireshark if you run it as a user you know it doesn't give you any prompt it just runs. You can't even like listen remotely to like a Cisco remote capture. It's Wireshark. I want to listen to packets on interfaces on my machine. That's where I'm running the network snapper. Where's my local interfaces? Oh well then you need to run this root. But if you do that you get a big scary message. It says I'm not going to let you run code in Lua. It's okay I hate Lua. But like this is not an industry but I actually think Lua is okay. You can embed it therefore it's interesting. But like this is this really common thing where we have a security model that assumes that I've got like 20 people on my machine and I'm the only one who should be able to be root to controlling them. Let me tell you I've got 20 people on my machine. I've got 20 people with roots on my machine. I'm in trouble. So let it snow. This is a problem. They kind of hide and fix. You have to like declare that it can get in the machine because they don't want you to know this there and then you go into this like advanced mode and up up down down the right left right B and B it's like start. And like you end up with this checkbox with multiple user groups and capability support. You know what that means? You uncheck that and all like the user root stuff goes away. You would think it's lovely but man I'll just code complaints. So this is my solution. This is going to be 0.2. I'm going to have like a key, I'm going to have a button, I'm going to have like three rocker switches and then you know what I'm going to have? I'm going to have root now and then like in my next prompt and in my next terminal window until I happen to like shut it down and don't say it's less secure. This is way more work than typing sudo. Now yes I'm being silly but so is the security model and so is all this stuff we're doing with sudo. This is not reflecting reality. There is an implementation for work. You can't just make all actions happen as root because software will complain like why are you shark complaint. And to do something weird and then bring it up without the code done because basically I'm recruiting. It's something I'm going to work on in the blue team village after this. Come by and hang out. Effectively what you need to do is allow action to happen as a non-root user but mysteriously all the permissions checks go through. It's okay it's secure. Those rocker switches with the flips is like an LED at the end. The LED makes it secure. Having kind of a baking room environment seems weird but guess what? It's what we're doing with virtual machines. It's what we're doing with containers. Do you know why doctors are so popular? Because app gets installed. Freaking works again. The first time you get an operating system that's totally designed by hackers. This is what's going on. Jupyter is probably, I don't know if you guys have ever used Jupyter. It is now how I tell people to start learning how to use code. It is a web-based interface for Python and data science. It is lovely. It takes all the gunk of go ahead and start playing with code into logo from back in the day. It's great. But there's no way inside of Jupyter to add a package because that would require root. So you've got the most usable environment is missing actually some key functionality. And because of us it's our fault we need to stop breaking the rest of computing. Now there is a reason why things are this way. You know, you can't just say why are people so stupid? Usually there's a reason. Usually when you give, people are just learning computing root access, the first thing they do is totally destroy their computer. This is true. And so when you get involved, this model will give you just a little bit of functionality and will kind of have like a tablet bar. You have to get this good and you can get like real access to the system. It's not a security model as much as it is. We don't want to get called in the middle of the night because the users got in and broke something again. Yes, you can do that and have a tablet bar for users or you can make it just really easy to fix stuff. Check out this little stunt. So I've been working on something called Inception, forging the universe. I wanted to see if it's possible to take a fully configured, fully running machine that maybe a user doesn't want to break but does want to try something on and forth a copy with the existing configuration with everything in place. Now I would love to do it with the present, you know, the exact moment of the flip. But here's how I got it working just booting into the system that has already been configured. So here we are, we got a stock gbox. Couldn't really have live demos here. It's okay, there's no time for them anyway. I promised that I was going to show some code. Here's why you don't show code in the middle of a talk. But let me tell you what's happening here. There's a super obscure out-of-tree kernel module called DATO BD It's the DATO block device. There are other ways to do this using the DMMAP or functionality in Linux but I'm using DATO because it's cool. DATO lets you basically say I need to make a backup. And you know when you back up a hard drive it needs to be like a point in time. You can't have like this file was what it was at the beginning of the backup and this file was what it was at the end of the backup Your system will not boot. You need to be able to take a snapshot DATO is interesting because it allows you to snapshot a disk and not have some other disk required to store content as well. To store the changes It's just a more stable variant. What we do in here is we take a snapshot of the running disk and it's a read only snapshot you're not able to write to it. But virtual machines don't care. They can totally be like read only that's fine. I'll put my reads from over here I'll put any writes over there. It's called a copy on write approach So I get myself a copy on read only copy of the disk. I make snapshots of my EFI partition in this new thing, UEFI. It's a disaster but terrible. We make a copy on write copy of the DATO environment. And then we boot a VM. And you know what happens you just get that system that exact system. And you're even it. You get to test whatever you want. You get to explore whatever you want. You can't break anything. It's lovely. Rude. Nothing bad can happen. When we're earlier I told you like you can context switch or you can just have other computers. All this is about persistence. All this is about managing how much damage can happen and where. I want to leave you guys on kind of a down note. I have a hard question. Why are we vulnerable to ransomware? Why is it a problem? Like why is it possible for all of our data to be deleted all at once? Why is it possible for anyone even maliciously to wreck everything? Like who is this a feature for? You might say oh you're going to run out of storage. Really? Like an 8TB hard drive 50 US dollars. Like your entire companies are not generating data passive. They could log every single bit 100 times over without exaggeration and not spend nearly as much money as a single ransomware cleanup. What's going on is we have all of this leakage of capability to lose data. We have too many people able to break stuff and as we look at the underlying capabilities of computers realize we're not just as hackers able to break things. We're able to redefine them so they can't be broken in the first place. So let me leave you with some closing thoughts. We should not be separating development and testing. Guys at the end of the day computers are capable of magic and we're just all trying to make the magic our magic and not some bad guys. Our hardest problems in security require a labyrinth between how we build systems and how we verify them. And our best solutions in technology are going to require understanding the past. How we got here, how we really got here even if it's from the 1890s. We're going to have to understand what is an unnecessary presumption. And what oh no we really shouldn't touch because we can scream this up. All that matters is how we protect users. And I look forward to seeing how everyone here makes the world better. Thank you.