 Hello, my name is Case Cook. This is the update on the Kernel Sol Protection Project. There are a lot of links and other things, some real eye charts and places. So go ahead and grab the PDF. It's a lot easier to follow along there if you want to take a closer look at some notes. Anyway, let's jump in. So for this talk, security is a big word that covers a lot of things. So from my perspective, the Kernel Sol Protection Project is not just access control. It's not just a tax service reduction or bug fixing or protecting user space or memory integrity. These are all parts of security, but what I really want to talk about is Kernel Sol Protection and the mindset that we have for doing this work. So I think the first step is sort of seeing what needs protecting. We're really looking at the downstream from Linux Kernel, the servers and laptops and phones and TVs and everything else. The latest numbers out of Android are that there are over three billion Android devices active in 2021. The majority of those are running 4.14 from 2017 with 4.19 slowly catching up. And the problem with this from a protection perspective is that the flaws that get used have a very long lifetime and they're even longer in downstreams. So fundamentally, to get the smallest possible lifetime for some of these bugs, you really have to be running the latest kernel and testing as close to Linux Next as possible. And this advice is really for the vendors as they generally are the ones who can do anything about the kernels that are running on actual products. Now it's a reasonable position for an upstream Linux kernel developer to take to say that downstream bug fixing is not their problem. Even the bug lifetimes in the upstream kernel are quite long. John Corbett did research into this in 2010 and found that the average time between the introduction of a flaw and it getting fixed was about five years. I started in 2015 trying to track this a little bit more closely and see what I could see. And looking at critical and high vulnerabilities, they started to creep up towards six years for a while. And I was worried that this number was just going to keep getting longer and longer because the start of get history for the kernel, those were a lot of these flaws were coming from long ago. So I had this concern that we had this huge long tail and it would pull the average down. But more recently, it started to fall again and we've stabilized it about five and a half years. It's been a while since the last critical bug, so I'm waiting for the other shoe to drop on that one since I'm sure there's something in the kernel waiting to be discovered. And since 2020, we added 20 more sort of high priority flaws, which is a recent high, I'm going to thank the pandemic for that for everybody staying on and doing tons of security research. It was falling for quite some time. You can see our high, the highest we've had was in 2017 was our highest growth. And it slowly started to fall. And then I don't look too closely at the medium and lows just because they those flaws don't tend to get sufficiently accurate like originating commit analysis where where did the flow actually come from. So if we take a look at the history of these critical and high CDs, we can see the kernel version is along the left side. The most recent kernel release is at the top and the bottom is the beginning to get history. And then just sort of looking at the CDs in order of when they were fixed basically so you can see that and then when they were when the flaw was introduced. So there's these long tails that for a while were reaching all the way back into the get history. And I was we sort of saw this this hint that we were beginning to lift up out of get history into more recent stuff and that we might start seeing a shorter cycle time on how long it took to fix all these things. And actually it turns out that that that appears to have happened pretty much for during 2020 is we had no critical and high CDs that had their point of introduction be before get history. So that's that's nice that we've actually started to lift that out. Unfortunately, we still have a couple of these that have really, really long lifetimes that are dragging the average down. We can see that we have fewer flaws going in, at least the critical and highs, and their lifetime is sort of getting smaller compared to earlier in the graph. But we still have some, you know, really ugly ones that took a long time to get to get noticed and fixed. So question I get sometimes is, you know, well, who cares about the lifetime of these flaws? You know, it's just about how long it takes to get them fixed once they're once they're discovered. And the trouble is that the attackers are quite motivated to find the flaws. And as a result, they're finding them way before defenders do. So and we know this is true, they are watching, they occasionally will boast about having found, you know, the days when they got committed to the kernel. So it's not theoretical. The actual flaws and the actual lifetimes based on when they were introduced has meaningful value to analyzing this. Bug fighting continues, of course. And we're finding all kinds of tools, you know, that the compilers themselves spit out warnings that get fixed before things go in. There's a lot of static checkers that we're using a lot of dynamic checkers. I'll call out CIScaller in particular. There's a huge dashboard where it's collecting almost a thousand security relevant flaws that it's that it's found that are rather I should say reports. It is up to folks to triage and and diagnose the reports to figure out the true cause of the problems. And these bugs are getting fixed. We on the CIScaller side, while it's growing by a couple hundred a year, we're also fixing an order of magnitude more than that every year as well. And you can you can see these going into the stable trees. You could ask Greg if you want to email him, but I looked it up for this presentation. So as of the 510-66 stable release, which is the most recent long term support kernel, there are 8463 fixes committed, which is an average of about 128 fixes per stable release version. So a lot of bug fixes are going in. As far as bugs go, we're just going to keep introducing them. The kernels written in C, it's we're prone to get stuff wrong because the compilers can only help us so much. And the thing to drive home from the prior graph is that the bugs in the kernel exist whether we whether we know about them or not. They are there, they're waiting to be discovered, and they might be getting used already by by attackers. Whack them all on bugs is not is not a strategy. I always come back to the analogy that Constantine gave in the 2015 analysis to between Linux development, like the kernel and the 1960s US car industry, where things were designed to run well, but they weren't exactly designed to fail. So in this image here is the is a 1959 Bel Air on the left and a 2009 Chevy Malibu. And I can't even see the crash test dummy on the collapsed driver's side compartment in the 1959 car. Like it was not designed to fail. It drove well and it was fine, but in a crash, the whole front end would be destroyed. Whereas the 2009 fair is much better because it was designed with failure in mind. So a similar thing needs to be applied to the kernel as well. It needs to handle failures, which in this case means attacks intentionally malicious manipulations needs to handle that safely. On the whole user space is becoming more difficult to attack. And containers paint a big target on the kernel since all the containers share the same host kernel. And fundamentally, lives depend on Linux. We saw a list of devices that are they're running the kernel now. So telling bugs is nice. There is some truth to the observation that security bugs are just normal bugs. One person security bug may not be a security bug to someone else. It really depends on a number of factors, you know, what was built in, what's being used, reachability to an API, et cetera. And there isn't really a common theme to the bugs that attackers are going to be using. Generally speaking, it's whatever is useful to them. Additionally, bugs may be an out of tree code. So this is an upstream vendor driver or something. And it's not an excuse for upstream devs to say, Oh, that's not our problem. Well, if there's an API or a language feature that allows for a bug to be introduced, that's not good. And we should do things to discourage that an out of tree code will have that bug. So we want to kill entire classes of bugs. This is more than just about the security piece of it, because generally speaking, this improves robustness. We can kill an entire class of bugs. The cases where that bug is vulnerable to some kind of security-relevant issue can be a subset of all the bugs that a class may introduce. But those bugs exist for other things. Just behavioral anomalies. So the general robustness of the kernel is improved when we're actually killing bug classes. And if we kill the class, it can't be reintroduced, which is really the only way to get ahead of the whack-a-mole situation. So the only problem, of course, is that we'll never kill all the bug classes. So we also need to deal with how they're used to exploit the kernel. We have to stop their exploitation and stop what they block their targets, reduce information exposures, basically remove anything that assists attackers. And I think the position morally is that we need to do that even if it makes development more difficult. This can be very tricky, because an attacker and someone debugging the kernel to try and fix something have a very similar set of tools as far as what they need from the kernel. So it can be a fine line to walk. But generally speaking, the car designer has to figure out what to do about the seat belt path and where the reinforcement in the doors is going to go, things like that. It makes their lives more difficult, but the safety matters. So another piece of this, of course, is that a big problem are the language features or perhaps lack of language features with C. The kernel benefits from being written in C. It's a portable assembler. We need the level of control. It's that way for a reason. Replacing C has been a difficult proposition for a long time. We are, however, to the point where Rust is sufficiently low level to do some of the things and provide the kind of speed that's needed without getting in the way. And we've got now actual working versions of this. There's the proposal to get Rust building in the kernel and we can start removing drivers and replacing them with Rust stuff so that they're in better shape. This is going to take some time. So I like to think of this as sort of meeting in the middle. We're going to continue to remove dangerous things from C itself. The kernel already forked C a long time ago. So my view is as we remove dangerous feature use from the kernel on the C side, we can start adding more and more Rust on the other side and eventually we can meet in a good place where a lot of the things that are more error prone are written in Rust and they're just as fast, if not faster, but they don't have the concurrency problems, the use after free problems, there's a whole set of these problems that is just gone for the Rust side of things. And so that's sort of the rationale of thinking behind the kernel self-protection project. And like NetDev is upstream and concerns itself with networking, KSBP is upstream and concerns itself with security, trying to make sure that we're getting rid of these bug classes, that we're getting rid of exploitation techniques and trying to clear the path for getting Rust in, whatever we can all do to help. It's sort of an umbrella project. I used to say our attitude was slow and steady and we'll get there. And I much prefer Alexander Popov's suggested motto, which is flexible and persistent and that we can adapt to the changing requirements in the development ecosystem and but more persistent. We really, really need to continue to remove the dangerous pieces of the kernel and the language that we're using. And we'll get there. We've got an issue tracker and everything's in there. You can see us slowly making progress through it. Please come join us. We've got a lot of little things that need to help too. So here is two years worth of kernel releases. I didn't do a status update in 2020. So September 2019, we saw the release of 5.3. This was our initial end of the implicit fall through work that was going on to remove that whole class of vulnerability where switch statements in C don't, there's no semantic marking for whether or not you intended to fall through to a switch statement. And we had roughly a one in 10 cases of these were a legitimate bug. So it was worth it to go through and just fix all of these because we could actually find them all and then turn on the warning so we never see that flaw again. And so 5.3, we saw the addition of that so we wouldn't gain any more. We were doing ref count conversions so we wouldn't have ref counters wrap around to zero. There's more PIDFD work so we don't have process descriptors as opposed to using an ID that might recycle. On XA6, we gained pinning of the CR4 and CR0 values which control a bunch of the CPU features like S&EP, S&AP, write protect. And so we did our best to make sure that that wouldn't get flipped when we weren't expecting it as that's been a common exploit technique. We also gained a heap variable auto initialization so if you boot with a knit on alloc equal one, all heap allocations will be zeroed automatically and that gets rid of a whole class of heap, uninitialized heap all abilities. More sanity checkup for the allocator. We got KSLR enabled by default on arm64 and we gained some documentation externally for how to deal with hardware security flaws like meltdown. And 5.4, we continued with the developments of PIDFD in the kernel. The kernel gained the lockdown LSM for trying to have a bright line between UID0 on a system and actual kernel memory in either not allowing root user to change memory contents as sort of the integrity barrier. And then if you want to go a step further, you could set that to the confidentiality barrier which means root user isn't supposed to be able to even see the contents of kernel memory. There's now a the tag memory relax. Just call ABI basically allowing tagged addresses through the ABI for the architectures that support hardware memory tagging and sort of clearing the path for that. Made some improvements to boot entropy, blocking rights to swap files. That was funny. You could actually write to a swap file even though it was being used for swap. That's not great. It can corrupt all sorts of things. One of the many sources of overflows has been string values trying to copy huge sizes. So trying to detect ridiculous overflows. If you see a size that's greater than it max, it means probably you wrapped around and somethings believes it's negative. Dropped linker support for sort of an experimental linker. Intel CSX was removed, support was removed and we continued refactoring refcounts. Perth has been sort of a persistent source of flaws. So there was a desire to be able to turn off with a much larger hammer. And so finally we gained hooks to do that from the LSM side. Also got refcount T used to have sort of custom per architecture implementations. A fast generic one in C was written that gives us greater coverage across architectures. We got some clean apps for exception tables and other linker script issues. KSLR was enabled for PowerPC32. SACCOM gained RISC-5 support and for user notifications you could actually continue a SIS call if it was deemed good enough to go through. That way SACCOM monitors didn't have to do those kinds of things on its behalf. Just say yeah, go ahead. EFI randomness protocol was implemented for X86. ARM64 had been using it for a while. Coverage for FortifySource was added for MIPS. And similar to the other size limits, the copy to and from user sizes were limited to in-max as well. KSAM grew support for dealing with VMAP memory. And in the world of VMAP stacks, that's nice. And there's a lot of VMAP stuff that's grown and used as well. So gaining KSAM used after free analysis of that is important. MIPS gained the ability to work with the GCC plugins. User fault FD started to gain some protections, but we'll come back to that in a minute. 5.6 saw the upstreaming of WireGuard. That was a long path and really required a lot of crypto changes. OpenApp2 SIS call gains, something that's long been wanted, is the ability to say, hey, look up this path, but don't go upwards. Don't be tricked by seeing dot, dot, slash, dot, dot, slash, dot, slash in your path. And certainly don't be tricked by simlinks that got created. So all those things have finally landed for OpenApp2. PidFD continued to grow. IOU Ring gained the ability to actually open files. We replaced the blocking random pool that existed forever. ARM64 gained support for an on-chip random number generator and the E0PD support, which is basically constant-time memory faults, so you don't end up with timing attacks trying to look into kernel memory sort of the way Meltdown worked. The idea being there that you could turn off KPTI, but turn on E0PD and you'd have no signal about what's happening with memory contents. PowerPC32 gained VMAX and VNAP stack support. As part of this, more and more architecture is doing the common stuff about protecting page tables. Things were rearranged so you get generic page table dumping for debugging. And then in earnest, we started replacing zero-length and single-element arrays with flexible arrays. This is a long, long-time coming ancient C code used to have a single-element array at the end of a structure and just pretend it wasn't a single element and would allocate more space than just the size of the struct and put a bunch of elements at the end, because zero-size wasn't legal. And then zero-length got added at some point and served the same purpose, but again, the compiler has no idea what's going on because it can't see or rather expects it to be actually zero-length and gets treated strangely. So flexible arrays were added to the C spec and those were empty brackets. No zero, no one, nothing like that. Going through the kernel and replacing all places where we have zero- and one-element arrays, so that we can actually start doing sensible bounds checking on these things, has been taking a long time, as you can see. We are still working on it and this got started in earnest in March of 2020. There's a lot of them because there's a lot of old code. 5.7 saw the addition of pointer authentication for N64. We gained BPF LSM, so you can, from BPF now hook any portion of the LSM hooks, which allows for some very creative things that you can monitor. There was a long history of exact deadlocking problems, like against PSTOR and signal landlars and stuff. There's a major refactoring of exact VAE. Made some improvements to the free list obfuscation with some feedback from a couple of researchers on what aspects of that were it was weak. Risk 5 gained the strict kernel memory protections, so you can't write to the kernel text anymore. If you're getting a write primitive as an attacker on risk 5, you can't just write to the code and change it. One of the sanity checkers is UBSan, the undefined behavior sanitizer, and it was sort of one big knob in the kernel, but it has a whole bunch of little pieces doing different types of checks, and the bounds sanitizer, which checks array indexes, is actually very performant and lets you do runtime array index checking. So that got split off from the main config UBSan and got turned on and led to sort of an ongoing work to find and kill off any of the problems we've had with out-of-bounds array indexes, and there's sort of a slow and steady set of reports on that every release of a couple more are getting fixed. Another problem is with how SNPrinF gets used to append, several SNPrinF calls in a row get appended, are appending to a single buffer, and you end up with some unexpected results at the end. So we've been refactoring that to use SCNPrinF, which will return zero if you've run out of rooms. You can't just keep on adding stuff. So yeah, and refactoring from prior releases has continued, the flexible arrays, the ref counts. In 5.8, ARM64 gained BTI and the shadow call stack. So BTI is where we get a sort of weak forward control flow integrity, where we can only jump into places where we actually expecting a jump target, and that can narrow the scope of execute control that an attacker might have. The shadow call stack support was another way of doing this, but in sort of in reverse, so it's for the backward edge side of this, but it's still that's a software piece. The concurrency sanitizer infrastructure got added, so we can actually start examining databases and proving those out. Gained a couple more capabilities, one for using sort of BPF things, let that off of Capsis, similarly for Perf. There's a bunch of RNG improvements, so a lot of these are driven by observations off the network. Various pieces of the network use random numbers, so trying to make improvements there has been ongoing after as well. There are a bunch of places where we were leaking potentially, leaking kernel addresses for processes that didn't have Capsis log got through that. RISC-5 as part of its kernel memory protection started reporting about any places where it encounters a writable executable kernel memory, and the exact refactoring continues. There was a refactoring of the PROC implementations so that there could be multiple instances of PROC, so you could actually remount things with different features different visibility characteristics. So setFS has been a long time problem in the kernel as far as how the kernel would choose whether or not it was attempting to write user space or kernel space. So the like the removal started in 5.8 sort of laying groundwork for what it was going to take to have explicit kernel addresses and explicit user space addresses without just looking at its value, which is what setFS would do. But read implies exact was a workaround for the transition long ago of actually gaining execute control over memory. And this kind of workaround was never needed for the architectures that natively supported execute permissions. So we sort of removed all of that because they were getting in the way. And we have the continuing refactoring. With 5.9 we gained our second gain the ability with its user notification filtering to actually inject file descriptors into the second process. The monitor could find the places where it needed to hand back a file descriptor, as opposed to just saying, no, you can't do that. You could say, here, yeah, that totally happened. And here's the file descriptor you should have gotten. And we gained three more architecture support for this account, which was I think the most we've ever had in one release. Clang, stack variable auto initialization support was added so we can zero in it stack variables. And like that entire class of initialized stack variable just goes away, which is a pretty big deal. There was a huge rewrite of the syscall entry and exit routines, which are normally very architecture specific, but they all do fundamentally the same sets of checks and things for validating state and dealing with interrupts and signals. And there's a whole bunch of really common code and fixes for various things were sort of out of sync between different architectures. So the idea was to collect all of that into one place, or it can all be done correctly. This also lets us have a single location where we can make some changes to syscall stuff in the future if we need to. There's some more slab hardening that got added. We added another capability for checkpoint restore, splitting that again off of capsis admin. DebugFS gained sort of a boot time visibility restriction. In other words, you could turn off DebugFS from the boot command line. First five gains, stack protector support. So I didn't have a canary before and now it can. Part of an API that was storing a bunch of function planners on the heap was task API. So the work began to sort of redesign that and get rid of it. XA6 grew its FSGS base implementation. That led to a lot of work around out-of-tree modules that were trying to use that, but could land a system into a very vulnerable state. So the idea was to sort of actually make that happen correctly in upstream, so we wouldn't expose. So out-of-tree modules would not end up exposing insane things to user space. Similarly, we wanted to filter MSR rights because those are incredibly privileged as well. The uninitialized variable macro was removed from the kernel. This masked warnings for a long time, but it also masked all kinds of other mistakes. So we just tore it out and actually fixed things correctly. It was a workaround we didn't need that was getting the way of other work. And there's, again, the ongoing refactoring. So 5.10 saw yet more around number generator improvements. This was for the p-random subsystem now, which one of the larger users is the network code. Save-set ID, LSM gained group ID awareness. It had been only UID aware until this point. The LSM gained more hooks for when the kernel reads various files for loading modules or firmware or other things like that. Now the LSM can more thoroughly examine what was happening there. Now that the SETFS work came along, SETFS was officially removed from x86, RISC-5, and PowerPC. So that whole class of vulnerability, that exploitation method, is gone from those architectures from this point forward. To deal with some of the refactoring around the SM print-off stuff we talked about earlier, SETFS has no view into the Cfile API. So we just added this another helper, and that would keep it from overflowing. The mount subsystem gained the no-synfollow option, which is pretty great. This is kind of like, oh, no-syn link in OpenAT 2, but this is for the entire mount. Whatever you're mounting, suddenly all the synlinks are disabled and won't work. The AMD-SEV subsystem gain register encryption support for virtualized guests. ARM64's memory tagging extension support got added, so we can actually do checking of the length of allocations because they've been tagged for a specific range. And if you try to access beyond that range, the tags stop matching and you'll get caught. An entire API for doing static calls was added, and this is to replace global function pointers. While the coverage is interesting, a lot of the global function pointers tended to already be marked read only after a knit. And for those that we couldn't do that with, the static call gives us coverage because now you actually have to go through work to do those updates because it actually does an in-place text update on a kernel. You can't just write to a function pointer, so that's nice. And here in 5.10 is when work started on dealing with implicit fall-through. Again, this time for Clang. As it turns out, Clang has a much, much more strict view of what implicit fall-through looks like. And there's now a whole bunch more of implicit fall-through work to do to get Clang coverage as well. And of course, the refactoring continues. In 5.11, we split up sort of the remainder of config UBSan because we were having quite a bit of success with bounds flaws getting fixed. So we opened up for other things like shift overflows at runtime, things like that. ARM32 gained KSAN support and also added poisoning to its single page so it couldn't be used as a target. And ARM64's ASAM gained the hardware tag support, so building on the MTE work from the prior release. And ARM64 saw a set of S removed. In the fortified string functions, before you could write past the end of a string but not past the end of the structure that surrounded the string. And for strings that didn't really make any sense, you just need to stop at the end of the string buffer. A really nice one is the unprivileged user fault FD says control. This now defaults to safe, which is to say that a user space, like a normal user space process cannot catch a fault for code that is running in kernel space. So this removes the user fault FD exploit method for gaining better time and control or slab layout control when performing attacks. We're still left with fuse as a path to that, but that one is much more easily controlled. The config page poisoning, zero, no sanity was removed since it's redundant to the init on alloc work and sort of got in the way of things. Syscall user dispatch was added. This is an interesting way to basically have better emulators. This was added for wine to deal with having a separate Syscall definition where you're not actually calling into the kernel. The actual Linux kernel, you want to do something else. So it was a way to dispatch in user space to a different Syscall handler that was defined by user space for itself. Second gain constant time bitmaps. So the idea being that if the composition of all the filters for a given Syscall always produced the same result no matter what, it's not variable in any way. There's no reason to actually have to walk all of the filters to get that answer every time you do a Syscall. You can just immediately return the result and that's sped up a lot of the more complex second filters. I would say around now is also when there was a bigger effort being made to replace string copy, string l copy and string n copy with string s copy in its family members. And then we continue to more ongoing refactoring. And 5.12, Ubisan which had grown all these things that we were actually catching real flaws from had the integer overflow checks torn out because if you use GCC8 or later it just doesn't work. Which is a problem. We're going to have to readdress that but we need help from the compiler side of the world. KFence which is sort of lightweight KSAN. We'll still catch about a bounds and use after freeze but it's not deterministic but at the same time its overhead is incredibly small so it's designed for production workloads to catch weird things. K compare which is used to compare different kernel objects became more available. There was the idea that perhaps it was a bit of an information leak but for the most part it's not and some of the graphic systems are really depending on it for being able to compare or file a script or some other things. Gained mount set adder which was for basically having user namespace aware mounts and doing user ID mapping. Per task stack canaries was enabled for risk 5. Before it was a global. ARM32 still has this as a global. There's a single global canary. So this work was sort of coordinated between the compiler and the kernel and we're starting to see more of that in some of the architectures. Another big deal was the Clang LTO support landed as a precursor to getting CFI working as a as a protection mechanism and again ongoing refactoring. In 5.13 we gained the Landlock LSM which gets us a much higher level bit of control over for example file system access that SecCount really was not designed to deal with. Clang's CFI for ARM64 landed so that finally gets into upstream what's been an android for I think the past three years as a as a protection mechanism against attackers gaining execute control. Also gained the percyscal kernel stack offset randomizations. There's a lot of attacks that sort of depended on being able to have a predictable kernel stack location. There was a lot of discussion around entropy and methods and how to do it and where to put it and all this other stuff. So there was a long path to finally getting that landed. We wanted to check rights in in PROC PID adder against the file opener not against what the privilege of the writer was as a common confused deputy issue. While most distros had already done this forever ago DevKman was officially removed in 5.13. You don't need to just see all of kernel memory just sitting in dev. SETFS was removed from MIPS, execute only memory support for ARM64 returned under EPAN which I think is an ARM v8.7 feature. Fortify source was enabled for RISC 5 and an interesting note is that x86 i32 the stack protector support was removed in the kernel unless you have GCC 8.1 or later. So if you're running i32 and you care about security which already seems strange you really really need a modern compiler to use the modern way of specifying stack canary locations which is what RISC 5 was doing a couple kernel releases earlier here and of course the ongoing refactoring implicit fall through etc etc. So this is now the most recent release 5.14. Again one of our number generator improvements. MFD secret syscall was added. This is an interesting one because it's mostly designed for dealing with accidents. An intentional attempt to try to pull memory pull the secret memory out of the process is is pretty straightforward with ptrace. But the idea is that or I should say the memory has been arranged so that the kernel can't accidentally read those things it needs to be able to needs to do it from the perspective of the process itself. And I think this leads the way to having more per process page table protections. I'm hoping we can build on that and get us close to something like xbfo in the future. RISC 5 gained vmap stack and stack comp gained the ability to add those file descriptors that I talked about earlier and also send a reply about it at the same time because that was racy and while not a security issue would confuse the set comped processes at times. Also started doing some refactoring around how memcopy operates. We talked about the inter versus intra object checking for strings for the string family functions for memcopy set functions. This is not nearly as straightforward so there was a lot of refactor needed to pave the way for being able to let that to be able to properly balance check those things. And ongoing refactoring. I expected for 5.15 we're through the merge window now so I can sort of look at what's in there already. There's another push for replacing open coded size arithmetic with struct size and that's beginning again. KV Malik was loaded in max. UV sand became available for RISC 5. SETFS was a roof farm 32 which was sort of like one of the last remaining architectures that we would see those exploits explicitly used on. L1D flushing API was added so it was sort of a large hammer for dealing with potentially unknown future side channel attacks that might be using the cache. The GC11 gained the call used register clearing and the idea here is another data lifetime reduction feature which is at the end of every function you clear all the registers that got used in that function that weren't already aren't about to get restored from the stack and that means you're going to leak a lot less from function to function. And of course we've got a bunch of ongoing refactoring just like all the others. Then plan for 5.16, stuff that's sitting in next. We've got implicit fall through enabled for clang so all the other implicit fall through work has finished we're hoping. The alloc size attributes got added to the allocators the compiler has more of a hint about what's going on that can it might be able to check things at compile time. There's a helper being added for dealing with flexible arrays in really weird situations you know alone in a structure or as part of a union some other places where normal conventions are difficult as well as a way to specify a subgroup of members within a structure and some additional memset helpers for dealing with that and then again ongoing refactoring and it looks like we've got thread info in the task for arm 32. There's a bunch of other soon and not so soon features that I'll just leave on this list that I need to read through if you want to. We've still got a lot of challenges. There's a lot of conservatism in the kernel. We don't want to change stuff but I think that gets better with having more testing in kernel CI kinds of things. We need to take responsibility for the sort of the APIs and language features that make the kernel a little unstable for when it's under attack and we got to sacrifice our time and be patient with each other. There's certainly a lot of complexity we need to deal with finding new ways to do things and finding ways to work together. So as far as resources, more developers, more reviewers, more testers, more backboarders, all this stuff is needed and that's sort of universal it's not anything specific to your security. If you have any thoughts please email me or use Twitter or I'm on that IRC channel and you can see the slides and join the project more directly. I hope you enjoyed the presentation. See you next time. Bye.