 Good afternoon. My name's Matthew Garrett. I'm going to be talking to you today about building a secure bootchain from the point where firmware hands-over-control to the operating system components all the way up to userland, and being able to verify at each point that nothing significant has been tampered with in the process. So some background here. I'm currently employed by Google. Just to clarify, much of what I'm talking about is stuff that I'm involved in working on. But none of this is a product. None of this is an official Google position in any way, while a normal source of disclaimers. Secure boot is important to overall system security in fairly fundamental ways. The biggest reason for this is that every layer of your security depends on its dependencies also being secure. If your kernel has no security issues whatsoever, which is a wonderful utopia that we can consider in an abstract way, even then that kernel is not going to be secure if the bootloader that loaded it was not secure. Because that bootloader could have been tampered with in a way that would then allow the kernel to be tampered with. If a previous layer is able to interfere with the operation or integrity of a later component, then any security guarantees you thought you were getting from that component are now gone. And obviously, every part of your operating system comes after the system is booted. The integrity of the boot process itself is the single most important part of overall system integrity. If the boot process is compromised, you have no security at any later point in the system. This has been demonstrated multiple times. There is malware in existence. Well, if we go back far enough, the original malware that existed as viral infectious software tended to be in boot sectors of floppy disks. You would put the disk in your system, you would boot the system from that disk, and it would copy some code into RAM, it would hook into other operating system components, and then if you inserted another disk at some later point, it would write itself to the boot sector of that disk as well. So this is not a new thing, but people seem to stop worrying about this sometime towards the end of the 20th century because this sort of malware was mostly replaced by stuff that acted at the operating system level. But that software was easily noticed and fixed up by antivirus software. So more recently, we've seen people start moving back to malware that does fit into the boot sector, the malware that is intended to attack this part of the boot process. We've even seen cases where this malware is being written by firms for sale to governments as persistently infectious materials, stuff that will continue circumventing any source of antivirus updates because it will hook the kernel, modify it, and stub out any attempts to read the boot sector logistically and instead return something that looks legitimate rather than something that contains the malware. And if your boot process isn't secure, you also lose out on many of the benefits of disk encryption. The boot process has to at some point be able to obtain an encryption key and something that may be protected by a passphrase. If the boot process has been tampered with, that key can be obtained by the modified boot process. That passphrase can be obtained by the modified boot process and then stashed somewhere and made available to attack us later. So the state of the art on consumer systems is pretty much UEFI secure boot. And this was something which we were somewhat legitimately concerned about in the free software world to begin with because it was unclear that we were going to be able to participate in this process and that systems were going to ship in a way where we had to give users the choice between either having the security or being able to run Linux and not being able to do both. Thankfully, that isn't where we entered up, but instead we now have a scenario where the firmware verifies the bootloader using a configurable set of keys. So admins or machine owners are able to install new keys to verify software. The firmware uses these keys to verify the bootloader and the bootloader having been verified then uses either the firmware's keys or its own set of keys to verify the kernel and then execute the kernel. So this sounds great because we now have a process where as long as your firmware is trustworthy, that's not necessarily true, but there's also someone at the scope of this talk, if the firmware is trustworthy, you know that your bootloader has not been tampered with and you know that your kernel has not been tampered with. So you've now got a mechanism by which you can get a known good kernel, which means all security that depends on the kernel being correct should be acceptable. That's something that is a meaningful base of trust. Unfortunately, this isn't really good enough because the early boot process is more than just the bootloader and the kernel. We have the Initram FS, which is responsible for the early stages of boot configuration. And one of the key things that the Initram FS does is configure and handle the any encrypted disk setup. So the Initram FS is what's responsible for obtaining the disk decryption key, what's responsible for obtaining any user passphrase for that decryption key. And right now, even in a secure boot setup, an attacker can modify the Initram FS and can drop in a module that will, rather than just decrypting the disk, will also copy that key somewhere else, will copy that passphrase somewhere else. So we're not to the point that we need to be in order to provide the security guarantees that we want to be providing. We have secured part of the boot process, but fundamental components are still not intrinsically trustworthy. And we actually have some similar problems with the kernel command line. So, for instance, if your system has an exposed DMA capable bus, for instance, if it has ExpressCard or if it has Thunderbolt, it's possible to attach a PCI device and then just DMA out of arbitrary parts of memory. And this is avoided by having an IOMMU that will be configured and restrict access to memory that isn't directly associated with that device. Which, again, sounds great, except that there's a kernel command line option that just disables the IOMMU. And so if it's possible for an attacker to modify the command line arguments that are provided to the kernel, again, a lot of fundamental security functionality can be disabled and they're in a position to just scrape secrets out of RAM. So, yeah, this is still not anywhere near good enough. Secure boot, as it currently stands, is necessary for us to provide security, but our implementation is not as yet sufficient. Which brings us to the question of why isn't the Inetrem FS side? And the answer is that it's dynamically generated and it's system specific. When you install certain components, that results in the Inetrem FS being regenerated and the components that are copies into the Inetrem FS differ between different systems. Part of that is that some of the information that ends up in the Inetrem FS is local configuration, like potentially information about the root file system and the arguments that it should be mounted with. But other parts are just things like, well, which boot splash should be used? Which keyboard layout should be used? Which language should prompts be presented in by default? And another part of this is that we, as Linux distribution people, tends to think of the distribution as a large number of integrated but decoupled components. So the Inetrem FS is sort of tightly associated with the kernel, but if any system components that is installed into the Inetrem FS is updated locally, that will often result in the Inetrem FS being rebuilt. So because of the way we've engineered these systems, because we sort of assume that any components upgraded should then have that updated component upgraded everywhere on the system, we are building things locally, even if that's not necessarily a hard technical requirement. We do it that way because that's the way we do things. So the easiest way around this would be for the user to sign it, that's a ridiculous idea. We're not going to do that. It's implausible to expect the majority of users to handle local signing. That's just not a realistic proposition. You're basically asking people to generate and take care of vital encryption keys. And I don't trust myself to do that. I don't see why I should trust anybody else to do that either. So what can we do instead of signing the Inetrem FS? And one approach is to use TPMs or technical protection modules. TPMs are small cryptographic chips that exist on the vast majority of consumer hardware these days. Now, when I say they're small chips, that's not actually necessarily true. It just means that certain functionality is available and until recently the common way for this to be handled was as a discrete chip. These days on Intel systems, the TPM is often in fact a software stack running on the management engine, which is an integrated part of the system chip set. And so it's not necessarily a discrete chip. It could be a firmware based thing. TPMs are often thought of as crypto co-processors that results in a sort of misleading impression that they're fast or you can offload tasks onto them. In reality, they're incredibly slow. If you can do something on the CPU, you should almost always do this on the CPU instead of on the TPM. The TPM is not there because it's fast or because it's good at cryptography. It's there because it is independent of the software that's running as part of the operating system. It will make its own policy determinations as to whether to release a secret. It will keep its own track of certain bits of system information. And a fundamental part of this is that it will record system state. Now, to be clear here, the TPM has no visibility into the state of the running system. Everything the TPM knows, the TPM was told by the operating system. The TPM is not a spy chip. It cannot reach into the running system and look at what you're running. It can only say, okay, I have a record of this information that the operating system voluntarily gave me. But based on the information it was given, it can then choose to either release or refuse to release secrets. It can choose to either decrypt or refuse to decrypt information. Now, how does it do that? How do you avoid this state just being corrupted? How can a malicious bootloader not just lie to the TPM about what it's doing? And the answer is that every single component of the boot process measures the next component and passes that state onto the TPM. And in this case, a measurement is just a cryptographic hash for TPM one devices, it's just a HR one. For TPM two devices, it can be a pretty much arbitrary cryptographic hash. And the TPM, when given that, does not just replace the existing state with this new hash. Instead, it takes the existing value, concatenates this new hash to it and then stores the hash of that. Which means that the value that the TPM has depends on every single measurement that's been passed to it up until that point. So the only way, unless you're able to just pretty much entirely break Shah one, to get to a specific value in the TPM is to provide it exactly the same sequence of measurements. Which, assuming again that there is a trustworthy route of trust, is great. If someone replaces your bootloader with an untrustworthy bootloader, then when the firmware measures your bootloader, it will pass a different measurement to the TPM. Which means that there's no way for that malicious bootloader to get the TPM back to the desired known good state. And this way, we can measure the Initram FS. So we don't need to sign the Initram FS, we can measure it and then put that measurement of the Initram FS into one of the TPM's registers. So why is this better than signing? The first answer is the list of valid states can be arbitrarily updated. There's no need to have any sort of cryptographic validation of the Initram FS. Beforehand, you can boot any Initram FS you want to, but if that Initram FS has been tampered with, it will end up with a different measurement and then you can have the TPM, for instance, have the disk encryption key sealed to the TPM. And the TPM will then refuse to hand that to the operating system unless the Initram FS was correct. Whenever you regenerate the Initram FS, you just give the TPM the new expected value and then when you boot that, the TPM will voluntarily hand this over. So that sounds pretty much ideal, but it's not really for a couple of reasons. And the first is that you can't actually prevent any code from running. You can only prevent secrets being handed over. So again, going back to the idea of at some point, a user is going to type in a passphrase to verify, provides additional information to allow the disk encryption key to be obtained. You can replace the Initram FS with one that still brings up the same prompt that fools the user into typing in the correct passphrase and which then immediately reverts the Initram FS back to the good one, stashes the passphrase somewhere and reboots the system. And if you want to get really fancy, you just splash up a fake oops first. At which point, well, okay, software's bad, computers are confusing, the system rebooted and now it works again, except someone has successfully stolen your passphrase. So that's not really sufficient. The other part is that it's kind of fragile. If you update the Initram FS, then you need to put the new hash into the TPM. If anything goes wrong during that process, then you end up in a state where the TPM refuses to hand over your disk decryption key. And then your system doesn't boot again ever. So yeah, that's kind of suboptimal. So what can we potentially do instead? And one approach is that, well, the kernel supports having multiple Initram FS images and each of them gets unpacked in order. We are to a large extent building the Initram FS locally because we are mixing local configuration and system binaries into the same thing. As they get unpacked in order, you can pass multiple images to the kernel. When a later one gets unpacked, it will overwrite the previous contents. So if we put all the code into one Initram FS and then loaded that last, that would overwrite any malicious codes that's been put into previous Initram FS images. So if we separated out configuration from code and put the configuration in one image and the codes in another image, then the dynamically generated material could go in one image and the statically presented material could go in another image and the second one could be built into the same image as the kernel and that could be signed. And then we just have some glue codes there that reads in any user-provided Initram FS images, unpacks them and then unpacks the trustworthy signed Initram FS on top of that. The distro would then be responsible for generating this image, goes through the normal signing process, and without having any meaningful kernel modifications, we end up with a way to provide a trustworthy mechanism for guessing early boot data with signatures. But how do we then verify the rest of the chain? Now, the fact that our file system is encrypted is a good way to make offline tampering more difficult, but it's still not impossible for people to get additional binaries onto your system. People talk about, well, okay, we should extend ELF so that we just have signatures inserted into every binary. That isn't sufficient in any way, shape or form, unfortunately, because a lot of the stuff that we run, a lot of the stuff that has meaningful security impact, is in interpreted languages. And you can't put an ELF signature on a Python script. You can verify the interpreter, but you can't even necessarily verify all the modules that the interpreter will then load, less loading the script it will execute. But the kernel does have support for signing entirely arbitrary files using something called the integrity measurement architecture, or IMA, or IMA. IMA is a combination of two things. The first is an extended attribute on each file that contains a signature in this configuration. And the other is a policy that tells the kernel when it should verify the signatures of files, under which circumstances. So you can specify there's a subset of files being measured, and you can also specify under which circumstances they should be measured. Now one of the ways you can say, okay, this should have its signature verified, is to tie it to a specific Linux security module policy. So you can configure IMA such that only things that run in a specific SE Linux context or only things that run in a specific app armor context should be verified. And then you can define policies that restrict access to various critical system components. And you can say that only stuff running in these security policies should be able to touch these components, and anything running with those policies should have its signature verified. Anything else that's running in a security context that doesn't need access to those components doesn't have its signature verified. And so you can continue downloading random stuff, building it in your home directory, and running it. And as long as it's not something that attempts to touch critical system components, it will continue working just fine. Otherwise it will be blocked by the Linux security module layer. Of course that means you need a mechanism to protect the security labels on these components which gives us the next level which is the extended verification module or EVM. And EVM provides an additional signature that is used to verify various bits of file metadata such as the file ownership, the security label, and any file level capabilities it has. So for instance, if something has cap net raw, then EVM would allow you to protect that such as if someone changes its to have different capabilities, the EVM validation would fail. There's a couple of problems with shipping EVM at the moment and the biggest one is that it's really designed to do local integrity validation and generation. It assumes that your LSM policy will prevent anything bad from happening which is not necessarily true. So the way this is supposed to happen is you have a symmetric key in the kernel, one that's potentially protected by the TPM and then whenever a file is modified or updated, a new hash is generated and a new signature written out locally by the kernel. Downside of that is that if your kernel is compromised even once, an attacker can then obtain that key and can then create arbitrary files that look entirely legitimate. The other is that signatures includes the INOS number. You don't know what the INOS number is in advance which means that you can't currently ship EVM signatures in packages. There's ongoing work upstream in an attempts to fix this. But the combination of EVM and IMA means that you can write a policy that says files that have access to critical security infrastructure must be signed by a trusty key. And for clarity, those keys are loaded from the IntramFS or optionally can be built into the kernel. So it is entirely possible to ship this in a way that still leaves it up to user control. Users can decide which keys they want their system to trust. And what's the future? Directly integrating IMA with Flatpak is unfortunately kind of difficult because you have, when you are getting Flatpak images, then if they're sharing run times, then well, what's the signature on something? If you have multiple Flatpak images that share files, the idea is that the deduplication avoids you having multiple copies of that file but in that case, whose signature is applied to it. There's some complexities around there. And the answer there may be, well, we just don't do this directly with Flatpak. Instead, the Flatpak agent, the runtime, is expected to be trustworthy and then it can do some sort of reasonable signature validation of the Flatpak images before it does anything else. We don't need the kernel to do this for us. We can just verify that some intermediate agent is trustworthy and then assume that it will do the right thing. And obviously, as with anything that is intended to restrict the running of malware, there's the potential for it to be used to restrict the running of the user's own free choice of software. And when deploying any of this, we have to be very careful to design systems in such a way that fundamentally, the owner of the system is still the one who chooses which keys their system trusts. And we have a mechanism in Shim that allows users to enroll new keys by proving that they are physically present at the system. Extending this such that additional keys can be loaded through that interface and then the Interim Affairs can pull those out of UEFI and load them into the kernel key ring may well be sufficient here. But that was allowed the user to fundamentally be in control of which keys they trust. Also, this is not necessarily something that we are, even once a lot of the infrastructure around this exists, this is not necessarily something that is going to be enabled by default. This is something that may well be relevant for users who have particular concerns about being targets, users who really want to have as much faith in the integrity of their system as possible. So we have three minutes left. So anyone have any quite quick questions? Yeah. Does... Does IMA and EVM protect directories, the contents of a directory? No. So you could add new files to the contents of directories, but they would not... Those files would not themselves have valid signatures. So if it's a thing where you're making security decisions based on the presence or absence of a file in a directory, yeah, that is something where you may want to think about whether that's actually the correct design. But so there are definitely cases where right now you would be able to violate security assumptions because, right, we do make that sort of decision and unless we have a rethink of that, there are going to be surprises. Anyone else? Regarding the inner-term affairs, why not just let distributions provide a generic one? I mean, there are generic inner-term affairs and we could just, for instance, in Fedora create one that boots on desktops. Right, so the... I think the biggest reason there is that sometimes there are still bits of user configuration that you want to embed in the inner-term affairs. If it's already possible to ship systems with a generic one that doesn't require any user configuration, then yeah, we should be doing that. And I know that a lot of the information that previously might have been embedded in the inner-term affairs has instead ends up on the kernel command line. So we're still left with we still need to protect the kernel command line. There's actually, in that case, we can take a similar approach of a bunch of security relevant arguments can just be appended to whatever the user provides is, which will then overwrite any insecure ones that were provided. And in that case, sure, we can probably get away with just providing a generic one without having to re-architect anything and building that into the kernel image. Okay, thank you very much.