 Hello everybody, I'm Tobias Miller and I have the pleasure to be talking about our work that we have been done at the University of Hamburg, which we have titled now Block me if you can subverting Linux's IMA. So this talk, this presentation, this work is about undermining or trying to undermine the security guarantees that you may be looking for when deploying certain security technologies and in this case IMA, the Integrity Measurement Architecture. And the very, very brief gist of this work is that if you happen to have a manipulated block device then, well, it may cheat on you and you may run a foul this batch device and it may actually cause you to execute the binaries that you didn't intend to run in first place. So keep in mind, this is academic work. Well, we have money, we have worked with a manipulated block device which may or may not be common in your scenario. So the work is in the academic context which means that we find problems where other people find solutions. We try to go, well, we wanted to see how far we need to go in order to break things that would typically work, even if that means that we break assumptions that you'd normally have. We try to, well, first break things and then tend to build these things up again under maybe different assumptions and as such we thought that the IMA, the Integrity Measurement Architecture, was an interesting target because we've seen it in a few projects that we are involved with otherwise. So we work with people in the medical sector and in the energy generating sector and the power plants and we've looked at how they use Linux and security technology and then we've come across this Integrity Measurement Architecture and we think it's very good technology and we've investigated how to actually, well, subvert the guarantees that people are trying to get from IMA and we've written this paper where you can see the screenshot of this paper then the bottom right. And this work has been performed with my colleagues which you can see there. And the context very, very broadly, very general is in the critical infrastructure domain. We focus on this area as well because it's a bit more interesting for us because it has a few challenges that other sectors do not necessarily have. Very, we have the impression that this sector is ridden with legacy so there's loads of old machines, long lived machines that are, say, difficult to update and upgrade and these devices that are being run, they tend to have low memory, low computational power and well as such they present an interesting target to us academics. We have here a picture of a PLC, a programmable logic controller is Siemens device, which, interestingly enough, has been subject to an attack or an attack with these Iranian centrifuges. This was the famous Stuxnet attack. It was very sophisticated at the end of the day it involved manipulated hardware. So somehow the victims in this case the Iranian centrifuge have been, they have somehow received malicious hardware which then cost havoc and damage and destruction to these devices. Of course Stuxnet was a very sophisticated and targeted attack. So the attacker in that case has had loads of, well, economic and economic power and was very knowledgeable in what they were doing. In the end, you may not necessarily be subject to such an attack so in that case you may not necessarily need to worry about manipulated hardware, in fact, in you. In case of Stuxnet however, it was a simple pen drive, your regular USB key, the pen drive that you tend to share with friends. This has been manipulated, the firmware in fact on one of these devices to then go undetected and well for Stuxnet for the warm itself to to spread around the world in fact. What it actually did was it costs the centrifuge to spin faster and more slowly than they were designed to or they were used for a normal use. And that caused them to wear out super fast and then they eventually got destroyed. It has been reported that about 1000 of those have been destroyed. Yeah, I would go as far as claiming that it's not necessarily about the damage there in this facility. So there has been some collateral damage involved in running this attack and you may very well be well cautious of this type of attack regardless of whether you think that you're that that another nation state will will be your attacker. So, there's other attack vectors as well. So it doesn't necessarily need to be some governmental agency that attacks you get very will be the manufacturer for example you could get manipulated hardware from the manufacturer like were a part is quite you could have bad firmware flashed on these devices before you even put your hands on them. And it doesn't actually need to be the manufacturer. It could also be that the device is being manipulated while it's in transit. And in fact, it has been documented that this happens. So there's an NSA program that well does exactly that it captures hardware while it is in transit and then manipulates it and then forwards it to the actual recipient. And then you have a bugged hardware in your computing. Well, the integrity of your computing cannot be maintained anymore. There's another interesting attack vector, which is that an etiquette. Well, if you have second hand hardware saying because you're in, you run an economic data center and you don't actually care about the latest and greatest hardware you just want to run your, in this case, controller for your machine and you may even need a second hand machine. And then the Attica may very well manipulate this. Well, this first hand machine before you buy it. And then when you buy it, it looks all great. But in fact, the Attica has infected you before you even got the machine. So the Attica managed to manipulate the machine while it wasn't actually worse yet. So, some of these Attica models they may or may not apply to you and someone more realistic than others. But we have luck, the luxury of just assuming that this sort of Attica exists and then trying to figure out how this impacts us and how we could potentially defend against such a such an Attica in the domain of critical infrastructures, you may look for certain security guarantees you may look for example to run only programs that you know are good. You may not want to run, you know, this malware that you've just downloaded by accident. And you may also want to prevent malware, if you ever run it, or if it ever gets under your machine to persist so you would want to, well, we've art to something like a clean state or pristine state. Well, or if that's not possible, then you would like to detect at least that there's something malicious going on something that's not how you thought it would be. Another interesting goal could be to convince someone else that you're good or clean or in a good enough state to, for example, be admitted to the network, or that your patch state is good enough, you may want to run well protocols that allow a third party to be convinced that you're good. And some of these goals are realized through IMA or related technologies. One probably a better known technology is is measured or secure with those two come tend to come together. And what measure boot is about is to hash each step of the boot sequence. The boot loading power is up first the farmer that's running on is being hashed, and then the boot load is being hashed and then before the car was loaded the car was being hashed, and all these hashes are stored in a tamper proof. Well, storage in some way that in Africa would have a really hard time manipulating bear in mind this is a very brief description of these technologies that involves so much more and especially the root of trust this can be very interesting depending on your scenario. But the gist of the measure boot concept is to hash each and every component before it's being brought up and record that that hash this measurement as it's called in a tamper proof way. You can also compare those hashes against the golden value against something that you know is good and only proceed if, well, if it matches well enough. Those concepts can be extended on to the operating system. That is, each and every demon that you run each and every well program that you execute or data that you load. You may want to know whether this these things are good before you run or load them. And integrity measurement Linux is integrity measurement architecture allows you to, well, to extend those concepts on to, well, the operating system. So you can make Linux hash each finally that you execute before running it, or that you would rather that you want to execute before actually running it. And I may also allows you to compare the obtained hash with your reference with the reference that you have saved somewhere. This, this IMA is allows you then for going further even and doing what what is called remote attestation. So you can run sophisticated protocols that allow an attest for to verify that you are good that you are the only running software that has been approved and that you are now admitted to save the network or to resources that would be denied to you otherwise. It's some noteworthy that IMA does not actually need a TPM like you can run the whole IMA without an actual TPM it all works locally as well. But of course, and you don't have the, the root of trust you don't have this, the guarantee that the values are have been stored in time for proof. It comes in handy for us because we can run our experiments and virtual machines without well messing around with the TPMs or virtual TPMs and just, we're just able to use the local logs. That makes things on our side of it easier. IMA on a very high level looks like this you have two modes of operation. One is the appraisal mode and the other is the measurement mode. The appraisal mode is, well, is similar in mind to secure good. That is, you have your hash each and everything and you, well, compare this against known good values which can be in or which are stored in extended attributes. And you can only have the hash values but you can also have signatures of those and then you can, well, use property to verify those those hashes rather than having static symmetric values. The measurement IMA measurement mode is similar to measure boot in that each and everything is being hashed as it is. You can have a policy as to what is hashed and what is not and those measurements you can then challenge like an external party can challenge those those measurements and then you can get a proof of the machine being in a good enough state. One thing that we noticed is that the way it's currently done. And I would go as far as saying that the way it's thought mentally. And there's an inherent race. So, if you read, if you read something of the days, but you need to hash it before, then there's the potential for you needing to read it twice. Once for the hashing and then once, once again for the actual execution or reading of the file. Well, however, it tends to not happen in practice though because there is the page cash which not only accelerates things but in this case it well also makes things a bit more secure. At the end of the day, the whole thing should be memory already from the first read and so you don't need to read a second time, but there may be circumstances where it actually happens that you need to read multiple times. And in this work we concentrated on investigating the circumstances and the conditions and boundaries of when this actually happens how we can force the second read of well the data that we want to read or execute. So the, well, the attack if you want to call it or the problem roughly looks like this. So we, we tell the carl to well open or execute a file for us. And then before it does that it wants to measure or hash the file. And then I'm a goes off and reads the file as it does for hashing it. And once it's finished and once everything is good enough, then the carl comes again for actually executing or reading the file. And then the read request hits the disk a second time. So this is the inherent time of check time of use problem of this architecture, the way we see I'm a being built currently. And we're, we're now trying to exploit this, this talk to problem here. So how do we go about exploiting this problem we have one big machine, which well is big enough for our purposes. And then it actually runs on multiple things for us and separate machines. And so we created our I am able to machine. And in that machine, we then ran the, the I am a targets the victims the guests that we try to attack. This makes things a little bit more weird than, than they need to be, but well, we didn't really have other infrastructure back in the day when we performed these experiments. I think the results are not affected by the setup though. But so just to let you know that maybe the setup causes problems down the line with our results, but I think it's it's solid enough. Now having our basic infrastructure we needed this malicious block device. We've implemented this malicious block device through QM the hypervisor of our choice. We could have patched from a directly, I suppose, and if we worked on bare metal machines. But, well, we don't have much experience patching heart or writing hearted from a first place letter and patching it up to the devices. But there are people out there who do this. So we know that people exist who will manage to manipulate firmware of heart is such that they will act maliciously. So we claim it's not totally totally off the wall to. Well, half malicious hard disk firmware that somehow gets under your devices. Our manipulated firmware has the goal of noticing the first request of our target binary saying, and then notice subsequent reads, assuming that then this comes from the operating system for actually executing it. So the first read, so the assumption of our target binary or executable is caused by IMA, so we assume, and the second read of our target binary is then caused by the operating the underlying operating system actually wanting to execute the file. We have patched your email in as of roughly last year. That was the latest release version that we obtained when we start to implement this functionality if you want to call it that. And it was relatively easy to patch QM such that, well, it has the desired properties. So there's this this file driver or this driver that backs block devices with with an actual file. And in that, in that file there in this in this driver, we can hijack the function that reads things off the off the file on the host. And, well, the code is relatively simple. So this is the bulk of our logic there's a bit of management piece a bit of overhead. On the bottom of this block but this gives you an impression that this is some is not like much code that we need to have there in order to have the desired properties. The main things it does is it counts the number of reads. So we, we have our target area that we need to define up front. And then we count the number of reads to that area of the, well, of the heart disk. And if we are in that area, we copy, well, OX FF in this case over to the to the original value assuming that the original value is with something else. Obviously works for our case where we control everything we know what we have to expect in the be nine value and now we know what to expect in the malicious value so we can easily discriminate those two. And it's obvious that these, well, these values would need to be adapted to an actual attack to, well, a real scenario. But, well, it works well enough for our case because with this we can measure how many bytes we could successfully manipulate to our method to our method. Another challenge with our naive approach is that depending on the file system, this method does not work reliably enough. So if you have sophisticated file systems, they tend to spread the extends all over the disk and you need to sort of track them and you need to. Well, make sure that you don't overwrite the actual file system metadata, rather than your actual well data that you want to manipulate. So, if you wanted to do it correctly then you'd need to account for actual file systems as well. That's not to say that we haven't used an actual file system but we have made it deliberately simple. Well, for us to not have to spend too much time and things that are not directly in our interest. So now we have a manipulated block device. So, let's check whether it actually works as it's as expected and turn out that it does. So we could manipulate the binary or a binary that we have there on the victim machine on the target machine. And we had a very simple. Well, very simple executable that would have some tiny states embedded in it. And depending on that state it would either print a green screen or if if the state did not match what was expected that would print the red screen. Well, and in our case we made it so that should be greeted with the recording. Well, screen there. We use this also to check whether I'm I was working as expected. It turns out that, well, Ubuntu comes with with good defaults but you still need to make it actually work so you need to hash over like each every file need to store the hash is in the in the extended attributes and so on. So we manipulated our binary and then we saw that IMA was actually rejecting executing this binary because it was not matching what expected. There is no screenshot now because it's actually boring it just prints something to the kernel log like IMA denied access something. So, we now know that our method generally works. Well, how much can we actually manipulate and under what circumstances how much method memory pressure do we need to have and how does that relate to the size of the executable that we need to have. So we have a small program that tries to assess how many pages we can manipulate. Basically, all it does is define and define an array with all zeros. And the size of this array we can, well, we can define upfront at compilation time. And then we cannot as well decide where we want this array to go in which, which section of the else binary. This is the definition of this array there. And because of the automation that we've that we've used, we were able to change this size upfront before we were compiling that was that was quite good. And this is the main logic of the of this test program. All it does is check whether the array is still zero or did the item in your race still zero. And if it's not, then we have found a manipulation and we record where this manipulation has been noticed. And then, well, we go over the array. And we, at the end, we give the total number of manipulated bytes in the relation of, well, in relation to the total size of the of the array so that we can have a neat value of how many percent we were able to manipulate. We use system destructured logging, which is very nice because it saves us a lot of headache when parsing, or now we don't have to parse some output which could be interleaved and everything this. So this is is very good. And then we have some sort of management around still so well this is the main logic of the program there's a little bit of logic still to create the memory pressure. Yeah, so this is not the whole binary but you can all find it on on GitHub I'll present the link later. So as for the memory pressure. We have used stress and G, which is a very nice tool with the many, many options and the test program that we've seen just before. So we've done some good weights for stress and G to be finished. And then in our experiments we've run one instance of a test for 50 times just to get some some stability in the results and to see where things may be less stable than we expected. And with this setup. We were we had all the necessary bits and pieces. We needed to glue it all together. So we have a medium size batch script that, well, creates the social machines automatically copies all the required scripts in all the service definitions and sets up the IMA and then all this. And we have tests for remember we have tests for two dimensions. The one is the size of the executable that we will try to attack. And then we can change with the size of this array that we have there. The other dimension is the pressure that we exert on the machine the memory pressure. So with with the stress and G, we can fill in. Well, number of, of megabytes in this case that it will allocate and will stress the machine with, and we run our experiments in these two dimensions. So as for our automation, we have made great use of great libraries that are out there and guest fish, which helps us tremendously in getting files in and out of the machine for not only our services but also then the logs out of the machines and we can analyze them. Use the, the virtual edit tool which helps with greatly with the manipulating configuration files in place rather than just copying them over, which, well, in our cases with more handy because then we could carry the state over and over. And one thing we were a bit fighting with is the offsets of the data that we are attacking. So remember, we have this executable, and we have this array in this executable, and the relatively simple task or the task that should be relatively simple of finding the the actual offset of this array in the, in the executable turned out to involve more tools than I expected and more grab and awk than I thought was necessary to, to finding this relatively simple address of the of this array. You can have a look at all this at this GitHub link. It's not pretty, but it works well enough. It produces these virtual machines and runs them it captures the output and the logs and it makes sure that all the, all the values are filled in for the various results, then produces a nice text representation of the results. So this is all should be all fairly reproducible and fairly extensible for, well, if you think you could run these experiments in another or better way. The results are or one slice of the results say is this so here we have a fixed size binary. And the, sorry, we have a fixed memory pressure 128 megabytes is here where we exert on the machine and be very the size of the binary that we want to run the executable. And then, as, as you could expect, if the executable is fairly small, up to 64 megabytes, then we see little to no effect of our manipulation, everything will be in the page cache only occasionally a few pages will be evicted and the pressure exerted on the machine, but up until that's like 64 mark 64 megabytes mark things are normal say no manipulation that we could observe. But if we increase the size of the executable, then 128 megabytes and then of course finally 256 and bigger, then we could manipulate more and more pages of this of this executable. We had 512 megabytes of RAM. So this is not like massive. It's not massive amounts of memory. I bet that the results will be similar. If taken into relation, if we had two gigabytes of RAM of memory, then I guess we'd have to increase the sizes of the executables accordingly and I, I guess we'd have similar results. So this is to accommodate our critical infrastructure domain where we know that the devices that are in use have, well, little memory. So this is some, this graph here shows the fixed memory pressure of 128 megabytes. This graph shows a fixed sized executable. So 64 megabytes in this case, and we vary the, the pressure that we exert on machine. You note that we have two lines. One is for manipulations in the text section and the other is for manipulations in the data section. So it's fairly similar. One thing to note is that Linux doesn't execute elf binaries when the data section is too big. We found that surprising. I guess there is a reason to doubt that we still need to learn and find out why that is. It works with the text section though. It's no problem, but the data section, it doesn't, doesn't run the executable. All the results can be seen in this table is a bit big. It's not very detailed. The details don't really matter here. What you can see though is again, as you might expect the top left of the half is all fairly zero. There's a fairly little manipulation going on. So the machine has 512 megabytes of RAM and I have an executable of 32 megabytes in this case, and I only started pressure memory pressure of one megabyte and we'd be surprised to find parts of our executable to be evicted from the page cache. Or not however, but also as expected in the bottom right where we have huge executables and large memory pressure, high memory pressure, then nearly all the content is gone from the page cache. So the interesting band, I guess, is there in the middle, there's sort of a sweet spot when the machines begin to trash and to or rather to evict things from the cache and where it then needs to hit the disk a second time for reading the contents back in. Another thing that's that may be interesting is that some results tend to be quite unstable. So we have standard deviation of relatively high standard deviation for some values, especially there in the 256 megabytes executable and there the 32 megabytes pressure. There we see relatively high standard deviation of our values of the 50 measurements that we have taken for this, for this particular run. We could run it another 100 times maybe it makes it better for it's like this one stands out because it's, we don't know maybe we just had a bad bad timing there when running the experiment another, another unstable result is there in the in the top with the 32 megabytes executable and 192 megabytes of memory pressure. Also, very unstable result. Maybe this is, this is where the sweet spot tends to begin and maybe that's, that's a magic value that is, well, somehow creates more uncertainty in the results than what we thought it would be. Another interesting thing is that we noticed that the first starting and the last page were always retained in the cache. So you remember we had recorded when we could observe the manipulation and which bite in the array. And from that we couldn't fire that. As we say the first time pages we have in all the experiments you've always seen being present. And so, as well as the last page. I don't know why that is. Maybe someone has a clue. So that's some that describes the the circumstances and conditions under which this inherent talk to race can be exploited to have a bad effect on the on the software that you run on your on your system that you try to protect with this with the technology. So I think, how can we, well, make the effects less severe how can we mitigate this problem. And we think that you'd somehow have to prevent I may hitting the disk. Additionally, like you somehow need to make sure that this hashing off the off the file content does not cause an additional read because once you hit the disk a second time it can discriminate the second read and it can then cheat on you. You could probably mitigate that in turn by doing random reads, like if you're idle and then you just request random blocks, hoping to sort of trip up the modified firmware on the on the hard disk. Another approach would be to go more into into direction that DM or FS Verity are doing. So as far as we could see they verify each chunks rather than each file. So this is inherently more, more granular is more fine grained in what it does. So I think that there have been patches on the way which sort of adapts this, this style of checking for my pages are being, or when the contents are being paid page back in. We haven't followed what exactly has happened there so it may, maybe that I may by now has a more granular approach to very fine things. So I think that somehow making sure that you also include the actual disk or disks that you're running or the firmware rather into the measurement process. Similar to what you do with the machine already right like your motherboard is being hashed as well like the firmware of that. So it's some, it's a bit unclear how you would ensure that the, the hard drive that you're having is actually running the firmware that it reports. At least when you have general purpose computers as we tend to have still with specialized machines it may very well be possible, but at least those that are being sold to consumers these days. Don't see how this could work. Maybe possible. I'm curious to ask how you do it though. So the things that we've discussed so far are, well, they have their, or they have their problems in the sense that the hardware attacks by by themselves are quite out of scope of I may. So we are exploiting something that I may does not actually put try to protect against. Yet we argue that this type of attack exists and that the attackers exist that could execute or launch such an attack. But still, it's, it's a bit unfair of us to sort of, well, exploit this while this is not something that has been actually tried to defend against. We also note that the binary sizes we have investigated like 300 whatever megabytes is not really something that you'd have, although we also note that it's increasingly more common to have large binaries. So I've just checked the doctor demon has something around 100 megabytes. And then I checked, I have some some rust binary slowly, which are easily in the range of 200 megabytes. So I guess it's not. At least for now. It's not unreasonable to to think of relatively large binaries. Although, we're typical. Debbie and package is not that big. But also note that in order to execute such an attack. You need to have very detailed knowledge you need to be a very strong attacker to know, like the environment that you're targeting and where the, the binaries like the data that you're attacking is on the on the actual hard disk in that case. That requires a lot of insight and motivation to. Well, to launch this attack. Depending on who you are, you may very well be be a target of such an attacker. What we'd also like to do what we haven't done yet though is to check other operating systems. So we were told that windows has a very similar thing, but none of us is a windows person so we actually don't know. And it would cost us a lot of time to actually check how to do this IMA style of integrity checking verification in windows but we were told it's possible so maybe one day we get around to actually do it. So to wrap up. We note that the attacks are possible indeed, but probably a bit unrealistic depending on on who you are. Pressure on your system with enough memory pressure. Your Linux IMA system goes to hit the disk multiple times, at least twice. And this entire can be used by by a malicious block device to serve you malicious content unsubstituted reads. In some cases, even a little pressure is enough as we have seen so there's some depending on the sweet spot range. There may be circumstances wherever you get surprised that your, your executable has been evicted from the from the catch. But, again, the attack needs to be very skilled and very knowledgeable in in your scenario and it's, it's more likely that you're not subject to such an attack than it is that you are indeed being affected. This work has resulted in a in an academic paper presented at this year's artist conferences, the conference for availability, reliability and security. You can find the link there and all the resources that we've used are there on this on this in this GitHub repository. The best scripts for creating the, the virtual machines, the, the way for extracting the results and, well, capturing all the, the required details. You find it all there it should be reproducible it's not pretty, but if you're determined and then you can get it to work and reproduce these results. If you have any questions, then I'm of course available after this talk, but as well of course via email with this address there. And I would like to thank you for your attention and of course thank my colleagues for this interesting work. And I hope you have many interesting questions. Thank you very much.