 Microcontrollers with secrets that they don't want to tell our next speakers have been convincing Them to open up despite a plethora of protection schemes Don't even try holding back your enthusiasm for Johannes and Mark so good evening and Welcome to our tales from hardware security research today We are going to shed light on three questions. First of all, why are we doing hardware security research? Then we will present you two examples on how we do it and then I will come to the consequences of our work So first of all, let's talk about the wire So my name is Johannes Uwe Meyer. I'm doing research on embedded system security and physical Unclonable functions. I'm employed at the Farnhofer Institute for applied and integrated security and currently I'm about to finish my PhD there Okay, so hello everybody, my name is Mark Schenk. I'm working as a security researcher Basically, I do everything that it's somehow related to hardware security So from analysis of security concepts to I'm such an attacks on microcontrollers Um We both both work Quite often with microcontrollers and work really a lot with them And we also and read the data sheets and sometimes they have really strong security claims in the data sheet about the security concept And you know a security researcher you the question arises Do they really hold? Um, but why should you care actually about from protection at all? So if you develop of microcontrollers, you probably want to protect your firmware, but even if you don't Care about IP protection at all you might want to protect your credentials on the microcontroller So let's say you have cryptographic keys on the device or privacy related stuff Then you want to protect these data against readouts and also you want to protect your firmware your microcontroller against data or firmware Manipulations so that your firmware does not run malicious firmware or work on manipulated data Okay, so but why we try to break these devices so there are multiple reasons for example in research projects We use microcontrollers and their security project and sometimes it turns out that the security concepts are not well Implemented or have weaknesses Another reason is that We have customers that want us to do a security analyst analysis of certain concepts Because they want to use these features in their components and their components and we am shoulder Evaluation of this concept and the other thing is of course scientific curiosity Sometimes we see that there's a new firmware protection out there in some product and we would like to see how this works and If this is really secure So but the question still remains Would anybody really try to break the security of your microcontrollers? So unfortunately the answer is that it's not only anyone who wants to do it That's even an entire market that evolves around Cloning embedded systems. That means we have companies that offer Cloning a device like you can get a clone PCB and they will also pull Firmware from a microcontroller and put it onto the clone device So you might perhaps think that's quite an expensive So we did an inquiry for one of our customers and found out no actually it's quite cheap It just cost around 6,000 us dollars per chip What state of the art microcontrollers and it will even be delivered in like 30 days You know, so that's surprisingly cheap and it's in we it's a real threat here so unfortunately Vendors of microcontrollers often mistake us as malicious attackers. That's bad because it's not like We are sometimes seen like I select a product from a mic from a vendor and I choose this specific vendor because I think I don't like them and Then I pull my cyber hammer from my drawer and hit the product until security breaks. That's not true We don't select a specific Product or a specific vendor and please keep in mind don't shoot the messenger The vulnerabilities in the microcontroller are already there. We are just pointing them out that they are present We are not bringing any vulnerabilities into microcontrollers here. We have an entirely different approach We do a scientific evaluation of security concepts So the science is the center of our research here. So first of all, we try to start with a Theoretical analysis, so we read the datasheet think about how does the security concept work? Then if we think that we discovered of law, we try to test it in practice on one of the devices And if this is really successful, we will Develop a proof of concept for the vendor so that the vendor can reproduce our Detected issues and maybe also can fix them in the next revision And then we will publish a paper and that closes the loop of this scientific evaluation of the security concept because then it becomes reproducible and the next developer Will not need to make the same mistake as we will see later on that Sometimes vulnerabilities are spread across several manufacturers so after I've Pointed out why we are doing this I will now continue with the how and bring one example of Research that we have actually done some time ago So usually if we want to start the questions always How do we start at all? So do we really smell bugs? So any questions to that is that it's not far from being incorrect because Sometimes you really have some gut feeling that this security concept looks a little bit fishy and then you investigate it further For example, I had these stm32 f0 micro controller board In my opinion, that's a quite powerful and good micro controller So I spent a lot of time developing with it So I got to know it quite well, and then I came across some security feature Well, I didn't know are they really that strong as it is promised here So and that was reason for me to just say well, hey, let's take a look inside So for the protection of the firmware the micro controller offers three protection levels We have RDP level zero up to RDP level two and the datasheet says that in RDP level zero You have no protection at all that is that what developers are using when you buy the micro controllers And so that you can do everything what you want to do with it Then we have the read protection in the level one and we have no debug access in RDP level two so And I can configure this micro controller by writing a specific value into a location in the flash memory So that's a permanent setting for example if I write 3 3 cc into that into that memory location This tells the micro controller to shut down its debug interface So of course how it is common for those security levels I only can increase them if I want to decrease them if it's even possible Then this goes hand-in-hand because a full device erasure Of the entire firmware. So this security concept looks quite good on a first Look, but then if you look closer in the datasheet you realize that all this explanation is written a subsection Of the flash firmware protection So does this RDP level one only refer to flash read protection, but what about SRAM? So I just tried it out. I took one micro controller loaded firmware to it locked it to RDP level one and tried to read the SRAM and Well surprise SRAM is still readable and all the data that may be in SRAM So for example some keys that are currently processed They are still accessible and you can even exploit it in a way that you can in some cases extract parts of this Of the firmware. So this is something that you must keep in mind if you select the RDP level one So people think well, we also have RDP level two. So well, we just use RDP level two. That's a good idea, isn't it? Well, let's have a look at this how the settings are stored on the micro controller So here we see the two levels RDP level two and zero And their corresponding heads a decimal and binary representation of the setting all other settings as we have seen Map to RDP level one. So I thought well It might be possible to ground grade RDP level two to RDP level one by just flipping a single bit So is this perhaps possible in some way? So the settings are stored in flash memory and in flash memory the data is Commonly stored on the so-called floating gates So if there are electrons on the floating gate that Represents the binary value zero and if the electrons are missing then it's a one so then I need a way to remove these electrons from the Flash del and then can that can be achieved by ultraviolet light. So if I illuminate this cell Then the UV UV light kicks out the electrons from the cell and flips our zero to a one Okay, but is this practically feasible at all? So first of all, we need a strong ultraviolet source This type here that we see on slide. That's a mercury vapor lamp I got mine from the dumpster from a university, but you can also buy them online for just a couple of bucks So they are quite easy to obtain and next so that you have access to the chip and not the flash memory you will have to Create a decapsulated IC. So you have to remove a part of the package You can do this for example by strong exit assets Or you can also go to a company or to some chemist that has access to the technology So that's basically is feasible to decapsulate this chip And then if you take a closer look and have a little bit of experience You can identify the region where the flash memory resides on that chip and by doing more experiments You can all find us in the paper You find out that only the lower right region holds those RDP bytes that Store your security setting and by putting a mask on to the remaining flash memory where the firmware is you want to protect You will just illuminate this region of RDP bytes for several hours and then you indeed Achieve a full security downgrade from level two to level one and thereby the Security of RDP level two is broken and that's only possible because there's a very weak fallback mechanism Where everything maps to RDP level one if I flip a single bit Okay, but in RDP level one I still cannot read out the flash memory But that is also something I didn't really trust because I have access to everything Except the flash memory That must be quite complicated as it is implemented here So I eavesdrop the communication between the standard debugger that is shipped with the microcontroller and the microcontroller in RDP level one so initially the debugger starts up the debug functionality the microcontroller acknowledges that and then an Entirety of 18 transactions are taking part between the debugger and the microcontroller So the debugger for example reads out the CPU ID reads out some meter data regarding the debug configuration like break points watch points and so on and as soon as I take control Over the debugger. I try to read out the flash memory and that's no surprise I cannot read it because it's locked in RDP level one that I don't have access So I didn't really have possibilities to use the standard debugger So I decided well, I will just implement my own debugger on another evaluation board and try out if I can perhaps mess around with the entire communication here and Since I don't I'm not interested in break points watch points and so on I skip this entire readout I know which microcontroller I'm talking to so there's no reason for me to read the CPU ID So I skip this entire config readout and then Just to verify that my system works correctly I try to read one of the flash addresses and we all expect this to fail because RDP level one is active So I should not have access to the flash memory But surprise we have some data there Okay, first of all, I thought yeah, I must have made some implementation error. So I checked it again But no, it was really some data that I've written into the microcontroller before So I tried to do it again to read the next bite But this failed again and I played around with this a little bit more and found out if I don't do any configuration in the beginning and adjusts and the first transaction goes directly to the flash memory I can read out a single word from this protected flash memory So, yeah, that's quite surprising So there seems to be a race condition so that the flash is not locked quickly enough and I can read out this data here We also requested a CV for this and it's already published here. So this issue persists in this microcontroller and I can even show you this in a short demo now. So on the left side you see we have the device under test That's the microcontroller and are the TV level two and on the right side We have the so-called firmware extractor that performs this attack and always resets the microcontroller We starts it and pulls one word after another so as I Prepared this beforehand. So as you can see here Using GDP. I cannot read out the flash memory. It just returned zeros here But now I will start up my firmware extractor that I have here on the table and The laptop now connects to the firmware extractor Yes, from extractor does all this experimental debugging stuff and Tries to read out the memory from the device on a test and as you can see here The first bytes are already being read and just to verify it I also put some human readable strings into the memory that are currently appearing here. So I will now Stop the attack and as you can see here We really can read out the contents of the protected flash memory and for the common chips This can be done in a few minutes or up to hours. So this is entirely possible And so we were also able to break our DP level one Thank you And the reason why this was possible is that we have insufficient permission enforcement and RDP level one debugger communication So that was our first attack and now mark will continue and show us another attack Okay, so I'm going to talk about an attack that we actually presented last week at the wood conference So this project started because I was reading a data sheet from my stm32 microcontroller and then at some point stumbled upon features called PC Rob and I thought okay. What the hell is PC Rob? I've never heard of about this feature and At first I thought it's some countermeasure against Rob, you know return oriented programming, but Turned out. It's actually the proprietary code readout protection So that's a feature that shall protect Your firmware on a microcontroller against other developers working on the same system So the basic idea is that you imagine you develop the library Let's say a motor control library or stuff like that And then you deploy this on your microcontroller and then you hand over your microcontroller to a different developer And this developer deploys the main application on the microcontroller and in the end The developer locks the device and the firmware on the device is totally safe and at the end you have your product So let's say a robot. That's quite cool, but During the development process your firmware is not protected right as we've seen in your harness lights So if you have access to the microcontroller via the debug interface you can easily read out and reverse engineer the code on the microcontroller and Here's where PC Rob comes into play So PC rope allows you to deploy your firmware on a microcontroller and then lock it into a so-called execute only memory and then again you hand it over to the developer to the other developer and This other developer can still use your library, but it's not able to read it out anymore or to modify it and doing a research you found out that That's not actually only implemented by SDM 32 microcontrollers, but also on kinetics devices from nxp and some device from text instruments So we analyzed all of these devices you can read out read the details in the paper We found also some interesting hardware bugs and these devices Okay, so how does this work? This is a simplified schematic version of a microcontroller on the left hand side You have the flash memory which is connected through a bass matrix to SRAM other peripherals CPU and in the right hand side You have to debug interface so the basic idea behind execute only memory is that You deploy your library on the microcontroller and then mark a certain region as execute only memory and From from there on it's not possible anymore to read out the code It's also not possible to modify the code only instruction fetches are allowed So you are able to do instruction fetches because you still need to be able to execute the code, of course Okay, so This from a protection or protect you against developers on the microcontroller the problem is that as a developer I've had have basically access to the entire system except for the execute only memory, right? So as an attacker I can execute the instructions in the protected code area and Can observe the effects of an instruction in the other components? So let's say I execute an add instruction and execute only memory then I can see that the register in the CPU core changes and If you do this and see all these these Difference in SRAM and CPU you can basically here recover the code that's actually protected and execute only memory and and thereby dump the entire code and That's what I will show now Last week we presented a demonstration on the STM32L0 and so we decided yesterday, okay Let's use a different microcontroller this time So for this presentation We use the device from Texas Instruments to the TM4C123 Okay, so What I'm doing now is I connect this microcontroller to my computer and And the first thing is I establish a debug connection to the microcontroller and Okay, let's see Okay, so let's connect to the microcontroller and there's a Library on the microcontroller that flashes the LED, right? so this is our library we want to protect against against readouts and reverse engineering and This library is Placed on this memory address and it's for the for the for instructions So that's that's the library we want to protect so without execute only memory you can simply disassemble this code So that's quite easy So let's Oh, you can use Yeah, let's enable execute only memory for this memory region Think this Yeah, okay Okay, so now execute only memory on this memory region is enabled and now we try again to To disassemble this memory region, okay doesn't work So all read attempts to this memory area are blocked and generate a bus fault, but When we use Okay, so let's do it again So we say that our tool should Recover instructions on this memory area and in this case for the for the two instructions and we also do a post posting step and Then we can see that we can actually recover the code and If you come if you compare the left and the right side, you can see actually be recovered the entire We can recover the entire code So the left on the left hand side you can see the recovered code and right hand side actually code It's inside the execute only memory and when you compare both sides. It's actually the same code. So here we go Execute only memory doesn't protect you from Okay, so We found vulnerabilities and microcontrollers. So what we do now, of course we do a coordinated disclosure process with the manufacturers and In theory, this looks like that. So you start your research You found the vulnerability then you inform the vendors and provide technical details of the of the vulnerability And then of course the vendor come confirms immediately that you found the vulnerability and everything is correct Then you write your paper Submitted to a conference and in the end you present your paper and your papers published So that's a theory in practice in practice. This Isn't almost all the case. Oh I forgot to mention sorry that for hardware security for hardware security bugs we provide the vulnerabilities or Provide information about vulnerabilities Usually 120 days in advance before we publish a paper to give the vendors some time to Mitigate or in some cases implement countermeasures if possible Okay, so now let's come to the first example here of vendor a So we started our research and soon afterwards we discovered our first vulnerabilities here And of course we wanted to inform the vendor. So this took a little bit more time because In the hardware area some vendors don't have something like a third where you can Write to the people that want to collect the vulnerabilities and want to fix them So it was a little bit difficult to get the email address of the right people But then we started our discussion with them But it somehow slowed down after a few days and weeks because we wanted to have phone conferences But they just didn't join them or they didn't have time on that and at some point They didn't even we spare on to our emails at all anymore So they basically ghosted us here so unfortunately There's nothing really we can do here. So we thought well, perhaps they don't receive our emails, but I Had a look into this and they realized we got a few a couple of Out-of-office replies. So the mail server Definitely received our emails and they just didn't want to respond to us anymore So since we want to proceed with our scientific approach here We wanted to publish our results nevertheless So we waited some time then the conference came around and we Submitted it to this conference. So then the time started where we have to wait for the review So our paper could be accepted or could be rechecked it in this case. You were lucky our paper got accepted And the conference program got published But then something very interesting happened They contacted us again and send us an NDA So well, they don't really like the idea of an NDA due to several reasons first of all I don't need an NDA with them. I want to publish these results. I want to have an open discussion That's no Nothing from my side. I really want to protect with an NDA here and also from the side of the vendor They don't need to tell us anything. We just want to talk to them what we have found. Okay, perhaps Discuss a few questions and that's it So we don't need an NDA from a technical point of view and there's also another reason where I also got into Contact with one of our Institute lawyers and he really said well That could be a really bad idea to send an NDA like two weeks before the conference where we want where you want to talk about Results your findings that you had so really check that this idea of signing this NDA and just said well We can talk nevertheless. We don't need this NDA here Then they're caught of course. They got quite again this trick of them Let's say didn't really work out here. The paper got published We got the CV got assigned and so on and yeah That's it for the vendor a Finally, it worked out. It took a little bit more time than the 120 days, but all in all it was quite successful in the end Okay, so let's see how to disclose a process one with when the with vendor be so again You start your research and then you discover the vulnerability and this time you have direct Contact with the manufacturer because they have so-called product security and incident response team So they have an email address on their side and in some cases in BGP keys and stuff like that So you have contact to the manufacturer So everything is good and then they started to deny the vulnerability So it'll provide them technical details and they say oh, yeah, we don't think that's a vulnerability Please read the datasheet and you will find out. That's not a vulnerability So, okay, so we read the datasheet again and we sent them again an email and said, okay We still think we found the vulnerability and they said again. Oh, no, we don't think you have This is a vulnerability. Please read it read the datasheet again and of course we read it again and Then we sent them another email and said, okay, we still think that's a vulnerability and tried to even more explain Why we think that's a vulnerability and they said again, no, that's not a vulnerability. Please read the datasheet and then then suddenly this happened and so Made a big big progress for the disclosure process any idea what happened here We sent them a proof of concept exploit so with that exploit source we were able to dump the entire code in less than a second and The funny thing was that of course we read the datasheet and again and again, and then we found an additional bug So with that hardware vulnerabilities possible to bypass execute only memory directly you don't need to recover the code Okay, so and then they asked us for a telephone conference and then they confirmed all the vulnerabilities and Quite good. So looks like a straightforward disclosure process from now unfortunately, and we sent them the our paper and then they asked us to We remove some sentences to rewrite some sentences about specific things They also asked us to add some reference to the to the datasheet which is quite okay. We did this But they also asked us to remove an entire section from our paper and Because it was too easy to understand so So this was a section about how to exploit the hardware vulnerability and We've refused us because we think that's important for the reader to to know how this works because then The reader and customers can assess on their own How how vulnerability this product vulnerable this product is and so we refuse this and Submitted our paper anyway. So unfortunately it was rejected. Maybe some of the manufacturers were happy because they have now a bit more time So we reworked the paper a bit and then publish it again and Submitted it again, and then it was published actually Okay So I mentioned that it's important for us to Describe things in our paper so that everybody can assess on their own how this works and The problem is that hardware issues usually not fixable So microcontrollers if you have a bug in a microcontroller you cannot fix it You have no microcode to update it like on on bigger CPUs You have no usually no software to to work around so if you have a hard hard work on ability that that's really bad and even this potentially potential damage for for Customers and and manufacturers on the other side We want to do a reproducible research. So we want to publish everything so that everything can verify our results So that's the reason why we included all details on the paper and also if you have the all the information published you can implement mitigation and counter measures on their own if possible and also with this information you can verify other products for similar vulnerabilities or Even the same vulnerabilities and other products. So for the last project execute only memory project We decided to keep all the information on paper But on the other hand we did not just we did not disclose the code I showed you So and also other code to to exploit the hardware vulnerabilities So here what this was our trade-off to say, okay? We put this information in the paper, but we don't publish the code to exploit us Okay, so we had four cases For all disclosure process. So the first one was the one that Johannes explained So what was there? What was the outcome of this process? So unfortunately the data sheet of this product was not updated nor the radar sheet So there's no information about this vulnerability in the in the product Description and the vendor also did not Request the CVE so we did this instead On the other hand communication with the with another manufacturer was quite good they even requested the CVE for their products and Also updated the data sheet. So now you can read in the data sheet. Okay, this security feature is vulnerable You shouldn't do shouldn't use this anymore So for the third case It went entirely bad because there was no communication at all We used the context that we had already from beforehand, but also again this case In this case we got no reply so far. They also of course did not request a CVE here And what's even worse is there was no update in the data sheet or a data So there's no official document by them telling that this feature that they are promising in the data sheet That's not really work out as it's described in the data sheet So in the last case D the communication was quite good unfortunately the vendor refused to request CVE on their own so this was done up to us because they wanted to take communication with their customers And users into their own hands. So there was also no data sheet update so far So that much about communication here, but they published an errata that in my personal experience is often Just overlooked if a chip is used So all together here in this hardware security Coordinated disclosure approach. We only have One case where this really worked out quite well in two cases. It's didn't work out at all in one case It's just very questionable And what's even worse here is that if you look at this row of data sheet update Despite the features are indeed broken only one or four Cases the vendor decided to really Add some information or pull this information from the data sheet in the other cases It's still the same text in the data sheet. So there would not be any Documentation that's easily accessible that this feature does not work as promised. So in our opinion, that's like the Yeah, this outcome is bad like three and four cases because there's no good documentation on that So now the question is is it really that bad? No, it's even worse Yeah Okay, thank you. So the problem is after we published our first paper on that I got a few emails from other researchers that are very interested in hardware security. So They told me well, I'm Quite interested in this trip. I think I found something there But how can I report this without endangering myself? so I also verified some claims of one guy and it's really interesting so other people also discovered very interesting flaws in some microcontrollers, but They were they did not know how to report them correctly and then we found out there's a lot of fear of reelection that the vendor could take so and that Leaves them at taking one of two options on the first Option they can just anonymously Do a full disclosure put this vulnerability on pay spin So there's no warning for the vendor or for the users at all and that is something that is very unfavorable And that's something we don't want and also it's not good for the vendor and for no one here And then there's a second option that appears just as bad and that's no disclosure because if there's no information Available about such vulnerability it can be exploited by the people that have found it and Perhaps the same vulnerability will be found in the next microcontroller series again And then when it will ability becomes public Several series of microcontroller will immediately become very weak and the protection can be broken so there must be another way here and that is the coordinated disclosure and That is something that is still not that good to achieve because There's still this that fee of legal action here. So we need to change is here on the one side. We need the vendors they need to cooperate with us researchers. So That we get the support so that we get responses at least that I've read our emails said they take them seriously And then on the other hand on the legal side, we need an international legal certainty So that we don't get sued here So that is something that's quite dangerous and there were cases like in the software security Where there have been lawsuits also in Germany against researchers who try to public published vulnerabilities and By this if we create this legal certainty here Then there's the chance to create an enhanced security by the collaboration of researchers and vendors So that for example firma cannot be extracted But also your secret data your keys that you might store in your microcontrollers remain secure here and researchers It's not just the scientific researchers here a research is everyone who is just interested in microcontroller security So every one of you can be researchers in this field Just get a microcontroller by a evaluation board have a look at this and then try to find out which Which security claims are really true, which are not true But to have this future where this can be just done and discussed openly and freely There we need this new developments and I hope that we are on a good way to that when we all also want to like Bring this forward also by this talk here to say we need the vendors For that we are not against them beyond on their side We also want to enhance the security of the products and on the other side We also would need some more legal certainty here and that could be the way to Yeah, better security for microcontrollers here. Thank you Johannes mark. Thank you so much. We have some time for questions We have a microphone angel over here We have a microphone angel over there if you have a question go find a microphone angel or perhaps the internet Finally has some questions Does he know yes the internet has a question? As security researchers, what would you recommend microcontroller vendors? Do you to increase the security of the security features? Well, that's a quite general question here. I think security must be Done from the very beginning. So if there's a product already on the market, it's mostly it's too late So it's more like There needs to be some analysis of the initial concept because often when we come across Some things like for example what mark has explained with a piece of rub that's something After we read the data sheet we thought to ourselves. Well, that cannot really work. So if someone got Got a hold of this data sheet before production or even in the very early development They could have warned the vendors So security is something you have to put in at the very beginning of the product So security is an essential part of a product nowadays Also some hardware vulnerabilities we found what were actually quite easy So one vulnerability is basically you use a different bus to access the protected code So if you Had some pen tester in the company or something like that you would have found this this one ability probably I See some questions over here microphone Now That's gonna be a quick one. Do the colors on the graph of disclosure timing have meaning to them Colors on which slide? Vendor A and vendor B Yes on this one Do they have particular meaning? No, no, okay. Thanks Next question Thanks for a talk Are there any vendors hardware vendors which have security bug bounty programs and Second question. Oh, well, what one question? Okay Does anyone have a bug bounty? I Don't know. I don't think so. I also don't know anyone at least in the common commercial microcontroller area, but also This security bounties care bug bounty programs can also sometimes counteract an open discussion of vulnerabilities because I've read some of these Guidelines or at least for software and it was often like If you report and use this about a bug bounty program, you may not publish these results elsewhere So the question is even if there were those bug bounty programs, would it really contribute to microcontroller security? So I think we have only enough time for a very very short question and first I'm gonna see is there a short question from the internet No, okay, then you are the lucky short question to ask her. Okay Very interesting presentation. I was wondering if any of the vendors actually updated their hardware to fix the issues No, not yet I think it's too early to say because the paper was published like last week basically and the other vendors I have talked to it's like more like well the new series will come out So I'm not informed about it yet, but it would be very interested if they came forward to us and told us well We fix it now. So would be interesting also for us, but as far as I know, it's not being the case yet So Johannes Muck, thank you again so much. Let's hear it