 All systems ready. I guess we should get going. So my name is Vitaly Wol and I'm gonna talk today about secure updates for memory constrained XIP system. It will happen along the way to decipher what XIP stands for and there will be some more abbreviations that we'll cover briefly too. But first of all I'm gonna talk a little bit about myself. Now we can see this is a company requirement to have a short slide about myself. I've been talking on ELC since 2010. I have never done such a slide now I think I should. Anyway I've been with Embedded Linux since 2003 so it constitutes quite a long time. Started working for Monavista in a dedicated team in Moscow, Russia back in 2003. And in 2009 moved to Sweden where it continued working primarily in Embedded Linux and Android field. And now being part of the Consulco I run a small subsidiary in Sweden called Consulco AB and do some development too yes. So today we're gonna talk about over the air updates which stands for RDA and XIP systems and how these two go well together or well not go well together. So we're gonna talk a little bit what RDA actually is, what XIP actually is, how we can make them work together and yeah we'll draw some conclusions. I hope to be fast enough to leave more time for questions because usually that's the most interesting part. So ODA stands for Over the Air Update. There's also an abbreviation FOTA or FOTA which stands for Fromware Over the Air Update and basically it's a very simple thing in itself it means that you don't have to connect the device being updated to any other device physically but rather the update is downloaded over the air and then applied using some special software running on the device being updated. This is nothing new over the air updates have been around for quite a while and while you probably all know different weird stories about routers updates that turned routers into something completely different like spamming the network with some malicious data just because the updates were not really secure the word tempered with and this is not really a nice story if we recall all the stuff that happened to routers in 2010 all these years. So even then people could understand that security is an important thing when it comes to over the air updates because there's a lot of stuff that can go on in between and when it came to mobile devices like your mobile phones your tablets iPods the updates over the air became a lot more secure when it came to mobile devices because when a device is leaking your personal data and you can see the company that issued that update well this is something to consider so companies became more aware of security concerns because basically they constituted threat for their money but that's basically nothing compared to various embedded devices of even bigger importance where security plays even bigger role like for instance medical devices some automation iot and the last but not the least cars so automobile you really don't want anyone to temper with your automobiles software so it's important to get secure RDA updates on one hand so it poses a threat potentially but on the other hand it's important to develop this functionality because it makes life easier both for you as a user both for you as a driver and for the companies and service centers because instead of driving to service and spending time waiting in line you just leave your car for 15 minutes and then the update is done anywhere for iot devices like telemetry devices somewhere close to the north pole it's even more important because sending a technician to north pole well you know this is a very costly operation so if it can be avoided it should be avoided and if that device is connected to the outer world one way or another probably via satellite then it can download and apply update using that satellite connection and it will be cheaper and faster at least in theory okay the updateers now if we're talking about free and open source over the air updateers there are several let's stand out i made a list it's not meant to be complete it's not meant to be a comparison i'm just listing the ones that i've been working with and those are os 3 which is used by a gel and some fedora distributions sw update which i believe is developed by denks and it's also widely used and it has an open embedded yacht integration at least in part air a uc which has a very nice open embedded integration and i think they're developing it in pangatronics and then there's android update engine and since android is kind of embedded linux anyway i think it makes sense to list it here especially given that android was pretty much the first one forcing the ab updates but we'll get to that so update your requirements so in order to avoid situation when you update your favorite device and it gets bricked or starts malfunctioning there are certain requirements to oday updates which are listed in this slide this is again nothing new updates should be failsafe so if something unpredictable happens like the power is off during the update it has to revert there shouldn't be partial updates there should be a possibility to roll back to a previous software state which basically implies that there's a backup version of the software somewhere so you have to have two versions of the software at the same time which may be complicated due to size limitations an otf there should be capable of updating all software and firmware you know in the linux world that means bootloader and kernel and root file system and databases and other data you can think of and finally the last but not the least it should be secure so up there should be able to distinguish between the real update the one that's certified that's authentic and the one that's been tampered with that lost authenticity or integrity and not apply the updates that it isn't sure about we're gonna classify oday update is a little bit there are many classifications basically we will concentrate on two by the update method you can say it's either a single copy or double copy updater so as you can see on the picture on the graph down below if we're dealing with a single copy then you have to reboot in order to apply an update so once they update when the update patch is there you need to reboot into the bootloader which detects that an update should be going on and then it calls into the software updater which is abbreviated as asw u in the left rectangle so and when it finishes at work it reboots into the bootloader again and then the new bright and shiny application software is there to be started in case we were dealing with a double copy the first currently running application software should have an updater application basically which would then apply update to the standby copy so if we're working off of the software b it will update the software a and then reboot the system uh letting the bootloader know that it should now start the software a so now we're starting the software a and that's the bright and shiny new one and we also have a software b as a backup copy the second classification uh has some relation to the first one of course but it's concentrating mostly on the execution context of the updater so first of all in case a reboot is required to apply the update it can happen using a special kernel and RAM disk initial RAM disk combination uh which in the previous slide will be the sw u or it can be a part of the bootloader in which case the sw u here in the previous slide will be a part of the boot in case there is no reboot required it's either the common user space application or some trusted application running off of the trusted application zone um and that's much of unknown as of now but that's probably the best thing uh but we'll get to that also um despite the fact that double copy oda or in terms of android uh the ab style updates although they take more space they have some advantages uh that are very important in terms of security and that is self recovery uh you always have at least one working version that you can roll back to um also with some additional security measures and precautions it's possible to not download the whole package completely and if you have a trusted channel you can download it in chunks and apply it in chunks and that's important if you have serious RAM constraints okay so that's the oda part uh now up to the next abbreviation xip xip stands for execute in place uh that's a software technology that allows the code to be executed directly from persistent storage historically that was nor flash uh now there's also q spi type of flash that also allows for execution directly from itself um when we're talking xip we're talking kernel xip first and foremost and in this case and in case we have uh uh double copy lta method applied the flash that allows for execution would contain a bootloader and two kernels the first one and the reserve one and the application root file system databases and whatnot will then most likely be on an end flash or emmc or something else some other storage now that you cannot run programs directly from but that's actually not necessary uh it's going to be a little more complicated uh if we want the application file system to be xip at least in part then we should move it to the flash that allows for xip execution and this makes the design slightly more expensive but we do save on rem in this case um and yeah we'll pass over to other xip advantages as i've said the first one is that we don't have the need to deploy a lot of drem usually rem footprint is up to 10 times smaller than if we don't use the xip technology and there also are some advanced designs uh when no drem is used at all it's all only the flash where the xip partitions are located and then some srem rather small one to run programs off of for those kinds of programs that cannot be run directly from flash uh also what's important especially in battery powered iot devices that have to work in locations that are complicated to access is lower power consumption in idle state because for drem there's also some power leakage you have to maintain circuits that are used in drem uh you have to spend power basically in vain to maintain drem operation if we can reduce drem sometimes it can make a lot of difference for devices that have to run for months on a single battery charge and also if implemented properly xip can also give shorter boot time because you don't have to copy uh code uh from flash to rem and in case of a fast qs backflash it's also the faster execution with that said obviously if everything were just shiny and bright everyone would have been using xip happily and there would be no need for a talk like this so there are still obstacles that are native to xip technologies um and we have to list them too so the first uh the first one is that you can't really write to flash and execute from it at the same time uh due to the nature of flash writes uh if we issue a command to write a flash then it becomes non-readable for a time being and in this case execution has to happen from some other place be that rem or srem or maybe some code that is locked in cache that's the different story but still you cannot write to flash and execute from it at the same time also we are very used to having kernels compressed and stored on flash storage having been compressed on front that is not going to work with xip the code at least should be uncompressed because otherwise it cannot be directly executed and also all addresses for execution should be defined at compile time and compiled in because otherwise it's not going to work you cannot modify anything on flash so basically if we're running a certain kernel then all the addresses are known up front so there cannot be any randomization of addresses and this may be a security compromise so when we pass over to OTA and xip working together we need to keep in mind that xip as such can potentially lower the security so security itself becomes even more important so OTA and xip why do we talk about those two together what's the point and the point is that they actually pursue the same goals xip technology as its main features gives us smaller footprint for ram and faster boot and smaller footprint of ram means less idle power consumption which is important for remote iot and faster boot is important for automotive as well as faster execution because you really don't want to have your multi-media car system start for five minutes as the first android ones did and when it comes to OTA its main features are easy maintenance which contributes to remote iot a lot so if you have easier maintenance for iot devices that are hard to access that's an important thing that will simplify the maintenance quite a bit and OTA makes it more convenient and cost effective and customer friendly when it comes to update of the automotive stuff so there are two major appliances that both xip and OTA make better but once again not everything is shiny and bright because if we're doing xip that's a very fine technology in the sense that if you do something wrong the chances are high that you break something so fail safety is crucial because it's easier to break a device and also as i've said it's actually easier to do something that poses a security threat also if we're talking about memory constraint system and if we're not talking about memory constraint system then xip is not that important but if we do talk about memory constraint system then integral update may not fit and then we will have to deal with the double copy mechanism uh and then we'll go over to how double copy mechanisms the existing ones according to the classification that we did before how those mechanisms are actually working with xip and well spoiler not that well so um gonna skip this slide we're gonna back to it so um the user space OTA which is basically what most of the f os s updates do uh is very simple when we don't consider xip so if we implement double copy user space LDA it's basically a common application that does some security checks and then updates the kernel and the user space which are not active at the moment and then you reboot and you reboot into the new kernel and new user space but in case we're dealing with xip as you can see on the picture both kernels are going to be located in flash so the kernel a cannot execute during the kernel b update even though kernel b is itself inactive so you have to disable interrupts and preemption during every single flash right and it's going to cause significant interruptions of service and if we're dealing with the user space that's also xip then we cannot execute directly from flash using the updater too so both the updater and all the libraries that it uses should be copied to rem before we can start the update and this makes the whole thing quite a bit complicated then if we go in a couple of slides back and discuss the init rd based over there updates is going to be looking a lot brighter because all this updater stuff it doesn't have to be xip you know the kernel plus the ROM disk the ones that we were talking about in a single copy update description they don't have to be xip so we just load them into memory you know you as a bootloader on reboot you load them into memory pass the execution to those and they do the update so this one actually works with xip pretty well but on the other hand using single copy updater with xip is a potential threat so I think we're ending up in some kind of weird state so there's something that works but we don't want it to to be deployed and there is something that we want to be deployed but it doesn't seem to work bootloader ODA just for the record is pretty much the same as the kernel plus init rd but a little worse I'm not going to spend time on this if you want we can take it as a question so then we go back to user space ODA and we know it's not that good for xip system but if we go one step further and implement updater well at least for ARM we can do that if we implement updater as a trusted application running in trust zone then it's going to work pretty well because you pass the whole execution to this real updater running in a trust zone and it creates some additional security creates some additional safety because it's not easy to tamper with that type of software and at the same time it since it's been clear that it's going to work from RAM the situation where you have to copy too much data to RAM to be able to run a certain application is not applicable here because in this case the updater plus the trusted OS will be self-contained the problem here however is that there is no free and open source solution that uses this method even though there are some in progress we are working on this but we're not there yet okay so summarizing everything what can we say xip execute and place technology can add value to systems using OTA solutions but it also has complexity that has to be taken into account and also xip puts certain specific requirements on updaters which may not be easy to fulfill sometimes existing free and open source updaters therefore don't play that well together with xip secure updates within the trust zone should be working fine but there are no known solutions for that yet so we have something to work on basically and yeah that's it my presentation is over so if you have any questions please yes yeah that's that's right you still you run the updater in RAM in case of trust zone but you load it and it's self-contained so it's easier to make it work because you don't have to consider all the possible links to libraries that actually reside on an xip file system while trust zone gonna gonna running something in trust zone gonna consume more power but um it's not i mean this is not a frequent case when you update for an iot device let's suppose that there are there are devices you know somewhere somewhere in between the ceiling and and the floor you know that hard hard to access and and those those devices uh supposedly get updates like once per year yeah but but there's a spike but that's okay any other questions and thank you for attention