 OK, I think we can start. Thank you, everybody, for attending to my talk. Just two notes about me. My name is Stefano Babic. I'm working for Denk Software Engineering in Germany. And U-boot maintainer for NXP processors for U-boot. And I'm the author of Software Data is the topic of this talk. And of course, I think I have not to say something about why it's important to update an embedded system. We have not to hide our hand in the sand. But believe me, there are still a lot of people, a lot of managers, that I think, why do we have to update our system? We have tested. We have developed. It works. This talk is not for these kind of people living in Alice in Wonderland. It's just for tough developers who choose the red pill if you have seen the robotics movie. So first of all, I want to show you because a lot of the time was asked, what I have to do to import or to use software data in my project. I think it's the wrong question because really should be the opposite. So we have an embedded system, a lot of variation. And important in the system is the application. So an update agent should not change the behavior of the application, should not add further dependencies. So really, software data is not an out-of-box tool. It's more a framework. And this framework should be adapted for each custom project. So I have parted some real examples of real products. This is one of my favorite products because of a project. Because when my parents or some friends ask me, what are you doing? I say, OK, I'm a software developer. I work with open source software linux. OK, you're doing something with computer. But this is a coffee machine. It's easy to understand. And everybody knows now, OK, I'm a coffee machine maker. The coffee machine has Linux, of course. It must be updated. But it's not an overware update. Why? Because the machine has milk. So it must be cleaned very often, as it does not smell very well. So there's a technician. Sorry. Someone is bringing new coffee, cleaning up the machine. But he has no idea about the computer. So the manufacturer doesn't say, OK, you cannot give him a computer and try to set IP address or something like that. It must be very simple. So the solution in this case is a software update is running. Check for USB stick. USB stick is mechanical. So the technician can insert in the right slot maybe not at the first attempt, but he can. And press a button. And the result is just a smile or a grin. So very simple. I have to drop all logs. It was nice for me. But I say the technician then is confused. Another example is these are 3D cameras. So they can understand the distance from objects. They are using industrial application. And we have a technician. We have a technician with high competencies because he has to understand which software should be installed on each camera and which configuration should be downloaded on each camera. In this case, the software update is running. There is an integrated web server. And the technician push the software, the new software, to the target. A different use case is in this case, we have this company makes something with power energy. And this one is a controller. And they sell all around the world. And in most cases, in countries where they are, I can say bad internet connection or quite bad no internet connection at all. So we have a GSM modem. And the connection is broken a lot of time. So in this case, the key feature is that software data does not restart to download the image when the connection is broken. It must be able to resume a connection. The connection can even take a long time. It's not a problem. But it should come to the end. And this is just possible if the downloader is not restarted again. Another approach, but I forget to say, in this case, the manufacturer put the software on a server. But it does not need to understand if all devices was upgraded to a new software or not. It's just I put the software, the device should take, but I don't collect. Another use case is using a back end. The back end or deployment system has several names. This is the Hockpit deployment server. It's an open source project under the umbrella of Eclipse Foundation. It's mainly driven by Bosch. So most of the developers are Bosch. And this one is a Bosch device. It's a master from Bosch. You can buy, at least in Germany. I found in a lot of discounts. And software data is running on the devices. And the water they are using is also some additional feature. For example, software that collects data from the device sends to the server. And depending where the device is installed, a different software is sent. They have a different software for UK, for Germany, for other countries. Another use case is when something goes wrong. So we have our device. Our device seems broken. So software data is used in rescue mode. So the footprint of software data and all of what is needed is very small and can be put also in small storage. This makes it possible to start the device in rescue system and put the software again. So at the end, which are the requirements we need from our update agent? So software data is an update agent. So of course, I can say we have three requirements. And then the update agent must be reliable, then must be reliable, and then must be reliable. So how can we reach this? First thing, an update must be atomic. So it cannot be that just a part of the update is installed and the rest is not installed. Or just a library is updated and the rest no, it should be a nightmare for the support, because it's just like as the customer calls and ask, the update was stopped, whereas, for example, libcool was updated. I don't know what is and something else no. The support, people have no idea, because it's untested. We have always with an embedded system, we're going from a release A to a release B. And there is nothing in between. Another big feature, so another feature that the update agent must have is regarding the verification of the update package. So it should be verified that the update package is coming from a verified source. Not that everyone has changed something, push to our device and our device is working. Of course, this requires much more to investigate. It's not just the update. It's also part of a secure boot. But of course, if we are not a secure update, even if we are a secure boot, we have not closed the circle, because we update the system and after the new boot, the system is broken. We need an attended update. So of course, it's not like a PC. There is nothing in front of our device. The system should be able to manage error, to have a failback, should be able to update all common parts of our system. So usually, we have bootloader, kernel, rootfire system, but not only. I can say we have FPGA. FPGA can have even storage attached. And an update agent should be able to update also what is specific for an embedded system. If something is going wrong during an update or the application is going up and check that everything went wrong, we should have a rollback. Rollback is not in all situations, but I will show you later a NAB approach. And the system should be able to restart the previous version if something is going wrong. And of course, it should take as much as possible of a most important low, even if it's not yet always possible. So as I said, there are different parts that we have to update. So not only what is common, but even FPGA, microcontroller, configuration data, calibration data, two common approaches. This is our most used approach, but the software data is not limited to this approach. I won't show you, but really, you can have much more of this. One approach is I call it single copy. Someone is calling rescue system or recovery, whatever. The thing is, you have a running system. In some way, you have a trigger, and you know I have to update the system. You switch, you reboot, and you communicate to Rebootloader that you need to upgrade. Rebootloader then starts a rescue system, a small system, for software data is in a RAM disk. This is running. This starts to flash everything, so get the image, flash what it has to install. And if everything is successful, reboot again and start the production system. If something is wrong, there is no problem, because it's like a transaction. So software data sets a transaction when it starts. If there is a power cut, Rebootloader is coming again, it knows there was a power cut. I have to restart the software data software that is running again until it is successful. The other common approach is a double copy, or AB, or whatever. In this case, we have two copies of our software. A big advantage is we can update Rebootloader offline. Because on the rescue system, we have to boot. Our application is not running anymore. And during the update, we have an offline time. We have not doing a double copy, but we have to reserve space for two copies of our software. So in some way, there is a trigger. Software data is started. And get the image and the flash. They stand by copy. When it finished, everything was OK. Reboot the system, switch between the stand by copy and the production running system. Rebootloader is in form of this and can start the other copy. And all where the system was running before is becoming the stand by copy. I said these two are the most used, but I can combine them. For example, I used this in a lot of projects. I mixed six. So I have a rescue system. This rescue system is very small, so the footprint is small. And they can put all together in an 8 megabyte flash, SPI flash. I use this also in the factory. Because the problem is also, how can I put all my software for the first time? And in this case, the SPI flash is programmed once with a SPI programmer. Then all devices are coming for testing. They are simply connected to the network. Software data is running, let's say, in factory mode. So it knows I'm running in factory mode. It connects with a server inside the factory, takes the right software, make partitions, set everything, and at the end we have a setup like a dual copy. At the end of the processor, there's a switch again, so it's not anymore in factory mode but in production mode. And for the live product, it's running with an AB approach. But if something is really going wrong, for example, partition table is corrupted, the system is not broken because it can still run in the rescue mode and can get a new software. So something about the software data. I started the project at the end of 2014. It's GPL version 2. There's a library. I will talk about the library later. This is less GPL. It's to be linked with custom code or proprietary application. An important point, I delivered the software data together with the BSP. That means, generally, an application developer needs to have a system running. So bootloader, kernel, and your building and everything should be already running. Software data is part of this. And the thing is that everybody, any application developer, is using software data to update its own target. Before having this, if there are 10 developers, they will find 10 different ways to update the system. Some of them will use the Uboot console and use the TFTP, get the software, flash everything, and so on. Some is running on Linux, and then copy with Linux tool. Something using something else. The thing is, this is then part of the continuation integration. So everybody is using this during the development. If there are some bugs, they are solved during the development and not later. And I can tell you that I have practically no bugs reported by the field, because all bugs were already discovered during the development. From the first approach, at that time, I just published. I just sent an email telling that I have started the project. The project was not yet completed. It's grown. And I just look in the history. There's at least 40 contributors that at least send one patch to the mailing list. I set a release cycle of three months, just because Yachter has six months, and they want to be quite faster. So I can decide which version should go into Yachter release. It's one of the listed Yachter data. Here in the link, you can take a look there. Of course, the other updated tool. And I try to ask how many are using software data in the mailing list. I really don't have an answer. But checking or googling, I have seen that a lot of projects are using now. So which are the main feature? An update must be atomic. And the reason should that update is atomic. It works with all common embedded storage. So MMSE, SD, Ronan, the UBFS, or some strange problem with Nender and Hemming, for example, and so on. It's a single image for multiple devices. So you can deliver a family of software, so a complete release for different devices with just one file. Of course, it's power of safe together with bootloader. There's a hardware software compatibility check. So it's ensured that the software is for your board, for that revision of the board. And for the revision, it's not listed. The software is not installed. Interface. I've shown there is local interface, USB, SD, or whatever. Remote interface, there's an integrated web server. Or it can pull the package from an external server. There is an integration for backend. Really, there is a general interface for backend. But currently, there is just one implementation. It is for the Oak bit. And also, as interface, there is this customer-client interface that means if you have a proprietary application to talk with your device. And this is also responsible to get the software. You can simply link your application to a library, that is LGPL, in this case, to communicate with software data to make the installation. Other feature, there is an integrated Lua interpreter. I choose Lua because it's safe, it's very stable, and it's very small. So I have the Lua interpreter even in the rescue system. We have support for current embeddable system. So Yachto, I do myself. Someone else has done the support for build route. There is a general interface for bootloader because you have seen we have to talk with a bootloader to understand what should be done. Currently, there is support for U-boot and for GAB. And there is a very small footprint. Together, with a bootloader, we can have a fallback. So the previous version is running when the new installed version cannot go until the end. It's both an embeddable update and a file update. There is a general interface to report the status of installation. This is important, for example, when we have a display and we are upgrading. And we have to show, for example, we have six steps. You are now third step and 30% of the step. This is possible. I'm using KBuilder to configure the system. And another important feature is that there is no temporary copy at all. So the package is simply streamed, or can be streamed, is also a configuration option, can be streamed from an external server directly to the storage without any temporary copy. This means that data is uncompressed, encrypted, verified, chunk by chunk, without copying somewhere. Regarding security, I can say I'm using the Libcurl library. So HTTPS is probably nothing. That means I can use all protocols supported by Libcurl. So thanks, Libcurl. And there is the possibility to set certificates for server and client verification, which is, for example, used for Oakbit. Images can be verified using a public key. Each artifact can be encrypted. So you can also mix. Some artifacts are encrypted. Some others, no. And this is quite a new version of privilege separation. So have you seen? I have multiple interfaces. And these interfaces are communicated with the internet and taking the new updated package. Of course, if I run these processes on the route, it's very bad. It's possible to buffer over for something and a hacker or some malware can get route privileges. So this is split. Just to install it, because we install it must write into the hardware as a master run on the route. All other processes are configured and can be run under another user and group. So first of all, we need a compound image. So we have a producer from Yachto, several artifacts. We have a kernel. We have a root FS, maybe something else. And we have to bind all together. I've chosen a CPO format because it's very simple. Really, I don't use a CPO. I just use the header. This is enough to pack all together. The first file is the most important file. Software description is a meta file and is a description of a release. In this file, I will describe which artifacts must be installed, where, and how. This file is passed. Of course, I have two, a typical two parser. One is a LibConfig. I choose the LibConfig because I like this syntax. So it's just a DTC or changes DTC. A lot of people comply about this because they say, OK, but we have no LibConfig on Windows. I try to say this is a mistake and they have to install Linux. But I have no success. So there's also a JSON parser. It works exactly the same as the LibConfig. But in this way, they can generate automatically with their own tool, of course. They can generate the software description for the release. There is another option is with a custom Lua. So you can call an external Lua parser. And in this Lua parser, you can decide your own rules. This is the current architecture. So you see there's on the bottom, there are processes working or assessing the internet. There's, like a bus, really is an IPC based on a Unix domain socket. The processes are running with other user. They send the update package to the installer thread. The installer thread takes the package, it's passed to the parser. If everything is OK, it's passed to an ender manager. Ender manager is a router. So decide for each artifact who is responsible to install. In this way, I can also extend. I have a handler for UBFS, for ROW device, for MTD device, to set the boot environment. Remote is, if I have some outside my system and so on, I have also the progress interface to communicate with the standard processes, which is the status of the installation, and of course, Lua interpreter, and some utilities. I just said that I use a K-Builder, which means you can run a bit bigger, me and you see a menu config software data to configure it. And this is the structure of software description. So we have a general header. Then we have something board specific. So I can define for different board a different entries into software description. And then we have section. We have section for images, we have section for file, we have section for scripts. Here missing another one for the bootloader. In each of them, I can write what I have to install. So really, we have the most important thing are three. So we have who. So that means the file name, where, and is the device. So I decide where it should be installed, and how is the type. Identifies the name of a handler. This is a case when I have two boards in the same package. So I can say first board, you are in a HMI. Take just the software described in the HMI for another board. Tape A, I can just send or inform this board. You have to take just this part. Another way to search inside the description is using collection. This is my preferred way to implement a double copy. I know there's other way. Some sets link, and I've adjusted the description. I was asked, which is my preferred way, this one. So I have collection. I have a complete description for each copy. So for the first copy, and you see this is an example with Ubi. In copy once, I have that should be installed in the volume root FS1. Later, there's the setup for the Uboot environment. The same is for copy two, but just in another volume. And when the system is starting, I know which is the running copy, the stand-by copy. I pass this information to software data. In software data, check this inside the software description and take the part related to the stand-by copy. Handlers. Handlers is the way to install a single artifact. As I said, there is already a set of handlers in software data, for example, for flash devices, Ubi volumes, HRs, or a terrible road device, environment script. There's pre-installed, pre-installed scripts. The remote handler is for if you have an only installer, this cannot be GPL, because if you link to software data, it becomes GPL. The remote handler is like a way that the software data works as proxy. And this artifact is passed to someone else, to another process. But of course, the rest of the part was already done by software data. So this artifact was decompressed, decrypted, verified, and then passed to another external handler. Another feature, this comes recently. The software description was just declarative and now is also executive. So it's possible to set a hook inside the software description. And they are calling a function in Lua. This function should be part of the software description. Because the software description is verified, it's also a way to change dynamically the way the system is updated without upgrading software data. Rollback, I think this feature is together with a bootloader. So we need a boot counter in the bootloader. When the boot counter reaches a free shoulder, the bootloader is able to switch to the other partition. And of course, when everything is running, the boot counter must be reset. Image, I said, can be signed. This is part of a build process. That means everything is in this meta software bit. Is the layer I provide. You don't need to compute hashes or to sign yourself. It works if the private key is on the file or the hardware key. Image is signed, is sent to the device. And on the device, you need the public key. With the public key, the image is verified and if verification is successful, it's installed on the target. Same thing with encryption. But with encryption, I just use a submitted key. There's no support for other, at least at the moment. You can decide which part should be encrypted. So if you want to encrypt just some artifacts or some other, or you can mix not encrypted with encrypted, a single artifact is encrypted during the build process. Yes, through is generated. And transfer to the target. The target knows because in this software description, there are some attributes I have not shown because I wanted just to tell you, which is the most important thing, that there's a list of other attributes. One of them is the encryption. You can say for each artifact, if it is encrypted or not. As well as you can even say, tell us of the data if an artifact is streamed or not. So you can stream all artifacts or just say, OK, this part is streamed. Another part should be extracted. And then installed, each artifact can be configured differently for the other one. So software data has a Suicata mode. I think the name comes at the beginning because I take the embedded server from the Mongoose project when the Mongoose project was completely GPL. And Suicata is also an animal quite similar to a Mongoose. So I think I have to stop with this. And this Suicata daemon is really a way to attach different back-end. So it's a daemon. It's running an interface, a generic layer. And we can have different back-end to communicate with server. Currently, and you see above, this is the Orbit server. Orbit server has a restful API based on JSON. It's planned to have multiple back-ends in software data, for example, adding Mender or some other. It's just a thing to update, so to increment and to add a specific implementation. Build process. I provide a meta layer for that. I know that someone else provides support for build route. I have not used, so I cannot say anything more. In meta software data, there is support for building the complete image, including signing, including hashes and so on. So the image is automatically produced by the build process. And at the end, I know that some customers build also some connection between their Jenkins server to push the image directly to the server. Or I don't know if they are using the Orbit server. Another way is to put the file in the Orbit server. This is when taken by the device in development. I just show you what's going on in detail because there's a lot of text and it's quite boring. The most important thing is to create your own SVU. There's a class. The class is SW Update. You have to inherit this class. Of course, there are some new variables, as usual in Yotto, as this software update image. And in this, you can simply define which artifact should be part. Just one minute regarding the roadmap. What I want is to extend the community, to use the software data as an update again, I know I'm late, to add dynamic lower handlers. So you can change the way to update the system. And you can push new handlers that are in your SVU. Support for other keys. A bigger thing is the delta update. So to provide binary data. Another thing is to chain handlers. So now there's a handler for each artifact. I would like to have, for example, a table go to a raw handler and so on. Adding new backends is also a very interesting thing. And add a new model with that. So I know I'm very late. I plan to have a demo, but I have no time. So maybe some questions. Do you need a specific configuration in Uboot to be compatible with the software update? The thing regarding Uboot is you need to be sure that Uboot and software data are seeing the same thing. So first, you have to set, you need a library. When you link a software data to a library, this is provided using meta software data, or you have to provide your own. And this library is generated when you are generating the Uboot. This is important because Uboot has a default environment. And it's not stored on a flash. You can also run without any environment in the flash. In this case, software data has no knowledge. So you have to link with this library. This library is produced with your Uboot machine and contains the default environment. So even if there is no environment stored in the flash, software data is able to change the environment and to save also for the first time in flash without breaking anything. OK, thanks. How do you make sure that the Uboot environment is not damaged by power off during writing? By? By power loss? OK, because we have two copies of environment. So Uboot has a config redundant. You have always two copies of environment. This is also something that should I say. You have seen that in the software description, there are some parts regarding the variables. These variables are not set in a sequence, but software that takes all of these variables, compute the new environment, and then store the new environment. So that means there is no problem. There is not because a variable is set, and then there's a power off, and the environment is in a stranger state. What is the footprint of the software update image? What is it? The footprint of the size? The size. The example was a real example. So that means I have a project where everything fits a bootloader, environment, a kernel for software data, and a RAM disk, a compressed RAM disk, everything in 8 megabytes. Thank you.