 My name is Marcel Holtmann, I work for Intel, and the talk will be about what I've done as a small hobby, sometimes last year, see how much we can actually swing Linux and run it on small devices that have actually limited RAM and limited size of flash, and what you can do to rethink this one. So a little bit of background, and you might realize that then why I've picked some of the choices that I've made when trying to this one. I've been working on Bluetooth since 2000, it's an awful long off time, and that technology keeps evolving and moving into the IT space, so it's becoming more and more important when you have power constrained and small devices. I maintain the Bluetooth Linux stack since 2004, so that has been over 10 years by now. While joining Intel in 2007 for OTC, the open source technology center, I've created Conman, a connection manager, a phone or complete cellular stack, pack runners, maybe not known much, but it works in the background for you to figure out your proxy support. Everybody running in an enterprise environment appreciates that your proxies get figured out automatically. A couple of years back, we've started open sourcing L, which is something like a G-Lip library for small embedded systems. L stands for embedded Linux library, and also we'll give a quick intro, a really short one, into IWD on your wireless daemon. The talk will be given next week at Open IoT Summit, so you can see it there, and we will be opening sourcing there as well. As I said, I've been working on Bluetooth way too long. I sit on the ITACCHER review board as Intel's counselor. I share the internet working groups. Some of the examples are picked because we're working a lot of getting IP and IP ready into the IoT space. While I can't re-talk about this, but the Bluetooth SICK announced this, there will be Bluetooth mesh technology coming next year as one of the big things where point-to-point technology changes its focus into a mesh-like technology and where that becomes more important than you can deal with small devices. The focus of today is really getting why not switching to an RTOS while stick with Linux, how far we can drive it. I'm going to start this off with a quote that Jonathan Corbett did at the Colonel's Summit in Seoul last year. Pretty much, the fear that he laid down was, if we can't shrink Linux far enough, then we're missing out on the IoT opportunity because at that point, Linux doesn't become relevant anymore and everybody has their own thing, and something else we're going to replace it. So after he gave that quote, I said, okay, look, what can we actually do to shrink it, where are our limits, where does it make sense, and where it doesn't. And so how far can we actually get? I know a lot of people don't really care. There are the big data center machines, the big iron machines, and God knows what. And they have RAM, pretty unlimited RAM. We're talking on gigabytes and so on. But in some cases, we're talking about megabytes and below. And we still want to run Linux. It used to be when you had a 2.0 kernel. If people still remember that one, that was pretty easy to run on a small system. You need to put on a floppy disk if you still remember these ones. Now the question is how far we can actually get. And while this is not a scientific scale, I ended up with something, if you're larger than 512 megabyte of your flash or RAM consumption and you're even considering doing anything else than Linux, then you're just wasting your time. I mean, you can find an RTOS that will support you there, but the reality is you can just easily install Linux and you don't even have to do much work. You don't have to tweak much. You will find an off-the-shelf, fully-packaged Linux distribution and you can just use it, put it on there, get on with your life and focus on what you really want to do. Write your application or get your use case done or anything else. So if you above that one, forget about this. Less than a megabyte, less than a gigabyte, really dead simple. 32 megabyte to 512 megabyte, it becomes a little trickier, but it's still also easy. And the reason why that number is on there, if you don't get a dedicated SoC, the smallest RAM chip you might gonna get will be actually probably already in the 32 megabyte size, if you're unlucky. So 8 to 32 megabyte, in some case, it becomes a little tricky. You have to do a little bit of extra work to get your Linux distribution that small, so it actually runs. But same thing here applies. You might actually not even have the choice between choosing 8 or 32 megabyte of RAM because the smallest RAM chip you're gonna get is 32 megabyte already. On the lower end spectrum, if you go less than a megabyte, Linux is really, really tricky. You have to do a lot of tricks. You have to do a lot of dedicated work. You really have to want to use Linux to actually get this going, and picking an RTOS might be actually an easier way to go there. And then you have the rumbling zone in between. There's really a couple of ranges where it can go one way or the other. If you want it to megabyte, who knows. Maybe Linux, you shrink it down and it will work for you, or maybe you're just gonna stick with an RTOS. 2 to 4 megabyte, Linux becomes a little easier feasible. 4 to 8, you pretty much have a choice on what to choose there. But that's a rumbling zone, and the RTOS has come from one side. They're gonna grow, and that's the most choices you have to face with when you actually pick an RTOS and you end up with needing a feature that it doesn't have. You actually have to build this, and then you look at Linux while you already have the feature. Best example of these ones are when you have IP support. You need an IPv6 stack that you need support XYZ feature. Your RTOS doesn't have it. Linux already has it. Where do you spend your time? Shrinking Linux or building that feature into your RTOS? Same goes for something like Wi-Fi connectivity. Linux already has it on your RTOS. You might actually have to build it, or maybe it doesn't work as well as you think because only few users actually use it there, because otherwise they're focused on other radio technologies like 15.04 and so on. So as I said, there's a rumbling zone. I think it's gonna shift a while. Earlier, I would have thought the RTOS will shift further away and will grow up into the larger space when it goes over 8 megabyte, but I think Linux has made a couple of strides to get towards the smaller space, so we will see where this goes in the next couple of years. While my scale was a little bit drawn down from what I've seen and experienced, there's also the fun one that Tim Burt put down in October last year. I called it his Serial Scale, and it pretty goes in the direction that there's a steady decline for the price of silicon. The silicon will get cheaper and cheaper, and he went out and actually spent a lot of time looking at it, and he found that the cheapest Android phone is $29. That's about a year ago now, and the cheapest computer was pretty much $9. That could run Linux. I think the prices will have gone down, maybe not as crazy as significantly, but we're already pretty cheap, and I was at a conference last week where the speaker gift was a Raspberry Pi, so they basically hand you some of these computers if you don't run fast enough away from them. So the estimation from him was that the cheapest Linux capability with an MMU, which is the important part, otherwise you have to jump to a lot of hoops that might not well pay off, is about $3, and that was a year ago. But the basic assumption is that if you want to get a computer in a serial box, like the toy thing, you need it for less than $1 with SoC, display, battery, and input. So it needs to do something useful. It probably has to display something, otherwise they throw it away right away. You need to fit the battery in there as well. So that's the price where this has come down, and once we're actually there, then Linux has reached the stage where it became part of like a throwaway toy kind of computer systems. And I was looking how we can get there. Well, the price of the silicon we have to wait out, but actually the further we train Linux, the easier it will be to not actually pick an RTOS and pick Linux. So that was a year ago. I haven't gone around looking up the new numbers for this year, and I don't remember Tim has been doing it one. But if I ever run into him again, I will ponder him about this to this again. It's an interesting talk. You can find it online. The slides are available. It's really interesting what he has done there. So I know that a small, I can't fit a Linux into the rambling zone that I mentioned earlier by just taking a kernel building and then putting on the hardware. So what do we actually want from this device? Because I wanted to pick a clear focus in where we can actually make a difference. So for me, if I imagine a small device, an IoT device, the first thing it has to do, it has to measure and access sensor data. So a thermometer or any kind of devices that measure any kind of things that you want to do or part rates or something like that. To make it useful in the IoT space, it most likely has to get its data that it's censored or to somewhere else. So we need to make sure that we have that support and it can run for a long time. Mainly with these IoT devices, my example is some people want to take this, build this ditch in the ocean for the next three years and then pick it back out. You can't go and dive down and replace the batteries. So these kind of things you have to consider. And every time I have these devices and I have these devices I have these devices in front of me, I will end up with most of the times of three categorizations. One is the really dead simple sensor and that's the Bluetooth hard rate monitor thermometer. Pick your favorite bed of Bluetooth device that does nothing else except measure and you can access the data over Bluetooth low energy. Really simple. Tons of out there really dirt cheap and you will find everything from find me tags to anything else. The second category gets a little bit trickier because of course on where you actually have IPv6 support in your device and that's generally an IP enabled note and then you play with Bluetooth low energy or 15.4 or threat as a technology going on to actually build your networks and that's also where mesh technologies come into play. A lot of these ones are also used with home automation in some countries where they actually run IPv6 networks for your home automation. And then you have the bigger ones where you have the gateways or your router or Wi-Fi access point or anything else and we've seen that some of the companies besides Wi-Fi are now put threat so 15.4 radios and Bluetooth low energy in there because they see the market is growing where these ones become the fundamental home automation hubs for you where you will bridge your phone into it or your other network and there's a couple of companies having these ones used as home automation hubs. So the gateway side, I don't think this is a problem. We have been running Linux on access points for the longest time adding a couple of more radios and supporting them is pretty easy and a lot of companies have done this. I've seen even NAS boxes for your home storage systems where you plug a Bluetooth dongle in there and all of a sudden it supports Bluetooth and they haven't really failed to do with it but it's so easy to do with it, just do it. The IP notes become a trickier and I will have a slide on this one later where unless you're as large as a gateway that's a trickier and the Bluetooth sensor is currently really not the space of Linux. It's doable if you have enough RAM but I will go through this exercise to know what it takes to actually bring this down a little bit so we could actually build a Bluetooth sensor running Linux and what sizes we get out of this one. With the size I had in mind there was obviously no way we're going to just take a Debian Fedora or whatever distributions booted and hope for the best. That's way too big and you don't really get there. So I ended up with picking a couple of ingredients and I just put the logo here. Most of you have ideas what projects are behind this. Any idea? I normally just really get that far. So on the right is the Muzzle C Library. So we needed a C Library for obvious reasons. I chose Muzzle because it's most standard compliant. We've done a lot of fixes for upstream projects so they compile with Muzzle and follow the standard really strictly and it also shrinks itself really, really small. There's more optimizations that are possible but by default it gets really, really tiny. And I looked at all the other ones that work as well but Muzzle seemed to be the most reliant one. Obviously we wanted to use Linux so we're going to use Linux kernel and while gummy boot strictly speaking doesn't exist anymore it existed when I did this project about a year ago. It's now consumed into systemd boot or whatever that is called now but the concepts are pretty much the same. The two lower ones are L, it's our own embedded Linux library which basically packages together things like main loops, hashes some crypto functionality etc where you would normally then take something larger like Glib or some other bigger libraries. It's nicely compartmentalized and optimized for small systems and we have been releasing this for about two years now and we finally started making some use out of it. And then I wanted to use a Bluetooth sensor so I picked bluesy and it's like look what are we going to need to do to make it smaller and integrated. So with this all in mind I wanted to take this and put this together and see how to build this differently and now it becomes the fun part. I actually decided to reimagine Linux completely differently. I hope this is recognizable. So I didn't focus on actually getting a boot loader in my boot loader was okay we're going to start with a UFI based system and go from there and with a UFI based system you can actually boot Linux out of it without actually installing something like Grab or any other boot loader. That's where Gummy Boot comes into play because it provides you an EFI stub that you can then use to bootstrap the Linux kernel. So fundamentally you're building a PE so Windows executable and booting this one. So you need the EFI stub that comes from Gummy Boot and then you link in a couple of operating system information that's purely informational and then in the front line you link your BZ image into that binary and then you link it into your binary as well and then you link this correctly together and then it becomes a PE executable that you can pretty much put on any UFI capable system and just boot. So UFI will find the PE executable it will jump to the Linux kernel it will jump to the in a RAMFS and then execute in a RAMFS. At that point still a little bit different forward since it's still how Linux would be doing it anyway. The difference is the in a RAMFS is actually not a full Linux system it's just a single process it's PID1 that is dedicatedly built to just do one thing and in this case for the example that I chose be a Bluetooth peripheral so we having Bluezy playing the PID1 the in a process. Nice advantage with picking UFI first award works on nicely on AI 32 and 64 it will also work on ARM it's just I work for Intel so obvious choices for Intel based systems but it will also work on ARM if you want to build it this way. The really nice advantage is you can also sign this image and use the secure boot facility of UFI and then you can just have a secure boot all the way through without having to do any extra work on this one because at the end of the day this is a binary so you only need to sign this once and you don't have to do any extra attestation to execute it. That's also the reason why the command line is fused into the image itself so you can't mess with that one. So to recap what is this one if I step from gummy boot or these days system reboot a label for your operating system that's pretty much optional but nice to have in case you want to put two images on these where which you would do for upgrade path or if you have two selections that you want to oh I want to upgrade to this one then you can use the authentication string which one got actually booted. The command line as I said if you want to sign this as important as it's in there then the kernel itself and then the user space. If you want to try this out you put this on inside EFI boot and then pick the right EFI magic name it's a FAT32 partition so you can put this on a SD card use B-Stick you choose it, you stick it in a UFI capable system for example Galileo or MinoBot Max and it will just boot. As I said earlier software update is just by replacing this one with a new version UFI already has a facility to boot a different system different image if it's available and fall back to a previous one so you can have the fall back cases nicely covered. Big advantage to secure boot you don't have to do anything you just have to sign your binary and UFI will take care of the rest of you. So, PID1 a little bit back on the PID1 so to actually get this down in size I had to redo PID1 so I couldn't just say okay, I'm going to take system B that's already 2 or 3 megabyte out of the question then I need the normal standard C library that's all not going to work so if you want to take the smaller and you know what you actually after then you can do this really simply by just using your right C library your right helper library and then the task you want to do have a project that actually emulates PID1 and PID1 has to do a couple of extra tasks that you can have it mount automatically or mount manually you don't have to do any extra work there but there's a couple of few device links that you need to create in the user space so we had to do this in Bluezy now when it is a PID1 you have to mount a couple of virtual file system a few dev tempfs etc to get this booted up otherwise it's just going to fail because it misses the things you have to set up the hardware normally Bluezy would only set up the best setups we wanted to have a watchdog since there's nothing else providing a watchdog you have to do that as well otherwise you can't really reboot your system in case something really goes wrong and then you need to start the Bluezy operation the interesting part is really that we actually don't have a file system so we don't have anything else so it stays as dev root and dev root is just a tempfs that the kernel has in the ROMFS and they're mapped directly with persistent data and here's where another trick from EFI comes in that you can use EFI variables to actually have persistent storage and in this case this is fine because what we need to store for these cases are we need to store a couple of states your bluetooth address or some persistent state that you might have fused in there can be only read only but it is enough to actually get you going in a way that it works out nicely so pretty much you boot on a FAT32 file system just get going create your extra links mount your file system and then off you go be linux system while you all run in PID1 with your main operations so from the idea to actually getting this done I actually did this initially in QEMO mainly because I was lazy and didn't want to tingle with hardware too much until I figured out that this is actually possible so it boots in QEMO the bluetooth controller that we needed is forwarded via serial port capabilities that you can take a USB dongle tick in your desktop and say look please forward this one as a serial device into my QEMO session and then you have your controller there so it turns a USB dongle into a UART controller and you can then just work with it works pretty nicely so in QEMO this ended up in this configuration that I just showed was less than a megabyte of size if I built a 64-bit binary and in the end when lazy I built the 64-bit binary since I run a 64-bit fedora OS so it's like okay I don't want to cross compile this I'm just going to see how far I'm going to get and I deal with using 32-bit later on if I have to but I already got pretty lucky with this one and say look this is already kind of small so it can only get smaller from there by choosing 32-bit but I actually wanted to try this on real hardware so I took a micro SD card in one of my minobots now you need a little bit more since the hardware is not redesigned for being small but even with taking USB and PCI support the controller is working and then take a physical USB dongle to put it on then actually use it it ended up in the 1.3 megabyte of size for a 64-bit binary I was going to build the 32-bit but at that point the flashing of the 32-bit bios into the minobot max was a little bit more trouble that I had time for that's up to some of you where I might want to try this so at that point we had a simple peripheral that can read a sensor data out and provide it over Bluetooth LE and the whole stack was available and we can use it as a 32-bit would have gotten us even smaller while this one all worked and was kind of nice and was kind of good to see that we actually can get really small if we find a dedicated use case the question is how we can go even further because this was not trying to actually change anything in the Linux kernel this was pretty much taking it, building it shooting right in greetings, putting them rightly together and then see where we get so what I got at is look if we want to go this small we need a dedicated PID1 and for that one we had to redo Bluezy in a way that we actually had to take things out so Bluetooth Classic support is pretty large especially in Linux since it's complicated and we support so many configurations for the main case because the desktop side had to be all mighty and magic and it did with a lot of crazy corner cases so we took that out and focused on low energy with this from a protocol a lot simpler but we never actually planned for supporting such a configuration so Bluezy was not really ready for doing this one so we had to actually build switches and configuration options to actually allow us to take the Classic support out same as if you run as PID1 you don't need any extra IPC you're the only process in the system you don't talk to anybody else except the kernel you don't really see out and make sure that we actually don't include any extra code sizes because of that and mainly is taking Diva support out using L and muscle were designed for this purpose so they came in really handy when you compile a static PID1 the next one was fundamentally okay we have to start off with a kernel that is already small so you need the tiny config kernel and you pretty much can just start with it's already really small and then you select what you extra want or you even deselect some things that these are providing a little bit later on we finally got the support for Execute in Place if you want to build a real piece of hardware using this kind of system you really want Execute in Place and not need the extra RAM that Linux needs to unpack the in-net ROM FS that's something you really don't want to incur so you want Execute in Place possible patches are there linked in the LWN article another interesting piece for size optimization is the LTO support and when you can do a link time binary writing and code compression excellent presentation from Tim Burt as well he did a lot of work in this area so if you haven't read this I haven't seen this one I really recommend reading this one if you're into this stuff he did a really good job in explaining some of these things the kernel is now the part of the process it's pretty easy to shrink down the kernel side gets a little more trickier as I said the user space don't need any IPC so the kernel doesn't need any IPC either we can take this out since we only run a single process we don't need any user IDs or group IDs we will be running as root and root only and that's about it we're running one process so all the complicated pieces of fork and exec are completely not needed either we have a lot of config options to disable system calls but there's so many more that are with default on that we wouldn't actually need so there's a lot of things you can actually take out additionally the whole command line passing is a lot of extra code that linux carries around that in this case you would actually not need since you know exactly what you're doing even the virtual file system it's pretty limited since we only have tempfs and we don't even have the fed32 we pretty much only have tempfs or virtual file systems for some control we wouldn't strictly speaking need a big fancy scheduler since we are a single process so the only thing that needs to be scheduled is PID1 and the few kernel threads that linux starts if you're on if you focus on bluetooth the networking sites needs not much there as well you need some basic networking support which when you do this you only select bluetooth and no other networking it's kind of a funky situation and it's a great way to compile parts of the net filter code anyway and links it in so there's a couple of extra things and as I mentioned earlier with the user space we don't need bluetooth classic or high speed support so there was a lot of code chunks that we actually could eliminate as well and there are plenty of more where you can when you start looking these things we can actually take away I just picked the big ones that were obvious and standing out and would have made a huge difference single user support went into 4.1 so there's a little bit of progress we put in the bluetooth LE only options so that's as well we have patches for the security manager to take it out mainly because in some cases you would actually not build the security manager because you rely on higher level security and then you just can say okay we just don't need it we don't need to put ECDH or AES into the kernel because we're actually not using it at that point in time however the biggest problem is actually getting these things upstream since some parts of the kernel community don't really like when you talk about size optimization or you want to make it smaller they want to reduce the number of configuration options it's like look either you allow us to go smaller or we're going to become irrelevant in the cases of the small devices as Jonathan Corbett mentioned earlier so there needs to be balance some patches get accepted other ones have a really hard time to making the case for it the areas we focused on initially was pretty much one of the fun ones if you try to run Linux as IPv6 only that's still not possible and the reason is not because nobody wants to a lot of companies not even an IoT space where pretty much the only thing you do is IPv6 but also in the data center world where they actually don't need an IPv4 network anymore because they've completely migrated but the code is so entangled between IPv4 and IPv6 that it will probably take a while to actually get this untangled and finally only boot with an IPv6 only system where IPv4 is no longer supported another fun one is that the IoT space actually doesn't need TCP or some of them do but if they are smart and care about their power efficiency and the battery lifetimes they will actually start using UDP because they don't want to have the huge state in window tracking of TCP but actually building something with config TCP no which you would think is a kernel option but it isn't is actually complicated we have patches for this one that actually turns TCP into a no op so all packets will pretty much be rejected and that was one of David Miller's concerns that he doesn't want to enable user space TCP stacks he pretty much once looked fine if you want to disable the code and not build in but then please behave that nobody can use this to create a user space TCP stack and we have patches for this one that needs to get pushed upstream at some point there's another area where once you enable config net which you need for NFC 15.4 Bluetooth it builds a lot of extra things as I mentioned earlier so sometimes you actually oh some code of the net filter is compiled and then linked in which actually shouldn't be so there's a lot of cleanup when you build a tiny config where one option drags and some other code bases that it probably shouldn't we haven't even touched that one yet to see what we can do there the more interesting part is when we look at things like IP nodes where you actually rely on DTLS so TLS over UDP sockets that you actually have to get in a TCP a TLS stack to actually have your proper security if you played with open open SSL or TLS then you realize that's actually pretty big library and have a pretty heavy runtime overhead not to mention it's complicated and sometimes they have also security flaws and has been forked three or four times by now into different project names that I don't even remember so picking the right TLS library is complicated some of them actually don't work as you would think some of them have blocking operations in areas where you actually can't block because you have to do something else so there's a couple of issues with this one at some point we decided maybe it's actually not worthwhile to consider open SSL or new TLS at all so we looked into what we actually need to get TLS into the kernel and parts of this one actually is already available TLS itself isn't but the one important part when you do enterprise devices is that you want to have the certificate management and the certificate authorization and authentication completely done and the fun thing Linux kernel supports most of this one or did support most of this one out of the box since it has to deal with secure boot and the chain of trust that it creates because of secure boot so the kernel understands X599 certificates it can verify them but there was pretty much missing a functionality we could control this from user space so what we have been building in the last couple of months is a way to be able to actually create key rings inside the kernel that have certificates as keys in there and they are very fight against each other and then that eventually you can seal them off and say this is my trusted CA that CA might come from a file system might come from a TPM might actually be just represented by keys that are actually inside some piece of hardware that you just reference as an ID so there's been a lot of work ongoing to make this actually a reality we have this all upstream so we can actually use this and we've put support into L in our embedded Linux library to actually drive this so you can just create a new key say it's a certificate and then it will handle all the rest behind it for you to make sure that gets added to the right key rings and the key rings become authenticated against each other so what this means at the end of the day eventually you can even say I have one process at the at the init phase that loads my CA into the key ring system and it can become available in every single process because the key rings can be referenced into threads and into process into threads individually and then you can actually use them to reference your CA so you don't every process at startup don't need to actually load the CA's again it's still an early stage there's a kernel tree on kernel.org that combines all the work of the different crypto subsystems the key ring subsystem and some other subsystems to actually make sure that this can be tested I should have put the link in the slides but I think you will easily find it and eventually we might actually get to having TLS and DTLS support inside the kernel TLS is something that a lot of enterprise people want DTLS becomes really interesting when you use VPNs etc because then you don't have to switch through user space to actually do the processing of the packet encrypt it and send it back or de-encrypt it and send it back to the kernel it's making slow progress but I think it's making a little bit of progress to get towards this one we will be also using the TLS side into our new Wi-Fi demon because when we talk about IoT and you talk about Wi-Fi and IoT you will talk about enterprise security you will not talk about pre-shared keys that you use in your home a lot of devices that have to be deployed then have to use enterprise security and that becomes even more important and we really don't want to deal with GNU TLS and OpenSSL in these libraries and we don't want to deal with functionality as much as possible and there's been a lot of work to be able to use that from user space as I said before IWD we open sourcing this one at OpenIoT summit next week and I will also be giving a talk about this one there and then showcase it so much for this one the big elephant when I gave this presentation end of last year for the first time was that UI is required and everybody doesn't have to be because the reference and protection of UI has everything and the kitchen sink in it including Wi-Fi and Bluetooth support itself for whatever reason but at the end of the day UI is just a spec so it's just a standard and an interface description on how you interface, how you build your bootloader or your system entity and then the operating system can use it so when I did this presentation first time last year it was pretty much okay this is something you can look into how much you can shrink it how much it extra overhead adds it to it the funny thing is actually that literally a week later after I gave this presentation for the first time Alexander Graf said okay I actually ported EFI support for UBoot and so you can use this on systems that don't natively support UFI so you can play with this one and get a few things working there as well and some of these concepts then apply as well so that was kind of nice surprise nice Christmas gift now it's still a big elephant but it's a smaller big elephant and people can play with this one even on systems that currently don't have native UFI support but they have UBoot support so some of my closing notes the interesting part with actually doing this with Linux is testing and verification of these things become really simple so I didn't even try to do this on real hardware for at least a month because I could just run this in QAML or I even could run PID1 straight as with a special switch in it say look you're behaving like a normal process so please skip the setup of the system and I could just test most of it all the way through you can use namespaces in Linux support to simplify your testing but your turnaround time for actually getting some of your code develop is way quicker because you take a Fedora installation Ubuntu, Derbian, pick your choice and then you just get going all the functionality is already there and you can use a lot of these things and you don't have to oh I need to compile this I need to cross compile this put this on a huge turnaround time which saved me a lot of time when I had to do some of these crazy testing interesting enough once it comes to wireless hardware they also some of these ones are really easy to emulate with QAML and then have it tested and then you only do the field testing once you actually have confidence in your code so it saves a lot of time instead of just having to deal with hardware issues you focus on the ones that work nicely and with Linux and the Linux sub system for Wi-Fi, Bluetooth, 15.4 et cetera and all the abstractions that it has you can retest with any hardware or virtual devices and then pick the real hardware and you don't really have to do any extra integration work because Linux have already done this for you literally KVM and QAML is your friend if you have to do any kind of these things without this a lot of this would be painful and horrible to test since you can also emulate it it gives a huge opportunity for continuous integration when you actually need to actually make sure okay I've built this now I want to make sure that I don't break it so you can easily integrate this in a CI system or even with hardware labs where you basically put a new UF by binary on it boot or that works put a new one on it boot and you don't have to do the massive installations pretty nice and easy to do I think the interesting part however is Linux developers are still easier to come by than dedicated RTOS developers so think twice before you jump to an RTOS project and have someone say oh yeah I'm just going to use the RTOS it's easy I have two guys in my company that know this stuff and then these two guys left and then how are you going to replace them a Linux developer is a lot easier and we have a lot more of them these days it has the dedicated RTOS for XYZ RTOS so that helps you maybe in the decision finding when you actually want to build something that is sustainable some of my colleagues actually took this idea a little bit further and integrated into Soleta which they can use to build many kind of variations on systems for your target system but it also can use this idea to actually build your UFI binary then put it straight on the system so in the end it's just a config option what your output will be will it be a full Debian system with systemd or will it be a UFI binary with this dedicated Bluezy peripheral code in there or something else so if you don't want to build this all needy greedy by yourself you can look into Soleta to actually trying to get something boot stepped and see how this is going to work stick it on a miniboard stick it on a Galileo board and have a toy with it and since of recent Edison supports UFI boot as well and a recent kernel so even the Intel Edison is a nice playground for this one or any ARM board if you choose to have one with that one thank you very much and questions please oh sorry I can't really see much so I think they have microphones so if you might come to the front otherwise I'm going to repeat the question actually you said that you removed the IPv4 and instead you put IPv6 so what if as I my limited knowledge tell me it's we cannot using IPv6 to IPv4 NAT is it possible for using software that there is no IPv6 to IPv6 so we haven't succeeded in removing IPv4 that's the work in progress the reason why removing IPv4 is interesting because the sensor networks only run IPv6 only so 6 low pan and some other stuff when you run 50 not 4 or bluetooth on a sensor network they will only speak IPv6 you will never see an IPv4 packet the translation is done on the gateway side obviously if you have a mixed network you have to run some legacy IPv4 you can't really do this it's only for that cases where the technology has no provisioning for IPv4 in the first place and that's 50 not 4 in bluetooth using NAT in IPv4 IPv4 to IPv6 but we cannot NAT IPv6 to IPv4 yes that's correct but it would be purely IPv6 network you will never see an IPv4 packet it takes time to migrate to IPv6 completely oh yeah it takes time the sensor networks only run IPv6 so they have no migration they started with a clean slate the enterprise side migrated thank you any other questions if I don't see any if you have questions please pick up because I can't see everybody in the room thank you very much and have a good day