 For those that came to DevConf, this is basically the same talk. The intention of this talk was to get an armed device, collecting a whole bunch of sensor data, writing it into the cloud and demoing it that way. In the time between I envisaged doing that and now, well, Fedora just keeps eating up my time. I've got a bunch of stuff done, but not enough to demo because what I wanted to be able to do was demo the sensor data of IoT, shoving it into Elasticsearch, graphing it over time and things like that, but then also basically demo an upgrade using Atomic on IoT, a failed upgrade, a rollback of said upgrade and various other things that I'll cover off in this talk. Unfortunately, that's partially working, but not really good enough with an appropriate set of slides to go with it, so I failed it back to the interesting talk and hopefully once F23 is out and we have a whole bunch of interesting new docker features on different architectures for F23 and I might get some breathing space to finish off that little project. But anyway, in the interim, we have what we have. Is it just armed? No. Is it just embedded devices? No. IoT covers a lot of space, a lot of devices, a lot of themes. We have the endpoints, which these days are generally small, tiny, embedded armed devices. We have gateways. In the case of home gateways, they are more small, embedded armed devices. In the case of industrial IoT or large scale IoT, they are generally big x86 servers with wireless controllers and various other bits and pieces on them. You have message gateways. M2M is machine to machine. You have messaging systems. These are both M2M, which is machine to machine, so you might have a sensor device that is collecting a whole bunch of environmental stuff and then you might say heat data in a data center and that might be then connected to another IoT device. That is controlling the air conditioning or the HVAC systems to adjust the temperature of the room in response. Then you have machine to data, which is more your login thing, so you can see the temperature of the data center over a period of time. You have data gathering, data storage and data analytics. Obviously, data security with various different breaches and various different public things that are ongoing. Right through the IoT industry and the Linux industry with things like cart lead and various other bits and pieces are obviously very important in IoT. I have certainly been discussed in the IoT industry these days, but I don't believe that yet at a cohesive, straightforward solution that is applicable everywhere. There is a lot of beginning to become a lot of awareness and a lot of discussion about it, but that doesn't appear to be like a single cohesive IoT security strategy. This is a fairly standard IoT device stack with libraries and APIs, protocols and obviously down to things like wireless communication and bits and pieces. This image is courtesy of the ARM guys because I haven't managed to find the better one. Covering endpoint devices, they need to be low cost, almost disposable devices. ARM devices are obviously driving this revolution because they're low cost, low power and persuasive everywhere, so there's already lots of tools, toolkits and various other bits and pieces to deal with them. You get Arduino ones which are very low end, small amounts of code, small amounts of RAM that can wear on your wrist or almost disposable. Or more capable devices like the Beaver Bone and the Raspberry Pi and other similar style devices. There's a number of OSs out there, Contiki OS and Tiny OS, both around the Cortex N series, which are those uberloaded power stuff that you're finding in things like electronic door locks and things like that. And then as you come further up the stack, there's Pokey Open Embedded of Yoctote which is a compiled from base but fairly generic supported across hundreds and hundreds and hundreds of devices at a number of different architectures which is more closer to the sort of space that Fedora would compete in on your Beaver Bone style devices. Sorry? Yes, I should actually add that. It was sort of just starting to... I actually have a device at home which is an IBM type thing for Embed with the IBM cloud IoT in my stack to play with devices, never enough hours in the day. So Fedora on IoT endpoints, you know, x86 obviously, something we're lucky enough to get a Raspberry Pi style x86 device in their bag. ARM V7 and obviously AR64 are incoming at some point with the potential. I think atomic images for IoT is the perfect fit. You can create a direct image, you can do a remote update on it. When you update, you go right, update seems okay. Set a watchdog, reboot. On reboot you have a process that checks local things like local sensors but also it's all very well having a device boot okay but if say the network device doesn't come up or the wireless interface is screwed, you still end up with a break. That's useful. So you boot it up, you run basically a test script to test all of these minimum features that you need and then you unset the watchdog and you go right, we're good. If that hasn't completed by the time the watchdog triggers, like say a minute or two minutes later, the watchdog issues a hardware reset and you automatically deal with, in the boot process, the rollback to the previous good. And there was a few months ago, there was an IoT home manufacturer that was producing these $100, $200, thereabouts, IoT gateways that had I think either Bluetooth 4 or Zinfi or something like that and they sent out to in excess of 100,000 gateway devices a firmware update that broke them all and they then had to send emails out to all their registered users saying we've just broken your device. Please put it in a reply post to this address and we will send you out a fixed one shortly. That's expensive. I'm not even sure whether that, I mean that's a situation that if it happens to a company you can send them broke. And so something like a comic would mitigate the vast majority of those screw-ups. We've got Infadora support for the 802.154 stack, Bluetooth LE and various other technologies. We've got enabled, there's still some upstream support to do a bunch of features, but 6 Low W Pan which is a stripped down IPv6 to go over small networks. There's some routing protocols and various other bits and pieces and as you go higher up the stack there's MQTT which is a cut-down message queue technology which we have support for through a bunch of different protocol stacks within Infadora including mosquito and rabid MQ. We would have active MQ support when the supposed maintainer actually gets around to updating it to a newer version. And obviously we have the ability to deal with firmware fixes and various other bits and pieces. No management framework yet for things like firmware updates and things like that. But the newer stuff that's happening around EFI firmware standardization so on would be... Yeah, but I mean that would be not so much for upgrading the OS. No, but I mean for expanding the firmware... Yeah, I mean there's a whole bunch of technologies there and I've not looked at it closely because it's a purchase that Red Hat has made designed for the mobile area but there's Feed Henry that does a certain bunch of mobile and when it's open sourced I'm sure that we can possibly adjust it a bit and it could be a whole bunch of that sort of stuff has the potential to be re-persuable for things like AOT because they're not that dissimilar to mobile device management. So 802.15.4 is an open base level wireless implementation. It's used for ZigBee and a bunch of other protocols. Google has adopted it as part of the Threat Group stack which runs IPv6 and various other bits and pieces. The group there is very industry oriented and not so much oriented to open source contribution that you've got even as a private member you've got $1,000 a year as a member fee or something like that. But they're starting to publish some more standards and it will be interesting to see if they do an open source like standard implementation of the lower levels of the Threat Start. I've covered basically the 6th low WPAN which was the IPv6 which is what Threat is using for their low layer routing. Gateways, I'll just cover this quickly but basically a bridge from these lower level wireless protocols that speak to devices in your house or devices in the factory to bridge it over to the standard network and the standard internet. In a lot of cases they're routers but they can also be things like an MQT teak caching layer or an N2T themselves so that you can have all completely local traffic and even to some degree a data collector and border. Because you'll have something like a doorknob you don't want to power up a hell of a lot because you want it to run on a AA battery for a year and so it'll power up, it'll query MQT teak but here I've got these cache messages for you, you know here's something to deal with or it'll power up because it's just unlocked the door for someone and it will send both logs back to a logging device to say Joe or RFID X has entered the building at this time. So gateways can function as a number of things and as I mentioned earlier they can be a small device in the home and a much larger higher end server to deal with hundreds of thousands of devices in a factory. So here's a basic IoT, you've got a bunch of modes meshed together a gateway and on to the internet. Messaging systems, MQT teak is message queue for telemetry a protocol that was originally proprietary, came out of IBM and it's got quite a bit of traction it's really nice and simple and seems to work fairly well. There's a number of projects including active MQ and mosquito as I mentioned earlier and a number of client implementations in Python, Java and various other bits and pieces as well as a number of sort of start-up hosted platforms. Robo and Q is one. I think there's one called OpenSense SOP BIOXP or something like that which is announced recently which is actually based out of London and so there's a bunch of interesting projects and everything from OpenSource Do It Yourself to hosted platforms. Constraint application protocol is your messaging queue system is very much for machine to machine and machine a number of different things. COAP is much more similar to HTTP so you'll have a client or a gateway or various other bits and pieces actually actively polling a code server. So it relies on a sort of stripped down version of REST low overhead and it doesn't compete with but it's useful alongside MQTT for different use cases basically. The lightweight machine to machine is just a standard messaging protocol that can be embedded in COAP or MQTT primarily on COAP for... It's basically an object model to enable standardised into device communications and it has standard dictionaries and the ability to define various bits and pieces and enhance dictionaries or you can push out bits to devices so they know how to communicate with each other. And smart objects, another open thing uses the previous or can use the previous lightweight machine to machine and it's for things like naming of sensors use input and output so you would use these sort of things to say I have a smart light, turn it on, turn it off set the hue, set the colour, that sort of things but different device profiles from thermostats to lights to industrial machines or whatever they happen to be. So using Fedora for data analytics and storage IoT is probably in my opinion one of the better uses or better verticals of big data. You can feed vast amounts of data in and analyse it and get vast amounts of data out. And so from this perspective a lot of the IoT side of things is no different than any other big data problem. And there's a bunch of tools we have in Fedora and a bunch of tools out there in the industry that answer this solution very well from Elasticsearch, Hadoop, Google Cloud Platform that launched a streaming API data analysis which enables you to do both streaming analysis and static analysis of data. So there's a bunch of interesting projects out there where this is suited very, very well for IoT stuff. There's obviously with Apache Spark, Apache Hadoop, Elasticsearch on top of Lucene, a number of awesome, awesome open source projects for this as well or most of which run pretty well on Fedora. And obviously, you then integrate into phones and bits and pieces like that for mobile learning. Hey, you're away on holidays and your front door's just opened. You know, why? There's so much stuff in Fedora for that at the moment. Although there was a discussion earlier in Flock about would it be possible for Fedora to provide a means of doing open source Android apps and getting them published into the Android app store or the Apple app store and things like that for people that are interested in doing open source apps but don't have the ability to deal with that. So that was an interesting sort of talk. Not really related to IoT but has the potential to be with regards to open source. And obviously storage as well. Various different databases and Elasticsearch and you know, again, similar to big data. Nothing particularly in my opinion Fedora, sorry, IoT specific when it comes to these solutions. I personally don't see a lot of value in investing a lot of time in doing an IoT specific thing where you can use a general thing and existing technologies that are known to work. But again, a thing that we do relatively well for Fedora for a lot of that already. I mean looking at the data grep data store with Fed message is a perfect way of seeing how we're using technologies in Fedora to deal with what is essentially a big data problem for data analytics and stuff like that and there's no reason that can be used for or the same style of system used for say an IoT style platform. Security. As I mentioned earlier certainly a massive big issue or looming potential issue in IoT. I don't think it will be long before we see some fairly large breaches about this if it's someone's home third but the startup might not be so much of a problem. Well the example that Armgate did was even with that break. So let's say that you compromise all the white faults in the city. If you adjust them all at once. Yes you can overload the power grid. Or is it really on fire if you play with enough further stats? Everyone always talks about the internet connected toaster in the UK. It's a known problem that at the end of a football match or in the middle of at a break of football match everyone goes and puts on a kettle and massively overloads the grid. It's not duking there's a hydro storage thing. There is a hydro storage thing which they can start generating power in about three seconds flat and it has basically up in a mountain a massive store of water and a whole bunch of pipes and it goes down into a lake below and they finish with it overnight once they use some of the generated power to pump the water back up the top. Is that about east enders? Yes. In the past it was east enders but now it tends to be any major thing. In the Arm case if you go and remotely turn on everyone's kettle Yeah. So as I don't know, I don't remember who said it but the internet of other people's things. And you know, actually security is not a new thing. Security on Linux is certainly not a new thing. Dealing with security issues on Linux is not a new thing. There's been a lot of recent, um, the Viper issue on KVM, or on QEMU cloud platforms both Zen and KVM Parkway, there's a number of recent ones that are a good indication of the way the security industry is changing and some of the, you know, Linux foundation and missionism to help mitigate that is a classic example of the Linux community reacting to security issues and trying to, and so there's no reason all these principles can't apply to IoT. SD Linux everywhere. Atomic images to enable easy upgrades with rollback for the inevitable, as the example I gave earlier, not well QH product update. You know, the... You know, the recent Intel we have this ancient old feature that is easily in the chip layer problem. I mean, you know, they have many in vast. Network security, I mean all these devices are connected and if you don't secure, you know, security is only as good as your weakest link. So ADO 2.154 has inbuilt security, automatic re-keying, various other bits and pieces. All these tiny little embedded chips generally have even a basic crypto offload engine to enable things like that. TLS by default on all applications you know, these are things that the industry in Linux has learned a lot about in the last 12 to 18 months and I would hope the IAT industry as they use Linux you know, learns from the industry at large. I fear that may be one of the major downfalls as they use just old stuff that's already running. Application security, you know you can secure everything in a low level but if the underlayers aren't secured, you know, as I mentioned just for easy updates and rollbacks, regular constant consistent updates. All things that, you know, we're dealing with in Fedora but you know, how many IAT devices? Gartner says 26 billion, Cisco says 50 billion, you know, there's I don't think anyone knows and the fact of the matter is when we get to 2020, there's going to be all these organisations or their equivalent ones reporting all sorts of different figures because I suspect we'll be in a situation that nobody knows. So is security a problem? Yes, every which way you look at it, there's going to be a problem. So there's a part of there's diversity in IOT platforms, competition, you know, various other bits and pieces to ensure companies that are delivering a base IOT firmware image that there is choice out there in competition and I personally think security should be one of those major competitions. I mean it's all very well being able to get data and if the device is cheap and all that, that's all very well and good. But if you've got 10 million devices out there that are suddenly compromised, you're going to have a lot of big, the companies are going to have a lot of big on their face. So I mean I think Fedora is a great building to build all this. We have a lot of the technologies starting to come together to enable an awesome IOT platform with things like atomic and security updates and some of the new rings and policy packaging and things like that. Of course there's still a lot to do, but the basis there, you know, there's enhancements going on in the industry like thread and proof and that and if we can sort of implement that and be able to test that and all the building blocks for that are already in Fedora and runnable. But you know, there's obviously ongoing enhancements everything from new packages, updating packages, tweaking bits and pieces, and end-to-end testing. Thank you. Any questions? Discovery is the other piece that's missing a lot of these conversations. So how do devices discover each other? And there's a whole bunch of consortia right now working on, you know, one of my friends working on the hypercap in the UK, which is definitely, you know, a few times. But there's a whole bunch of them like how do smart devices discover each other? Well the thread stack also implements bits and pieces like that and there's talks of like, you know, embedding RFID in the devices. So you basically go and put the device on top of the gateway and they pair using RFID and set a security key. And there's going to be a lot of competition there. There's thread group, there's three or four related, but similar, but slightly different consortia. So I think there's two outcomes that the door of the newsletter, one would be an actual IT evidence. Yep. Using the technology we already have, so there's no point in being able to live. And the second thing is the comprehensive not boiling the ocean reference example. Yep. Where you say here's a little device generating some data, it's aggregated, processed using whatever technology you want to do, spot something on an X86 and power it over in the club. And to an example. Yeah, completely agree. And as I mentioned at the beginning of the talk, I was hoping to have the basics of that in place, but you know. We can talk a little bit about doing that. I mean I would love help. I mean I have a whole bunch of ideas in my head how to do that and just not enough time and cycles to do that. And people want a little project or a big project or a series of projects to come and help out and start to play. That's formalised. That's formalised some kind of IoT scum words. That's it. I mean we should, I thought the other day that we should actually just get a mailing list as it starts to start this discussion. Yeah. We don't have that done by... Exactly. It was by half already. So yeah, I mean there's things like that. I mean we've got dozens and dozens of building blocks there that are being used in the industry that we've already got in Fedora. We have a superbowling opportunity that we are not addressing right now. So we should be addressing that. Yeah. I've had all these ideas in my head. But I am one person and one person doesn't create an IoT stack. Unless it, I mean I forget who I said it to earlier. It's like I wanted to do this demo and if I had two weeks to sit down and just work on this it would be working. It would be beautiful. I mean it won't take that much time. But I know a number of you in here are very aware that actually taking two weeks out to hack on a prototype and get it up and running. Yeah, I wish I had two weeks spare to do that. You know, I'm sure there's people in the business that would say, yeah, we will give you the two weeks to go and do that. But there's going to be 50 other people in the business going where's all these bits and pieces that you're also doing.