 Hopefully, we're all in the right place. I'm here to talk a little bit about the basics of IoT development, just all the way from the beginning, prototyping phase, all the way out to production. That's obviously a very broad topic. So this talk will end up being a very high level survey of a lot of things you need to think about as you're looking to move into the IoT development space. Given the target of this conference, if you have been doing IoT devices for many years, you probably won't get a whole lot out of this talk. I appreciate you coming anyway. But for those that are new to the space, hopefully this talk will give you a good overview and some things to consider as you start thinking about how you're going to design your systems moving forward. So just a brief overview of what we're talking about. We'll start just talking about what IoT is and what are some of the markets that it's targeting. Most of us, I'm sure that'll be refresher information for, but it's always good to start with a little bit of motivation for why we're talking about the things we are. Then we'll talk a little bit about the hardware and software things that we would want to look at when you're designing an IoT system. And finally, we'll kind of go to a little bit higher level and talk about some of the overall system design considerations in the IoT space. So a bit about me. I've been in the embedded Linux space for about 10 years now. Primarily focused on Yachto. Longer than that, I've been in the general embedded space. My current role is primarily as a project lead and a solutions architect. I'm not deeply involved writing the code on a day-to-day basis, but I am a technical customer-facing engineer on the Mender.io project, which is an over-the-air update client and server for embedded Linux devices. We've got a booth out there. So if you are in the IoT space and have a need for updates, which we will certainly cover here in a little bit, feel free to swing by. And we can give you some more details about what we have. So let's start with the definition of IoT. When I was putting these slides together, first thing I did was what everybody does and go to Google and type in IoT. And I got a few choice links here. Business Insider claims it's a network of interconnected objects able to collect and exchange data using embedded sensors. So a lot of words there. Then I found this other one on the IEEE, which was a lot more content. It was 86 pages. I read about the first paragraph and a half and stopped. So the link is down at the bottom. Feel free if you are looking for if you're having trouble sleeping or whatever. Download that, and you should be in good shape. After that, I went to the font of all knowledge today. I went to Wikipedia. And Wikipedia has a very similar definition, a network of physical devices, vehicles, home appliances, and other items embedded with electronics, software, sensors, actuators, and connectivity, which enables these objects to connect and exchange data. Again, lots of words. But it comes down to a couple key characteristics. In the internet of things space, we're always talking connected devices. It might not be always on connection, but there is some kind of connectivity to these devices. Generally speaking, you have some number of sensors that are able to detect characteristics of their environment. You have some number of actuators that are able to affect change in the environment. And in a lot of cases, you have cloud infrastructure involved. That's not strictly speaking required. There are plenty of IoT applications where there's no connectivity back to the cloud. But more and more these days, things are moving back to the cloud. So what are some of the typical IoT applications? First ones that come to mind are the consumer devices. Obviously, the Nest thermostat and the competitors there for the smart home climate control systems. Smart lighting, I've got a few different things, one from IKEA and one some random that I picked up on sale on Amazon one day to control lights in my house. Home security, there's a lot of those. I know the Ring Doorbell is a fairly common device that's advertising quite a bit now. And there's other connected alarms and that kind of thing. And connected automobiles, obviously a very big space. I know there's a lot of people here in the automotive space and automobiles are getting more and more connected every day. I saw a stat somewhere that a modern high-end automobile has about 10 times the number of lines of code in it as the 747 jet. So think about that next time you're getting into one of these vehicles. There's a lot of code in there and there's a lot of connectivity going on. Other markets for Internet of Things industrial applications in the operations center, a lot of factory inventory management. It makes it a lot easier to deploy a large fleet of fairly low-power devices to track things that previously would require fairly high compute capabilities. So a lot of factories are retooling and adding new capabilities to track things at a much more granular level through the factory processes. Seeing some in enterprise, supply chain management is obviously a big deal. I'm sure we've all seen headlines of issues with supply chain management and when can I get the next greatest phone and when are they gonna have enough LCD panels or whatever. So being able to track the supply chain as the parts you need go from one place to the next and make it all the way to your manufacturer device is pretty important. Medical, obviously there's a lot of things going on in the medical space and then business operations. I've read somewhere about an elevator company that's using IoT enabled services to help improve reliability, to decrease downtime, decrease the time that people are stuck in elevators when there are inevitable issues. But the ultimate goal for all of these, especially once you get outside of the consumer space is obviously lowering your operating costs moving forward. And finally, one area that some places are doing better than others is in the municipal space. Obviously, traffic control, public transit, there's a lot of possibilities there, especially with the connected automobiles. I know I'm from the Tampa, Florida area and I know they've got a project going now. Somehow using connected automobiles and then sensors on the streets to help control traffic flow and do extra studies and things like that. So there's a lot of interesting applications. As I mentioned, cloud infrastructure is increasingly important in the internet of things. You see all the logos on this slide. There's no end of providers for cloud infrastructure. At a minimum, they're used for device control and data store in your internet of things applications, depending on exactly what your workflow and use cases are. The cloud infrastructure may provide AI and big data services in addition to just simple data aggregation. And some of them also have overall fleet management, device dashboards and that kind of thing where your operations team can actually see at a glance a view of your entire device fleet, be able to be able to make changes, trigger updates, turn devices on and off that kind of thing. So this is a diagram I put together to kind of help explain what IoT is because there's a lot of buzzwords in there. You see at the upper half of the screen is the world of atoms, that's the world of real things. And then we have our lovely little border there which is called the edge and on the inside of the edge is the world of bits. And that's, you know, the IoT kind of spans that boundary. You have a number of devices. The device is in green here. One of the architectures is the devices themselves on the edge will connect directly to the cloud. So the device, and that's a lot of the devices that folks at this conference deal with, they're running a full embedded Linux system. They have all the connectivity they need. They're able to connect directly to the cloud and they can do whatever they need on the device themselves. Some of the lower powered devices things running, some of the smaller RTOS is that kind of thing, might use, sorry about that. There we go. Some of the other devices might use a gateway between the device. So some of these devices will be running a small real time operating system with limited connectivity, limited communications capabilities. So then they would pass the data back to a gateway which has more capabilities to actually deliver that data back into the cloud infrastructure. And the third typical architecture we see is that the gateway is sometimes optional even for some of these smaller devices. They can communicate between themselves directly. There may not be actually any cloud connectivity needed for a particular use case. So in that case, if your devices can communicate directly to each other and share whatever data is necessary, you can get away without even having the gateway in that instance. So talking about some of the connectivity options that you'll need to consider when you are planning your design, the first couple on this list, I think most of us are familiar with, we've got the short distance, generally considered personal area, NFC and Bluetooth. Although I've seen claims that Bluetooth can go up to 30 meters, I don't think I've ever managed to get my Bluetooth headset to go more than about six or eight feet before I start getting dropouts. So depending on the amount of data you're transferring, obviously you're gonna get better or worse distance on some of these things. Moving up into medium distance connectivity, venerable Wi-Fi and Ethernet, I think we all understand what that is in the pluses and minuses of those. Those are obviously fairly high bandwidth, but especially if you're dealing with Wi-Fi, you've got some extra configuration that needs to be tracked and that kind of thing. Moving on from there, these are some of the emerging technologies that are increasingly in use in the internet of things space. These are typically longer distance than within the buildings. City scale is what they're looking at here. You've got Laura and Laura Wann, which are protocols that are governed by an industry alliance and there are a number of cities that are rolling out these kind of networks. And then SIGFOX is a commercial entity doing a very similar thing. So if you've got a fleet of, say, weather sensors you want to deploy across the city, the devices can communicate over some of these protocols and not have to have any kind of gateways or anything. And the big advantage is, of course, the battery life on these things is measured in years. They're very low power. They're not really intended for high bandwidth applications. You're not going to be streaming multimedia data over these protocols, but that's not really what they're designed for. And then moving out from there, looking at even larger scales, nationwide, statewide, then you start looking at things like cellular and LTE with the obvious costs associated with that. So when you're talking connectivity, some of the, you know, you've got to figure out what your bandwidth needs are, what your latencies and that kind of thing are, as well as what the costs are going to be. Obviously with Wi-Fi and Ethernet, typically that's not going to be a metered connection, but obviously when you get into cellular and LTE, you're generally going to be paying per bit in that kind of scenario. So moving up the stack a bit, we have the higher level IoT communication protocols, and there's definitely some overlap between things here, but these are the kind of things that are typically layered on top of the base connectivity that we discussed in the previous slide. So HTTP and Secure HTTP, REST APIs, that's a pretty common mechanism that's used today, but there are some other things in the IoT space that you will start to see as you ramp up in your research tasks. Six LowPan is an implementation of IPv6 over some of the low power WAN protocols. That's not one I have any direct experience with, but I know it does come up from time to time. MQTT is a, it's a more structured protocol than say something like a REST API where you get to define everything. MQTT really follows a public subscribe model. It's very lightweight. Most implementations typically run across your standard TCP Ethernet links, and there was sometime recently an actual standard published for that, so that's getting to be extremely common in the IoT space, and it's very easy to use. There's libraries for just about any language you may want, and it's very easy to get started with that. Then there's another one called ZeroMQ. It's similar in its, you know, it does have the published subscribe model, but it supports a few other models. There's more push-pull and router dealer kinds of things that are implemented in there, and it is an open source protocol, so that makes it very applicable for most of the Linux-based IoT designs. And then you'll start to see things like ZigBee, which is primarily used for home automation. I think that's a little bit lower level protocol than something like MQTT, but it's definitely growing in popularity. And then one that I have had conversations with folks about, which is kind of a bit of a stretch to have on this slide, but it's the DDS, which is data distribution services provides for global data space distributed with proper access control. So if you have a fairly large amount of data that you want to have distributed and have proper backups and that kind of thing, this is certainly something to consider moving forward. So hardware. First thing typically in the design is you're gonna start thinking about hardware. First choice is, do you have an MCU versus a system on chip? MCUs are generally lower powered systems, not gonna be running Linux, probably will be running an RTOS of some kind. I think probably for this audience, we're probably mostly looking towards the SOC level, but I know there's quite a few folks here doing things like Zephyr and other real-time operating systems that are more appropriate for the MCU level chips. From there, once you've decided on the basic chip set that's part of your system, then you start looking at onboard peripherals. What hardware does your particular use case require? Most of these chips are gonna have a variety of onboard peripherals, and if you're buying things you don't need, obviously you're paying extra money, but if you buy something that doesn't have what you need on it, that's not good either. So that's the next basic step in deciding on the chip is mapping out what your use cases are and what peripherals you may need. Then start to look at the hobbyist versus the commercial vendor type platforms, Raspberry Pi and Beaglebone. We use those a lot in our day to day, but typically lead times in inventories can be tricky if you're gonna be producing builds in tens of thousands or hundreds of thousands of devices. You may be better off going with a commercial vendor that's able to actually satisfy those numbers. Other things to consider, are you gonna have battery power or is it gonna be hard wired power because some chips are gonna be running, obviously, lower power than others, so that's something to consider, and obviously price always comes in. And the last one that I mentioned here is the form factor. Boards like the Beaglebone and the Raspberry Pi are typically standalone boards. They'll be mounted in a case somewhere, but the boards themselves are fully functional without any add-ons. Then another common design that we see a lot is the system on module, where you have a baseboard, which is typically provided as a reference design by the manufacturer, and then they also provide the system on module where for your design, you would take the system on module unmodified, but you would create a custom version of that baseboard that would have the exact peripherals you need on it. The nice point about these kind of designs is you can customize in the baseboard the connectivity between the psalm and the baseboard is well-defined so that you know that if your board works with their baseboard, that as long as you use the same protocol and have the same pinout, you'll be able to plug that into your custom design. And it also gives you the flexibility most of these vendors have a wide range of system on modules that are available with different MCUs or different SoCs on them, so if you start with a lower powered chip and at some point you decide you need to move to a higher powered system, you have the ability to simply swap in a new psalm and it should in that case work pretty well with your existing designs without having to retool your designs. So once we've picked hardware, now we need to start thinking about the system software that we're gonna be running on these devices. Three main choices typically are full blown OS which I think is probably the choice of most of the people in this room versus an RTOS versus just simply writing bare metal code directly write every line of code in the system do it a bare metal control loop. Then once you've chosen there, obviously if you're in the Linux space, you have a choice for system development tools, things like Yachto and Buildroot which are very common and there's a lot of expertise here, a lot of folks talking about those kind of things. Then OpenWRT is another option and finally you have things like Debian and other desktop class OSs that have been repurposed in the embedded space. And then from there, you also have some additional deployment strategies, hypervisors and containers. Obviously there's a lot of talk of that, a lot of activity in the industry today with a lot of asynchronous multi-processor designs where you're running maybe a hypervisor and you're running an RTOS on one core and a full blown Linux instance on the remainder of the cores and things like that. So there's some very high level decisions that need to be made about your system software deployment strategies. And finally, depending on the industry you're in, security and safety are a big concern. ISO 26262, it's a automobile functional safety spec. It's pretty hard to satisfy something like that in Linux but a lot of the RTOSes will have that. So that's where the AMP type design will come in. If you have some areas of your code that need that level of certification, you can implement those in an RTOS on a sequestering core and then have your Linux system handling the rest of the connectivity on the other cores of your multi-core chipset. And then obviously things like SEA Linux, App Armor and SMAC are our Linux side security frameworks that can add additional security over the traditional split model of users and then who can do anything and everything. So moving up the stack a bit more, now we start talking about application software and this is definitely outside of my area of expertise. So most of these things are just names to me but application frameworks that we see commonly, things like Node-RED, Node.js, those are very common in the IoT space. Eclipse Cura is, I believe it's more of a development IDE type plugin that works on the development side and then Qt, obviously a lot of the graphical embedded Linux applications are running Qt-based APIs. And then from there, how are you gonna develop your code? You can use Eclipse or some kind of CLI or what are your developers gonna be looking at on the day to day? Typically if you're in a commercial RTOS or even an open source RTOS, they're gonna have one mechanism for working. If you're in the Linux space, obviously it's gonna be very dependent on the build system that you use, the packaging system that you use and that kind of thing. Language availability, something to consider. If you're in the Linux space, that's usually not a big deal. Golang is obviously increasing in popularity. I see a lot of people talking about Rust these days, although I don't think we've seen it a whole lot just yet in terms of actual devices going to market. And then finally, just look at third-party package availability. You know what your use cases are. You know if you need specific libraries or specific protocols, a lot of that is gonna help drive some of the other decisions that you might make based on where the libraries you need are available. If they're in the Yachter project, great, that's an option. If not, if they're provided as packages for Debian, that may push you towards a Debian-based system and likewise for other packages and protocols you may need. So just briefly mentioning a few things. If you aren't running Linux, a couple options I mentioned, the bare metal embedded control loop, very low level, writing everything yourself. And then one step up from that is an embedded RTOS, things like Zephyr and Minute, include OS, free RTOS. Those are all things that we hear in discussions with the IoT developers that they're considering. And then there's commercial offerings such as Nucleus VX Works and QNX. Obviously the cost models are gonna differ very between them. Sometimes there'll be royalties, sometimes there won't. Sometimes there'll be a development upfront license. So that all has to be taken into consideration as you're planning your system. And I did wanna mention Windows IoT Core here. I wasn't really sure the best way to classify it other than as a desktop class OS in the area I live. There's a lot of Windows IoT development going on. So I've actually been to several meetups with folks that use it and they really like it. I don't know enough about it to really comment, but it is an option. Moving on, if we assume that we are working on a Linux system, I briefly mentioned some of these, but you have quite a few options here. You have your desktop class distributions, Debian, Ubuntu, things like that. Couple options with those. You can go with the direct install. Typically, most of them are available just in a disk image that you download. You DD out to your SD card or your EMMC and you boot. And typically there will be a package manager, apt-get, something like that. You install everything you need, all your dependencies, and then you can either package your application as in that package manager format or whatever you decide is appropriate to get the code onto the device. Another option for the desktop class distributions is packaging strips. I know that in the Debian space, there's DE Bootstrap and other options that actually allow you to kind of script and customize these systems offline. And then the output, of course, is typically the image that then gets written to your storage media. One step down from there is the embedded distribution builders, things like Yachto, Buildroot, and OpenWrt, which I mentioned previously. We'll go into a little bit of what those are in just a moment. And finally, there's some hybrids which are new to me. I know there's folks here that are giving talks about them. So things like Esar, which is an ELBE, which I believe are basically just using BitBake with special customized recipes to actually pull binaries from the Debian build system. So you're actually not doing a full build every time of each package like you do with a typical Yachto BitBake build, but you're actually still able to control it and set up your designs and your configuration in a language, configuration language that you may already be familiar with. So Yachto Project, for those that aren't familiar with it, the quote at the top of this slide comes directly from the Yachto Project website. The primary focus of Yachto is the recipes that tell the system how to build all the packages that are part of your system, how to build the images and that kind of thing. The primary output of a Yachto Project build is a package feed, which is really just a directory somewhere in your build tree with a whole lot of .rpm files or .deb files or whatever packaging format you choose. The secondary output is the boot images and this is the actual bag of bits that gets installed on your storage media and it contains typically the entire root file system bootloaders and that kind of thing. Yachto generally builds all components from source although there are some things like the Linux firmware that are typically just binary blobs that are installed and the Yachto focuses on mechanism not policy and that kind of goes back to the quote about it's not a distribution, it builds a custom one for you. There are sensible defaults for most things in Yachto so that you can get up and running really quickly. However, Yachto is focused on allowing you to make changes to things like the init system so you can switch easily between sysvinit and systemd and things like that. So the focus of the Yachto team is not to enforce policy on you as the system designer so you're able to customize it however you want. Moving on to build root. Build root has a similar objective to Yachto. It's focus is on a much simpler view and they build simpler systems by default than the Yachto project does. They don't support package feeds in the same way that Yachto does. The primary output of build root are the images that are installed on your system so it's the root file system, the kernel, the boot loader and that kind of thing. It does again focus, it does build everything from source and its default is to focus on simplicity so if you download the build root sources and you build without making any modifications you're gonna get a very bare bone system that is enough to come up to shell prompt and connect to a wired ethernet and then you need to start adding things back in turning on package configurations for all the packages that are installed and that kind of thing. And finally just want to mention OpenWRT. It's a fully writable system with package management so it's very similar in concept to things like Debian and Ubuntu. Its primary focus is networking obviously based on the name, it came out, it originally started as a replacement firmware for the Linksys WRT family of home routers. It's since been expanded to support just about every commercial router available. If you look at their support page it's pretty impressive how many devices are supported. It does, it's primarily a binary distribution, there's more of a separation in the OpenWRT world between the bits that you run and then the build system than is for example with Yachto and build root. So most users of the OpenWRT won't be messing with the OpenWRT build system. You'll just download the pre-built binary for your particular router, install it and you're good to go. And they do provide network available package repositories for OpenWRT unlike Yachto and build root where especially with Yachto the package feed is part of the build that you do which you can then make available on the network but there are no generally available package repositories for those whereas with OpenWRT when you install and boot up you can actually go to the package manager and install new packages on your device at runtime. So looking up at a little bit higher level considerations for when you're deploying an IoT device fleet. Things you might need to think about what are the lifetimes of your devices? Typically consumer devices, the lifetimes are anywhere from six months to two to three years although I think the router I have at home now is four to five years but it's getting long in the tooth so I'm getting ready to replace that. If you're in the automotive space you're talking at least a 10 year lifetime once the systems are deployed and typically the lead times for new designs in the automotive space is between five and 10 years so right away you're looking at 15 to 20 years total lifetime from prototype to production on the automotive space. So that's definitely something you need to keep in mind. What kind of fleet do you have? Are your devices managed by you as the system designer or by some central authority or the unmanaged? If you're selling consumer devices that are going into people's homes you're probably not gonna have a managed fleet so these devices will be out there they'll be out there on their own they need to do their updates and that kind of thing but if you have a device, a smaller deployment or a managed fleet you probably will be able to have visibility into what the devices are doing over their lifetime. What's the operating environment? How hostile is it? If it's deployed in say coffee shops with open wifi you know you're gonna be getting network pings on a fairly regular basis. If it's in a fairly controlled lab environment then obviously that's a different consideration. What's the power and connectivity like? Obviously if you are battery based that's always a concern you need to make sure that you can handle unexpected power outages. Can the users modify the software? Do you have a system with installable packages where the end users of the device are actually able to manipulate the software that's installed on the device? That adds a whole level of security and complexity that if you can get away with a completely read-only system that you don't have to deal with. And then what is the end user interface? Is this an appliance that goes in a cabinet somewhere and nobody ever looks at it? In that case you can get away with doing updates when your software determines it's right but if it's something with an end user interface generally you don't wanna install new software or do reboots or anything without prompting the end user. And then as we mentioned before the bandwidth is always something to consider. Both the network bandwidth to your device as well as how much cloud compute capabilities you might need for that cloud structure and the back end. So in security we'll talk a bit about this. The quote at the top, the SNIOT stands for security. I hear it a lot, don't know exactly who came up with it. But the Twitter handle you see there is the best guess I could find after searching for a bit. But I think it's obvious that all security, all software has bugs. Not all bugs become vulnerabilities but when you have one to 25 bugs for every thousand lines of code there's liable to be some vulnerabilities out there that will eventually allow your device to be taken over. So just in general, as general advice use well-maintained software, keep it updated. You don't want to be the only one using some branch of Uboot or a Linux kernel that hasn't been touched in a number of years. And then just follow general security practices, principle of least privilege. Don't run something as root unless it needs to be root. If you do have something like SE Linux or SMAC you can get a little bit more fine-grained control over that. Kirchoff's principle basically says that the only security of an encryption system is the key. So don't rely on security through security. Use a well-known encryption system when you are doing a crypto in your devices. And just this kind of self-serving thing here at the end. Over-the-air updates are a must-have these days, especially for any of these devices that are connected. As I said, you're gonna have vulnerable bugs which will eventually become vulnerabilities and if your device is connected there are people out there scanning and they will take over your device pretty quickly. And I'll just leave this slide up. This is I think fairly well understood. It kind of just goes to the point on my previous slide about the number of bugs out there. But with the average remediation time being 110 days, well beyond when then there's a 90% probability of a vulnerability being exploited. Obviously we as an industry have to get better at getting these things fixed and getting the fixes deployed to our devices. So briefly about patching and updates. You see the number from ABI research there. One third of current recalls are for problems that could have been fixed over-the-air. $35 billion in savings in 2022, according to one automotive survey, obviously in the automotive space. If there are vulnerabilities that you can fix over-the-air that's significantly cheaper than spinning up all your repair centers, having your customers bring the devices in and have them flashed over USB. So if at all possible, definitely when you're deploying IoT devices or any kind of connected device over-the-air updates should be considered a must-have, especially given some of the lifetimes and the expense of accessing these devices when they're deployed in the field. All of these things put together to tell us we need to do this. And when you're looking at updates, just some high-level design criteria, robustness, how does the update mechanism ensure that you don't get brick devices in the field? If the device is on the shelf next to you, it's easy enough to reach up and hit the power switch. But I was talking to somebody yesterday who does devices for undersea usage. It's a little bit more expensive to charter a boat, get a dive team go out, send the dive team down 200, 300 feet and hit the power supply. So you don't want to be deploying an update, have the power cycle at that exact moment and end up with a device with a corrupted root file system. So the robustness of the update system is critical. Security, obviously, TLS, image signing, all industry best practices, I think that's all fairly standard. Are the updates atomic? If you're doing apt-get update, typically that's not an atomic type of update. It makes it very difficult to know exactly what set of software is on any given device. If you're doing full image updates, it makes it very easy to say, my fleet of devices is all running the exact same bit of software and the automatic rollback that goes to implementing the robustness. So if there is an issue with an update, is your system gonna detect it and automatically rollback to a known good configuration? And finally, how expandable is it, is the update system for your particular workflow and use case. The update system isn't gonna know about your database structure or what peripherals you have. And so there needs to be mechanisms in the update system for you as the system designer to be able to customize it and say, okay, on a new boot, I wanna validate the database and I wanna check this peripheral over here and make sure everything's working. So the expandability is something to consider when you are adding this into your system. And with that, I think we've got just a few minutes for questions. I got a couple links here that go into a bit more depth on some of these things. I would encourage you to check them out. There's a couple articles and a talk I gave last time that goes into a bit more depth about things like Acto and BuildRoot. So if you have an interest in that, feel free to take a look. And with that, I'll open the floor for questions. We got mics on either side. So if you have a question, just come on up and make sure that everybody can hear you and we get it recorded. All right, I must be in between lunch. Oh, we got one. I evidently not. Such as what, I'm sorry? Suso Linux Enterprise? Okay. What's less? Okay, yeah, so the comment was, and you're right, I didn't mention it. I mentioned Debian, I mentioned Acto and BuildRoot. There are a number of commercially supported desktop class distros such as Suso Linux that are supported and available for embedded slash IoT use. So that's certainly another consideration when you're looking at your system software strategy is, who do you call when something goes wrong? If you're starting with a completely open source thing, you might get help on a mailing list, you might not, but if you get a fleet of 100,000 devices in the field, you might want to consider some kind of support arrangement so you get somebody to call. All right, let's see if this side's working. Nope. I have a question about your product, Mender IO. Does it allow to enforce an atomic update of systems that consist of several Linux nodes? No, not today, it does not. Our product is focused on a single node. There, when you're talking about multiple node systems, you definitely need some kind of higher orchestration layer that can handle that. We've talked about ways to do that and certainly some of the expandability and plugins that we have can certainly can be used to implement something like that, but that would require a custom development based on what we have today. Thank you. All right, thank you very much.