 Go ahead and start. Hi, I'm James Pace. I'm from Runtime. We're going to be giving an overview myself with Marco Kiskila. I'm going to provide an overview of Runtime, very brief, and a bit on Apache Minute, an open source operating system for constrained MCU class devices in the Apache Software Foundation, and then get to how we're using a protocol called OIC 1.1 exposing it as an application layer, an application framework, and how we use it for management. So it's been a lot of time there. So a bit of background on Runtime and Apache Minute. So a couple of years back in late 2014, a few of us from a company called Silver Spring Networks set out to build a cloud-based management and monitoring system. So at Silver Spring our backgrounds were having built and deployed and managed over 23 million connected devices, mostly in the utility space, but also in industrial IoT and smart cities. And so we built systems on chip. We designed systems on chips. We built out modules. We built all the firmware that went on these devices. We drove two transceivers. One was a 2.4 gigahertz transceiver running Zigbee, a protocol stack that we co-designed, and an 802.15.4G frequency hopping spread spectrum implementation. Narrowband over 900 megahertz that provided really great coverage for these utility applications. So we put these devices in things like electric meters initially, then they gravitated upstream to medium voltage and low voltage equipment, things like transformers and capacitor bank controllers. And then ultimately as we were leaving, we were there for nearly 10 years, things like street lights for a canopy for smart city coverage. And so it was something of an unheralded project. We conceived the protocol standardized in the IEEE. It's called 802.15.4G, and then we deployed and managed it. So our largest footprint was over 5.5 million devices connected reliably at Pacific Gas and Electric, about 4.5 million at Florida Power & Light. And so we learned a lot about building embedded systems, but we also learned a lot about managing millions of connected devices that needed to be off on all the time and reliably connectable. So when we left, we saw this thing called the IoT was happening and we realized that we'd been doing that. And we thought we would apply some of our learnings on the management side to constrained devices. And so we started looking at a lot of consumer applications and what we realized was that the state of play, the landscape hadn't really changed much from the perspective of a developer building a connected product. So at SilverSpring one of our first decisions was to choose an RTOS. We chose one from Mikrium called UCOS. We modified that heavily. We chose a TCP IP stack from a small company. Modified that heavily. We had one poor guy maintaining ICV2 and trying to upstream his changes to the family business that never saw the light of day. And the story goes on. We developed our own bootloaders, secured our own bootloaders, did the over-the-air upgrade scheme. When we got out into the non-utility world we noted that people were basically building their own bespoke OSs in the same way. So one observation was that single chip solutions were becoming more and more powerful, more flash, more RAM, more processing power on Cortex-M class devices. And yet they couldn't run Linux or Android and yet they deserved kind of all of the functionality that you might see from a general purpose operating system. When you download Linux you expect to have protocol stacks, you expect to have bootloaders. In the embedded world you cobble all of that together and then you maintain your own bespoke OSs for the life of the product. And if you've got multiple products you may have multiple bespoke OSs that you're maintaining. So in order to build a cloud based management and monitoring service, client libraries and an agent style approach just wasn't cutting it, it didn't go deep enough so we decided to embark on an open source operating system called Apache Minute. And so the stack diagram there is basically represents what we're doing. So we have a real-time scheduler, a real-time kernel. We don't really differentiate hugely on that. But what we do have are all the components that you need to assemble a product and maintain it over time. So that includes things like a flash file system, it's Apache 2 licensed the ones that we saw out there were mostly GPL'd in a side. In embedded products everything is statically linked and there can be troubles. Some legal groups don't necessarily like the obligation or the perceived obligation of contributing back code to GPL products. So everything here is Apache 2 licensed. We include a TLV, lighter weight TLV storage mechanism with a circular buffering scheme, a secure boot loader that we've broken out of the project and are collaborating across multiple OSs on. A rich instrumentation of the microcontroller, the transceiver, and tasks in the OS stitched together by an OIC 1.1 logging infrastructure and the world's first Bluetooth low-energy stack down to the controller level which has really kind of been our differentiator to date. So it's a pretty expansive OS. We're getting a lot of traction particularly from companies building connected product with Bluetooth low energy because of all the configurability that you have. You have the opportunity to set breakpoints, direct access to peripherals. You can set your connection pools you can size them correctly. We're doing 32 concurrent connections which are important in a lot of real-time location service applications. We can trade off throughput for the number of concurrent connections. You just get a lot of options that you don't typically get when you buy a chip and you inherit a licensed binary from a vendor. And there'll be more protocol stacks coming soon. So we a little side note here. When we say OIC is the acronym for the Open Interconnect Consortium and what this diagram represents is kind of an evolution of the standards bodies or the specifications bodies that are emitting these specs back in the day. It didn't seem like too long ago. There was something called All Scene. It was perceived as competing with Open Interconnect Consortium, OIC. OIC rolled up UPNP. Those two organizations merged and became the Open Connectivity Foundation, the OCF. OCF is a Linux Foundation initiative. It has, I think, 11 or 12 Platinum members. It's a pretty substantial organization. But on the side, and this is sort of interesting, there's a separate Linux Foundation project called Iotivity which its purpose is to provide reference specifications, reference implementations for OIC and the successor spec which will probably be named OCF. So there's an Iotivity Linux implementation that's been out there for quite a while. But there's also Iotivity Constraint which is a microcontroller class, very parsimonious implementation that's completely different from the Linux base one. You had a question? Exactly, yeah. So right now when we're talking about it and Iotivity Constraint is based on OIC 1.1 but the successor specification will use the appropriate acronym provided they don't change the name of the organization again. But we're really happy with it. So this is my last slide and I'm going to hand it over to Marco. Why OIC and Apache Minute? We were referenced by one of our chip vendor partners, a customer who really liked Apache Minute, the totality of it but they wanted to use OIC 1.1 as an application layer or an application layer framework and their idea was, wow we really like the fact that you guys have a controller level implementation of Bluetooth Low Energy. If you could go all the way up to the stack to the application layer we can develop apps for that but also we, by having this implementation that deep and by virtue of the fact that it's an OS it provides hardware abstraction and what they get is leverage across chip vendors. So as we port this to other chips, so we're initially on the Nordic NR-52 we followed that with the NR-51 and then we're also porting to the NXP, the old Free Scale KW41Z. So now these companies can actually have optionality and pricing leverage against the chip vendors and I think you're probably going to see some cheaper remote chip solutions coming out over the course of the next year or two. So this company wanted to do that, we took a look at it and they're also very frugal and they're using the older generation of the Nordic semiconductor chip, the NR-51. So the NR-52 has a bit more flash, bit more RAM, a slightly better processor but the NR-51 is really constrained and that's what we're talking about here is developing a constrained environment. So we started to look at the protocol, we thought there's actually some really, really admirable aspects to it. In particular it uses CoApp a constrained application protocol, it's a RESTful protocol that's been specified by the ITF then there's also CBOAR, a concise binary object representation for JSON and we like that because it has respect for small frame sizes. So a lot of the transports we're talking about going over, Bluetooth Low Energy has incredibly small frame sizes but also things like ZigBee or Thread which operates on 802.15.4 radio has 127 byte frame sizes so having that built in was great it was goodness because most of the application use cases around all scene or open interconnect consortium when these specifications were being defined were really for home use cases and transports of a Wi-Fi which has like a 1500 byte MTU. So seeing that that was great we started to pair that down and then we also had a proprietary logging infrastructure. So runtime when we initiated this and you'll see Apache Minute has a huge focus on this, we care about the upstream facing interfaces, we care about secure boot loaders and how you do over the air upgrade in a very consistent way. We care about abstracting away instrumentation so that we can manage the system as if it's a, you know, as if it's a, I'm beating myself up, as if it's a server in a data center but it could be in a lock or a window latch or a wristband and so we care about managing those resources and so we defined a proprietary logging infrastructure and then we thought well how do we squeeze more out of the space and actually this OIC implementation looks quite good for doing logging and having a standard there is actually quite good as well. So Mark will talk quite a bit about how we plumb that into the system and use that as a core of our logging infrastructure because these systems, you're not running SysLog or SNMP or any kind of traditional IP based management protocols on top of these constrained devices and constrained networks. The other thing that we liked beyond just the efficiency of the protocol itself was as a solid security framework we didn't want to reinvent that and service resource, service and resource discovery. So some of the use cases that we see beyond home networking or industrial IOT applications, sensors mapping different sensor APIs into OIC so that an upstream device can simply discover all the functionality of a downstream sensor is great. So when you're talking about industrial IOT you typically have these gateway applications, they call them the edge but there's an edge beyond the edge and that's these smaller devices. And one of the challenges in really getting industrial IOT out in a deterministic manner and at scales you need to manage all of these devices. The gateway shouldn't just be just dumb protocol translators and trying to figure out what these things are doing at the other end. The connectivity needs to be reliable and deterministic and the device itself needs to be manageable. So sensors are a great application but then there's things like every light bulb is going to have a Bluetooth chip in it one day, a Bluetooth carrier and it's probably going to have some sensors in it too. So we think that there's commercial and industrial use cases that will be really well served by this. So with that I'm going to turn it over to Marco who will talk about the implementation and really kind of where we're pushing things and some of the things we'd like to see. I'll give you this. That better? All right good. So if you attended Kisha's talk yesterday, some of this is going to be a duplicate data. So I was going to first talk about OIC a little, what it looks like. Just high level, basically what kind of protocol it is and what does it entail. So the goal of OIC is to provide two different places where they give rules how you do things. One is how to abstract resources, how to export them to OIC framework and another one is how to map these resources into protocols underneath. Protocols like HTTP, co-op or both of these are restful. So there's server, there's client. Then also it includes rules about resource discovery. You should be able to do that with your protocol mapping. Rules about what the protocol needs to be able to do. That's the basically get set and then there's also this notify mechanism. Basically what that means is you can register to be notified if a resource changes its value. So it's not just client asking server what's the current state, the server can also tell that there was an update to this. And then the IOT with the reference applications like James already mentioned. This is that same thing in the form of a picture. There's a client, there's server. These logical operations create, retrieve, basically set the notify. They go from client to server. Underneath there's the specific protocol OIC runs over and there's co-op here. It took very well the HTTP if needed. So the resources What's a resource? Light, bulb could be a resource. There's an example of a resource in that box. OIC declares that you have to have certain fields always present in a resource. You have to be and the client should be able to discover what these properties are. Like this light resource is retried. You can change its value. Does it look like it's something that there would be another field saying if you can register to be notified when it changes. You can see it looks like JSON there. That's how OIC publishes its data models as JSON. This is over the air, this is encoded as seaboard. There's key value pairs. State is on, for example, key value pair. So resource discovery. This is interesting to the client. Especially if you try to write generic clients you don't know exactly what you have on your network. Being able to do this eases up the deployment quite a bit. Depending on the underlying protocol you have different things. Co-op runs over UDP. So it uses UDP multicast to initiate the query. Responses come back as Unicast. That uses MDNS to publish information about this service. With Bluetooth you advertise it in the Bluetooth advertisements. You say that UDP is present on this device and that's how the client knows that this is a guy who can talk OIC. After that it can go ahead and ask for the resources that the device exports. Resource discovery with UDP multicast is just a co-op request to asking everybody to respond to OIC RDS URI. The response contains a lot of the data that the device has. You can also filter if you formulate the request you don't get all the OIC clients responding. You can limit what type of devices respond to you. After you discover these guys and you want to know more you use these to other URIs to get more information about the device. And then operations on resources. This is standard restful interactions. This is the acronym that OIC likes to use. The most common use case would be get set and notify. The notify indeed there's certain resources you can register observers for. That server can send you data about changes in the resource. Then IOTVT there are reference implementations you can just run with this if you have a bigger system. This is a great library because it's multi-platform Unix I think there's iOS there, Java for Android and so what we used the big IOTVT library for was to write the clients on our Unix machines. But to get it to fit into a small system you can't it's way too big. There's IOTVT constraint which we initially took pretty much a CIS and ported it over. It worked okay for NRF 52 which has more memory than these 16K guys. Then we had this request coming in that we had to be able to fit it into a smaller device. After that I had to take it under heavy modifications. Basically just to cut down, mostly to cut down the RAM use. Our implementation indeed is based on that. We do IPv6, Bluetooth, low energy and also we run co-op over serial port if when appropriate. We augmented this code with the advertising tests to the constraint framework. Yes, we did. There's a gap. So the biggest problem we had for running the IOTVT constraint was not so much the text size code size, it was definitely the memory size. It had it uses memory pools which is great. We like that. It's much easier to size your memory to match your application. If you have a leak, you can easily figure out there's only limited users for it. It's a great recommendation. All that is good. Unfortunately there were a lot of those memory pools and what IOTVT constraint is it has implementations of features that you would normally find inside an OS already. So I systematically went through the code and replaced their use of their own implementation with calls to minutes matching code. Decided for timers, control flow, and a few things other than that. There was also the biggest code savings had to do with data buffers themselves. What IOTVT constraint expects is all the data to be contiguous and them being in control of that memory pool. These entries are large. It has to hold the full frame. So minute has its own network buffer scheme and buffs which are linked together. So I had to go through the code and replace all the calls. All the places where data was read in instead of just expecting the data to be there, replace it with a call that actually can follow a chain of mbuffs. I basically abstracted to use this disk and to go memory buffers. And once I did that, I could share this memory with our Bluetooth stack. Therefore, overall, less memory footprint. Also, zero copy, which is nice. And another thing that yielded significant benefits was the way the payload of the request was decoded. The implementation in IOTVT constraint, the way it does this is it takes the whole payload and out of that C bar, which looks like this, and it creates a tree of objects. And all these entries come from memory pool. The data gets copied to another block of memory, basically the values. And this is done automatically before calling the resources, get or set handler. So I replaced that with the version that actually does this decoding on the stack. The user provides pointers to where the data should be encoded, should be placed. So I could get rid of all that memory. So now, if it's temporary data, you don't have to hold on to it at all. If it's data that you care about, you can start it later or just decode it directly where you want to retain it. Right. And it was not just IOTVT constraint that we had to modify for these for our network buffers. Also, tiny C bar, which is a library, which is small footprint library for decoding and encoding C bar. Also, with that we replaced the bottom half of it where it asks for more data instead of just expecting a flat buffer. You have get set routines for getting or setting different amounts of data. And that can deserve the data to tiny C bar either from a flat buffer or from this non-contiguous memory buffers. Yeah, that's right. Yes, we did talk with Kishan and Diego about doing that. Some of it. So, first of all, when I started doing this, it was just whether we can fit it no matter what we have to make this happen. So, therefore, upstream was not concerned. Some of this data is definitely something that can be pushed up, like the tiny C bar changes. I think that would be valuable overall. A lot of people do have used this chain network buffer. That can definitely come up. The fact that I moved the abstraction between the OS and the IOTVD constraint, that's also something that can be pushed up to IOTVD constraint. So, yeah, but we'll see what the maintainers of IOTVD constraint will do about these individual things. Yeah, it is it was interesting results. So, here, I think the salient lines are the ones that talks about the OIC prior to restarting the optimizations and after. So, 16K of RAM. Bluetooth stack there in addition to OIC and you need to leave space for the user application as well. Very important when building these frameworks is you can fit your stuff, but how about the actual code that does something there. So, anyway, we got good savings on RAM and as a result of relying on OS more also the text size went down quite a bit. And now there is, I mean 16K of RAM there isn't much to start with, but at least there will be some left to play with afterwards. All right. And what next? So, we are using OIC as our management protocol as well. So, we export anything that you can either for debugging purposes development purposes or for managing devices on the field. It's all OTA it's done with OIC. So, we are going to look at contributing these data models back to OIC and also we'll try to help upstream the changes that we have done to the OIC itself. So, this changes I think there is a completely new class of devices that you can run this stuff over. People like cheap, they don't want to pay more for the MCU that they absolutely have to. Yeah. And we try to be good citizens. Yeah. Brian? Yeah. I think it's TVB. So, we've talked to Diago and Pichang. And I think that the constraint of implementation may not be obligated. Super set of certifications that I see that he does. So, I think the thing that need to happen is first of all there's a basically the way you do this over the only gap, the only 4.2 implementation that needs to be codified and standardized. In fact, there's been a working group in the OCA for that but I'm not certain how much progress has been made there. However, I think having a reference to communication showing an actual application or a set of applications and particularly managing the devices using OIC intrinsically to the OS, I think that will help drive that conversation. So, we hope to have that in Amsterdam, here in OCA. And then, so, we think it'll be a certification. I'm not sure that it directly lines up with what's been done in the past. So, other than the memory impact, do you think there's been any performance in the past studies comparing bare metal and wireless running the OS? Not so much. The problem with running just bare metals without OS here is interactions with the Bluetooth. In this 16K application we don't... well, all the packages are written such that you can call them as libraries. In theory, you could get rid of the OS altogether. The only exception is the Bluetooth controller code that sort of runs on its own context. That's why it wants to run. And, you know, it's not running in an interrupt. So, you know... That's right. Yeah, that's right. Yeah, even with Wi-Fi, you know, I mean, you don't need an OS. You know, it's very flexible. Right. Right. Yeah. Yeah, take it for a spin. I think for me, you know, the number of cycles has not been that big of an issue. It's always about RAM more than, you know, how fast the response comes back. So, that's why like... You know, you can build the system however you want. So, what we have made possible is, you know, you can either have just two tasks and, you know, that's it. There's the controller and then there's everything else. And you basically have no context between except to the idle task. Or, you know, if you don't have Bluetooth, then you can have just single tasks. That's how I, for example, run the OIC over Wi-Fi on 7D21. Right. So, at the moment, the question is, you know, OIC Discovery co-op over UDP, this multicast, this is a LAN-based thing. So, how do you do this from the Vans side, basically from the cloud? So, at the moment, the model we have for this is existence of gateways. So, therefore, the entity who talks OIC to the endpoint would be present in that local LAN or in close vicinity. We haven't thought about it too much, you know, how it would look from Vans side. That's right. So, if you look at the... Actually, there's a... So... No, not really. Right. Yeah. Yeah. Yes, absolutely. Yeah. Bluetooth, you know, there's no LAN. You are not going to connect to LAN with Bluetooth. So, therefore, not the immediate call. So, the current implementation of Bluetooth connection with OIC in IOTVT actually uses TCP-style headers on the co-op. So, you know, that didn't exist in the IOTVT constraint. So, I added it because I just wanted to talk to normal libraries, you know, to perform all that goodness. So, interoperability. Sure. I love it. You know, in my previous life, I was working at the IOTVT company that was using RESTful stuff. Not OIC-based. Notify was definitely used. Yeah, it was okay. Well, scaling usually was not an issue because the number of users or clients connecting was not that large. Yes, then there's an issue. Yeah. This was more of a home use case. Right. Right. Right. Yeah, that's true. But, you know, if you want to do this securely, then you know, I mean, doing security over multicast, you know, especially on a constraint guy, you know, it's something or another has to give. The first answer is that's certain to answer it. But it's how we use it. How we use it is our infrastructure. Do you want to talk about that and then I can talk about the customer use case? Sure. So, yeah, the I assume you're asking about the updates to the IOTVT constraint that how do they trigger into this code base? Oh, into the device. So, we have a OTA framework which existed before we started doing OIC. So, secure of firmware updates with a company bootloader. The bootloader is now being used elsewhere as well, you know, forked off as its own project. But in addition to that, you know, you need the process of getting the images to the device and you know, deciding when to switch over and you know, being able to fall back, you know, in case something happens when you try to boot it up. You know, that stuff is already in existence. What we did was we just took our existing implementation of these things and you know, mapped it over OIC. Actually, before, in our previous revision of this, everything was JSON. We had to do JSON because JSON is, you know, it's not, you know, condensed. However, people who write clients, you know, on various platforms, you know, for phones and, you know, laptops, you know, they have very easy libraries available for passing JSON. Luckily, Seabor is more dense, so you know, it's not as costly over there and pretty much you get the same abilities in terms of ease of implementation for the client. I think I sort of started talking about different things. But anyway, upgrades, yeah, you can create these binaries. And how would we maintain this source base? This is a part of the Apache minute source tree. So, you know, whenever we need new features, you know, it's going to appear there. And you know, we intend to follow hopefully us and the IoT with the constraint implementations to converge at some states and, you know, we can actually just play with the same code base. Until that time, you know, we'll hold on to this one because, you know, it works for us. And so, we do OTA. We drive that with the OIC Bapings, but some monitoring. So, we have Bapings to how to object objects that are defined to abstract with microcontroller, so they're really protectives. We do have MIP support, not a PIC32 port yet, but as a progress, about four to five versus five ports. So, the abstraction there is good, and you know, OIC is useful, whether that's, you know, doing logging, upstream to a management device or a local system. Same goes for transceivers and tasks in ALS. We have signed up. So, we're also members of the central project and we signed up to do the logging infrastructure. We hope to leverage that on possible projects. Stepping back, and aside, for us, it's important to have all of these common upstream management interfaces irrespective of what the OS is. We don't think there's going to be any singular OS that has 100% of the market tomorrow. It won't be Zephyr. It won't be Apache Monday. There's others like Riot who's still here. There's Kotiki. There's RTOS. If those other OS initiatives, or RTOS initiatives, structure adopt these components. It's easier for you guys and product developers to work right management systems. So, that's been important to us. So, we want to see OIC based on structure adopt, you know, across all of these initiatives. In terms of the customer, it's a pretty simple use case. It's a consumer-issue use case, but they basically just want to discover objects on a consumer device using an asset. So, they have a framework already that they have on iOS and everything with devices, and that's what drove them to adopt OIC. I mentioned before, I think there's a lot of industrial applications and sensor applications that will be lost by this. Right. So, yeah. Yeah. What to thank the IOTD constraint, guys. It was great to start with the reference as opposed to writing everything from scratch. It was great that protocols came with this kind of code.