 Hi, my name is Jason Kreidner. I'm a board member at the Beagleboard.org foundation and Beagleboard is a non-profit that makes Linux and Zephyr now, newly. Zephyr based open hardware platforms, right, that run open source software. And yeah, we certainly love all sorts of other projects like, you know, BSDs and all the, you know, wonderful non-Linux foundation projects, but we have a special liking for the Linux foundation projects with Linux and Zephyr, right. So we really appreciate the way this community works together. And you support each other from hobby, self-interest projects, you know, projects for other people all the way to professional product designs. And my experience with the Linux developers is they deeply care about educating the next generation and making sure that we're preserving the knowledge base for people in the future, right, with wonderful tools like Git that allow us to look at the development history and understand where things come from. I say all that because in this talk, I want to talk a little bit about how we preserve knowledge, right. How do we pass that on? And different ways we could do rapid prototyping and leverage software I think have a significant impact on that. There is a scourge on the face of the earth. No, there are some habits that I don't think are necessarily healthy for the open source development world. But I want to talk about a number of different approaches, including ones that are very effective, but maybe not the best idea. And I think that it's also, I want to talk some about the benefits of using both Linux and Zephyr together. I don't have enough time to really talk about making the trade-offs like going between one or the other, but they're both really fantastic projects. How many people use Zephyr already? This is a Zephyr, yeah, okay, wow. So you know how it can make really, really tiny images, but yet it's very, very capable, right. There's a lot of features, but you don't have to bring in the features that you don't need. But sometimes there's some lift that you can get from a Linux system in the loop somewhere. And that's some part of what I want to talk about. Okay, so I'm going to look at specifically these six different kind of mechanisms for rapidly prototyping an IoT type of solution, right, you know, Arduino code on Zephyr. Anybody know that you could run an Arduino code on Zephyr? One, all right. So you can learn something new. MicroPython, probably a lot more of you knew that you could run MicroPython on Zephyr, not so many. Yes, there is a MicroPython port to Zephyr. I think, and I think it's actually really, you know, there are some ports out there that I think are somewhat more complete, but the Zephyr port for MicroPython I think is actually really good. And there's some nice integrations in with Zephyr, Zephyr RTOS as well. I'm going to talk about doing Python user space code, but on the Zephyr device, that probably doesn't make any sense yet. But we're going to use some Python code to control peripherals on a Zephyr device. So you can kind of recommit the same atrocities that people do with Python libraries on Linux devices today. And then we'll clean it up a little bit with some Linux kernel drivers. We'll just really just kind of talk about the native, the benefits of native Linux drivers and the roadmap towards how many of you have heard of Graybus? Not so many, just handful. How many of you have heard of Project Ara, the defunct mobile phone, modular mobile phone? It's interesting, those weren't all the same people that were close. Anyway, so I'm going to go and kind of look at the pros and cons of the different approaches. And, you know, we make these little boards with software on them, that will kind of be my focus. But this really doesn't matter that it's these boards. The idea is we've got a microcontroller running Zephyr and a, you know, a computer running Linux and, you know, some sort of network connection in between them. In our case, it's a sub gigahertz wireless network connection. And, you know, just looking at the, why do we want to do that? Why do we, what's the idea of why just having a Linux system and why having a Zephyr system in there together makes sense? Linux nodes, you know, are by nature a little more memory hungry. You know, you know, we can strip down Linux to run on all sorts of things. But the Linux that we are probably most know and love, you know, offers a lot of capability to do things remotely, right? To use the kernel itself to get visibility to what's going on inside the system, right? To do more complicated networking solutions with multiple different types of networking stacks and routing. And, you know, to do a lot of Linux is designed to work for servers, so not necessarily sitting at your desktop as well, right? So yes, it works great on the desktop. But it's designed to be used remotely, right? That's how a lot of people do it every day. You've got shells, very powerful shells you can use and very, you know, lots of logging capabilities, you know, virtualization, you know, access to large storage and network based storage and a lot of features that you'd want. But of course, for these data nodes and for a lot of things, you need something low cost, you know, and therefore Zephyr makes a really good solution for that. And while Zephyr's drastically growing in capability very quickly, there's probably still some point at which that data is going to touch a Linux server, right? It might not be on premises. It might be in the cloud. But I think it's just worth paying attention to that Linux to sensor node with Zephyr platform. I think there's just a lot of work that we can do to try to simplify that experience for developers. So it's probably going to go to Linux anyway. So the work that I'm going to kind of show this are the two get repos. The first one is the Zephyr. We have a hate to use the word SDK, but it's just a patched version of Zephyr with some kind of a little bit more bleeding edge stuff and a bunch of example projects that are compiled under CI, right? So the projects that I have here, either they're building under the CI now, or they will be soon so that you can easily just kind of pull down those examples and run them. And then the Linux, we were using a Debian-based system and using a bunch of shell scripts and Debootstrap to build that system, right? But all that customization is stored in those couple of repos or the repos that they reference. Yeah, so Arduino for Zephyr. So last year under Google Summer of Code, there was a Zephyr plus Arduino port done. And so they took the Arduino core and so you've got the Arduino library that essentially wraps the Zephyr calls, right? So you map all of the different the index pin IDs, right? Like you'd have for the board IDs on an Arduino into where you'd have like the GPIO bank and index, you have the chip and index IDs in Zephyr, right? So they do a mapping. Therefore, you can just call the Arduino functions and reference the pin numbers by the single integer index or the analog IO index equivalents and do GPIO. They also tie into the PWM, SPI, UART, I2C, and ADC subsystems of Zephyr, right? So you can just use the Arduino calls for all those subsystems. The build environment, at least as I've seen it as it's out of this project right now, is still the Zephyr build system, right? But you are compiling the wrapped, if you're from... How many of you have written Arduino code? Yeah, so the setup and loop structure that you're familiar with, right? And the header files that you're used to including for the different functions, right, are the same. So that code is the same. But when you want to build it, it's not under the Arduino IDE right now. Right now, you just do West build and you build it like all the other Zephyr projects that you have. It's just now you have an extra set of code in the loop. So this is something, if you're not familiar with Arduino, a lot of people that aren't doing computer science and electrical engineering and embedded systems every day are very familiar. So that's a great benefit. So there's a lot of code base out there. But I would argue that's also a really bad reason to use it, because there's a lot of people can do it and a lot of people really probably shouldn't be doing it. And this is all example code that nobody does like real QA on, right? They just publish it somewhere. I'll touch on that a little bit more. But the nice thing about... So why is it better on Zephyr than just on its own? Well, there's not much of a networking stack in Arduino, but we know that there's a really nice networking stack in Zephyr. And because it's really just Zephyr, you can make all the calls to all the Zephyr functions and you can still configure everything with the def configs. You still have full flexibility. But now you can copy and paste Arduino libraries into your code. And of course, I think one of the really nice things that you've got all this networking stack, but you also have MCU boot and MCU manager. So you can do remote management. You can do firmware over the updates when you have Zephyr underneath the hood. So it gives people a very familiar programming environment with Arduino, but that powerful capability of Zephyr under the hood to do remote updates. So this is what it looks like as an example, right? This need to look at that one a little bit more honestly, but it's not perfectly clean yet. I think it was an intern job over a summer with some good mentorship. But you're able to go and just take the familiar Arduino API and use that and you can dump out to the UART port just as you would expect with an Arduino, right? But here you're running Zephyr under the hood. Even things like the LEDs, all our Zephyr projects are required to kind of declare LED zero. It'll grab that as the equivalent of like pin 13 and Arduino for blinking the LED, right? So Blinky just works on Zephyr boards, right? So that's, I think this is, I think this is where I kind of stopped, but I hope some of this was good news. I hope. I don't think this is good news, right? So, I kind of, I went back to my original drawing, right? I'm going to keep solving this same problem six different ways, six different kind of programming paradigms to go and solve this problem. This is just an OPT 3001 sensor. It's actually built into the board, but I kind of drew it on here anyway to make it look like it was something else. But I want to talk to that sensor. And so it's like, well, how do I do that on Arduino? And, you know, like every good engineer, I just Google it. Pause. And, you know, I'm returned a few things. Well, one of these is actually showing up on an Arduino website. Sounds good, right? But it's a third party library that seems to just kind of be fetched and dumped on here that hasn't been touched in six years. And that's what you find on the Arduino site to talk to an OPT 3001. For me, this is scary. I don't know how it is for you. But, you know, I had every hope that this would still work, right? This Arduino has been stable and used for so long, right? This should just work, right? No. Not really. I don't know how much of this is honestly in the limitations of the code itself, but I never could get the OPT stuff to run within the... Even though the wire examples work in the Arduino port to Zephyr, I could not find a working example just out on the Internet of Arduino OPT13. Yep. I had a little better luck with Python. So the micro-Python port for Zephyr, I think, is really interesting, right? Because you get this self-hosted read, evaluate, print loop. I think I figured all the... I remembered all the letters. But, you know, you have like bring back the basic prompts, right? Nobody's old enough for that except me. And what you can actually do live coding, so you've got, you know, some benefits that you'd have to, you know, from a dynamic system. And there's just a lot, a lot of emerging examples coming out for Python, regular Python and micro-Python to go and build on, right? This seems to be kind of the new Arduino in a lot of ways. And, yeah. And of course, those benefits that we talked about that from Zephyr Arduino getting, remote over-the-air updates, you know, rich networking stacks, right? Those come with the Zephyr port for micro-Python as well. So you have socket commands. And although the socket wrapper is not very complete, the reading of sensors using the I2C interface actually turns out to be pretty easy and does work. But the... So I can use Python code that reads the OPT-3001 sensor and make that work. And I can do listening examples that work fine. And I'll show you that. The... I'm doing an IPv6 packet transmission for the architecture that I was building here. And that did not work in the micro-Python port today, right? So you couldn't generate the packet template in order to fill it in. And there's just a couple of wrappers that are missing from the port because it must not be being used enough for IPv6 socket networking. So we... So I've got a version of micro-Python compiled for the Beagle Connect that I published. And if we look at how we can use that, there's this really wonderful thing that you can actually just leverage the Zephyr sensor drivers, right? So yes, you can do the micro-Python and the Python code to get that to read the OPT-3001. But I can also just do this little bit of code, nor the humidity sensor kind of shoved in the middle there. But those few lines to go and read the value. Again, I couldn't get it to transmit the data. Oh, I left out the one that I wanted. But I could easily get it to receive the data from the other ones that are using just a pure native Zephyr to broadcast the packets, right? So the IPv6 packet reception in micro-Python worked great. Any questions about micro-Python? Because I'm going to try to drive into the gray bus stuff, right? I think I've got about halfway in my time. Micro-Python or Arduino stuff. Is that of interest? Is anybody wanting to do something with those? Or those just seem like too much of toys? Okay. Yeah. And understanding what... Oh, sorry. Tim was talking up front. So the thing about Arduino and micro-Python, circuit-Python, is that it's a very easy on-ramp for very young, new developers. And I'm talking as young as like three to five years old sometimes. There's other things like scratch and so on. But I think it's important from that perspective that we can have young people on exactly the same hardware in the same thought process, but with a little bit easier lift for the first learnings. Absolutely. When they're understanding these wire interfaces to devices for the first time, those programming languages give them a fairly simple paradigms for understanding the I.O. and they even have block-based generators generating both of those code sets. And people are building great virtual machines that run on top of them that even abstract things more. But I think it's interesting to kind of get this stuff running here so that there's a roadmap to move them past that, right? To move them into a more native OS driver environment. Explaining the need for drivers, I think is something interesting. Question? Tom Hanna, Tom Ogerman Holdings. I wanted to emphasize that again from my personal experience, the value of systems like the Arduino and the micro-python. I mean, I'm an electrical engineer. I've been retired for a few years. And before I retired, I was financially done. And before I retired, I had a nasty fight with Massimo Banzi at the conference, basically saying that the Arduino is a toy and I trained cadets on 8-bit assembly. And while this is true, we should always keep in mind that the low-level entry basically makes electrical engineering accessible to non-traditionally ease. Like for example, my wife is a Java developer. A Java developer doesn't do assembly. But with the Arduino, she was able to automatize half of our apartment, for example, without even just I came back from a trade show and one day I saw it was automatized. This is the kind of value which especially an electrical engineer often doesn't want to see. And I mean, I'm going to get hate mail for this from other electrical engineers. But I really think one should keep always this in mind, the non-traditional, non-EE embedded guy. Thank you. I actually agree with you. But I think that we need to understand their place and understand how do we teach people about why we want driver models. A lot of the other R-tosses out there don't really have driver models. You can't do something like use a single sensor interface to go and read whatever sensor that you might have connected. Those sort of subsystems aren't there. And those really can provide a drastic simplification of the rest of the code for a little bit of headache. And ultimately there's one code base. So this is not having the problem of wondering about the quality of the random library that I pulled down. I mean, the great thing with all those Python libraries is I can debug them. But God, why would I want to, right? When I know that Zephyr and Linux have debug drivers that just work. And yeah. So that's... Just one question for myself. I agree with the value of MicroPython and Arduino. So because of that value, I wanted to ask if you're aware of the current situation in terms of contributions and maintenance for MicroPython, because you've been... The specific portent effort. I don't know the details of that. I've seen a regular reoccurrence of commits that are coming in. And some fixes. I think a lot of the things that it needs to be kind of first class are not that hard to go and fix. I would love to see some people motivated, because I think it provides a very useful gateway into operating system-based programming. One of my fears is that you generate too much unknowns, right? I think kids these days, you know, when they think of Arduino, they think it's like maybe how I grew up thinking about computers. It does exactly what you tell it, right? Because that's how I grew up, but computer always did exactly what you tell it. These days, a lot of your computers don't necessarily do exactly what you tell it, because there's too many other people telling it to do different things. And the one... Like the counterbalance to me in some respect of putting in operating systems and putting that into these... The Arduino and the MicroPython programming environments has introduced things, a lot of things that are unfamiliar or unknown to the developer. I think that with Zephyr, however, it is a reasonably easy to understand codebase. Once you get past understanding how the device tree maps, you get the generated device tree in your output, right? Once you see how the config stuff maps and you get the generated configs, you can figure out exactly what code is generated in your output, right? So... And you get the benefit of all the other eyes looking at it. So I think you can generate very predictable systems with tools like Zephyr. I fully agree with you that Zephyr provides the more predictability. I fully agree with you. You are running into open doors, but please, think of the cadet. Don't think of the situation from a professional electrical engineer's point of view. Think of, basically, think of somebody like my wife. She had access to a full lab for many years and she was always, no, I've got my Apple, I do my Java, and this is it. And then this person doesn't care about the deterministicness of execution. She doesn't care whether the window blind goes up in 10 or in eight seconds, but she will care after she has done the first experiments. Forgive me for being blunt. I see it like an entry-level drug, the Arduino and the MicroPython. It gets the people hooked and then they automatically move to something different. I think where the determinism really matters is in trusting that the computer is running the code that I've generated for it, right? I'm sure a lot of people here have generated assembly from their C code so that they can figure out if the C compiler is actually generating what we thought it was going to generate, right? And we can do that, right? But bugs in compilers are still there, but they've gone down, right? But I think where I'm trying to emphasize is I don't want people to lose that idea, a computer just does what it, you tell it, right? Pulling the plug because it's doing something funky is not the answer to the future, right? I think we need to be very conscientious of the technical debt that we're building on top of. So it's a project ARA. Project ARA was an attempt to design a modular cell phone. This is a different type of technical debt, right? We throw away our cell phones every couple years to try to get another whole platform that we don't have access to and understanding of what's going on inside of. But this was an attempt to sort of break down the problem a little bit and make each individual module something that you could upgrade or replace or even design and put in, you know, and a modular basis on your own, right? With published IO standards, things that you can actually go and monitor and look at, and ultimately you could replace individual parts of it when there was something potentially more open than something else. And unfortunately the project was killed, but the good news is that Graybus out of the Project ARA project is upstream. It's in the Linux kernel. We can leverage it and we can do some fun and interesting things with it. So they had the need to make iSquared C and SPI and UART essentially hot-plugable because that's what all the little embedded electronics are using. And so they needed a way that they could plug in and get a new iSquared C bus of what they needed to talk to as an iSquared C sensor, right? So they just put microcontrollers. The microcontrollers can be very cheap, cheaper than the sensors and the memories and the other things that you might be interfacing to and make them work together. And yeah, it's in the mainline kernel. The reference is Unipro, but there's been some, it doesn't have to be, right? Because it's it's just a swappable portion in the driver code where you can put as inchly any transport you want because now you have packetized raw bus transactions. You can also have high-level sensor transactions or display transactions over Graybus, but you can also just have an iSquared C read, right? SPI read and UART reads and writes over Graybus. And now you can put them over a network like I'm doing here where you can put them across other wired interfaces, most of this is prototyped through USB connections. Yeah, so I think we're all familiar with how nicely USB works on a computer, right? You plug it in. It notices you've plugged something in. The computer asks, what did you plug into me, right? And the thing that you plugged into it tells us what it is. It then loads the driver to match what it's talking to and it works. But that's not so much so in embedded world. In our world, right? You connect up over iSquared C, then you write some device tree and you recompile. And if you're using Linux, you're a little luckier because you can just recompile part of it. You still have to reboot because we don't have upstream dynamic device tree overlays. Frustration, right? It doesn't tell you what's connected to it. You've just got a bus and you better know ahead of time what it's going to be and you have to code up support for it specifically. So Graybus kind of fixes that. It essentially assumes a microcontroller on both sides of the loop. I put UART here. It really doesn't have to be UART. It just has to be some way that you can talk to the microcontroller. In fact, this microcontroller doesn't even necessarily have to be there on that side. But you do need this microcontroller here on this side to break out the commands into the various transactions, iSquared C, SPI, GPIO, PWM, A to D that you'd have on a typical embedded system interface. When you use Graybus, you connect the device, you probe what it is, you have something now to answer you and tell you what's there. You send back a manifest description. It could be anything, but you just need something to go and tell it and there's a specification for Graybus manifest. We've extended it a little bit to handle a whole combo microbus description and you load the driver and everything works. Native Graybus itself really just says you can either have the sensor that you're talking to or you can just have a raw bus and there's more. But with the extensions for microbus, it'll actually define what it is you're talking to and load the driver. So this is the software stack that we built up to prototype this on. This is not the way it's going to end up. So we have a Beagle Play and a Beagle Connect Freedom. So the Beagle Play is running a Linux and it has also a microcontroller on it to do the network transport. But it takes Graybus and it introduces this thing called GB netlink which is you just use socket style interface to just to communicate with a service in the kernel. So you can trap those Graybus accesses and move them up into a user space app where we can route network packets. That's the real reason it's there so that we can do a Vahi-based discovery of devices and manage the security keys and all the things that you want to do that are a little bit hard to do in the kernel. And it's a user space app. This has to go. This is prototype stuff. But that's what we're doing today to get it all going and then we're going back through a six low pan network over a sub gigahertz network. We've got an IEEE 15 4G MAC interface over UART and then a radio interface to descend standard TCP IP over six low pan on a sub gigahertz network. And that's what we've got running here. And then the sensor node running Zephyr and the Graybus for Zephyr application which is also just available as a module if you want to plug it into other things to answer the Graybus requests including providing the descriptors and then driving the bus transactions. So to start the user space service you run a shell script. It pings and makes sure that something's actually alive and answering on the other side. And it loads up the Graybus service and now you're talking Graybus to devices. And it exposes the thing I'm trying to merge too many things in this concept here with the Python stuff because my goal here was to take the I squared C dev interface that gets exposed, remove the driver that we automatically load for the kernel for it and just get down to the ITC dev. But you kind of quickly get into a lot of the problems, the quality problems again from not having something that's centrally maintained from experts that are actually trying to make products out of things. You get stuff that doesn't work and hasn't been touched in ages because it worked once and nobody's actually using it. So when you go to PyPy and you pull down the OPT 3001 package and PyPy, it just doesn't work. I pulled down the examples from how they had it. I had to then not include my little yeah, anyway, it just doesn't work. I did a little bit more searching because I wasn't quite happy with this. And I found another one that wasn't in PyPy. It was just a standalone piece of Python code doing the ITC dev transaction. I had to patch it with the line of said because it didn't even provide the ability to specify which I squared C bus that was going on. Yeah, you know. And once you do that, you run the little Python script and being I'm using I squared C dev transactions over gray bus to read an optical sensor. Let's go the other way to the other option or I could just use the existing Linux driver and with the manifest description that's on the the micro bus boards that says what driver I need to load and which pins are what. I can just utilize the Linux driver that's already there and use IO info to provide me the output of the optical sensor. I appreciate that people like to write Python code. I don't know why. I'm overly snarky there because I really do want people to have a gateway to learning to program and moving things but I really think as a community we need to be thinking more about how we onboard these people and teach them about why having a single upstream be it in Zephyr or be it in Linux is really, really important. Having a device driver model means I'm getting shaped data. I just did IO info. I don't even know what kind of sensor. I don't care what kind of sensor it is. It's just going to show up and it's going to tell me what type of sensor it is and it's going to tell me it's going to put it in standard units that I can just use. And then I can have all my Python stuff to go and grab that and put it up to the cloud. Use your hacky Python stuff for that stuff, not the stuff that's actually talking to the device driver that when it doesn't work your whole system just fails. Anyway, yeah, this is a rant presentation. So that's Linux. Linux, you do the great bus stuff and the stack is fairly complicated. It's not super stable today because it's relying on a user space driver. We're rewriting that and we're putting the G-bridge stuff into Zephyr as well. And so when you come back in here, the G-bridge function will be in this microcontroller as opposed to in user space and it's going to manage the routing table and the automatic discovery and the addition of the devices. So this coding is in progress so that we can have a nice stable G-bridge interface somewhat akin to what you'd have in the Unipro hot plug approach. But let's say distil to me is kind of rapid prototyping stuff because the power for this, okay, I thought I was getting a five-minute warning, a one-minute warning. This is not power effective, but it's really, really great to just see something is working right away. Kind of the right next thing to do is to do a Zephyr driver. So we see the sensor works, we know the hardware is great. We can just, I missed the paste for what the the Zephyr driver interface looks like, where there's an OPT-3001 sensor in Zephyr, since you guys program Zephyr, you know what it looks like. And you can just use the Zephyr code to grab that and put it out on the same network. But instead of relying on the gray bus thing to ask me for transactions, I just do something like broadcast UDP messages. And it's this many lines of Python to listen to those broadcast UDP messages and to get all my sensor data updates over UDP messages. So this is the easy way. Gray bus also has a sensor interface, so that you could potentially augment that to use, to kind of keep the best of both worlds. The rapid prototyping, but then you just go and add the driver to Zephyr and pass whole sensor messages. So you can do all the sleep functions and all the timing things that you need to make a real product. But now it's also super friendly, debuggable and hackable as well. Yeah. Do I have any time for questions or am I going to get up and run? All right. Well, thank you very much for your time. I hope you do something with it.