 So good afternoon, everyone. My name is Andreas Verber. I will talk about a proposed socket API, or actually several ones, for the Lora wireless technology. First, let me start with a few words about myself. I work at ZUSA as a project manager responsible for the ARM architecture. I have been involved for many more years in the open ZUSA ARM port. And more recently, I have also become kernel maintainer for some of the more niche ARM socks, such as Realtek and Action Semiconductor. I'm listing a number of other things that I've also done in the past, but I will simply skip over that. This talk will specifically focus on software interfaces for this particular technology. I will not go into the exact details of how physically the modulation in everything is going to work. Everything and more that you ever wanted to know about this, you will probably find in this talk, and some that have been referenced before, from FOSDEM, which is also going to go into a bit reverse engineering of how the things actually work that you normally don't see, except if you have SDR hardware. Lora appears to be short for long range. This is one of the low power, wide area network technologies that are being hyped in the IoT world. This means in particular that you have a range in the, say, tens of kilometers or two digit kilometers, but you are sacrificing that with a low data rate. So this is mainly used for transmitting sensor data to some form of gateway or some other listening device, and it's not, for example, being used the same way as Wi-Fi for having arbitrary network and traffic over that interface. It is mainly today being used in the unlicensed sub gigahertz short range devices here in Europe or internationally the industrial scientific medical bands, but more recently it is also available in the 2.4 gigahertz ISM band that Wi-Fi, Bluetooth, and all those other technologies have been using traditionally. What is interesting about Lora technology is that you are not, per se, dependent on a particular network provider that would be for payment of fees giving you a gateway that you can actually connect to, but rather you can set up both the sending device and the receiving device on your own if you want. Not strictly necessary, but it's possible. And most importantly, the hardware modules and devices are available in great abundance and are fairly cheap, so a couple of dollars for particular models. There is a lengthy list that I have compiled as part of going through writing drivers and evaluating drivers for this particular project. And most ultimately, why would anyone do it? Well, it's simply possible to do it, so let's simply give that a try. When I first received through some crowdfunding campaign a bag of components that was literally my first start with this technology, so it was like a half-assembled PCB and that basically raises the question, you've got the chip, the technology, you have a basic understanding of how it works, but how do you actually make it work with the Linux system? And from there on, I'm going to dive into how to deal with this from the software side. So first of all, a look at what hardware is available. So there's different types of radio modules available, so usually you don't get the actual transceiver chip itself, but you get some module that already has some clocks, amplifiers, radio connectors on them. And the one that I started with was simply a module that would expose exactly the chipset interface that the vendor had created for this, interfaces that this would be using mainly spy, but it could also be in some other cases UART or USB interfaces. These chips that I will go into more detail later on do not have any permanent storage of any address data or identifiers that would be used in the communication. So this data always needs to be supplied by Linux in order to be able to do any meaningful networking. Similarly, these transceivers provide the physical layer of the network communication and in order to have any particular framing and routing and in particular Lora 1 data communication, then this needs to be implemented in software. So if we're using it in a Linux system, that needs to be somewhere implemented in Linux itself. Then there is a growing number of modules that do not directly expose this raw interface from the vendor, but rather come up with their own interface. To some degree that is obviously taking care of some of the, let's just say, interrupt handling and other interaction between in particular the receive path and buffering, but also the main point why people are doing it because this allows the vendors to certify a particular firmware stack that they deliver on this module that then passes certifications from the Lora Alliance to be compliant with the protocol. And depending on how exactly the firmware interface gets implemented per vendor, that also determines which of the actual chipset features will be available to the user in the end. And finally, this can be a trap at some times. There are also modules that do not actually expose a particular interface that could be consumed by Linux, but are rather intended for doing your own microcontroller development directly close to the Lora chip, where then you would need to use a particular vendor API in order to make use of the radio functionality with or without a Mac on board. So I'm now going to go a bit more into Linux kernel interfaces, how that looks like today. So up here, I've barely sketched that there are, you know, some generic kernel subsystems that I assume most of you will know anyway, maybe even better than myself. And down here, for example, I've sketched the spy subsystem and there in particular the spy dev module would be used in order to expose a generic interface to a spy device that has been attached. Alternatively, if there's a UR interface being exposed, then you would have, depending on what host system you are working with, a whole range of various TTY devices up here, or if it's, you know, like a USB stick that you're attaching to your notebook or a MPCI card, MPCI, sorry, then you would have, you know, some of those USB drivers down here being used and ultimately they will simply in the file system be exposing some device, you know, here dev spy 0.0 or depending on how many you have and how many chip selects, the number obviously will defer, but you will have some device and the API to use those devices then is the traditional, you know, read and write calls or in particular when it's spy, then probably rather I octals to have, you know, both read and write in one call without talking to the chip select line and that is basically it. So given that, there's a number of issues with that. So it appears that most of the vendors have their own forks of certain packages, but there is no central package that we could simply take and include it in our open zoos or slash products in order to run this kind of software because there's hard-coded constants like an if dev, are you like in the European region, are you in the US or somewhere in Asia in order to define various frequency bands and settings. Then another issue I've run into is that many of such more hobbyist project that you can find on GitHub are actually consuming GPL libraries that are incompatible with the library of the lower stack that they're using, which is then something that we would not be able to redistribute and in particular, the spy dev module that we saw on the previous slide does not react favorably to when being used with a DT compatible string of literally spy dev, but the spy maintainers have explicitly said that they expect spy dev only to be used in certain cases when it is not feasible to have a kernel driver and as such, when you actually use this string spy dev in your device tree, you will end up with a warning message with stack trace in your kernel log, which obviously may serve to at least confuse people or get people into a panic mode when they see that. So what has been recommended, because well, if you're developing your own kernel and compiling everything on your own, then you can much easily in, say the spy dev driver, add a single line to define an additional compatible string for your particular use case, but if you're like Zuzu, someone that provides a district to other people, then that is not possible for the end users and we don't want to have a downstream list of devices that would be using this particular driver. So one way to work around that is to simply hijack a defined and compatible string for device, that is something that actually the spy maintainers have been suggesting, that is of course also contrary to the concept of device tree descriptions. And finally, when you're using a generic spy dev layer and device to access in your applications, then you don't necessarily know which device you are actually talking to. So that means that in many cases, you will have some code in user space that is trying to probe which device is actually connected to a spy by, like reading certain registers and just trying to have some heuristics of detecting is it one lordship or another and then making decisions, for example, is the reset low active or high active, one of the funny differences there between some. And ultimately therefore, I've come up with the idea of simply moving the drivers for lordships into the kernel and to thereby allow, I get into that later on, to write generic packet forwarding applications that do not depend on which particular library or device they are being used with and hopefully create a community that those can be essentially maintained and then packaged and don't have to be duplicated all the time for each network provider. Now, thinking about you have the spy driver in the kernel and you have your user space somewhere down here, how should they actually talk to each other and what are the requirements that you have? You will want to expose basically all features that the chip set provides technically to your users because if you don't do that, if you provide only a very small fraction of those, then people will not adopt a new interface and simply will have to go with the old route for making use of them. There are multiple proprietary protocols and well anyone can pretty much develop their own proprietary protocol. So one design goal here would be to allow people to use that because otherwise if they can't do it with that interface, they will have to find other ways to do that. So basically that translates to having the, just call exception in all headers that are related to the implementation here. And the basic, well the big goal here would be to be able to reuse protocols. So you don't want to have a, again this idea of code reuse that you don't want to have an implementation per vendor chip set and protocol but rather that you want to have one implementation of for example Lora One or any other protocol maybe simply reuse 802.15.4 something else on top, six low pan something and just layer that on top of a generic network file layer and not duplicate that all the time. Yes and obviously it should work with all or at least most of the chips that out there and the idea that seemed most appealing to me at the time was to use sockets because that as well what we all know from networking in particular if we've done that with ethernet or wifi and so on and hopefully that will make it easier for people to adopt such a new technology. Now this is my very first journey into network subsystems so there has been a lot of new things for me. What I've been implementing and sending out an RFC to the NetF mailing list has been in the net subsystem to have a Lora module that is implementing those basic sockets and then in the more concrete device subsystems, driver subsystems in particular in spy have a number of drivers corresponding to the chipsets as well as in the serial device subsystem for any other devices that are using a UART serial based protocol. Ultimately that will then mean that if you look at IPA or the file system in this case, you will have network devices that get instantiated by the drivers and I've simply chosen for now to just call them Lora with a running number. So assume here there's a 1276 driver would be exposing one Lora zero device here and then another driver would be exposing further device and so on. Now what's different from the previous model of working with Ioctols and ReadWrite is that through the socket API, we can instantiate such a socket construct or even multiple of them. We can then bind the sockets to a particular socket address and then send and receive via those sockets and we will be using socket buffers to represent an individual packet that has been received or is to be received via this interface. Now looking at the particular chipsets in order to figure out how to structure those things, the SX-1272 and SX-1276 following our signal channel devices, they support two selectable modes, one FSK OOK and the other one, the Lora one that I'll concentrate on for now, I'll be getting back to that later. Those modes can be switched via a special sleep state which means basically that it loses all data that was previously configured if you want to switch between them. Similarly, it is using a state machine for switching between receiving and transmitting. This has to go through an intermediate standby state but it means similar with the single channel statement above that it is a half duplex interface so you can only do one at a time always. And for receiving a Lora mode, it has 256 bytes, up to 256 bytes of data buff available for FSK 64. There are ways using interrupts, I've been told to enlarge that up to I think 2048 but it gives you a rough idea of how much data we're actually dealing with here at the moment. Similarly, there is a new generation, SX-1266, I have one here, so this is an Arduino shield with this chip set on it. Ideally, this is basically the radiologic on there, the rest is just for bringing it out to the Arduino pins. Similarly, different modes available, a state machine to define what you're actually doing at the moment and you always have to go through some intermediate state. In this case, there's not just a spy register interface available but a spy command interface that in turn can be used with a read register, write register command in order to access registers on the device. Data buffer in a similar magnitude and also this is slightly, well, this is quite similar to the one that we've just taken a look at. The difference here is that this one is actually using the 2.4 gigahertz band and as such has slightly different modes that it is supporting but otherwise from the register interface and so on is still quite similar to the other one. What is much different is the SX-130X family, so these are referred to as concentrators and basically what you can understand them for is they are one of those SX-12 whatever chips cut in half and you can now have two separate radio chips, SX-12, 55, sorry 55, 57, 58 that you simply have the first part of those transceivers that simply output the data that they received on I and Q lines and you will then on these 13-hour X chips have an ADC that is getting that data in there and has some packet processing logic in order to emulate 49 channels out of two transceivers that it actually talks to and this one again much like the original SX-1276 has a register interface, however in this case it is unfortunately not very or no longer I hear very well documented and there is reference code available on GitHub how Semtech have implemented the user space interface themselves. Data buffer slightly larger but is also dealing with the data of 10 channels in one buffer and one thing of note here regarding the driver implementation is unlike the other chipsets this one is dealing with binary firmware that it needs to load into the chipset in order to for one calibrate and then afterwards operate however they are under BSD license so are at least not a problem to redistribute. So much for the actual Semtech chipsets then based on those chipsets I already mentioned there are a number of modules that are implementing via some microcontroller or a UART serial interface of their own the serial device bus that we briefly saw in the image earlier on allows to attach an internal driver to a TTY device it will then not be accessible to user space this is available for things since 4.12 and the way that you would feed that information to that bus infrastructure is by having an additional child node in the device tree be below the actual UART device node in the device tree. So that provides a callback for receiving data depending on how that particular chipset you're dealing with is operating you might receive individual characters be transmitted via that callback and need to individually buffer those into a sensible format that you can then process or it might be sending you like a whole chunk unfortunately you don't really can control that so basically you need to do some buffering on your own and obviously there are APIs available in this order framework to also send data back in order to communicate with that device. Now while everyone writes about AT command interfaces I have found that in practice they can't even agree on what line ending they use and they defer on whether they are case sensitive case insensitive and if they're case sensitive whether they want to be lower case or uppercase so basically I have, although I've dealt with probably a handful of them by now not come up with a standard interface that I could get them into one form but if rather needed to have one driver per interface protocol that was being implemented. In some cases it is not a textual AT command protocol but rather some random binary command where you then have like some command code that you're sending if you're lucky also like a length value that you can deal with unknown commands and you will need to then possibly deal also with check sums on those binary commands and ultimately there's two ways that the communication can work as far as reception goes. One is that you have external GPIO lines that will signal you as kind of an interrupt whether data is now available in the buffer or what the state of the buffer is or you could have in your textual AT command interface maybe some asynchronous notification that even without sending something to the interface you may be getting some notification via the callback that an event has occurred that there needs to be processed. USB unfortunately has been a slightly bigger problem so far, one of them so in theory you could just take a USB serial driver and attach a sort of bus and sort of devices to that bus. However, today USB serial devices don't appear to have an actual device tree node attached to them so there was a proposal to fix that. There was still some discussion and differences of opinion for how exactly to go about that connected to that same problem is in your device tree how would you actually tell that you have a particular device connected to USB? So one was to have a special thing serial node under the device bus that would then get simply numbered. Another one is what you may know from for example the Raspberry Pi is using the USB vendor ID comma product ID schema in order to pass through like for example the MAC address from firmware into the kernel driver obviously for that you need to know on which part of the various USB hubs and controllers is the device actually connected. With ACPI it becomes slightly more difficult and I have to say that that is not my field of expertise. I did at some point find a command line option to overload ACPI tables from for example your RAM disk that would allow you to add information to the ACPI tables describing such options because obviously if you're running a notebook you don't want to flash your notebook firmware with custom modifications in order to make such a device and detect it if it's not already made product coming to you. Also similarly still connected to that if you have USB ports well you usually don't just have one what happens if you plug the device into a different USB port than before would it still be found that way or would that be hard coded to a particular position maybe also connected to how the exact USB hub topology looks like internally to the controllers and devices. One idea was to simply use USB drivers and try to detect that a particular device as you enumerate it and you know you can see the data in LSUSB and so on. If it doesn't only have a particular interface but a particular combination of vendor ID and product ID simply try to have a driver loaded with higher priority that would then attach to that device and somehow reuse the code that is already there for the generic USB devices. An alternative would have been to use a line discipline so that means you simply leave the TTY device and the USB drivers all alone and simply use a user space tool to tell the TTY device to switch to a different processing mode. Unfortunately today we don't seem to have any form of bridge that would make the Zodaf framework work together with these line discipline callbacks. So basically that would mean that you would have multiple different implementations depending on how you would actually connect the same chips at which obviously is not ideal. If anyone has any ideas on that I'd be very welcome to hear your feedback on that at the end. For now another challenge we're facing here since we are talking about the physical layer well data link. We are talking about packets that come with some preamble to signal the start and also the end of the packet but there is no particular metadata that is just driving it so there is no addressing or anything going on. Also because there is no addressing it always gets sent to like all devices that can receive it and then the receiver needs to decide whether it is for them or not because that is on the Mac layer only. The only way to filter at this physical layer is by so-called sync words which is a sequence of one or more bytes that basically give you a very limited addressing mode to distinguish various packets. And therefore my idea here was to use the relevant radio properties that determine whether you can actually receive a given packet as the address that you are sending or receiving from and let's take a look how that would look like. So for one, because there is no particular addressing you can't do any automatic routing of packages that means we need to explicitly say, okay we want to use the lower zero or the lower one interface to send or receive our data respectively then there's the frequency that we're sending at or receiving at a so-called spreading factor that determines the exact shape of the radio signal then the so-called bandwidth and the mentioned sync word in the case of LoRa that is one byte. I don't pretend that this list is entirely complete but something like that would be the idea to use as a socket address specific to LoRa and with that said, this is how the basic idea looks like. So you would have on the the PHY driver that we were talking about here on the very bottom you would have a LoRa protocol family using the datagram format of sending data and you could then have the user instantiate that socket device and from user space define what content you are actually going to send via that interface and corresponding to this LoRa socket family you would then also have a netlink module which allows you via different sockets to configure well, the PHY layer down here. And similarly, there has been work ongoing by Jan Hong-Pung to define a LoRa1 protocol family. Similarly, there's two different modes so there's alternatively the datagram mode as unreliable transmission as they call it and then the reliable mode which would be a sequenced packet and corresponding to that also we would need some way via netlink to configure and interact with the LoRa1 layer below that. And there's two ways to go about that. One would be if we're using such a module with an MCU on it and then we could simply reuse that hard Mac if available and implemented by the vendor or if we want to have it all in Linux then we would need a module that translates the LoRa1 packets down into the LoRa format adding any needed header information. By extension, since I already showed LoRa1 without going into much detail what exactly that is so basically that is a framing protocol that has addressing modes it has joining operations for exchanging credentials with gateways and then obviously while sending, receiving in those two different modes and it defines a set of so-called data rates. Those data rates are like a meta description of all the various channel protocols that I was showing for the LoRa layer below so it actually supports two different modes. If we're looking at the LoRa mode here first then that would obviously be the frequency again the spreading factor and the bandwidth that is being defined as a particular data rate I think it's like zero to 15ish something like that and some of those data rates are actually using the FSK mode and also then have a frequency associated with that and the bandwidth so the spreading factor is something that's specific to LoRa here similar to TCP IP there's also the concept of different ports that you can have and since at least currently we are bound to a specific network interface on the LoRa layer this is also what we may need to use on the LoRa1 layer and depending on what data rate or I think it's not even, well there is a specification for regional parameters of the LoRa1 protocol and at least in that document there is associated with the various geographical regions a definition of which sync word to use along with that. This I mainly already mentioned on the graphical slide so we're using the generic link layer, sorry the generic net link layer to send the commands in an extensible way. What I have implemented so far is access to the frequency in the case. This interface that we are designing here needs to work for all chip sets and modules that are out where one such thing is that initially we've been dealing with single channel chip sets when dealing with the SX-13OX then we have to deal with multiple channels so it may be necessary to add an attribute which generic net link allows to be able to specify the channel that we want to operate on because simply it does not have separate receive and send buffers per channel but rather a single one for all of them so it seems to make most sense to have only one network device even if there's multiple channels in it. Currently the net link implementation in my staging tree is making assumptions that all the lower interfaces have actually been created through my lower def module with certain setting set there and data structures available. Obviously that is not something that we can merge as is in case someone wants to use some SDR chip set in order to synthesize the signals through different ways without going through those and same chip sets. And then there's also the question of course we can implement all kinds of configuration interfaces by network layer but which ones should we actually implement? So there are certain properties that are actually hardware properties like if you have an antenna with a center frequency of 868 megahertz then it may make sense to provide that information via the device tree layer along as the hardware description whereas the exact frequency that you're going to use from my perspective certainly makes sense to make configurable via the net link layer so that the user can easily upset that or well as we've seen before possibly even as part of the socket address. Then on Lura1 it's basically the same with different operations on it. You have for one the data rate that I already explained in some more detail that should be able to read and set. There may be operations such as join that we could implement via net link and use that in order to bring a network device up in a particular usable configuration. And depending on what device you're actually dealing with this could be delegated to the existing NLura module for using the Lura layer configuration of frequency, bandwidth, and so on and so on. Or if there is a particular command to have that operation performed directly then it could also go directly there. Another topic here is that in particular when we're thinking about the sub gigahertz channels which is what Lura in particular as widely deployed in Europe is being used for then there is currently not to my knowledge any database that would contain the regulatory information of what is allowed in which country or region in particular. That means we have a 14 DBM transmit power limitation here in Europe. In the US you can go I think up to 20 or more. There's also a duty cycle limit that you are only supposed to use up to 0.1% whatever of the available time on that frequency. And there are ways to circumvent that but at least there is no database that would be exactly spelling that out in machine readable form in order to automatically feed that into the configuration interfaces that I was just talking about. So the idea here would be to find some way to reuse existing technology such as wireless recta B or CRDA maybe. I'm not so much into the details there yet. In particular since the 2.4 gigahertz modules could then be able to reuse the data that already exists from Wi-Fi and other technologies for the 2.4 gigahertz frequency bands. And yeah, if we have the data available of what can be set and if we have interfaces available for actually setting those configuration bits then it should be possible for user space to simply feed that into those interfaces. Currently what has been implemented for many or well some of the drivers that exist today is the transmission path for packets because that is fairly easy to do given the state machine and model that I was describing. However, there does not appear to be a way to proactively or not direct way to proactively receive packets only when you want to. So that would mean that once you bring up a network interface you would need to start listening on that interface so that once a network packet comes in you can then notify any available listeners if any and only interrupt the listening for new packets if you actually want to send out packages yourself. With transmitting that is fairly easy as you only have to make sure that only one person is accessing the register interface at a time otherwise you would simply be able to use locks to rule out that two people are doing that at the same time. However, what happens if one person wants to listen on one channel and the other person on another channel or if one person wants to listen for lower messages and another person wants to listen for FSK messages. So basically we would need a way to detect whether the current configuration of listeners is actually possible and if so find some way to error out on those that are incompatible with what is already going on. Another note here also is on ethernet. You have a well-defined frame format with this I think it's called ether ID where you have a field that indicates a protocol number that you can then use to parse the packets and transmit them to a particular layer. This is not possible here so that means in order to detect a lower one packet one would actually need to try to parse all incoming lower packets and try to convert them to the raw format. This is how it looks like with some of the some of the other surrounding protocols in context. So there are some proprietary protocols shown here. If we're looking at the FSK then there are some existing stuff like 80254 that could be reused. BLE in some cases is also based on FSK and the various other protocol formats here. So some of those protocols are also available on non-Laura modules so that is a potential naming problem that we need to go to. There is a generic protocol family for packets that could in theory be used to transmit lower packets. However, that would take away the duality of different frequencies and settings that could be selected so that is not really an ultimate selection only if we want to go through net link for everything. I will skip over that and just say a few words that I have an interop setup for some of those drivers for the European technologies where various modules have been provided that are being used on various single board computers. There are some hints for how you can go testing yourself. And yes, I'm working towards a V2 patch set with what has been shown here today implemented that will be sent out hopefully fairly shortly with the goal of stabilizing the ABI and then starting to get that merged. These are some of the vendors that have contributed in particular, Ben here from Laird has been doing some great contributions for the Essex 30.0X and a whole number of vendors have been contributing hardware to develop these interfaces. So there is a number of companies that are interested and there's also some competing technologies that are already shown like ZigFox and NB IoT that may also need some interfaces where they are on the same module and yes, USB is an issue that we need to solve. And since the time is running out, please come to me after the talk, find me somewhere or contact me by email if you're interested in a topic. There is unfortunately no dedicated mailing list for this yet. If anyone can help get that set up then please also get in touch with me. Otherwise just email me or reach out on the NetDev mailing list. No time for questions so thank you very much for your attention and enjoy the next talk.