 Hello, everyone. Oh, sorry. I guess it's time to start the talk. Oh, thanks. So, a few readings, yeah, especially the FreeBSD source commit mailing list, probably my only best for walking on the Spark64 port and also generally on Ambulance, beloved device drivers, but that's not what this talk is about. Apart from working on FreeBSD, I'm currently also a master's student and scientific staff member of the University of Applied Sciences in Regensburg. And as part of that, I had to work on a research project involving so-called ESA-NOT microcontroller reference design boards. And these boards turned out to be rather interesting. I decided to delve some more into them beyond what was actually needed for that project. And that's what this talk is about, finally. So, before I start, I still want to set some things straight first off. I am in no way affiliated with Ignite, Gimbiha, which sells these boards. I just regularly bought them. I don't get no money from them. Also, for me, it's rather, yeah, unusual to give a talk without writing a paper. So, I ended up having a lot of information in the slides and the whole talk is a bit more workshop style. I hope that's okay for you. Also, this is my first regular talk at BSDCon, your BSDCon, so hopefully you'll be here with me. As you can see here, I have a FreeBSD logo here, but the talk isn't solely about FreeBSD, but at least, in some way, FreeBSD centric. So, that's why I kept the logo from the slide templates. So, to finally, Gimbiha, this is, yeah, rather intentionally crowded slide to introduce the abbreviations. It's from another talk I gave on, yeah, introduction to embedded system prototyping. Probably for this audience, I don't really need to explain these abbreviations. Expect maybe for, if you're not familiar with embedded system with JTAG, which is debugging interface, but also used for flashing microcontroller boards and an alternative to set this in-system program via an SBR interface. So, what are these? Yeah, Ethernet boards, that's a whole family you can see here, that's now a bit unhiny because I have to put away mine. Yeah, you may wonder if there's no Ethernet 4. I don't know what happened to it. There's just one, two, three, and five. There never was a four. So, Ethernet 1 is designed from 2004, and still sold. It costs about 85 euros today, and Ethernet 5 is the 2011 design currently sold from 185 euros. Obviously, it's way more what, for example, Raspberry Pi costs, but also targeted all the ends of these boards. It's different than other boards. For example, the Raspberry Pi doesn't even have an RTC chip, a real-time clock chip on it. I'll talk about a bit more about hardware of these boards in the next slides, but some things noteworthy here is the Ethernet 2 is basically the same as the Ethernet 1, except for more RAM, RS485 interface, and it's also certified for the full industrial temperature range from, I think, minus 42 plus 85 degree Celsius. Yes, that's right. And the Ethernet 5 is really interesting. It was the first member of this family to also have USB connectors. It's possible to power it over Ethernet. It also has things like an image sensor interface, and it's also certified for an extended temperature range. In general, the operating system intended to be run on these boards is called NatoS, which I'll introduce later. However, for the Ethernet 5, the Ignite Gambia also is debuting a Linux kernel and, yeah, user land based on the Armstrong distribution, but the necessary patches for Linux are not in the Linux mainline 3. So, comparing these Ethernet 1 and 2, as I said before, are based on the same microcontroller, which is an ATmega 128 from AdMal, AVR architecture 8 bit. As you can see here, it's up to Ethernet 3. It's rather tiny in RAM and flash, so really intended for embedded systems. And with Ethernet 5, which finally is a, yeah, rather powerful ARM 9, based microcontrollers, they put a lot of, sort of, relatively a lot of RAM and flash there. You can also, yeah, for example, run Linux and other interesting operating systems there. In terms of frequencies, 1 and 2 are running with about 15 MHz. If you don't know, you need exact board rates, you can also run at 7 at 16 MHz. Ethernet 3 clocks at about 74 MHz, and the 5 is 180 MHz. Flash on the Ethernet 5 is, you know, yeah, divided into three parts, or there are three components. An internal one used solely for the first stage bootloader, four megabytes, external one for configuration data, the rest of the bootloader, and another application as well as kernels. And there's another one gigabyte flash chip intended to be used for a file system. First, comparing these boards, yeah, there's not that many to say, that much to say about satellite. You don't need to wonder why, also based on the same microcontroller, Ethernet 2 has more GPO lines because there is an additional multiplex on it. And, yeah, so for NVRAM configuration data, here are in brackets because it's shared with, or it's actually the flash chip also used for the bootloader, stuff and kernels and applications. So, if you're looking into a better systems, obviously power consumption is interesting. So that's what I measured in Rarity for the Ethernet 1. It's about 1 watts with the application I had to use, or I developed as part of the set research project. And the Ethernet 5 booted from an SD card with an Ethernet link up is about 2 watts. So now, the real interesting thing about these boards, in my opinion, is that all layouts and schematics are distributed under a BST-style license. In this case, you see the license of the Ethernet 1. It's the same for the rest of them. Actually, I tended to use it, this schematics to build my own version of an Ethernet 2. Unfortunately, that project was cancelled. Yeah, these are all the links for the respective layers and schematics files for the EGPCB design software. The reason I collected them here is that they're a bit tricky to find because unfortunately, most of them are only found on the German version of the home page, but not on the English one. Something I forgot to say, one nice feature of these families is that they all have a pin-compatible expansion port. And there are some ready-made expansions you can find. For example, the MediaNut, which is an MP3 decoder. But if you go around, you'll also find other projects like GPIB, bus interface used for connecting measurements instruments. There's also a template available, layouts in schematics template, if you want to build your own extensions. Yeah, so what about NatoS? It's rather simple and small, but rather usable, real-time operating system. It provides all features you expect for a real-time operating system like cooperative multitasking, especially interrupt response times and so on. It has support for all the typical communication protocols. For IP, that's currently only IP version 4, but IPv6 is in the works. One nice thing also is that sort of user land, which is in real user land because it's using the same aerospace, it's largely POSIX compliant. As you will say later, it's also rather simple to get used to it. And for myself, I also ported software or applications between NatoS and POSIX, and it's straightforward. And of course, NatoS itself is also beastly style licensed. One nice feature is also that kernel is modular and like library. So if you build an application based on it, you don't have a kernel with an application with unused parts, it's only links in what actually is used. More features, obviously, all the hardware of the Ethernet family of parts is supported. However, they also support some additional microcontrollers and additional platforms. For example, actually the Gameboy Advanced is a supported platform. Some resources for NatoS you should be aware of is, yeah, with the sources. And, yeah, as of now, the current stable version is 4.10.3, which is also the first stable version to include support for previous host as a host, but it's actually too little to add support for other hosts like NetBeasty and OpenBeasty and so on. So Wiki is also rather good with lots of examples, better ground information and so on. And one really, really useful and useful thing is the API reference for NatoS, which explains about any function you can call in detail. If you want to build NatoS for Ethernet one and two on, in this case, FreeBeasty as a host, you need the following FreeBeasty parts. I didn't actually look that the Net and OpenBeasty has the same parts, but I assume they have these rather common parts. First two are the tool chain for the RVR microcontroller. So you need traditionally an Lipsy for RVR, AVR. You need a tool called AVRDude for flashing these parts. And, yeah, you also need some glue stuff, sort of. It's talking about Lua. It's also possible to actually run Lua scripts on these boards under NatoS, but that's something I haven't tried myself so far. Installing NatoS is rather straightforward after you download the source and, yeah, run configure. Two things you should, yeah, disable in my opinion is the graphical version of NutConf. I don't think you actually need, really need them. The command line version is basically fully sufficient. And using the, or adding the graphical versions just as a lot of dependency bloat. Another thing that's causing additional dependencies is Nut Discovery tool. Basically, that's a tool you can run to discover boards via ethernet and configure static IP addresses of these boards then used within NatoS. If you read that feature, you actually, yeah, need to build Nut Discovery apart from that you probably can save or spare set dependencies. One thing you need to do is after installing it is to add, it's a binary path to your, yeah, path environment. So, two tools can be called. In general, if you then want to build or, yeah, stuff for ethernet one and two, there are two methods, so-called entry method and the build tree method. Entry method is sort of the old way, like we configured kernels in free BST value, CD user source on AMD64, for example, Conf and then run config on the kernel config file, change the object and then build it. So, it's basically building the object within the source tree. The build tree method is something like CD to user source and run, make, build kernel and build world. In my opinion, the entry method shown here is more appropriate if you want to change NatoS itself because you don't constantly have to change directories from source tree to object or build tree. So, if you want to configure NatoS for ethernet one and two, you typically run a nut setup script, which is an interactive script that's on the serial giving for ethernet one. This is pretty much what you use unchanged. Just depending on the ISP programmer you're using, you probably want to select another ISP protocol. In this case, I'm using the USB SP protocol, which is rather common. You find several ISP programmers using that protocol. This is one of these. It's a USB to ISP programmer self-bit. You can get the watch for free from Haka Lighter button if you send them stamps and sends about five to six euros in components. The only problem is you need to bootstrap it because it's also using a microcaller itself. So, if you have to, for one time, borrow an ISP programmer. If you have run that nut setup script, this results in a bunch of files to be created. You probably should check in if you're using a local source repository. There's one COVID currently. NatoS 4.10.3, obviously only supports GCC 4.32. However, currently in 3B, you have 451. It deprecated a bit of stuff currently still used by NatoS, so you have to manually add a hack to the user.conf file, which is adding to the hardware definition for these two macro definitions and also these flags. Those are currently deprecated and not yet in NatoS replaced functions are still available. You also need a walk around for back in the NatoS build system, which is the last line. My fix for that got accepted upstream, but it's obviously not yet committed. Then to finally compile NatoS, you just run gmake, clean, all install. One COVID here is also if you're changing NatoS sources yourself, you have to rerun that last step and relink your application because NatoS build system unfortunately doesn't catch changes in NatoS with both. If you then want to compile applications, there are several example applications brought with NatoS sources in nut app. For example in HDB, DMN or on RS232 terminal server program, you just a CD to set directory, run again gmake, basically all on it. Once you have done that, you can just run gmake burn to flash it onto the board via an ISP programmer. You have to watch out to have necessary permissions on the device nodes used for flashing. In theory, it should be possible to do that via deafness rules for reasons unknown. I didn't manage to get it working, so I just set you a bit of AVR due to for... Yeah, that's the typical hello world example, how it looks like in NatoS. It's a NatoS application. As I said before, it's rather familiar. If you know POSIX, you have underscore IR control for settings, in this case, you are at speed for the output device and you have also stuff like F open and regular open and so on. Here you also see how you register hardware devices within NatoS. In this case, you have a macro for the debug or the standard console interface. The following two parameters in general are address and interrupt. You need these parameters, for example, for some Mac controller, but not for serial devices. Creating Mac files for NatoS is rather simple. Basically, it's similar to the BST Foo MK files. You basically just have to define the project variable. It has to match, in this case, helloworld.c. If you have additional sources, you would add them here. The rest, you can just copy and paste. If you do that, you just have to watch out that our M lines here begin with tabs and not with spaces. So far, about using NatoS for Ethernet 1 and 2, for 3 and 5, we basically need just another tool chain, which are the ARM-ELF binaries and ARM-ELF-GZ. You get these within free BSTs ports by combining the cross binaries and GZZ ports, setting target arch and the target ABI. Then installing NatoS is the same as for 1 and 2. Yeah, exactly the same. Then in this case, I'm showing how the build-free method works. It's a bit more complicated than the NatoS setup one. You basically run NatoConfigure with a bunch of options, which then creates a build-free for NatoS itself, which is called nut-build. Yeah, in this case, 50F. Unfortunately, again, you need a hack to make it work with current free BST ports, which is the change in NatoConfigure in case the name of the target binaries. I'm not really sure about the background here. Apparently, GZZ and the binaries guys decided to give some different names and free BST ports haven't really caught up with that. You currently have to do that by hand. Maybe it's a better way to use the NewLip stuff. I'm not sure how the part is called. NewLip, no. Yeah, whatever. Maybe it's a better way, but for now it's a hack that's working. Unfortunately, NatoConfigure MK file says, yeah, automatically create it, don't change, just ignore that. Yeah, once you have done that, you again can just change the build-free for NatoS, run G-Make all again, and you end up with NatoS compiled in the build-free. Then you have to do the same for an applications build-free, also running nut-configure with that arguments. Same hack again required here. And then if you want to build also example applications, you just CD to the NetApp, build-free directory, run G-Make again, and this results in a lot of .bin files you can use on your Esonaut 5. Basically, also similarly for Esonaut 3. So how do you flush your NetButton Esonaut 5? Basic procedure is to use TFTP, so you'll at least need a TFTP server. I'll also advise to config a support via DHCP that's a snippet you'll need for the IC, DHCP, DEMOND. That's especially handy if you later on would like to NetBoot 3 BST in order to give it a root path to the NFS root directory. So yeah, running TFTP, DEMOND is rather straightforward currently. With the BST overriding systems, all you typically have to change in the NET config is a path to the NFS root, root directory you're intending to use, and afterwards just enable it in RCConf and start it. If you finally send connector 0.0 part of an Esonaut 5 with 115 KBPS and 8n1, you'll see typically you see these boot messages, which first line is SAMBOOT, which is a first-stage bootloader residing within that integrated flash. So you can see that you can see that the second is UBOOT stuff. So that's also, as Rafa pointed out, we are a bit chitching with my title because, yeah, so far we have NUTWARE switch, BST line instance and the hardware, which is BST line instance, but UBOOT is unfortunately GPL license. Then you get a bit of DEMESSECH like stuff. And finally, you get an auto boot sequence if you actually hit any key to stop it. You get to the UBOOT prompt and then you can configure and do different things. For example, alternatively to using DHCP, you can set a static IP address and a static IP address for the TFTP server. With SAFE.IF command, you then can save this environment variables. And also with printf, you can print all the environment. So how do you netboot EZNUT5? First off, you need again to have, yeah, obviously an app to add application. In this case, I'm referring to the hello world shown earlier, which also works for the EZNUT5, not just the EZNUT1 and 2. Then after putting it in the TFTP boot directory, you basically set an image name and run the TFTP boot nut script, which will cause application to be loaded via TFTP and then to be executed one time. If you want to do that every time you power on the board, you can additionally set the boot command variable to run that script and save the environment again. Alternatively, to netbooting, you can also store that in the respective partition in the four megabytes external flash using the TFTP install nut script. Then, yeah, flashing takes a bit, but afterwards you can, yeah, either run it again one time from the flash using the flash boot nut script or again set boot command to do that every time you power it on. Yes, that's for not to ask so far. As for 3bc on EZNUT5, yeah, I, as off set revisions, I added support to it or this is the last bits of support. It's also going to be in 9.1 release. Basically, what it took to get 3bc running on EZNUT5 was adding support for some 9.xe family of microdrollers that was rather simply to do because we already support the core. These microdrollers are built on self-just, yeah, additional flash and apart from that, it's basically the same. Then, there was a client for a new driver for the real-time clock chip used on this board, also not that hard to do. Then, the real funny part was to fix a lot of drivers at, yeah, they were buggy in 3bc, but also needed workarounds for silicon bugs of, yeah, certain admiral microdrollers, not only is the 7.9.xe, but they're also applying to other stuff. Yes, most interesting stuff was TUR as a two-wire interface controller, how Atmel calls their I2C controller, which just had no chance of working anywhere at all before because it just didn't send a stop. I'd also like to thank Ion Lepore from Symmetricom at this point who especially contributed fixes required for MMC, but also did, yeah, contributed some other bits of these fixes and reviewed several of them. All in all, this should make 3bc the first operating system besides NADOS, of course, to support Ethernet 5 out of 3. As I said earlier, there's official Linux support, but it's not in the Linux mainline. So, how do you compile user-land, 3bc user-land for Ethernet 5? Yeah, this example refers to 9.1, but it's also the same for head. It's straightforward how you build within a cross environment. So only thing to watch out is especially the head to pass a demo production macro because otherwise some malloc debugging support will just blow on 128 megabytes system out of, yeah, the bottom sort of. So default kernel configuration file I'm shipping within 3bc is set up to do netbooting. So if you don't want to do netbooting but boot from an SD card instead, you have to replace in the kernel config file the above lines with root def name option. And yeah, once you have done that, you can regularly build again a cross-built and kernel by again taking a target to ARM, giving the kernel configuration a config file name and run build kernel. Then once you have built word and kernel for the Ethernet 5, you have to run NFF root to install that, yeah, objects by running the install kernel, install word and distribution targets. Again, straightforward as you do cross-building, the only thing to watch out in this case is to set kernel underscore ko to kernel dot bin because that's what you want to, yeah, boot on the board and not a regular binary called kernel. Afterwards, you probably want to disable sandmail totally and enable SSHD on the board. It's done by calling in sys comments and once you have done that, you're basically finished with, I think, an NFS root appropriate for Ethernet 5. I'll prepare that for 911 RZ2. While on Torbord, you can download from the people3vst.org and basically that's also the same for building a root that is intended to be placed on an MNC or SD card. Yeah, then if you want to netboot3vst, obviously, we have for one time to make sure your NFS server host is running, for example, like in this case. Yes, on the Ethernet 5 itself, you have to do a bit of magic, setting environment variables and sort of adding scripts. Most important one is running the power man on comment to power on peripherals on the board. You also need to set some variables used by other parts of the UBOT support. In this case, start and end are not really needed for netbooting, just for flashing, but the same sort of script is also used for flash, booting from flash later on. Then you have to create a sort of script to boot3vst via TFDP and in the end save all the environment. Once you have done that, booting3vst via network is as simple as running TFDP boot3vst, or if you want to do again that on every power up of the board, you can again alter the boot comment variable to do that automatically. If you want to install 3vst on an SD card, basically what you do is buy a new card and sets how they typically look like to have part stuff on it. These are the usual comments to delete what you find there. In general, you can do that either by for one time netbooting an Ethernet 5 bar via network or you can use one of those MNC SD adapters or reader servers called also service for writing and connect it to a regular PC and do that stuff there. That's how you create then UFS partitioning and file system used for 3vst on the Ethernet 5 and once you have new fasted and mounted the file system on the SD card, you sort of install regularly the userland spits built earlier by either setting the duster or copying what you have over. Then again, if you want to boot from that card, you have to again create some magic. The first part up to creating select 3vst is the same as I've shown earlier for netbooting. The only difference here is that it's a tftp install 3vst script and a flash boot 3vst script. Once it's done, you can run tftp install 3vst to boot fire tftp or load fire tftp kernel and then it's flashed which takes a while about I think 20 to 30 seconds. Also one problem here is that Selenux partition we are abusing forces is limited to 2.6 megabytes currently. That's a rather tight fit. Once you have 3vst on there, it's a bit my opinion simpler to flash this kernel. You can just use dd to set device. As I said earlier, it takes a while for both methods to complete. Also we have to watch out to use the right kernel for SD booting. As I said earlier, you have to set the root step name as appropriate and also we have to fetch the kernel.bin file for that. Again, I've prepared such a kernel which you can download from people 3vst.org. Once you have flashed the kernel to the board, it's again simple to run flash boot 3vst and again by boot command ordering, you can do that automatically on boot. Yeah, to do this for 3vst on the ethernet 5, that's not that much stuff to do. Currently we're missing a drive with a power management controller which allows to turn off certain parts of the board or peripherals on the board. It's not hard to do. The main question is what will really happen if you just turn off the clock of the ethernet controller. It's something to try and maybe not provide. Then getting new boot loader to work is certainly very desirable. As I said earlier, we currently have a tight match with the kernel in flash. Using new boot loader with the augmented probably also speed up boot times quite a bit. I only looked at it for a short time. It didn't work out of the box, but it could be all sets required to get it working is to build a new boot loader image with a config API stuff enabled. However, in general for 3vst ARM, there's quite some stuff to do. So eight drivers, network interface drivers for the embedded Mac should be improved for one. It's laid out for all the controllers which need to copy the received data to Mbuff with especially the same mine version of microcontrollers. You can directly receive into Mbuff. That's something you really want for speed. Also, I discovered reliability problems with the driver. For TCP, it's just working fine. However, for UDP, for example, when copying large amounts of data via NFS, it sometimes just wedges and you can't recover. I have no idea for what the problem really is, but that's also something that needs to be fixed but isn't really, isn't at five specific. There's a lot of stuff to do in the SPI land. Mainly, we're currently lacking a way to let client device drivers say what the hardware supports in speed. Currently, we just can assume that only the lowest one possible that is supported. We need a framework to tail some upper layers. What's the maximum hardware really supports? Also, for the 1891 SPI controller implementation, there's some strange bug I haven't mentioned down so far that when actually using speed supported by the hardware or higher speed supported by the hardware, I get to get data corrections. The only working variant currently is to run at the lowest possible speed. That's also not very desirable. There's room for improvement in the machine-dependent bus demay implementation, especially regarding cash flushing. There are some corner cases not caught currently. USB is rather problematic at that point in time. Some while ago, there were some changes which blow on the arm and mid-support of the water because it's not really using bus demays way intended, but even before that breakage, it wasn't stable on arm. It worked for simple stuff like RS232 controllers, but if you try to mount a file system or use a file system via USB, sooner or later, that stuff blows up. Also, thankfully, we finally got the NAND framework, and as I said earlier, the Ethernet 5 ports provide one gigabyte of flash for our file system. That's currently not usable within through BST, and yeah, we also need a driver to hook up the at-mail part with that framework. I haven't looked at how complicated it would be to implement that. Hopefully, we'll get to it sometime, or maybe some of them would also be great. Yeah, that's everything for my part so far. I'd like to thank the organizers to select my talk, and as I said earlier, I am the power for also contributing some fixes here. Does anybody have any questions? Yeah. Are we still need to mall lock production for 9.1? No, maybe I wasn't really clear about that. In the stable branches, we're, by default, disabling it, but if you build a head, as you probably know yourself, you really need that it won't work. I just wanted to give sort of a universal command which works for both stable and head branches. Sorry if it wasn't clear before. I'm also sorry if the talk was a lot of information in less time, as I said earlier, it would be nice if I had a chance to write a paper for it and put more information there. But thanks to the video guys, you can download the talk probably later and take your time to have a look at the slides and listen to the audio stream. Thanks for attending.