 Good evening everyone The next presentation will be in English, but the translation will be available in online stream But not in this room. So if you prefer a German version, please go outside and watch the stream Anyway, it will be out FOMO and FPGA inside your USB port and the talk will be given by Tim Mifro Enzel and Sean Xops across and I will introduce himself further. Thank you Hello everyone Hi, I'm Tim and I self-identify as a software developer I'm almost a open-source developer and I'm almost an Australian I Also have a lot of projects. In fact, I give a talk called Tim has too many projects and you can find a link to it Right there One of my big projects that I've talked about previously at CCC is the HDMI to USB and Tim videos project and another one of my big projects is Simbi flow, which is an open-source FPGA tool chain Kind of like the GCC for FPGAs and I'm Sean and coincidentally I have too much work But one interesting fact about me is I tend to get cold easily So I have lots of jackets because the weather was just up there and it's gonna get pretty cold tonight So I brought a lot of jackets Fortunately, I live in Singapore Which is on the equator and so getting cold is not so much a problem but one of their advantage of Singapore is that it's very close to China certainly closer than Sydney from where Tin hails and This is because I like to call myself kind of a fuller stack developer I like to work on various Projects various hardware projects low-level hardware projects We gave a talk at CCC a couple of years ago about this. This is a project Fernvale, which is an open-source Media tech mobile phone CPU that we open sourced essentially the hardware and the software is all open then there was The Chibi Tronix love to code Chip that it's kind of like an embedded educational platform for teaching Programming to seven or eight year olds. It's it's one step up from learning Blake basic electronics on how to blink an LED I love the book that we would did that went along with this chapter three is where we introduced multi-threading to him and then there is the Novena open-source laptop, which we actually did a presentation on I did a full crowdfunding campaign But throughout all of these I never ran the crowdfunding campaign myself and I never did any of the hardware myself I did a lot of the hardware bring up a lot of the factory testing But I didn't actually have a project or crowdfunding campaign of my own I Like to describe Sean as my hardware guy He tends to work with bunny who is significantly harder hardware. So he is bunny software guy Which I find funny So as I mentioned, I have a couple of big projects that I've been working on the HDMI USB and Tim video stuff I started in 2011 and Who knows when it will be finished maybe 2025 if I'm lucky and that's a long time and Doing really big projects which take a long time can be hard So sometimes I do smaller projects as well and today I'm going to tell you about the inception of one and this one actually is literally a smaller project So my family have computers and like everyone they deserve to be secure and My work gave me these things called UB keys which Help prevent phishing and I was like I should get them for my family And then I noticed something wrong I love my family, but I don't love them that much especially since I've got like five computers each being related to me And that adds up really quickly and I was frustrated with this because I kind of have a little bit of a hardware bent I know what's inside this device. It's a little processor. It must have literally less than ten dollars worth of parts in And it also rub me the wrong way that this is a security device that is closed source So like any hardware geek I went on Digi key I sorted the arm MCUs by price and came up on this happy gecko CPU from Cy Labs It is literally two dollars fifteen USD in individual quantities So you can imagine how much cheaper it must be in mass quantities and so knowing this existed I then created a schematic on a weekend. It was actually pretty fun because it's such a simple schematic I then created the PCB. I did the PCB in schematic in tandem. It was kind of fun I then Put one together by hand and I worked I had the device and you could make it as a hobbyist Yourself, it's a six mil six mil which is pretty much anything on the planet and that was really good and I started giving these to people and people started wanting them and The thing is as I said, I wanted smaller projects but people kept wanting them and kept asking me to do a crowdfunding campaign for them and Proud funding is not a small project and so I'd worked with Sean through a couple of things and On the off-charts. He might be silly enough to accept I sent him an email saying are you interested in doing this for me? And of course I said no I have too much work No, I said yes actually. So what I had been doing up to this point I wanted to do my own crowdfunding campaign. So I came up with this Which was supposed to be a do-it-yourself game controller Adapter board and so you could see it has a whole bunch of inputs and the idea was You would wire this up together with alligator clips and then wire that to like a joystick box and then plug in a wireless USB dongle into your PC and it would be Really easy to design it yourself like maybe a dance dance revolution mat or something like that The problem I ran into is that it just didn't make financial sense Overall, I didn't see there would be enough people wanting this to actually turn it into a product and so I was kind of You know, I was at the prototype stage, but I wasn't sure that this is a project that I could go to production with Then Tim came to me and said hey, I have this thing. That's at the prototype stage. What would it take to make? Tomu a production ready board And I thought oh this seems interesting people seem to really like Tomo They seem Attracted by the fact that the thing is so simple yet. You can do so much in such a small package and there were a few issues with The design at that point mostly due to the fact that it was a couple of weekend project that Tim put together And there wasn't really much of a community behind at that point For example, there was no good bootloader Like when you have a hardware project, you need a bootloader in order to put new code on it That's what made Arduino so successful was they have that serial bootloader that makes it easy to put new code on there The one from psy labs was a they have a serial base one that shows up as a serial device It requires a custom driver It's open source, but only compiles under the arm. I a are commercial tool chain And it requires you to short a pin to ground or to get to ground when turning it on which is not that Do-able when it's in your USB port So there was a binary patch that was made to turn you know a jump if not equal to a jump always Which really you can't ship a product that sort of thing So what I decided to do was work on a DFU bootloader DFU is the device firmware update. It came out of the open moco open mobile phone Project it's driverless and simple it works on all the platforms even windows 10 actually nowadays without a driver And it only needs end point zero so unlike the serial bootloader Which needs two extra end points two and a half extra end points it works without a whole lot of complicated hardware or software It needs a plastic case originally the design had this image on The left which was a 3d printed case. It's it's very simple You kind of have to put it in the USB port at the right angle Another option was just fold a business card over and jam it in the USB port. It's fine for a hacker But you can't really do a product around it So I designed the 3d case on the right there and had the factory produce that This 3d case was actually my introduction into free CAD which is an amazing open source 3d modeling program you all should go use it if you want to do 3d modeling one of the cool things and I'll go into this Later actually go into it now is that it can export step files Now step is what the factory is going to want to do use and they'll take that step file and turn it into this big Piece of steel. That's actually about this big for the Tomo They'll turn it into a chunk of actual steel That requires multiple people big bars to actually lift up And it looks like this. This is what it looks like when you actually take us a slice out of the steel tool The cool thing is I asked the factory when they produced this Tool hey, can you send me the step files for that tool? And so I actually stuck those inside the Tomo hardware repository So if you ever wanted to know what a steel tool looked like that produces a piece of plastic You can see that in these files for the one that we actually used for Tomo So we launched the campaign and it was success success We raised more than enough to cover the cost of the steel tool to cover the cost of making the bootloader And the great thing is now there are a lot more Tomo's It's now available for mass manufacture it has a lot of Support software and everything that you would need to quickly get started, especially if you're not familiar with how to program a microcontroller So you don't need to worry about serial bootloaders and have to figure out how to upload something with X modem Like the original bootloader had you do no need to short out pins or fold over business cards And we're at a point now where if you want to solder your own from hand you can too and You can use our bootloader and our software and it just makes it easy to get to know hardware from a More baseline level Things I learned from this crowdfunding is hard crowd campaigns are hard Novena was my first But this is the first one that I did all by myself and I learned that Tomo customers Are really awesome. They're really understanding and You know job well done You know, that's it. Yeah, we're done, right Right well so As I said at the beginning I have two primary projects. I'm working on at the moment The second primary project is this project called simbie flow, which is an open source FPGA tool chain and I've been looking for ways to get more people into FPGA development and I came across a cool little FPGA dev board by This guy called Luke Valenti who goes via the name tiny FPGA And he created this tiny FPGA board, which was pretty cool And so I'd already successfully done a smaller project by palming off all the hard stuff to sobs So I got thinking could I do it again? Could I Create a FPGA version of the Tomo? At this point Sean was still busy doing Tomo stuff. And so I contacted Luke and said You want to give it a go and he was like, yeah, that seems pretty cool. I'll give that a try and created this design here and that was really cool because You can see it's almost exactly the same form factor as the original Tomo, which Means that we should be able to reuse the same case And it's the same size and it's much smaller than his original tiny FPGA So it's an even tiny FPGA and he even constructed one by hand This is well beyond my skills as a soldering person but he managed to build one by hand and so We had a Tomo FPGA Device that we knew worked. We could blink an LED with it And it still fit inside your speed port. So this was really cool And when looking at the pricing it looked like it would be cheap enough that I could give them out to people Without having to worry about their cost The other thing I had found at this point was this thing could circuit Python by Adafruit Circuit Python was really cool in that it made People who were previously too scared to be embedded programmers Into embedded programmers And I think that's really cool. So I wanted to emulate some of that in the FPGA land So I created yet another project called foopy, which is FPGA micro Python Circuit Python is a fork of micro Python to kind of replicate that ease of setup and To enable people to start the FPGA development System by just starting with Python stuff and we managed to prove out the FPGA micro Python on some bigger FPGA boards and Everything looked like it was ready So I asked Luke whether he wanted to run a crowdfunding campaign for this but turns out Luke is quite a smart person and was like A crowdfunding campaign is a lot of work But it turns out I knew this other person who was also really smart But not actually that smart because he didn't learn the first time and so I sent him a message and said Do you want to do another one of these? Um, do you want to do another crowdfunding campaign for a device that fits in your USB port? And he grumbled a bit But it seemed cool enough like if you look at the specs for the FOMU It's significantly more advanced than what you could do with the Tomu and so This brings us back to Sean who actually gets to do all the hard work Yeah, so earlier we did a lot with Tomu to get a production ready because there was a lot that needed to be done to actually Get it to a point where we can mass produce it Fortunately with FOMU we had a good starting point. We had a plastic case we could work with already But there was some work that needed to be done For example, this is the PCB design from the original FOMU that Luke put together It is very cleverly done. It is doable in essentially a one-layer PCB with relatively lax design rules, but if you zoom in this is a close-up of the actual FPGA Footprint you can see that some of the pads are round some of them are flattened and oval he does things like Running some power wires to some data pins Just because he has no way to put a via underneath it to make it not energize that power pin And so the problem with this is if you actually go and produce this it will work Reasonably well you'll get some failures right away But because as you put it in and out of the USB port it'll start to flex and slowly come across come off and crack So it might work right away, but eventually over time there might be issues with reliability Which is not necessarily what you want to do have with a product. So I went and redid it a little bit It's a four-layer blind buried vias, which is not such a common process Here in at least in the US. It's very common in mobile phones, for example We if you even if you find cheap phones in China, they all use this four-layer blind buried vias just because all of the modern chips require it because I was going with a More advanced process now I could stick ESD protection down Every component on here has a second source and that's important from a Manufacturing standpoint and that if the crystal that we use is no longer available We can go with a different supplier and put a different crystal that done and it should work exact the same I was able to run all four wires for spy so we can actually do quad spy on this The LED has a 3.3 volt power supply going to it rather than a 2.5 volt power supply Because I actually had the room with this four-layer design to be able to run the power pins And I did length match traces, which is not so important on such a small Low relatively low speed design, but I did it just because I put all the test points on the top Which ended up not being entirely necessary And the thing is this is 0.6 millimeters thick So we actually have a custom Depanelization jig that we put the boards in and then have a CNC drill go through and mill them out because if you just bend So normally when you want to separate PCBs you either do V scoring Where you've essentially run a drill across it and then bend it in half or you put mouse bites They call them and then you snap it off like that These weren't really options with with such a small PCB because the factory was afraid that if they bent it First off it'll leave some material on the side, but also it could cause components to flex and bounce off So we had a custom deep panelization jig which I'll mention in a bit So the factory management I made a mistake Hardware is hard. I screwed up on one of the Schematics and accidentally forgot to run power to one of the power rails and accidentally ran ground So x-rays are really cool And I had bunny who I work with and Tim mentioned that he does a lot of hardware I had him for scale. This this is what a tomu a foam is So this is that PCB up there He managed to get a flywire underneath one of these pads underneath the BGA In order to run power to it to make sure that my revised design with the revised schematic would actually work So the takeaways here are hardware is hard and x-rays are cool and being in the same time zone as the factory really handy because It means it's a much tighter loop actually talking to them But sometimes things go wrong This was an example of somebody not Knowing how to work a CNC drill. I think they misprogrammed one of the axes or something like that It honestly looks like somebody took the erase tool in real life and just tried to erase the fpd the PCB This is a sad mistake that that the factory did they only did it once well 17 times But they're never gonna do it again. They promise Managing the factory is hard So to mention that I was really excited about FOMO That's because it has this lattice ice 40 family of FPGA, and it's got a fully open-source tool chain And I mentioned the work that needs to be done. It's got an open-source tool chain But if there's any people here from the from a distro packages You know that patching packaging stuff is hard because this is a list of all the tools You need to run in order to actually produce a working bit stream If you have an embedded CPU, you just need a compiler for this we use GCC the risk 5 has a has a Sci-fi has a tool chain they put out. So we just use that tool chain from them Then you need a synthesizer. YOSIS is the one we use but there It was difficult to find Pre-compiled versions of YOSIS that were current for platforms like Windows and Mac The same story for the place and route we use next PNR Which is brand-new. It's only what about a year old I think a little bit longer than that A little bit longer than that But it's not available in any of the common packing systems and certainly not available for Windows or Mac For packing we use project ice storm and that takes the output from the place and route and turns it into a bit stream There's no pre-compiled binaries available from the vendors for that for any of the platforms We use DFU utils for loading DFU now that does exist for Windows and Mac, but installing it can be a bit of a challenge Especially if you aren't familiar with homebrew And for interaction we have wishbone tool, which is a tool that I wrote and we'll talk about that at a workshop That we're giving later So we had to package all of this And you could see finally after a lot of work on the image there on the on the right Finally Travis is reporting that everything passes. So yay The installation process there is you downloaded and you set the path That was the thing we had to do By we we mean Sean I just looked at it For we also needed a copy of Python now Python is great because you can get it just about anywhere And we use this software called me Jen my Jen me Jen I'm never sure to how to pronounce it It's it's weird living on the internet because you see these words all the time when you never actually pronounce them I say me Jen It outputs verlog as part of its output stream, which is great because you could take that code and then Inspect it in case you're not sure what it's doing or run that into a Simulator such as Icarus verlog or something along those lines It has native finite state machine support multiple clock domains. There's none of this distinction that you get in Verlog of like register or wire and it's actually real Python code that compiles Certain things down into synthesizable verlog code. That's an example of what a finite state machine looks like somewhat difficult to do in Verlog super easy to do in me Jen We also use this CPU called Vex risk V Vex risk 5 5 5, okay So the nice thing about risk 5 is that you have several designs to choose from we use Vex Because it has a lot more features. There's Pico RV. There's Minerva The great thing about risk 5 is that the compilers are all upstream. You can go and get clang I believe they're getting support for risk 5 sci-fi packages GCC It's so easy to just change which compiler you're using and suddenly you're targeting a whole new platform Vex also has on-chip debugger support So they have this debug port that it's designed to be used in verilator in simulation but we take advantage of that in a different way and One advantage of risk 5 versus some other CPUs that we used to use such as LM 32 is that The code size is more compact. So if you're limited in the amount of ROM you have Switch to risk 5 and you should save. I think it was about 30% of code size Just by changing your compiler and your CPU But compilers are still not as good as arm Right now they'll get there and if you look at some of the synthetic handcrafted assembly code It seems like they'll get there eventually Which brings us to the magic of FOMO is that You can just treat it as a risk 5 as a user You don't even have to care that it's a CPU that it's an FPGA when you get it It's running software. So there's no need to replace the bit stream and do all this FPGA Stuff just treat it like a CPU one with 128 K of RAM and a two megabytes of storage You can use GCC which you're familiar with Presumably you can write bare metal C if you want There's a port of Zephyr. That's currently is it going upstream or is it getting up? The base support is upstream and a bunch of the peripherals are upstream as well I think the USB is missing still though And you can treat it as just a DFU device to load devices or in the future We'd like something with you have to Or just treating it like a normal mass storage device to drag your binary on to the disk and have it just work And that's something that the circuit python people do is that they just appear as a mass storage device and you drop your file onto the System and you can program it that way you don't need any software or any Thing apart from the ability to mount a mass storage device right having it Just show up as a USB mass storage hard drive is a huge enabler to getting people Over the hump of just getting started because it's something friendly that they know On top of that Tim is very good at convincing people to do things for him and he found this project called renode Which is kind of like QMU on steroids It is something I have yet to try but it supports a whole bunch of SOCs and he's It's it's great for for you know debugging a continuous integration and it's designed to not just simulate the CPU but to simulate everything around the CPU as well and so it's To manage to get them to add light x support upstream FOMO support so it simulates the vex risk V It simulates many of the light x peripherals such as the timer the interrupt handler They're working on doing simulation of the USB stack, which is going to make debugging the the software a lot easier So you can actually synthesize or compile FOMO software and run it in renode And in fact the upstream renode config now ships with FOMO as one of its configuration settings Now I mentioned the USB stuff to borrow a quote from Carl Sagan When first you're inventing writing the bootloader You first have to invent the universe. So with Tomu. We just had to write the DFU aspect of it But with FOMO we had to do it all from scratch. We had to create the USB itself So we there was some sample code originally from Luke who got it working within a single 48 megahertz clock domain But there was a lot of work I had to do to get the bootloader working on FOMO We ended up with this thing called Valenti USB It's written entirely in Python and Mijen It's has a few different CPU interfaces available. One is one that actually shows up as the CPU I'll go over some of the other ones later One of the interesting design decisions we had to make is the UP 5k is small like most of the time you think of an FPGA You think big outputs a lot of heat when I plug this into my USB battery pack It keeps turning off because it thinks nothing's plugged in As a result it doesn't meet timing with everything in the 48 megahertz domain So we had to slow it down We had to pull as much USB stuff into this slower 12 megahertz domain to get it to meet timing and Unlike most of the other ice 40 boards out there with the exception of the tiny FPGA. We have no native UART So a lot of the ice 40 things talk directly to an FTDI. We don't have a native FTDI So there's no way to talk to this native. We don't have a native anything support. We have to create it all from scratch Which leads to some pretty interesting things we can do If you look at this this is a schematic of what a light-x chip looks like you could see there's a wishbone bus at the top And that brings everything together brings together the CPU the configuration status registers like a serial Timer IRQs the RAM is on the wishbone bus the ROM is on the wishbone bus But you'll notice that on this there's also this thing called a bridge that we can add which Light-x supports this interesting thing called multi mastering where we can actually have two or more masters on the wishbone bus And you know I saw this and I thought ah Wonder if we could make a bridge out of that out of the USB So I put the slide in there because there was a talk given by CERN about light about ether bone Which is a protocol to tunnel wishbone over ethernet and the only source I found for the actual protocol definition was a talk so in keeping with the tradition of that here is my slide Describing debugging over USB It's really simple. It's two separate commands. One is read one is right and it's a setup packet and if you do it right the actual Risk five running on foam movement has no idea. This is going on. This is completely separate from That it doesn't know that there's a debug command, but through this we can Read and write anywhere within this wishbone bus Now remember earlier I said that the vex has a debug port well It turns out that is on the wishbone bus as well So we can do really cool things like run gdb over these USB bus to debug the CPU That's running inside your USB port over the USB in the USB port running on the FPGA. So it's kind of wow This is really cool and this works and this is an example That we're going to talk about from the workshop where we're actually debugging a blink code You could see the back trace you can stop you can set break points it's You can take it one step further and actually debug the USB code over USB if you break in the right points Which is kind of a fun trick because we don't have enough space in there for a serial console Anymore because it's full of USB, but we can get full online debugger support And The Reno team is also like I said in the process of adding USB emulation So now they can actually use Valentino USB as a target One step further when you have a serial port on Your FPGA board you can send serial data back and forth and get information that way But we can actually take this design one step further and actually cut out the CPU entirely We don't even need a CPU in our design because we have this bridge and we can access all of wishbone and With light x you can actually do really cool things like this where this is essentially what a GPIO block looks like there's Really nothing more to it than this a divide this defines a storage register and then it gets some pads that get passed into it and then it assigns this the backing from The output from the storage register into the pads and that's it That's that's really all there is to get this working and if you plug this in It will generate a file called CSR dot CSV Which will give you memory addresses which you can then peek and poke over the USB bus And it's it's super easy to get started and super understandable in how it all works Again going one step further Reno supports the FOMO bridge so you can actually run a virtual CPU on Your PC and communicate to FOMO via this bridge So remember how I said we removed the CPU earlier You can kind of put it back by running it on your PC and then just memory map everything from wishbone over to the bus And so here's an example of what it looks like when you run something on your PC. They're running You could see the communications running on the side and they're actually blinking an LED By poking memory into a virtual CPU on the PC that gets routed over this bus as memory writes over USB I mean, that's just that that's really cool So to wrap it all up running a crowd campaign is hard videos are hard No, we know if my first Tomu was my first by by myself But I can say that through it all Tomu and FOMO customers are all awesome So, thank you back to Tim so What was shown is the Tomu and the FOMO and I've been fairly successful in convincing people to do this work I haven't actually done much apart from management and poke people Sean here has done a lot of the work of making this stuff happen. And so we have the Tomu family and so Why stop there? Maybe there are other People on the internet that might want to do it. So we have the Tomu The key thing about the Tomu is you can build it yourself Then we have the FOMO which is configurable hardware There's this other group on the internet who made this thing called the solo key and They were interested in doing something similar And so we talked to them and they developed the somu which is the security Tomu it's very similar to the original Tomu except it uses a STM secure Microprocessor so you can have more confidence that you could use this as a security key And they have you have to support including phyto to support. So that's the somu None of our systems had any way to communicate with the outside world and I noticed this guy called Femto that made these really cool tiny Beacons and so I suggested he make a ESP 32 Tomu and Crazy enough. He did There's a design that exists and he's working on it. He calls it the FEMU I prefer it to call it the one move personally for the wireless Tomu But it's his project. So he gets to name it And so we've got a bunch of Tomu projects out there Um, but what about a Bluetooth a bomu? There's this great little chip called the NF the nrf 52 840 That would make an awesome security key with NFC and Bluetooth And we even have a logo already to go for that when I did the logos Or when I convinced somebody else to do the logos for me I suggested they do a Bluetooth version And what about something else I haven't thought about? There's plenty of cool people in the audience that might have ideas So you can reuse a lot of the stuff we've designed if you reuse the form factor You can use the cases Sean has designed So I would encourage you to try and build some stuff that fits inside you as people I think it's really cool And I've been known to help fund these projects and help develop these projects Because while I don't have a lot of time myself I have a IT salary and very little expenses And I love to see people doing open hardware So I would love to see you make More Tomu related family things Um and so we did the Tomu and the Tomu but there is more of it more of the family We would love to see even more And even more so at ccc camp We will be running workshops And when I say we I mean Sean will be running workshops Because he hasn't learned yet The workshops will be at the hardware hacking village The thing is if you come to me and show that you have The formu toolchain set up I will give you a free formu To play with and it's yours to keep That's the great thing about the formu project is it's cheap enough I can do that The other thing is if you do more than that if you send me pull requests, I'll give you even more hardware um I also have non tomu fpga hardware here That I'd love to not take back home This hardware is a little bit more expensive um If you're interested in working on simby flow or one of these other Um fpga boards, I have them here to give you um If you send me pull requests Um, I will give you hardware. That's kind of my deal Um, and so I'd love for you to come and contribute to simby flow. Um I think fpga is a really interesting technology Thanks to a huge amount of work that shawna's done We actually have a device that I can give out that is cheap enough that will get you into fpga Development or risk 5 development if you don't want to go into fpga development um Or just embedded python development if you don't want to go into risk 5 development And I think this builds a really interesting pathway That previously hasn't been available And I'd really like to thank all the people who I've collaborated with Including shawna who's been a major person in making this possible But including others like luke who did the Original tomu design florent who does a lot of the litex code The white quark and sebastian who do me gen and end me gen The reno team for the reno support I think one of the really important things and one of the things I've really loved about the tomu project Is the fact that It's a family of people coming together to do cool things and everybody has different reasons for doing it My reason is because I want to see cool things exist And it really makes me happy to see them happen. So Hopefully this talk has enthused you that If you stand near me long enough, you'll find yourself doing a project for me And that will be an awesome experience and you'll get to learn lots So, yeah, um, I have A bag of formus here. Um, this is about 200 the one thing That's a little bit disappointing about formus is that you get like a pack of a thousand. It's only like this big and this high Um, so, um, yeah, and I still have plenty more that aren't in this bag Um, so I feel like I'm very unlikely to run out But running out would be a good thing. I will make more or More I will get Sean to make more and buy them off him. Um, so Yeah Thank you very much