 So I've been attending the CCC since 30 years and what I always appreciated coming to this conference or this Congress as they call it is that it brings people together to collaborate on projects that probably have not seen themselves, seen each other so in real life and it's nice to put a face on a name, it's nice to actually work across the table from each other and people come from all over the place, over the years we have people hailing Tim for instance all the way from Australia, Bunny from Singapore and it's great to see this sort of thing happening. What Tim and you know Tim, Mithro, Anzel and Bunny from Singapore are going to talk about is how the CCC has been able to bring together a diversity of people to enable products, projects that really couldn't have happened without this sort of collaboration and with that I would really like you guys to help me welcome Buddy and Tim. Thanks. Thanks everybody for coming to attend our talk. I know it's a very exciting night tonight for the Congress. There's a lot going on. We're going to title the talk is Snakes and Rabbits. I'll let you map that on to what you want to map it on to. How CCC shaped an open hardware success depending on how you define success. If it's shipping a device, okay, but if it's you can ship the device five years from now, that's yet to be seen. But in order to sort of lead into the talk, first I would like to, we're both going to introduce ourselves a little bit to sort of position the relevant parts. So who am I? I'm Bunny. I do hardware. I deal with atoms and atoms are hard. They have some difficulties. I'm going to tell you a little bit about why atoms are hard, not just because I like to complain, but actually the challenges surrounding atoms help motivate a lot of the decisions that we're going to talk about throughout the course of this talk. So first thing is that atoms are generally owned or people think that they have some right to some atoms. It's very rare you encounter a piece of atom that someone doesn't have. Some feel they have some right to tell you what to do with either a government or another person. So you have to deal with vat and tariffs and all this sort of stuff. But the consequence is that as an engineer, it's oftentimes much easier to spend your time engineering, to avoid convincing people to pay more for something or to have to move things around. And that ends up becoming a core constraint on a hardware project. Another problem is that atoms generally tend towards entropy. They don't exist naturally in perfect crystalline silicon. They don't exist in nice orderly rows of capacitors on your motherboard. And so a lot of effort is put in the engineering process to simplify things, to deal with testing and validation. Atoms are generally not in the place that you want them to be. So for example, tantalum ore is found in the countries highlighted in green here. Those countries are not necessarily renowned for their ability to turn out tantalum capacitors or to put them on circuit boards. So you have to go through this whole process of acquiring the atoms and then getting them in the right place and refining them. And so moving atoms is actually quite expensive. Any time that you get a deal that says you get free shipping, generally shipping is not actually free. You're either giving it something in privacy or you're hurting the environment somehow. So there's always some sort of consequence to it. And from the standpoint of sort of like, you know, from the open hardware and just the harder playing standpoint, you spend a lot of time thinking about the like the reverse logistics. If someone has a problem, you can't just patch hardware because you're in the wrong place. The hardware is in the wrong place. That's a very difficult problem to solve. And more so just even getting your human self to the right place, going to conferences and so forth, can be very difficult. So atoms are generally not where you really want them to be. And atoms also just have inconvenient truths about them. So, you know, RM is a nice abstraction. You can remove things. It's forever. It goes away. We have garbage shoots. Those are also abstractions. They don't work like RM. The garbage goes somewhere, someplace else in the world. But, you know, we deal with the inconvenient truths, just like, you know, everyone here, you may like to think that you have free and open software that's secure on your laptop, but and it all starts from the reset vector, but in fact, we all kind of accept the fact that there are firmer blobs that run inside of our machines that we can't do anything about because we have to move on with our lives and do some development on something somewhere. So, despite all of these problems, I still love atoms. Actually, it turns out, I can't explain it, but I was actually browsing through a decompiled genome of mine, and I found a certain comment annotated to one of my communications saying, does not learn well from mistakes. So, back in the day when I first burned my hand on the soldering iron, it's probably helped that I had that because I kept on soldering despite the, you know, the inconvenience posed by hardware. Over to Mithro. He can tell you a little more about his background and what he does. So, hi. I'm Mithro, and I self-identify as a software engineer. In contrast to atoms, I deal with bits. And bits are interesting in that they're a lot easier than atoms in some ways, but they're also hard as well. If you're a hardware engineer or maybe a venture capitalist, you kind of think of software like this, this kind of blob of ones and zeros. But in reality, software is written by humans, and humans are not robots and suck at doing repetitive things, and developing software is actually very repetitive. There's lots of creative parts, but a lot of it is repetitive. And so software actually really looks like this. It is full of bugs. And I have written lots of bugs. I get paid very well to be a software engineer, but even best practices can't prevent bugs. So all software should be thought of as being full of bugs. And sometimes bugs are really big. I have written big bugs before. The thing about software that is very different from hardware, though, is you can fix these bugs after the fact and make it so they never existed in the first place. And this is awesome because I write lots of bugs, and I prefer you didn't notice I write lots of bugs so I can fix it afterwards. And this is really awesome. And that's why atoms scare me quite a lot. Because it is much, much harder to patch atoms, and there's always some type of trace that you've done this patch. It is really hard to patch hardware. I guess that's why it's got harder than the name. And so I didn't do much hardware at all until I discovered these things called FPGAs. And FPGAs are really awesome for a software engineer like me, because they turn what is normally a hardware problem into a software problem. And being a software engineer, I love software problems. It lets me fix problems in my hardware after I have shipped it to a person, which is great because, as I said, I'm a software engineer. I create bugs. And so I live in a very different world to Bunny. So it's interesting that we're both up here sharing a stage with very different backgrounds. How did this end up happening? And so Bunny is going to introduce where this story actually started. So a little bit about the origin of how this whole journey that we're going to take you on begins. Once upon a time in a hardware startup long, long ago in a place far, far away, and Adam Wrangler had a problem. So this is a roadmap from a company called Chumbi. It's defunct now from 2011. And it kind of goes up and right. But one of the things you notice is the price keeps on getting higher as you go up in the right, the screens get larger. And what it turns out is that glass is very expensive. It's very inconvenient to have to buy a large piece of glass and ship it in the product. And if you want to get a device in everyone's hands, one thing you want to do is get rid of the glass, right? So we're thinking, okay, well, how do we make it so we don't have to ship LCDs, pieces of glass inside of our product. They're very expensive. Well, we would like to, we observed that everyone already has a perfectly good piece of glass in our house. It's a TV. It's a very large piece of glass. It has people looking at it all the time. Wouldn't it be great if we could somehow co-op the bits that are going on to that piece of glass and put our bits on top of it? The problem is that most digital video links to LCDs, these pieces of glass are encrypted. They have a lock on them. You can actually modify those pixels as they go between the devices to the screen. So I noodled on it for a little while. There was a lucky break with a certain key being leaked and so on and so forth. And it came up with a solution of a man-in-the-mill attack on the link and an encrypted overlay. I'm not going to go into the details of it. But the net effect is that you basically have a chroma key background, which has like, you know, the pink is the chroma that you're going to get rid of, maybe a little banner with news on the bottom. You have your video over here on the right-hand side and they get merged together and you end up with an overlay even on top of an encrypted link. Out of this, a product called Netv was born. Unfortunately, the company didn't make it, but open hardware for the win. What happens when a company goes down its open hardware, the hardware still exists out there. And I'm able to give a talk about it at 28C3 in 2011 and able to sell some versions of it so people could develop on it. And so if you're actually interested in finding out the details of how this is executed, you can actually follow the link or look up the talk on the device. So the good news is actually this talk was recorded and the recording of it ended up being an important role in Tim's decision and a project that he was developing at the time. And so, handing off to Tim for the fork. So for people who are good at picking accents, might have noticed that I'm Australian or maybe you're paying attention in the introduction when that was introduced. And this is Australia. You might notice something interesting about where Australia is on the world map. It's kind of near nothing and no, New Zealand doesn't really count. The other thing is that Australia is actually huge in itself. It's like the size of United States. And that means us being physical atoms, as Bunny likes to go on about, it's really hard to get places. It takes like 24 hours in the air to get over here to Congress. And I don't particularly like flying. I end up doing a lot of it, but I don't really like it. And so I got interested in how could I get the content at conferences like this without being there. And so I became very interested in doing recording of user groups and conferences. And in 2011, I started doing this as a hobby and I started a project to do this and make it easy for everybody else to do this. And this is kind of roughly the setup that you would use to capture a conference. And one of the really important parts of at least technical conferences is the slides. And so you want a really nice, clean capture of the slides so that people can read the text because people are terrible at putting teeny, teeny text on the slides, as apparently I'm guilty of in this slide as well. And so when we started, we had these things called Twin Packs, which are very good at capturing VGA signals. They kind of did a VGA pass through and it worked really great. Things were really good. We were capturing lots of conferences. But then something terrible started happening. We started seeing more and more venues had switched to HDMI and we were unable to use the analog VGA capture hardware with these systems. And so we went on the lookout for HDMI capture hardware. And there's a VGA solution. Why isn't there a digital solution? And I searched everywhere and bought a whole bunch of different hardware but was not able to find hardware that was able to tell me what is going on wrong here. No matter how much money I'd be willing to spend, I was not able to get hardware to give me useful debugging information that allowed me to understand at a low level what was going wrong with these digital signals. And so this was giving me quite a lot of headaches because we had this really awesome recording system and we were increasingly being unable to use it. And then when I was pondering this, I came across a talk recorded at 28C3 about this HDMI situation and I was looking at this and going, well, Bunny has done a HDMI man in the middle. But he's deliberately designed the hardware to make it so that you can't capture because he's dealing with encrypted video and allowing a person to capture encrypted video would be bad according to the law. And so I was pondering this and I realized in my use case, all the video signals are unencrypted because if the presenter is sending encrypted video, what are they doing presenting it to everyone? And so I also discovered from Bunny's talk that when you look inside any TV, you find an FPGA, which as I previously mentioned converts hardware problems into software problems. And as a software engineer, I know all about making software do things it isn't supposed to do. And so I sat there thinking and went, let's make it do exactly what Bunny didn't want it to do. And so I was like, we should make this a capture device and created the HDMI to USB project. It's basically an FPGA-based HDMI capture project. And we started trying to develop this and replicate some of the stuff Bunny had done. But we discovered something. I actually lied to you that software and Verilog, which is the language you use to program FPGAs, are very different. And Verilog makes me do this most of the time. When trying to take Bunny's code that he had customized for any TV, we tried to use that, and we were just making very little progress. We had, you know, hundreds of thousands of lines of code and still very few features and nothing kind of worked properly. And so this was really sad. And I thought Bunny was a totally different type of person to me, able to make magic happen. And then I discovered this thing called Megen. Megen is a really interesting project that has actually been presented at Congress. I didn't find it through the Congress, but I later watched a bunch of videos from Congress on this technology. And what it takes is if you imagine this logic design, you've got a bunch of combinational logic and a bunch of synchronous logic. It lets you write a bunch of Python statements that describe the combinational and synchronous logic. And then it generates the Verilog for you from that. And it also runs through the FPGA tool chain, which is a bit painful, but if you've seen my other talk, I'm fixing that problem as well. But we were actually making progress. And even more importantly, Megen and Litex and the MESOC environment came with a whole bunch of really good cores that did mostly what we wanted already. And so we snapped these cores together because Python is very good at making things composable. And so we were able to compose these cores into a full HDMI capture system in pretty much four weeks compared to the roughly three to four years we've been working on the Verilog solution. And so that is how we became successful in developing this piece of hardware to capture conferences. And we now use it to capture many different open source conferences around the world. And because this was reasonably successful, I thought people at Congress might be interested to understand how we did this. And so I submitted a talk to Congress. And for some reason, they accepted it. And so at 33C3, I gave my first talk ever at Congress. And I talked about the project. It was recorded. You can go and watch the recording. And it describes how we did this recording setup using Python and how HDMI works and all the type of problems we ran into. And I was here at 33C3. And it happens that Bunny was also here. But he hadn't been idle for the last seven years. So, meanwhile, Tim was doing all of his amazing work on the HDMI to USB. Seven years has passed on the any TV design. And one of things that happens in hardware after seven years is that you have this inconvenient issue called EOL, end of life. You get a when a manufacturer says they can't make a product anymore, you get this notice that says it's EOL. So these devices here with the red arrow on them are things that I can no longer buy to produce more any TVs. For example, I can't even buy the original LEDs they use on it. I can't buy the memory chips. Can't buy the Wi-Fi. It's modular. Okay. But I can't buy the CPU anymore. For example, that CPU chip is no longer made at all. So there's it's sort of begging for a version two to come out. However, overlay is just so 2011. I mean, we have phones now with incredible video processing capability. But unfortunately, it's legal for you to even replicate some of the things you can do with your phone as shown here. If you just take a real-time translation app point at a piece of Chinese you can turn it into English. In order to do that with a TV signal you would have to break the crypto on the link and that would be a crime. And so overlay is kind of where it still sits. So it's not really interesting to do the overlay. And it turns out that, how do I get the video to start? Click here. Oh no! I broke it. Sorry. Should have just started. Apparently the internet link is not working. Ah, okay. Well, it's fine. Anyways, we can skip the cute video. Anyways, the problem with creating a v2 overlay is with overlay is that actually it's not lawful to go ahead and do these types of things because there's a statue called the DMCA. And actually the internet is broken so we can't even get to the next slide. Cloud-based presentation software. Ah, there we go. Excellent. I blame hardware. Okay. Sure. And then in July 21st, 2016, something happened. This is actually the team from the EFF. And they made an announcement about a lawsuit that was filed on behalf of Matthew Green and me against the United States government. And one part of the lawsuit alleges that section 12.1 of the DMCA. Friends, people like you and me from expressing ourselves and asked for the right of people to access encrypted content through devices like perhaps a v2 of NETV. Thus paving the way for me to get started and develop on NETV2. And so here I was able to start development on a v2 of the device. This is actually what the first cut of the v2 looks like. It was a mini PCI Express card. It was very small, meant to make a very compact device that would be nice to sit beside your TV or whatever it is. And so at 33C3, I didn't even know that Tim was going to give his presentation. I had brought some samples along and I decided, oh, we should probably do a little bit of user validation and see if the, you know, people would actually want to use a device like this. This is even like meet the need of the community and the requirements. It's great because both of us were here at that time. And then we had a little discussion about things and this is where things starts to merge together from instead of forking. And it turns out that hackers and very tiny board is not really a good combination. Hackers really want you to break out all the GPIOs at the end of the day. And so trying to target a very small form factor, very sort of consumer friendly product from the get go was not going to get the community very involved. I had to go back to the drawing board and come back to the start. But this is a good thing. Like this is why you come to conferences and you talk to people to make sure you're not doing the wrong thing because it's very easy to convince yourself that you're building absolutely the right thing because you spent so much time building the circuit boards and fabbing things out and buying thousand dollars of parts. But you want to find out before you get into mass production because then you're constantly making mistakes at a rate of, you know, some, you know, exciting threshold per day. So I went back to the drawing board, tried to listen to what people wanted and came out with what I called any TV to MVP minimum viable product. And so it ended up being a full sized PC express card, which is a bit more friendly. You can put into a PC and has more HDMI ports. It has like a built-in ethernet port for debugging, has an SD card for you to go ahead and load like, for example, if you want to put Linux on it and a file system USB port, a bunch of stuff that is actually not even mission critical to the core application of the device, but is really important for the community of people who want to develop with it and make other things, for example, like a video switch or something like that out of it. I also developed a case that could go around so people who didn't want to necessarily, you know, bolt on the seats of their car before they drove it, could go ahead and get in and turn the key and start driving the car. And so this hides the fact that it's actually a PC express card and you think you just get like a nice piece of hardware you plug it in and it does what would be the overlay. Of course, the hardware is all open and hackable. The hardware, the case itself was designed with a little bit of extra space in it with the regular peak array mounting on it, which is actually derived from the Novena project of before. So if you, if the one or two people that will develop a Novena peripheral can go ahead and plug it into here. The front bezel was actually left intentionally completely blank in design so that if you had a laser cutter or 3D printer you could manufacture your own and put your own cutouts in it so you can add ports as necessary to it. And even we went so far as to making the whole test infrastructure very open. So it's very, very rare that you can get actually the manufacturing tests for the product. Actually, ZOBS had developed a wonderful open source infrastructure called Xclave for doing the tests and you can actually find out the details of how we test the product even. But very importantly, we now have it with Python. And so I love this sort of XKCD comic about like, you know, Q-ball is flying around and he says, how are you flying? Wow, Python, right? I go ahead and I, you know, tried, he convinced me at 33C3 to go ahead and try, you know, try out Litex. I put it in and I'm just shocked. I like, I like build it and I'm in a replous shell. I've got a BIOS booting. Everything's coming together like, like, wow, thanks, Tim. That's great. And then Python, right? Just stack trace. And then I'm like, I have no idea what I'm doing. Like, I don't, I'm, I'm a hardware guy, right? So what, what the heck is going on? So curse you, Tim. So, but the reason why I end up sticking with this is, is this is, this is the slide here. These are sort of utilization of the, of the FPGA itself. So this is the picture of the die. On the left-hand side is the proprietary tool provided by Xilinx. The results of a compilation. On the right hand side is the real compilation with Mijian. On the left-hand side, all that's on that chip is the core to drive the memory and a little bit of debug logic. On the right-hand side is the full application. It's the memory. It's a CPU, a RISC-5 CPU, three video files, the HCCP engine, an Ethernet engine, and all the debug stuff, all built onto that. And you can see that on the left-hand side is just jam-packed, just trying to hold just the core for the memory and some debug logic. On the right-hand side, there's room to spare to go ahead and add other stuff on the inside, right? So as a hardware person, you will go at great lengths to try to not have to solve, you know, the atoms are owned by people problem. It's probably easier to figure out how to debug Python than it is to convince people to pay more for a product. So I was like, all right, fine. I'm going to go ahead and deal with Python for the next few years. Another sort of minor advantage is that Litex and Megen builds faster than Vivota, about 10 minutes for that design versus 3 to 45 minutes, although as a sort of a little asterisk, the proprietary tools have gotten better, much better over time in build time. So the current status, there's a crowdfunding campaign. You can go to this site here to go ahead and if you're interested in getting one and playing with it, it supports the classic overlay mode, exactly like the old NETV did, and there's also sort of a Libre mode, which if you happen to have conference recording applications or you have writes to the video that's not encrypted, you can go ahead and implement this sort of video processing pipeline. And currently the status is simple amount of programming. I came to the Congress with several of these boards. Actually, I gave them to Tim. If you're interested in sort of developing on this, you can talk to Tim and maybe he might give you one of these boards to play with if you want to sort of learn more about Python and developing sort of stuff. And so now we're at the situation where through these interactions we had, Meejin is now helping NETV2. Open for the win. Right. Yay. But so nothing's really free in life. Good abstractions are powerful, but they do have costs. And Python is really great for building things out of components. And was the reason that I was very successful in doing the HTML USB was that I could connect together the components to, for example, compose a car in a bunch of different configurations very quickly. And it allowed specialization, the people who really knew how to make the tires could go off and do the tires, and then I could just plug the tires onto my car. And the people who really knew how to do seats, I could reuse their work. And this is really useful for someone like me who doesn't have a lot of time and doesn't want to have to worry about those details. And it's what allows us to get that efficiency is because we can compose components in a much more flexible way, we can also choose to not include them. And that means that our cores are much more efficient because they don't have all these optional extras that you don't need if you don't need them. But there is this cost of its Python and Python is great for building things out of components. But there's a bit of an impedance mismatch between hardware and Python. It wasn't really designed to make hardware and describe hardware. You can kind of make it do it. But there are things that somebody like Bunny who's a hardware person might expect to exist that don't. Example that Bunny always complains to me about is that in Megen we only have a logical zero and logical one. That kind of makes sense. You have binary, right? But it turns out in hardware you have things like a logical unknown or a high impedance value. So there in hardware they frequently model more than two states on their binary. You know, they're quite crazy like that. But the reason they do that is because this is what a signal looks like in theory. But in practice it kind of looks like this when you get to high speed. And when the signal looks like that, some of the things you tend to think of don't really hold. A not equal to B, A not the same as B are two potentially different things. And that's actually an important piece of knowledge to capture and something that somebody like Bunny really cares about. What Python is allowing us to do is that it's allowing Bunny to specialize on this. And there are some things he can't do because of our abstraction. He can't code Xs and Zs at the moment, which makes his life a lot harder. But he's able to make a really awesome tire that the rest of the community can use. And he gets all this other stuff for free. So even though his life is a little bit harder in doing the stuff he really cares about, his overall more efficient. There's also some other learnings that Bunny has discovered maybe about how well people test things. Yeah. When you get code for free, you can pretty much expect it hasn't been necessarily well tested. So, you know, number one principle when I was teaching hardware, you know, in college is that it's never the chip it's always you. So if you have a circuit and it's supposed to blink and it's not blinking and the student comes to me and says, I think my 555 timer is busted. I'd be like, no, I'm sorry, your circuit's wrong. It's not the case. You have a bad part. And it's literally cases like 99.999% of the time we have like a wall in our office in Singapore of the wall of shame of all the parts we ever encountered that were actually bad. And we had this one production run where we've probably gone through about a million parts and there's exactly one part in that wall that we can found that was legitimately a bad part of a million from a manufacturer. So it really is the case that it's in hardware, it's typically a case that you screwed up. In software, not quite so. I think I'm being generous here by, I would say, 90% of the time it's a problem in my code, but 10% of the time about four or five orders of magnitude more commonly it's a library bug or a problem inside the core libraries itself. So the reality check is that if you're going to go ahead and ship a product using a hardware product, in particular where it's difficult to patch things, you're going to need to plan to spend several months extra in your validation cycle. So for any TV2, I actually built in four to five months extra of an 18 month design cycle to go ahead and make sure we're testing it. And people would say, well, why don't you script that away? Why do I have to do it? It's literally the case that I have like a dozen HDMI sources at home. I had to find the worst three or four offenders. I had to find cables that were the crappiest cables I could find. I was like plugging them in and out. I have calluses on my fingers from taking these things in and out every single time I would do a release or a build to make sure that we haven't broken some corner case on the cable. And then it says, oh, why don't you just put that into a switch and automate that? Well, you put the switch in, it changes the analog behavior, the wires. You're not testing the actual system you're trying to test. So it's literally physically plugging cables into boxes to do testing for this. Another one is I built the first 100 boards, and I did not expect all of them to boot. I did not expect 40% of them not to boot, right? But I had, in my experience, knowing working this sort of stuff, I expected problems. Turned out there was a small issue with the DRAM core. It's well validated on a small set of hardware. But when you go ahead and scale up to larger aggregate and things to our test infrastructure, we're able to catch a corner case where a large number of them boot and in that sort of few weeks that I built into the schedule to handle these issues, we're able to work around that and issue a patch and get those boards working again. So if you don't just sort of take this sort of stuff and run with it and say, hey, we're going to go to production, it's all going to be great. You're going to have to expect to send some serious time validating your IP. The good news is, hopefully, that's several months saved for a future. Meejin Litex users, the bad news is that I hear there's end Meejin coming and that might throw away a lot of the effort that's been put into it. And so at the end of the day, FOS is great, but dot, dot, dot. You know, when we talk about the closed source company is they do have thousands of employees, billions of dollars of revenue and just maybe, just maybe some of that resources being spent to get to that 99.99% correctness rate on the IP. Right. So when I get an IP block from these guys, I can actually put it in and just go to the races and pretty much confirm that I'm going to be good. In contrast, Alphamax, the company that's making the NETV2 has one employee, only $78,000 in revenue so far you can check on the crowd supply page if there's more. There's still a lot to do. We're hopefully shipping next month, yay, that I'm here at the Congress, giving a talk instead of working on that. So please don't hurt me. I would hope that the company can last longer than the trial itself to try and make it legal to build the product, which is going to be several years. But the thing is that it's pretty scary to go ahead and try to incorporate some of these things and get them to the market. But you need to make sure you've built in enough time and check all of the open source code that you're getting to the level that people expect Harvard to work. Lesson number three. We're not quite done yet. Yeah, we've still got this lesson and one more lesson. Yeah, two more lessons left. So as Bunny said, convincing people to pay money for you giving them their atoms is actually quite hard, especially if you're honest about what you're producing. When Bunny did any TV, he did it mostly through having to work at a company. But these days, crowdfunding is actually a great way to develop this any TV was crowdfunded. The HDMI to USB, we did our own development board, the Opsist, that was also crowdfunded. Some other projects I work on, Zobz's other software guy does the FOMU and it brings together the like-minded people and allows them to put their money behind their beliefs. But it's very important to look at who you're targeting. If you compare the Opsist versus the any TV to communities, the Opsist community, we were very much targeting developers and people who are already involved in the HDMI to USB community. They were OK with the first thing they did when they got a new board was, you know, flash new firmware on it. Or before every conference, do some testing and get going because that was part of setting up for a conference. Whereas Bunny is really trying to ship something that you plug in and it works first time every time, which is a very different set of expectations. He's got like a nice case. I have no idea how you make a nice case. One day I hope he'll give a talk on that as well. If people want to hear it. Yeah. And cases like that scare me anyway. So it's very important you look at your community and target the right way because if we were to try and deliver the Opsist board as a consumer product, people would be very unhappy. Whereas if any TV to can probably fit into the developer category somewhat. But your aim is to hit more consumer market. Right. And so the the last learning I want to share and I think we're almost out of time is that open anything really takes time and patience. So open source software, open source hardware, they all takes time. So if you think about like the mobile phones that you have, you almost expect one to arrive year after year like clockwork, right? You have to have something over Christmas. And you and people think that, wow, like, you know, those guys just crank out phones once a year. The truth is actually it takes several years to build like a mobile phone. And so the big companies do is they have multiple teams of people running in a pipeline. So it's like two or three teams, one team will finish and then the other team will release the next product and then the next one will release a product and the while the other one's starting on the next one, right? So that's how it works in these well-funded close source companies. In open source, you're lucky if you can get one team revved up to work on anything, right? And so if you're going to go ahead and say, we're going to build like an open source phone or something like that, the problem is that by the time you finish it, a lot of times it's not that interesting a project anymore because it took you several years to get to the conclusion while all these other people are out investing you with the current technology. So open source projects are more, I feel they're more like, instead of like this, you know, glitz and glam of like trade shows and whatnot, it's more like a force, a walk in the forest. Like you're trying to plant trees, you're trying to build community and deep roots and a whole ecosystem. It takes time. You can't force that out there. And so picking a project that you can drive a wedge into over years is very difficult. And it actually turns out that it's perfect that, I mean, this sort of community of people who have a need for doing either video overlay or for doing recording of conferences is a community that we don't have to sort of deal with sort of the consumer cycle. We can actually, it's been literally seven years, this project has been in place and it's not like someone's gone and scooped us or tried to do something, I would invest this or something like that. So open source takes time. And so picking the right sort of project as the medium to express what you're doing is very important. The other half is that it takes patience to do open source. So in the software world, the sort of attitude, if it works for me, it's good enough. If you don't like it, fix it yourself. PS, give me your fix, that's in the license. And by the way, you also can't sue me to fix it if it doesn't work for you. So this is like the BSD two clause license, clause one and clause two are up top. And there's this other thing that we don't talk about in the license, which is bigger than the two clauses, which is the you can't sue me clause. It's a very important part about open source software. You can't do this for hardware. I can't just be like, hey, did you not read the shrink wrap? It didn't work, you can't return it, whatever, like fix it yourself, get a soldering iron. There's actually consumer protection laws that prevent me from doing that. And the situation is even worse than that. If I were to say, I'm gonna sell you a toaster with a single slot, right? And that's the single slot toaster on the left. And you open up the box on the inside, you find it's actually a two slot toaster where one of the slots has just been epoxy closed is it didn't work and they had disconnected the element. You'd be angry. You would say, this is not a really good toaster. I'm gonna send this back to you. However, how many times you've gotten a piece of code where huge sections are if-thift out, features don't work, and you compile them out, whatever it is, and you still run it. You accept that sort of stuff, right? So there's a very different standard of release between what you have in software and hardware. And it takes a lot of patience to do the sort of development. Because there are features that are not gonna work on day one that you're just gonna have to sort of figure out and get them working later on. So it is really super scary trying to ship hardware built with open source software. I lose a lot of sleep about like, what if a bug inside Ledix or Megen causes a functional problem in the product which causes a lot of returns or whatever it is, that's the end of the product. I literally wake up in the middle of the night and I get out my oscilloscope problem and I'm like, oh, thank God that wasn't like, that wasn't a hardware problem. That's actually like a software problem. You can go and patch that later on or whatever it is. Despite all of that, I still really love doing hardware possibly because I just don't learn very well from my mistakes. The end, we'll see how this all goes. Yes. And I think we have a final lesson that Tim would like to share. So you've kind of got two different perspectives here from a person who really cares about hardware and a person who is a software engineer and is much more willing to admit his human. And so I think one of the really interesting things about CCC is that it brings us both together and we're both given talks at this conference. We've given in some ways very similar talks before and now through CCC, we're actually collaborating together. So I think we should give a big hand for the CCC organizers who enable us to do these things because the things we're working on would not be as good as they are without this conference. So I'd say a big thank you to the Congress organizers. Thanks for bringing us all together. Yeah, and everyone who comes to attend, right? Like the year you were coming and that we kind of met was you were just attending. Yeah, that's right. Yeah. Okay. So I think we have time for questions now. Yes, and even though they asked you to applause yourself, I think we still have to thank them for giving us a reason to be all here at the same place. So we have about 15 minutes time for questions. We have four microphones and let me explain what a question is first. Yes, you may not actually know it. A question is like one or two sentences with a question mark behind and no, it does not include your life history. We have five microphones in the room. Could you please step up to the microphones? There's one here, one there, another two there and one all the way in the back for the cheap seats and the bleachers. So let's see, do we have somebody on number one? I believe that is one. Could you include, yeah, that's a specific question for one of us. Oh, yeah. Do you're asking, otherwise we'll both assume that it's us or maybe the other person? So I have a question for Bunny. So, good, good, sorry, another interruption. Get very close to the microphone. Is this good? Yes. So do you like hardware or modern software? Do you like hardware or like software? Who like, which one do you like more? Which one do I like more? It's like asking, which one do you like? Which one do you like? I like hardware, come on now. Yeah, I'm a hardware guy. I think that, so, my, how do you say it? There is a, there literally is some weird chemical reaction I have in my brain from hardware, like when you give me a really good piece of hardware, my pulse raises and I start breathing heavy. I'm very excited to see really good hardware, really well engineered hardware, right? The problem is that the reality of being a hardware nerd is that if you ship really cool hardware that doesn't have software on it today, they're effectively door stops, right? There's a lot of products in the world that are just door stops because the software sucks on them, right? And so what you end up realizing is that software is sort of a necessary evil to make hardware do something very useful. In fact, what you find is that the good news is that for hardware, one of the things I love about hardware is that they're atoms and so they're localized and I can see them and we can point to the problem. The problem is that this signal is not good. You have to have a good oscilloscope, Provert, thank you. But you can actually do that sort of thing. Software is like this incredibly complex thing and you can spend hours sort of plumbing through and still never find the end of the bug or whatever it is. But I have a lot of respect for what the software people do. I try to surround myself with really good software people to help me get through the dark nights when I can't get my code to work. And I think that ultimately the people who are going to buy the hardware have a better experience for having sort of thoughtfully designed software on it. So we also are taking questions from the internet. I don't see any questions at the moment but maybe there'll be some. But I do see another question on microphone number one and do speak closely to the microphone. My question is for both speakers. So my question is about when design, if we're to design a new project or product that is hardware based, would you recommend if we wanted to last and maybe get picked up and ported to different architecture or different layout or kind of live on, would you recommend using an FPGA instead of trying to pick an SOC that you find really cheap? And does that, for example, in your case now, does the FPGA kind of protect you, protect the longevity of the project? I could take that one if you want. Well, I give a talk called my theory on FPGA development and lesson number one on that is use an FPGA as a last resort. If you can do it in software on a CPU, do it on software on a CPU because the development environment is just so much easier, so much faster and you can be a lot more productive. But there are problems that you just can't do with a CPU and you are limited in your flexibility. I'm hoping that projects like SimbiFlow enable FPGA development to become as easy as the rest of the software and then maybe that will change. But at the moment, I would say an FPGA should be a user only when you have an application where it makes sense, you shouldn't just use it as a SOC. But I don't know, with the little risk five cores on the ISE 40, some of that is starting to become maybe I should use a FPGA rather than you don't need a 100 megahertz CPU for all your designs and maybe the flexibility around IO is better. I think that Tim's spot on, the only thing I would add to that from your longevity question is that yes, in fact, the CPU we picked for the original NE TV, you can't buy anymore, right? And so NE TV 2 uses a Raspberry Pi instead of a custom SOC because that someone else is solving the SOC problem and evolving that and keeping it up with the times. And in fact, today you can still buy a Spartan 6 FPGA and run the original NE TV 2 code base if you wanted to. So there is a bit of like those harder descriptions can live on even if for whatever reason you can't buy the other parts and bits to it. FPGAs do get end of life as well though. But much longer time curve than like a lot of these SOCs. Some of them are just a flash in the pan. So yet another question on microphone one. Hello, thanks for your talk, I really enjoyed that. I saw hints of lack of sustainability or maybe not the most sustainable development practices coming together with venture capital and product designs and maybe that open hardware could be an alternative way of doing things. I was wondering if there were any further thoughts that you had that kind of came out of where open hardware could go to make a better world in that regard. I want to be clear as open hardware is a hobby for me. I don't make my living from open hardware. I have a well-paying software engineering job. So I have a very different perspective on open hardware to somebody like Bunny who's actually, you know, has to fund his lifestyle from it. So I'd be very interested to see what Bunny says. You could say that I don't make a living selling open hardware. I just don't spend as much money. I reduce my cost footprint to the point where I can sustain myself on a little bit. I can grind out on top of the margins of open hardware. I'm still struggling to answer your question. You're contrasting the VC-funded model versus this open hardware model. It does tend to be the case that VC is go big or go home. So you have these ideas that you're getting investment for the point of trying to give returns to investor, thousand to one, and so you're making promises that are just ridiculously huge. And so the landscape is littered with dead bodies of people who have tried to deliver thousand to one, ten thousand to one expectations on investment. That system is rigged for failure at the end of the day. They're looking for unicorns. That being said, open hardware, because they're sort of more hobby-style projects, they don't die, but they oftentimes become zombies. And so the question is, can you get enough momentum in the user community to make it relevant and interesting and to move forward? And that actually takes a ton of effort, building community, getting people involved, answering their questions, sitting down with people, acknowledging their problems, making them feel valuable, and then bringing them, onboarding them, and making those links, it's actually a very time-intensive process, but I find them very rewarding, and they don't tend to blow up in the way that VC things do. So it's kind of apples and oranges a bit. One of the things, for example, you'll notice today, we don't have a good open-source mobile phone. A lot of people want one, that it's free and open and liberate, you can inspect all the blobs and say, we don't have one, partially just because it's such a huge investment to build one of those, we can't get enough money through an open hardware model to build something that meets people's expectation of what a real mobile phone looks like. So there are things that open hardware just can't do, because there just isn't enough resources to deliver on the consumer expectation. So microphone too? Okay, so I was kind of surprised that the Vivado tooling was doing way worse than the open-source tooling. So what I've seen in the past is that the proprietary methods work better and more reliable, so my question would be, did you investigate why this was the case? Yeah. The theory's on this. Yeah, there's a lot of different theories. My opinion is that when you're somebody like Xilex, you need to cater to people who are going to buy your FPGAs, and so they optimize for those people. That is not really somebody like me or Bunny who is a small volume type group. They more care about the people who are going to make 100 million of something with an FPGA in it, and so they optimize for those users, and those users are less, sometimes less concerned with how big the core is and more concerned with things like reliability features and having all those extra things that fit in the checkbox list of a DDR core must have, you know, these features. Whereas with things like the Lightdram, we can only build and use the features that we care about. The other thing about the Python stuff is that if you turn it off, it never appears in the Verilog, which means that the FPGA tool chain doesn't have to figure out that that thing is not used and remove it, and so that definitely helps as well. I definitely would say that Lightdram is not necessarily more reliable than MIG, as Bunny has kind of been investigating, but like it's definitely a lot smaller, and it definitely has at least a certain set of features enough to deliver some products with, and it's also kind of interesting that a company that makes more money from you buying bigger FPGAs has non-resource efficient designs. Nothing comes for free, right? So you got that software for free, and you're paying for it with the hardware, right? So what's their incentive to go ahead and sell you less hardware? So I'm really, really sorry to microphone one that we don't have any more time. We do have to wrap it up at this point, but I'm sure that they'll be available for chatting during the rest of the conference. Could you please all help me give these guys a round of applause for this fantastically interesting talk?