 and service point the display a tale of four generations and the speakers are Peter Stugge and Felix Niklas and give a big applause. Thank you Thank you Thanks for having us. I'm Felix and this is Peter Hi, so Yeah, we're gonna tell you four tales about the display The beginning of the display was that we that one of our club members boarded on eBay and We didn't really know what it was used for before it was done by a metal works company Who had an electric department that built the electrics for this and the metal shop? They built the hardware around it the nice frame and They sold it off. I think because it didn't work out for them so We got it on eBay and It was at in the beginning very very slow And at first I wanted to show the outline This is how it looks inside of our club. It's mounted on the wall So we're gonna talk about the hardware about the first until the fourth current generation Gotta talk about some software and work that went into it font and tethering And then about the APIs that we have the UDP and web circuits and send some fun applications that we ran on it Peter you want to start with that? All right, so hardware Yeah, first generation Was delivered with the 6502 somehow controller board I Wasn't I wasn't there when that was being used, but some people managed to get it working and send some some serial protocol commands to the controller and We're able to watch the characters be drawn on the display. It was was as Felix said really really slow So this is made up of a whole bunch of these LED Matrix for LED modules eight by eight little LEDs No slides Yeah, there we go. Thank you. So let's get back. This is the controller. This is the controller, right the 6502 and These are the the eight by eight LED modules that are on this display and in total there are 20 20 rows going down and 56 56 columns so that of these LED modules eight by eight in Total that's a thousand hundred twenty LED modules and with eight by eight that gets us 71,000 and some LEDs So it's a 71 K display So that was the first generation and then we got to know the hardware a bit and some some people in the Berlin CCC they made a New controller for this because the 6502 wasn't wasn't a lot of fun and was was as we said quite slow And of course it wasn't on the network. So we needed to get on the network Second generation was an Atmel X mega. I'm afraid we don't have any photos of it Yeah at this time There was also the beginning of a protocol designed for sending information to the display It consists of UDP packets that have a 10-byte header containing five 16-bit words in network byte order. That's important to us First one is the command we'll get into those in a little bit And then there are four parameters that may or may not contain some value if they're not used Or if one parameter is not used then the value is just zero After these 10 bytes there may or may not be some data payload also in the UDP packet these are the commands in the in the beginning or the the first The first yeah, the first commands that were implemented I'm sure there are a few so you see there are some gaps I don't know actually what the commands there were or the numbers there were anymore but clear obviously clear screen text sending code page 437 text because that's a fun encoding and Then we had some some brightness commands these These LED modules the 8 by 8 modules It was mentioned on the slide behind each of them is a maxim max 72 19 led driver made for this kind of matrix LED matrix display and in this led driver. There's there's a digital register for setting the brightness and We exposed that using some some commands Let's show a demo of that. This is a demo one of the members of the club did to test just changing the brightness since we can only do 8 by 8 Brightness changes. He still found a way to to do a nice Application of that because from movies obviously it doesn't really work to do grayscale or like color tones on 8 by 8 pixels, but in the end you used 8 by 8 Graphics to do this nice depth map. Yeah, so this to explain what we're seeing here Here the whole display is filled with plus signs So every one of these 8 by 8 modules is showing a plus sign and the only thing that is changing is the brightness value or the light intensity of These modules and that makes for a nice Even lower resolution low resolution effect on the low resolution display Going back to commands. Yeah. All right So that was character brightness setting it on individual modules and brightness for the full screen In one command just a reset command because that that helps sometimes then we had a fade experiment where A graphics kernel runs over the display and sort of reduces intensity Which looks like a fade it didn't get used very much and The main command the most important thing. Yeah, the most important new feature in the In in our development was to support bitmap transfer so that this this display could also show arbitrary graphics and The very first command because it's easier Received the bitmap data in a way that fits how the hardware was built and That's not really so intuitive or so convenient for software developers. It turns out Is that here or is it? Yeah pixel data in a vertical layout The the bitmap data was sent with each byte being eight pixels high or tall instead of Eight pixels across which is what we usually deal with in in software So still we could do bitmap. We could display bitmaps. That was a lot of fun and and a good start right and then We of course we want to go fast, right? so we we found out actually the the hardware the Factory hardware is is the construction is quite quite limited or quite slow Because not only because the 6502 is a relatively slow machine, but also because of how the whole data transfer was Implemented in in this display So as the arrow shows here the 6502 processor would address every single one of these 1,120 led modules at a time and in sequence Meaning that the Well the other 1,119 led modules were just not doing anything Most of the time right there's only one One module receiving data So this was changed in generation hardware generation 2.5 where The so actually we weren't using the 6502 controller than anymore, but but anyway, it's just representative here the the main The main controller the master controller sends data to these new row controllers that were designed in ncccb by some members and they have distributed Video memory so each row has them that has a video memory for the contents that is there and also parallelization the master controller sends data to all the row controllers and Only to the row controllers and the row controllers then take care of sending data out into the led modules or the led driver chips That is what it looks like There's a bug in the hardware that we fixed there But that was that was good that got us some more frame rate in in the bitmap bitmap Transfer and then we Well, then that wasn't enough it was still too slow for us So we then made an experiment with a biker controller That is a bit more powerful. I will fair bit more powerful than the xMega Cortex M3 arm microcontroller. I believe 96 megahertz and the thing about these particular ones is that they had Ethernet Mac and Fi built into the Cortex M3 microcontroller, which was quite unusual but very suitable for our project We still had the main problem that the graphics the bitmaps would have to be in this weird order going up and down instead of how We're used to it from left to right. So Yeah, so this was this is within one one row you would have to send these vertical Vertical bytes and that wasn't wasn't that all fun And we we decided to solve that in in generation 3 we implemented a few new commands to tune to new bitmap transfer commands To support just the the regular linear bitmap transfer that That is used also in in the in a regular PC or computer To to represent bitmap data where you send the pixels after each other Horizontally and and then you move to the next scan scan line and then so on So there's more processing power and we use that power to make it easier for developers And also the this arm architecture had some nice Instructions to to do this bits This bit transformation like to rotate eight by eight bits was was What was fun and easy and this the 2.5 G? new row controllers they also they Jumping back a bit. They made some optimizations as well to sort of reduce the The data transfer that needed to go into the rows because we had some flickering and we didn't have completely stable data transfer but reducing the the throughput helped with that that Allowed us to to still update a lot of Changing information, but areas that were all the same. They didn't get updated at all and to save power also if one of these eight by eight modules were Had no LEDs on then this particular maximum chip would be set to power save mode. So one also wanted to Improve efficiency and and reduce power consumption when we started out with the gen 1 hardware we Somehow we have some measurements we believe that we measured it to be over one kilowatt with all LEDs on and We've improved that with each hardware generation and we'll come back to power in a little bit But it wasn't perfect yet. There was something with the internet. I remember. Yeah, so This this bitmap linear command and the linear windowed Command where you can only update you cannot choose to update only a portion of the of the screen we We had the issues with this Cortex M3 that it did have a built-in ethernet Mac and Phi So network connectivity was easy and there was an example code for an IP stack and and all was good, but Unfortunately the hardware only had two kilobytes of packet buffer memory so if we if we take these 71,000 pixels and we want to or bits and we want to send those in one packet We have a bit of a problem because that's some nine kilobytes of data And with this microcontroller only having two kilobytes of packet buffer Some packets are gonna get lost so that was a bad scene and We we worked around it by instead of then sending one big UDP packet with the full image data We would manually Do what the IP stack does so the we send one UDP packet with nine kilobytes of data The the network stack divides this up because the network can't transfer that big packets And it splits that up into six or seven packets of 1,500 bytes or whatever the MTU is Sends that out to the display, but the display only received the first and the third and the fifth because in between it's busy copying the the packet that it just received into the into the display memory So what we did as a workaround and first Well to to overcome this at first was we did this splitting the video frame or the the I revealed the secret now splitting the the image frame Into several packets already on the sending side not not in the IP stack or the network stack but actually in the application so we made a video or image transfer application and there we cut up the the image in Little packets and we built in an artificial delay between the packets and that really that really hurt we that was painful We didn't like that at all Not a nice because you have to understand the goal always was to get more FPS Of course, we need to we want to make it go fast and we want to we want to play Want to be able to play video on the thing that was that was always the goal The second generation had a few FPS like two or three or so With the bitmap transfer commands and with the third generation We we were able to get it up to some nine ten maybe twelve FPS on a good day But we weren't happy because that didn't look so nice still so then we we took out the sledgehammer and we decided We'd try to use a big old bone black Because it has the PR you which is the programmable real-time unit So the bigger bone black many people know as a Linux Linux board similar to another fruity board that I won't mention and It is a Linux board. It's a it's a nice roughly gigahertz System on a chip, but inside of this. There are also two co-processor cores. So two 200 megahertz Little co-processors that are completely separate from the arm running the Linux system And you can write little program snippets for these and they run They run quite fast 200 megahertz and they can do IO single cycle IO GPIO well writing and reading and That we figured would be a very good way to to increase the performance the last bit that we needed and So we had already we'd taken several steps we'd already optimized the data transfer going over this The ribbon cable that you can maybe see through the through the front of the display It used to be serial with a very first hardware with the second generation Hardware it was still serial with the 2.5. We changed it to parallel So one byte would be transferred at a time instead of just one bit and the the row controllers would then take care of sending that That data out serially one bit at a time because they have time while the other row controllers get their new data And with the bigger bone back we continued that so we write some pru firmware to To talk to the row controllers and send them out the data The big the the pru has a shared memory. So the Linux system just receives Bitmap packets or really command packets Looks at the command and does whatever is necessary and saves the bitmap data into this shared memory Does nothing else does only that in a loop the pru is running Independently a firmware that takes the bitmap data out of the shared memory and sends it out the bus to the row controllers so even even more parallelization and With this we were able to go all the way up to 40 FPS. So that's Yeah mission accomplished somehow Let's show some pictures of the insights Right. So this is the the current state of things We have the big about black up top left with a cape on a bread on a breadboard with I think there's some level shifters and maybe some iobuffers I don't remember anymore exactly what's all on there But but yeah to to drive the Drive the bus going down and you can also see the power supply Here this is quite a small power supply. It's not enough for the whole for the whole display It's only powering the big a bone black And this is part of the the other well the the real power supply. Let's say for the LEDs Is there one more? There's actually there's two of those. Yeah, they're two exactly Right the two of these power supplies Are needed to to drive all of the LEDs. I think they are I don't remember anymore I think they're 500 watts each but we don't we don't actually need that much and We have two because there wasn't a single one that was high capacity enough and fan less But they've been in the hardware since the beginning that we didn't do this, right? We did this we mounted these power supplies There were other power supplies before that were really inefficient. So switch switch mode power supplies from the 1990s We're replaced with some some modern Modern switch Who did that nice wiring? Yeah, we did that in in ccv. Yeah, okay Yeah, the the the red and black and the but these copper Copper bars and the fuses they were already there. So they they're a factory and this little board in the in the Center there is an ad converter to monitor the the voltage Output of these these power supplies because we wanted to see okay when we switch on and off all of the 70,000 LEDs at a time Do the power supplies actually keep up with that and and yes indeed they do them. It's a stable 5 volt Yeah, this is the bottom right corner where there's the power connection and Ethernet connector and there's a nice engraving Showing our logo. Yeah, we had to had to tag it Okay, so let's go over to software Like as you might already feel I I don't know that much about the hardware But it was always a pleasure and being in the CCC and getting like hearing about all seeing all that's being done by the Different members of the club. Everybody's just pouring the heart out with where what they were good in and I Do web development? So I only was able to do stuff after they deliver the web server on it and Gave me web socket, which we will be talking about soon first Let's quickly talk about the font So the font system is a CP 437 I Am too young to really know this I think this feeling But I you might also know this from the boot screen because that's the same system being used there And MS-DOS and MS-DOS use this I know that So there's fonts out there obviously why would we need our own because we have our own one member built one and the problem is We have eight by eight pixel modules, right and There's eight by eight fonts. That's one of the standards. There's two different types one is higher. It's eight by sixteen or something That's obviously wouldn't fit but eight by eight would fit perfectly, but there's a problem our modules are one next to each other and When you put text on there, there's no space in between So that's there was the needs that we needed to solve that we have to have a gap so in the end a Seven by seven pixel version of this very basic font was created and this is the bitmap of it This is the basic texture set that's in there right now and the I don't know if you see it on the screen, but there's a light gray Area on the right of the seven by seven areas. This is the gap next to it and Marco who was working on this he he created the font. I think in Windows MS Paint MS Paint he said like in the virtual machine and now in wine I think because all right This is still the best thing if you want a pixel gimp is not good enough for that And then he created some scripts around to to export it and to on the one hand create a C header That is being used on the big alone and he also output it a Web font that we were able to then use in the web editor the web interface of the display to see almost the same Directly on your browser and then the next big Thinking was put into dithering and I don't know if you know the concept, but I mean let's say we have an image frame and It's color obviously we don't have color in there, so we need to make it into Something oh a step one point two four We need to first convert it to black and white But then again, I told you before about the brightness. We can only change the brightness in the eight by eight area so We couldn't do like gray in gray in there, right? We can only do green or Black or nothing so you need to the ring which is a technique to Create that effect by just using space by just using less white or in this case green dots It's the same technique almost the same that printers are using because they also only can print black and not black So let's zoom in a bit to see maybe see this a bit more this is like a different image and Mark would put a lot of love in this he even built like his own love 2d like a Lua application to test a lot of different Dithering algorithms. There's Floyd Steinberg, which is a very basic one But he quickly saw that this is not good enough. It's not fitting the the high-quality standard that He was aiming for and so this was the application he built where in the top left corner is there the original image It's actually in color and there were different ones for testing and then you could compare two different dithering algorithms in the middle and bottom picture and In the end he settled for I don't have the name here It's starting with Oh, and I think it's a Russian name. Oh something something as a base algorithm And then he had also some more pre-processing And to even to improve the results further he does some blurring in the beginning Because in the end you won't see all the details with the dithering So he blurs out like the blurring will remove that the details that won't be shown anyway and then he does some sharpening afterwards to Make those features that are gonna be shown a bit sharper And and you see it in the bottom left It's not a really good representation because it's scaled and when you scale different stuff, then it's bad But you see the results on the screen Packages or I UDP packets. Yeah, so Nine kilobyte bitmap frames. Maybe I said this already. Let's see. What's the next? Yeah, we have to speed up a bit as well so yeah UDP packet thing was web sockets from my because I never touched UDP before and With web sockets I was suddenly able to send pixels to this display in the browser using the JavaScript The web socket API is not the full API is it's not the same as the UDP API It's just for sending pixels So I was mostly working with it having a canvas and then just sending the canvas data the pure pixel data directly to the To the display there's a simple Demo application showing the web sockets Implementation or how to send stuff there. I don't know if it's big enough to see but in the end web sockets Is I mean it's a two-way Communication based on TCP actually for the web to to send stuff somewhere And you just build up the connection Say that you're gonna send array buffer data and You grab the pixels from the canvas at first I did that before I drew a rectangle on the canvas in this case and Then go through all the pixels pack it up into bytes and just send it out And it was actually quite simple to the applications with that. It's a very simple very simple protocol Yeah, but it worked to work fine So let's talk about applications and the CCC B What we what we do with it? So the basic one is a welcome screen It's since it's mounted on the wall usually whenever we have open days. It's on and this is what it's showing It's not always on obviously, but usually this is also the boot screen The one of the first one of the early the early implementations was when we didn't have the bitmap yet I think correct me from wrong then we The first movie was a ASCII Star Wars movie. There's apparently something where you can tell in it in and it just sends you This data It was nice for a beginning But then obviously which you already see live video Mark built a nice cheese through a plug-in so that you care that that takes care of all that dithering All the pre-processing we were talking about for you So you could just send your movie that way and it would correctly send it to the server There's one small thing we didn't talk about in between At one point we optimized also the the data load by compressing it as well, right? That's true. Yes, it was between third or fourth generation So that was that was added into gen4 into the fourth generation So these bitmap commands they they were one of them particularly only uses two of the parameters two of the four parameters in the utp protocol and then we introduced Some sub commands Supporting different compression algorithms there Dennis was was busy quite quite some time working through all the different Compression algorithms that he could could find and settled in the end on set standard for for for decompression speed in particular, but still a reasonable compression rate and Also something you might think about is the the lines in between Also one of the steps that of the G stream plug-in is to cut out some pickles so that it doesn't look stretched In the end so the way the way that works the G streamer plug-in says to G streamer Please deliver grayscale images to me that are actually that have more pixels than fit on the display and in these bitmap transfer commands then the the gaps Where the the metal is are are copied over so it's it's a little bit of it's unfortunately not zero copy, but it's not enough data to be a problem and it looks a lot better than than just Asking for the exact pixel resolution of the display then the the images is Compressed together and not not really recognizable and then when you connect to the API add to the IP of the Of the display you would see this web interface Just set a basic text mode where you could or it has a basic text mode where you can just type stuff on the screen And there's a small canvas drawing application in there as well to do some drawing on screen Then one of the fun things So it's a Bitcoin info screen somewhere in between with live updates Tixle flight schedule like this one a lot and there was also one for the is that one up next for the Yes, this one is really good. So on the open open nights there's the Fridisch Trasse station right nearby the club and Towards the end of the evening because it's weekday the trains stop running at some time And it's very nice to have this like sort of countdown when you have to leave to catch the last train Nice catch of time. I just realized Not bad and Also really fun project was a multiplayer tank game where you would use your phone or your laptop You would connect to the IP where you would get a controller in your browser and On the phone you could for example also use your touch movements to to steer your tank and then shoot each other And this multiplayer game and the display was the the map and then last up There was a nice demo application for the iPhone where you would just stream the camera to the display and All right, so Credits thanks to everybody who helped Helped work on this project everybody who did work on the project. It wasn't just me and then Felix but a whole bunch of people at at CCCB and I Always had a lot of fun working together on this this thing and I I sort of missed that a little bit. It's been a while since since somebody did something with it, right? And Yeah Let's hope that that changes I gotta say I don't miss it because it's always so Dangerous when they are handling it with 150 kilos in the club. It's really really weird So thanks to you too for now. Do we have questions? Siege and Beis is licht Then bit in my internet You mentioned that the modules can be dimmed. That's where you did your first animation with Does this ring algorithm also use the dimming of a module or is it just let on or off? The the dithering as far as I know and I the dithering doesn't use the the intensity light intensity because that's only possible to be controlled in eight by eight and I mean you you could I think it was actually attempted, but it didn't really look great because the pixels are Well much smaller than the area that you can control the intensity up and it didn't look so nice So you would also have to send several packages like one for the dimming one for the pixels right two different commands I mean we could have added a command that that's both to do to do both, but it didn't didn't look so great Yeah Okay, we have another question here in the front, please Thanks for a talk. Maybe you have other ideas of upgrading other old nice devices. Can you share? Other ideas of upgrading old devices. Well, what do you have a device? I? Mean the screen is so you upgraded it is now much better than the original one Yes, and maybe you have ideas for more projects If we have some future projects, you mean right like this. Yeah, okay at the moment. No, unfortunately, I mean with there's So there's been some work on on other display technology some flip-dots projects we've done but Not so much in the club not as a club project more more than the same people who worked on this also did some some flip Hacking but yeah, not at the moment if you have suggestions, please just send a mail or yeah Buy something send it our way But but don't send us your waste right don't send us old waste and we're not going to do this or do we now we don't We have one more question in the back and that's the last question, please Or no more question. No question. You're just standing there perfect Can happen, okay, then thank you so much and give a bigger applause to our speakers