 any of you do as well. This session will go over Richard's journey to hack the Quincy drawing bot. He wanted to hack this robot to draw Pokemons for his son. And that's sort of the root of this. And he has promised that at the end of this session, you can win one of these Quincy robots. So we'll have a section. Well, we might have a truncated Q&A to make sure we can have that session so you can win one of these. So thank you very much for coming, and enjoy. Richard, thank you very much. Yes, I will be raffling off, too. And you need your phone for that. So I hope everyone has a phone. So you should be good for that. But yeah, good afternoon. Thanks for visiting this talk. My name is Richard, aka D. Enneke. That's my nickname. And I guess I am one of those, like many of here, self-proclaimed amateur hackers, coders, makers. I do many things. I actually make lots of projects on the background. Actually, if you could see what it is, it's one of the projects I made many years ago. It was a 3D scanner based on a lot of Raspberry Pis, 100 Raspberry Pis joined up in one network, because I wanted to have a 3D scan of my son. So my son often is the driving factor in my life. Having children, I think, is really very insightful on the things you want to do in life. And I wanted to have a scan of him. Unfortunately, my son literally cannot stand still, not even one second. So all the regular scanning methods like a connect scanner and those kinds of things weren't possible. So when the Raspberry Pi camera came out, literally at that time, I thought, well, let's buy some of them. I started with 20 and got too many more. And I can now make 3D scans of my son. This project and many of my other projects are actually officially released. They're available for you to see and watch. I have a page on Instructables.com, where you find most of my hardware projects that I make. And anything that I typically do with code, I also tend to directly put it on GitHub and share. And I'll come back to that, actually, why I'm doing that, because I actually don't do that just so I give other people benefit. But actually, I get a lot of benefit out of it myself. And that's why I'll come to that in the end. I think it's really important when you share things to do. So but this particular project came about, that I came about this lovely little Quincy drawing robot. At that time, this thing was that cheap. I bought it for 29 euros. And I don't know how you can make a 3-axis or 2 and 1 half semi-axis robot with an ARM processor in it, stepper motors in it, a camera in it, a battery in it for 29 euros. But I guess you can. So I actually bought a few to start thinking around with it and those kinds of things. Now, I did put an asterisk with it, because, of course, nowadays, due to the shortage of electronics, the prices have gone up. So if you start googling around on an Amazon, or you can still buy them, they're still on Aliexpress. That's where I bought a few of them as well. But the price, unfortunately, has gone up a little bit. And what can this drawing robot do? Is it can actually not just draw. It can actually teach people, children, or whoever, how to draw. So it comes with all these cards of things you want to learn, a dog, a cat, a fox, a squirrel, or whatever. And it will take step by step, show you how to draw. Then you take each step and you try to copy and mimic that. And it works in a simple concept. You show it a card. There's a camera inside the robot. Fearful was maybe NFC or something. But no, it's a simple camera. And actually, on the back of the card is a QR code that it scans. And therefore, it knows what instructions should follow. So I did this, of course, with my kids. We presented it with a card. And as you see, the robot draws it actually in front of so facing the right direction for the kids. So every step it does. And in this case, we show it a, I should make you guess actually what it is. But anyone want to see what this is? It's a squirrel. Or when my son drew it, I'm not sure if it's really a squirrel or something like that. But so we did the squirrel thing and we did the dog and the fox thing, I think. But then, of course, my son came. Daddy, great, but I'm not really a squirrel fan. I am a Pikachu fan. Can you make it, please? Draw Pikachu. Okay, we can at least investigate this opportunity, so I did open up the box, found out what's inside of it. This is the motherboard of the robot. It actually has an ARM processor. Very well documented ARM processor, by the way. You'll find this processor on many small embedded chips and devices that are out there. So documentation is rightly available. And actually one thing that you also see actually very nicely is there's some pin headers here on the left side. And I'm pretty sure I didn't do the trade ship because I actually didn't go down that route. But I'm pretty sure that's the interface to directly overwrite the firmware and write your own firmware and that kind of stuff. So actually, I'm quite surprised how easily accessible they made this thing. But I didn't want to go down the hardware route. I wanted to just see if I can actually leverage the existing software that was inside of it to see if we could make it do other things. And I said, how does the thing work? It takes a QR code and it does something. So I just started to reverse engineer from how does that work? So if you take the QR code and you scan that, you see that there's some code pops up. In this case, this is a card of a dog. And it always starts with exclamation mark A. I assume the A stands for action or something like that, no clue. Then a dog, zero, zero, zero. Now in the back of the unit is actually an SD card slot reader. So there's a micro SD card in there. And actually if you put that SD card in your computer, it's just normal in English of less file system because a lot of things in China not always makes it a lot harder troubleshooting. But it was actually everything was super nice in English. And as you can see, there's a direct relationship between what you scan and it goes to a certain directory. Now if you then go inside that directory, you'll find some files. And some of the things very clearly made sense already. .snd, well, I guess that's the sound file, the things it talks. And if you did analyze it, that's just a raw web file. So that's actually easy to use and create and lots of tools available for that. And then there's this magic secretive PMD file and an IAS file. And the PMD files, there are many of them. And there was always per directory, there is this one IAS file. And this one I was able to open up. And again, looked very simple, realistic, didn't need actually documentation. I think I could figure this out because I knew when I showed a card to that thing what it does. It starts talking and starts giving instructions and then start doing certain drawing things that I have to redraw. At the end, actually, like, if this is a dog, at the end, actually, you have to write down dog and you have to put the show the cards, D, O, G to the machine. And that's actually the last three steps you see here. So that's a difference between a P and a C. With C, I guess capture or something like that, you actually have to show it the D. So it knows that you showed the D and says, oh, great, you did the D thing. So this was easy to think. I can write this, not too complicated. But yeah, how to do I draw? That's clearly in that PMD file thing. Now, of course, we can look inside that PMD file. Unfortunately, that's where the journey of easiness stops. Yeah, this was just abracadabra, whatever stuff. It definitely wasn't JSON or human readable or whatever stuff. Now, first thing I do, and many people should learn that, if you don't know something, you just Google it instead of asking someone else. So I, of course, go to Google and say, hey, what's a PMD file format? And I actually think it's a, so Google will tell you that it is some page maker, whatever, out of 1980s or something like that. But unfortunately, Google is, in this case, wrong. Because, well, that did exist and had the same name. It's not the same file format. So here I really have to start figuring out, so how do you take a file of data and try to figure out what it entails and that kind of stuff. Now, one thing I was sure of, it contains data on how to draw something. I shoot numbers, something with numbers or something like that. So what I did is I wrote a very simple program and that would just take every single byte or a few bytes, combinational bytes, to form a number and show me what picture comes out of that. You think, yeah, well, simple. So my first attempt resulted to this. Yeah, they didn't give me the dog that I was looking for or something like that, so. And even though I've been programming for quite a while, and normally when you program and you work with integers or floats and the different types of number systems, you never hardcore actually know on how it actually is being stored in a file. At least I never did. So I had to do some research on that and I found out that there are actually many ways on how you can write a number into a file. So of course you have two byte numbers, you have four bytes, how many bytes you use to create a number. You have something called little Indians and big Indians, depending on if it writes from the right to the left or the left to the right of numbers. So, well, instead of me trying to guess all this out, I just started randomly guessing and throwing that to my program and see what came out of that. And at some point, this came out and still doesn't look like a dog. But I guess there was something with these numbers that can't be artificial talent or something like that. I was, I think on a relatively good way. So I did a lot of more troubleshooting and actually I use a hex editor to figure it out because the main thing that I was struggling with is that so the numbers are four bytes, but my file size divided by four was not a whole number. I'm like, how's that possible? And in the end, I figured out well, each file starts always with the same byte. So there is actually the first byte of each file is just an identification that, hey, I am this PMD file, whatever thing. Did more and more troubleshooting and then I found out actually the second byte tells me how many segments are in the file of the steps of what it has to draw. And then per file it had another byte that says how many coordinates or numbers, at least there weren't. So then I figured out, wasn't sure yet how to draw it and all that kind of stuff, but what I did do is already directly document this on GitHub. Now, I'm pretty sure this thing is not very popular in the world. I don't know how many people have one of these things. So I didn't thought that this would reach a million people audience or anything like that, but for me it's easy just to document also for myself always a great place to find my information back if I stop a project and later on move on. And so I wrote down what I found, wrote down how I think the file format is on, and I went on with my journey. I tried to start figuring out how can I take these numbers and how does that machine turn that into an actual drawing action. And I was getting closer and closer, but I was really struggling because what you see here is so in the set of cards, there is a card for a circle and a card for a square. So I took some of these code snippets and I just put them into one file. So it would draw two squares and a circle for me to like calibrate. So I can also figure out what the numbers will be like, at least what I should be expecting and that kind of stuff. But every time my software was drawing the data from that, I got something that looked like it, but wasn't the same. This really baffled me, this was really things. But that's the nice thing about sharing without me asking, suddenly I get this email from some random guy in Canada, saying, hey, I saw your GitHub page. I was thinking about one of these robots. I like to see if I can help and that kind of stuff. And he was already writing code as well. He didn't even have the robot yet and he was already writing code for it, which was amazing. But he did order it, but it took a long time for him to get from AliExpress. And he was also getting the same strange behavior. He was doing random coordinates things. And instead of getting a square, he was getting this kind of, I don't know, fish shape thing or something like that. And then I started thinking, or we started actually thinking, it's like, I've always been thinking that coordinates, X, Y, that must be in the file. But if you see how the robot works, the robot has two arms. And it literally is like me as a human being, the arms are not on the same beginning point. There's one on the left and one on the right. And what this entails in is this concept here. So you have the arms, the shoulder joints at the top. There's two and a half centimeters in between. There's an 11 centimeters for the upper arm, 10 centimeters for the under arm, and that's where it comes together. So I was thinking maybe these numbers were not X, Y, but there were angles of these two shoulders. Now that sounded very possible, but then the real challenge started. So I don't know who's here good at math, but that's mathematically quite difficult. At least my math skills definitely stopped there. How do I calculate if I knew what those two angles were? Having two and a half centimeters apart from each other, they're joined together. If I move this angle here, where what is in the end the X, Y coordinates of that endpoint over there? Now I teach Coder Dojo programming lessons every Saturday and one of my fellow Coder Dojo colleagues is actually a math rock star. So he drilled these up for me on a piece of paper. And I'm like, dude, well, thank you very much for your effort, but how do I translate that into code? And he also, so he somewhat thinks that he figured it out, but he was really also struggling, but the original point is not zero comma zero. There's one on the left and one is on the right, and how do you put that in math and how to work that out and that kind of stuff. So I was working on this by trying to learn myself math. Now that's not an easy thing on my age. So I really started to give up. I'm like, this is not gonna work. I need some serious help here. And then I actually did something that I do for lots of other more typically graphical things. Let's see if I can actually hire cheap help to sponsor me with this, to figure this out. So I actually went to Fiverr. I don't know if you know Fiverr, the name stands for five. Normally you pay $5 for someone to do something for you. It's in reality not always $5. But I just posted the thing there. Hey, I have this, I think, super difficult math problem with a drawing on it, literally that drawing. These are the stuff who can help me write the math so I can actually understand angles towards coordinates because that's really what I was after for it. Some really nice guy, I think from Australia, I said, yeah, for 12 euros, I'll do that for you. Well, I said, I don't have to do my own. I don't know if this is cheating with hacking, but if someone is willing to give me this math stuff for 12 euros, go for it. And the guy actually spent two, three days on it. We had a long conversation. Every time he came back with his math thing, I was trying it out and the picture didn't really, but we sorted it out in a couple of days and we really got some really, it was very clean code on how to actually get from angles to x, y location and also the other way around from x, y to angles because that's in the end actually what I wanted. If I want to make a drawing, I can take any j-pack and then I want to say, well, on x, y, I want the thing to go down, but what angles does that represent? Now unfortunately, that way back, reverse, is not one-way street. If you imagine, I can move my arms in multiple ways while the point still stays at the same thing. So I just brute-forged that. I just figure out on all those what the angle should be, had a map for that and figure that out. So I actually wrote a piece of software then to take an image. I could draw a line over the image, in my case Pikachu of course, and started writing the steps down. But again, my newborn friend from Canada was working with me on this project as well. I wrote my program in a .NET Windows program. He did a much better job. He wrote something web-based. And we have now a Quincy online editor where you can go online and you can upload an image and you can make this, you have to trace it. And because you can't auto-trace, well technically of course you could, but what you want is you want it to say, okay, this is step one, if you want it to also be an educational tool on how to draw and that kind of stuff. But for me, that wasn't a quest. For me, it was just about getting my kid to be happy. So, start of May, this is in November. I showed him the robot. We tossed it a different card. And magically, as what he asked, took a few months, lots of complicated math. But it did drew Pikachu. What he really wanted. And of course kids, they're like, well, fantastic. He was very happy. I did it in the end for his smile. See, that's what, as a dad, what you do it for. The kid is happy. He doesn't appreciate really how difficult it was or whatsoever. He moved on to the next project. He didn't care anymore whatsoever because it was six months later or something like that. But for me, it was a fun journey. I met new people. I did learn a little bit about math a little bit more. And it was really, really, really nice. And that's why we really want to share this with you, right? Without you having to hack this robot. If you do things and that kind of stuff, one thing I really recommend is always when you do projects, share. On all my projects, I've always gotten so much nice feedback, but also interaction on other people's ideas. And it really evolves the project and that kind of stuff. So use an Instructable.com. Use a GitHub. Use a Thingiverse, whatever stuff. You find lots of things that I designed on Thingiverse as well. And I always get a lot of feedback from that as well. So it's very valuable to share things. And yeah, when things do get hard, don't cheat. Don't bill out. Just go cheat and get help. And you have to pay 12 euros for it. Go pay 12 euros for it, I guess. It helped me move on my project. So that was my experience about this thing. And of course, I wanted to give you guys a chance to win one of these things or two. I actually have two. So we're going to do if the screen works. I guess we'll have to cancel out of this. There we go. We're going to do a little Kahoot. Three questions. That's going to be short. So I don't know if you all know Kahoot. On your phone, you go to kahoot.it if you happen to have the app. You don't have to tell your real name. You can be any name. As long as you later can show me that you're that person if you win. Type in the code. We do three questions. And the top two will automatically get, well, you can come pick up your Quincy drawing robot. Let's see. I hope you all have good connectivity. Everyone in? 21? Almost? Good to go? No, almost? Hints? The questions are, of course, about robots. So I don't know how much you know about robots. Good to go? No? Slow phone? Yes, good to go? So it is about three questions. You get points for scoring the right answer, but you get even more points for scoring faster the right answer. So don't want to rush you because the wrong answer doesn't give you points, but there we go. So first question. You answer it on your phone. Where was the word robot first used? Was that in an ancient manuscript, in a play, in a science research paper, or in a news article? Where was the word robot first used? I think it was somewhere in the 1928 timeframe, something like that. It was, yeah, it's not really, it was Slavic or something. It's not a country. It's a dialect, but in Russia area. And it was in a play, in a play about human slavery to mechanics or something like that. Okay, so you still have chance, don't worry, two other questions coming. Number two, who wrote the famous robot law, right? One robot should obey, should not hurt human beings and should try to keep itself alive. Is that Isaac, Esimov? Is that Isaac Newton? He or she a Ishii guru or tombaker? Good guess, everyone. All the other ones do have something to do with robots, by the way. I'll let you figure that out. There you go, see if there's, well, still many people in the top, so final, final, final question. There we go. What is the name of the robot in the movie Short Circuit? Is that the T100, the T200, Johnny number three or Johnny number five? Oh, many people like Terminator, I guess. The T200 is a Terminator, they kill people. It's not a drawing robot or something, it's a funny robot. It is, of course, Johnny number five. So, we'll see number three, but unfortunately, number three, I only have two, so sorry, number three. Number two and one who will get the robot. So, number two, Cresim, whoever Cresim is, there you go. And number one is Logan W, who's that? Who's Logan W? In the back, there you go. So, one for you, there you go, congrats. And in the back, another robot expert. Thank you very much, thank you very much. And thank you so much for that. Now, because we had the giveaway, there's not much time for questions, we could maybe do one. First, I wanna check the signal, who just won a prize. Is there a question from, okay. So, there's room for one question from this microphone here, if anybody wants to ask. Or you can talk to Richard right outside the tent as soon as we grab here. Absolutely, I'm doing a robot workshop tomorrow. If you wanna learn to program a robotic arm, you can come to my workshop. Yeah, so from building with popsicles to tomorrow, working with robotic arms, there's a lot of, you can get a lot of skills from him.