 So I want to thank you for coming up and I realize it's tricky to get the elevator and get all the way up here to the 23rd floor, but I do appreciate the fact that you spent the time and effort to try and get here my name is Mike Anderson. I'm Chief Scientist for the PTR group and we are a company that has been in the of course we've been using Linux now for quite some time actually just after we were founded we started using Linux and we had traditionally been a BX works real-time operating system kind of shop so when the embedded when they first mentioned the concept of embedded Linux we all laughed. It's like, ah, there's no way you'll never be able to get it to fit because we were used to using boards that had 128k bytes worth of RAM I mean so when you're trying to deal with boards with very small memory footprints like that Linux just wasn't even in the cards at that point Fortunately a lot of that has changed over the years and of course with the introduction of the real-time patch that's changed quite a bit as well, but in our particular case we've been working a lot a lot of the business that we do is with NASA We have several projects right now that are all space-based robots Many cases we're actually using lidars for doing ranging on the robots There are some nasty little treaties about causing space debris So the fact that we're coming up underneath a low-earth orbit satellite and grappling it to try and refuel it on orbit If you make a mistake at those kinds of speeds, I mean orbital velocity is what six kilometers a second or something like that So if you make a mistake at those kinds of speeds, it just creates this huge debris cloud And it becomes that movie gravity that everybody saw right we know what happens with that so we absolutely want to worry about things like that and We've done a lot of embedded ports to Both our toss linux and bare metal pieces Of course tomorrow I have a presentation on iot messaging Well, we'll get into some of the messaging techniques that are used in the iot and how that's applied So lots of things that we're involved in we've got a little over 35 satellites on orbit right now That we're maintaining some of them actually date all the way back to the 68 302 So that chip hasn't even been in production for the past 20 years So and they're still flying so remarkably enough things can still last Yeah Yes, they've already been uploaded so you can download them off of the the site Myself, I've been in the industry now about 40 years And I got my first job as a programmer on an altair 8800 So with with floppy disks And it had front panel switches and we used to load a program in the front panel switches in order to get it to read the floppy disk so it was basically a jump instruction that would jump off to the rom and Of course in in those days Uh, we used to use a radio an am radio We would set it on top of the cpu Because it would make noises as the cpu would do things the cpu was so slow It was a 2 megahertz processor It was so slow that if you didn't have a radio to tell you that it was still working you would think it had died Fortunately with the front panel switch models you could actually watch the blinky lights But uh, then when they got a turnkey model there were no blinky lights. So it's like, how can you tell if it works? We had a floppy disk That was infinite storage because you could always pull a floppy disk out and stick another one in Infinite storage 253 k bytes at a time All right, enough enough down the history there So what we're here to talk about we're here to talk about light ours and their use in navigation In particular for robots That will be inside the house, although not necessarily limited to just that We'll talk a little bit about some of the drone mounted light ours and things of that sort as well So we'll talk about light ours. We'll talk about what options exist for light ours and then we'll Get into why would we put this thing on a robot? Why would we go to the trouble of doing this? And uh, we'll talk about how successful it was or not as the case may be and uh, we'll get into a summary So let's talk a little bit about light ours What exactly is a light are? well, it is uh, kind of a A munging of what we used to call a radar, of course radio detecting a detection and ranging Light are is light detection and ranging So, uh, unfortunately, there are lots of different ways that this is put out there. It's either light are lowercase i light are All uppercase sometimes they call them laid ours It just depends on who happens to be selling it as to exactly what it is. They call it For purposes of this presentation, we'll just call it light are with a all lowercase Uh, it's effectively nothing more than a light ranging mechanism of some sort now light ours themselves We'll use time of flight measurement. We'll see exactly what that looks like here in a moment We've got a picture of that coming up And we're trying to determine. Okay. I send the laser the laser pulse out. How long does it take to reflect back? Simple concept really nothing more than that But the fact that you can do that at extremely high frequencies that is very high data rates Means that you can get very fine granularity on the kinds of distance measurements that you're pulling back The big problem is that when we deal with light ours In general if we're dealing with a commercial light are that is for the Not for the scientific market. Let's put it that way They're usually in the 600 to 1000 nanometer range in terms of their light frequency Unfortunately, this puts it right in the eye absorption range So never look into a light are with your good eye No, actually in order for us to really be successful with light ours If they're going to be in this general range between 600 nanometer and 1000 nanometer We need to use class 1 lasers and class 1 lasers are eye safe. You can look into them. No problem with that. So They you do end up having to go with class 1 lasers, which unfortunately limits the power It also limits the distance that you can get out of them. So they do have some limitations associated with that When we start looking at some of the commercial light our units the commercial light our units typically run in the 1550 nanometer range And that is outside of the eye absorption range, which means they can run much higher power Which means they get greater distance and that's one of the advantages But the problem is as soon as you pop up to the 1550 nanometer range You now the cost kind of goes exponential on you and we'll see some examples of those costs. Yeah Yeah, uh, so we'll I'll show you some pictures that here in a moment So because the light are is basically a time of flight calculation Between when the laser pulse goes out and when it gets detected We're basically able to then measure time because we know the speed of light and all that sort of good stuff So we can actually determine how far away an object is But the the interesting thing about the way lidars typically work is there's almost always two Eyes on the lidar you have an emitter and you have a detector and they're offset slightly And the reason they're offset slightly is for triangulation purposes So we know the distance We know the angle between the two eyes and therefore we can calculate using trigonometric functions Exactly how far away the object is So that's the basic idea behind how a lidar works So it's not really all that complicated and it's very similar to some of the other modalities that we see for instance ultrasonic Infrared all basically work in the same principle So where do you actually find lidars being used commercially? Well, the first picture that you see there is what's called a flash lidar And flash lidars are basically cameras And what they do is they take a picture and each pixel is actually either 24 or 44 different laser pulses That then create the pixel and what they'll do is they'll do use faults coloring on there on the measurement for the actual distance To get the colors out there That is uh, basically that's kind of the the staple for the use of lidars 3d maps from airborne platforms. There are of course orbital satellites that use lidars for doing mapping purposes There are also of course now both aircraft and drones drones are becoming a big market now for the lidar space But the other place that we see lidars used quite a bit nowadays is in self-driving cars And if you've ever seen for instance, my daughter goes to Carnegie Mellon And uber has self-driving cars in Carnegie Mellon in the Pittsburgh area right now So you see them go by a lot and they have this little spinning dome thing that sits on top of them And that's the lidar that's trying to figure out exactly where the vehicle is in relationship to everything else So the lidars are actually becoming more and more popular certainly coming into much greater use Especially when we start dealing with actual distance and trying to figure out where we are in 3d space So there are some single point lidars that are out there Of course, if you've ever had one of these fat maxes or one of the other measuring tapes that are out there They have laser based measuring tapes Uh that basically do the exact same thing and again, you'll notice in this picture There are two eyes. There's an emitter on one side and a detector on the other And of course it doesn't project the marked laser beam out there into space That's thanks to walmart They they decided they were going to put that measured laser beam out there because people think that yeah It's going to show them exactly how far away it is whatever And those types of single point lidars Unfortunately, they're not really made to be interfaced to they're made as a sealed unit So you can buy them they they sell for less than a hundred dollars You can buy them and tear them apart and if you're good enough with your reverse engineering skills You can actually figure out a way of being able to hack into it and use it as a as a laser source Unfortunately, that is way too much trouble for what it's worth So fortunately, there are other alternatives that are available to us now You see the the one that is this guy right here This is from garmin There is actually a company called pulse light that had sold that particular type of lidar And they were selling it for an incredibly low price of only 150 a pop And it was so popular that garmin decided they were going to buy it They bought the company which unfortunately put them out of business for several months while they went to a new production facility And unfortunately that was right in the middle of the of the robot season So we couldn't buy the units really a frustrating and In this particular case this Unit can use either i2c it can use PWM is the other option that it has available for it But it runs off of 5 volt and so it's fairly easy to wire it up to an arduino or something similar like that and start using it to do measurements It's reasonably accurate. It has a range of about 30 meters So for small areas rooms and things of that sort. It's not so bad And at $150 it's fairly cheap all things considered Now when we want to go beyond that if we really want to start getting into All sorts of interesting applications. We start getting into scanning lidars Now the one that we had back up here This one you can actually mount that on a spinning turntable I actually tried that and of course wire management is an issue because now you've got a wire that is now spinning at some rate And balance is another issue because it was never made to be Never made to be spun like that and so after trying three or four different attempts and Using servos and stepper motors and a few other things. It's like uh, no, this is too much trouble Okay, that's an interesting concept, but let's move on So uh scanning lidars these typically move in a 360 degree range They will also like you see here. They have their two eyes. They're emitter and their detector And these particular lidars can be very expensive unfortunately This one actually here This one is not available yet. It's a model. It'll come become available sometime they say In late march early april time frame It was actually a Kickstarter campaign and if you take it apart it actually has one of the Garmin units inside of it Interestingly enough now this one is cheap for lidars. It's 350 dollars Um if you can actually get it so I'm waiting on mine to come from spark fund. We'll see how that works out This one up here. Uh, this one is $1,300 for that lidar And so lidars get very very expensive very quickly Especially as I said if you're in the 1550 nanometer range But all of these are in the smack in the middle of the infrared laser range So they're uh, unfortunately don't look into it It would be a bad thing They uh, most of these are controlled either through i-squared c pwm spy or in many cases serial So if you uh, oftentimes when we're trying to actually spend the data You know spend the lidar and get the data off of it. Uh, it requires quite a bit of transfer So it puts a fairly significant load on whatever thing you happen to have attached to the lidar In terms of being able just to absorb all the data that it produces So, uh, there are a couple of other things that we see of course commercial drone lidars There are a number of manufacturers of these that are out there these days It's uh, the the way the lidar is put together. These are of course the 1550 nanometer lidars typically and uh, they're Relatively heavy and relatively power hungry. So time aloft with these types of drones is very short Usually on the order of an hour or so before it runs out And that is if you can afford the hundred and twenty thousand dollar price tag for that little drone Um, it's not a big drone. This is a little drone 120 thousand dollars. So, uh, actually it's in euros. It's a hundred and 18,000 something or rather euros. So, uh, oh, but there's 10,000 euros in tax So it's not quite that expensive All right, uh, so that's kind of a cool object but again kind of out of the rage of the typical hobbyist There are uh, there is one source for a cheap scanning lidar And it is the nido robot, uh, cleaning robot So it's like the Roomba except in this case it happens to have a lidar mounted inside of it and uh, if you take the the robot apart Then you'll find the lidar sitting right in here and that's what it looks like when it's extracted You can actually get just the lidar unit refurbished lidar units off of ebay for about 120 dollars so and then of course you have to figure out how to Interface to the motor and how to interface to the lidar and how to get the data off of it Fortunately, there are some tear downs that are available on the internet Of course spark fun has one that goes into the details of it There are a couple of other sources out there that if you really wanted to try To scavenge one of these nido lidars The downside of that particular lidar is it has a range of only five meters So it's not very far. It doesn't have much of a way of distance. It's fine for a cleaning for a vacuum It's fine for a vacuum But for some of the applications that I had for it not quite so good So uh, and I again, I just kind of looked at it and go I'm going to have to figure out how to how to get into those jst connectors and all the rest of the stuff that goes along with it Yeah, let's see if we can move on and get something else So why are we really interested in a lidar? Well, we want to be able to do very fine ranging inside of a home for instance or inside of a building And in order for us to be able to do that ranging We actually need to combine it with another piece of information And that is we will combine it with an inertial measurement unit an imu And the imu has got like three degrees of freedom for the gyro the compass and you know for a magnetometer and for an accelerometer So using those nine degrees of freedom nine dock board We can then combine that information using sensor fusion With the actual distances that are being measured by the lidar and construct a map of what the room looks like So that's basically what the needle robot does It runs around and uses the lidar to figure out what the room looks like and then it figures out how to sweep the room Uh, if you happen to have a rumba, especially one of the earlier rumbas You notice that it just moves at random And it gets it basically it moves in such a way that it hopes that eventually if it runs If it doesn't run out of battery, it will have swept the entire floor There's no particular rhyme or reason for the way that it actually operates, which is infuriating But it's interesting to watch the cats chase it So it does have some interesting advantages there The actual measurement and the implementation of what they call slam simultaneous localization and mapping You will then describe it in this wonderful little short math function here But basically all it's doing is it's taking the summary of all these distances that it's measuring And then it factors in the navigation from the inertial management an inertial measurement unit To figure out exactly what sorts of things it's encountered And eventually build a map of the facility that it happens to be located in And that's one of the things that uh, where I was trying to get to for this particular application Uh, for what it's worth google has just released the slam algorithms that they use in their self-driving cars I've taken a look at it and definitely it's not optimized for hobby type lasers, of course And uh, it's Unfortunately, uh quite a bit computationally expensive because of course they're trying to solve a slightly different problem Keep cars from running into each other At very high speeds So most of the robots that I work with don't move any more than about 35 mile an hour And um, you know as a consequence, uh, that's a little bit overkill for my particular application But once we tie it in with an inertial management a measurement system an inertial measurement unit Then we can do the data fusion and be uh use that to create an internal map Of everything and because we have the inertial measurement unit We know exactly where the robot is at any one point in time The robot doesn't know where it is when it first starts It just simply starts scanning and then as it moves we have the uh accelerometers We have the gyroscope so we can then figure out where the robot is pointing at any one point in time And then use that to help construct this map that we're trying to build So eventually what happens if we happen to get enough laser points We can do one of these things they call a cloud a point cloud And what point clouds allow us to do is to have a 3d representation of whatever it is that we've just scanned Now in order to do a point cloud it really isn't that complicated If you happen to have a pan tilt mechanism say with servos or with stepper motors then you put that Garmin LiDAR on there and you just simply start scanning And you go scan step scan step scan step can't scan step and then after about five minutes or so You have enough data to then be able to draw one of these point clouds And that's kind of cool Unfortunately, you can't really do that in real time So it is a bit of a nuisance from that perspective But it is still kind of cool when you go to your your students and you go, okay Well, let's build this thing And describe what it's supposed to look like and then they graph it and they actually can see somebody's face In the point cloud, which is kind of definitely very cool These can typically in this particular case it was imported into blender And they use blender to actually do the graphing, but you can use processing You can use python matlab so many different solutions for this Excuse me Yeah, yes, yep So lots of different options that are available here Now that's kind of the background theory on LiDARs now. Let's talk a little bit about What we do in order to create a robot to mount the LiDAR on now Creating a robot is not something that you'll just wake up one morning and say I'm going to build a robot today unless you've done it a lot And fortunately over the last eight years. I've had a great opportunity of working with the first robotics competition And that in the first tech challenge and so we've had a lot of opportunity to build robots Now most of the robots that I build are 120 pound 35 mile an hour kind of robots So they're much bigger than what I really wanted to build here I wanted to build something that would be able to navigate around in a home Without running into things and without crushing the cat If they happen to get too close So when we're creating a robot, we really have to understand several different aspects. First of all, why Are you creating the robot? What is it supposed to do? Um, you know in my particular case, I have a very specific sort of mission for it So I understood exactly what I wanted it to do in some cases You just build one because it's cool and it's fun to do And it's very educational if you happen to have kids who are also willing to sit down with you and And actually go through the process, but we need to worry about things like how you're going to move it What's its mobility? Is it wheels? Is it tracks? Is it a hovercraft? Is it a drone that flies? How do you move this thing? We need to understand the motor controllers So if our motors are let's say, uh, you know, five amp motors Then I have to have a motor controller that can handle that kind of sustained current I need to worry about not only the power draw is it a nine volt a 20 volt a 24 volt whatever kind of motor it may be But I also have to worry about the steady state current If I'm running at full speed, what does it pull? But then I will also have cases where I'm going full speed forward and I suddenly want to go full speed backwards So now the amount of current draw at that is extremely large And so it's not at all uncommon in the case of the first robotics Type of robots to see 80 amps being pulled off of the battery at any one point in time So and it's really interesting for students when you when you explain to them That that car battery that you're using on the robot will light them up If they manage to get across both the power and the ground side of that battery at any one point in time so We have to think about What kind of motors we're going to be using are they brushed dc motors? Those are the simplest motors But they also don't have a lot of precision associated with them unless we're going to then put an encoder on them of some sort Brushless motors are usually three phase motors. They're easier to control But you then have to have an electronic speed control in order to manage them So there's a completely different sort of electrical harness that you have to have for brushless motors How are you going to power them? I mean the battery chemistry becomes a factor We can certainly use battery chemistry is like nickel metal hydride. We can use lipos lifey batteries Now lipo batteries have a great characteristic They're basically flat until you get to the end of life of the battery and then the battery drops like a rock Unfortunately on big robots if you pull too much current out of a lipo battery too fast it catches fire Uh, one of the reasons why the tsa is not terribly fond of lipo batteries as they get transferred around um Because that fire problem is kind of an issue, especially if you accidentally short the battery Which I have seen students do on occasion. They uh, they they didn't realize that the power and ground was actually connected to each other When they plug the battery in and it does a halt and catch fire. It's amazing um The of course which controller you're going to use now when we're using these motor controllers the motor controller has to be controlled by something So what is the something we're going to use? It has to be very reliable in the sense that the duration of the pulse Most of these are going to use pulse width modulation The duration of the pulse we can't see much jitter out of it if we do we've got a problem Now, uh, we'll see examples here. I've got an example of how linux sees these things Of course, we also have other issues associated with the connection. How are we going to control it? Are we going to use wireless bluetooth, wi-fi, zig b? How far does it have to go? Does it have to go through walls? All those sorts of things have to be taken into account So as you're going through designing a robot you have a lot of Pre-work that you have to think through in order to be able to make sure that you're going to have all the answers when you get there Now linux and motor controllers A lot of the inexpensive motor controllers are going to be using rc hobby servo type signals. These are typically 50 hertz Pulse width. I mean 50 hertz refresh rates and pulse widths in the anywhere from 0.5 to 2.5 millisecond kind of pulses and these pulse widths Uh, if you're dealing with just a standard hobby servo, you know so a servo is typically only going to have 180 degrees worth of motion out of it And servos are basically encoder based motors That will be able to repeat their location. If I say go to 170 degrees It'll always be able to go to 170 degrees assuming that nothing is blocking it for some reason So the servos themselves are pretty tolerant of pulse width variations So if you're using linux with the real-time patch Then you can probably use linux with a real-time patch on a hobby servo It'll probably be okay But As soon as you start getting to some of these bigger motors and bigger motor controllers Then any jitter in the motor controller in the pwm signal that's going to the motor controller I've actually seen motors destruct themselves because the jitter was so bad So in that case If you want to try and avoid getting yourself into any significant trouble Then you need to think about how am I going to generate a reliable pulse width modulated signal to keep this thing fed And keep the motor controller from winging out and then causing the motor to blow up on you So, uh, this typically means we're going to either use something like the beagle bone pru Or we're going to use an arduino or some other microcontroller Well, we can dedicate the controller to the actual generation of the signal so that we don't lose sync on the pwm So now, uh, with all of that, uh, what exactly did I want this thing to do? Well, in my particular case, there are so many new lidar models that are coming out I wanted to have a test bed that we could then set there and plug it in and Determine whether or not this particular lidar is going to be easy to work with hard to work with I needed to have something that my students and my robotic students would understand Just be able to look at not be intimidated by the robot and be able to then focus themselves on the actual lidar unit itself and not worry about how everything else somehow worked I also wanted to start playing around with some slam approaches. So the slam models Are really important as we start seeing for instance in the first robotics competition They keep introducing new obstacles on the field every year the game is different So you'd like to be able to create a map of what the field was so that you could then find out where your robot was At any one point in time in this particular year's game We actually have several blind spots on the field Because there's an airship in the middle of the field and we can't see past it So having the robot being able to determine exactly where it is at any one point in time either with the lidar and the and the slam algorithms You know really helps the students to be able to drive also With the the amount of data that's coming off of these we're also starting to see kind of Problems with the students being completely overwhelmed. There's too much data coming off the robot And it's that classic problem of I can't be the pilot and the weapons officer at the same time I have to have somebody else that helps me just to keep track of all the stuff that's out there We already have that in first. We have driver one and driver two But in this particular case Even with that they're starting to get overwhelmed with the amount of data that's coming off the robots So one of the other things Yesterday I had an opportunity to talk about using Alexa and creating an echo based device That would allow you to interact with it And I have several friends of mine whose parents are getting into the geriatric time And they don't want to be moved into a home. They want to be able to continue to live in their own home But if they are living alone You need somebody to be able to say hey, have you remembered to take your medicine today? Oh, by the way, did you eat lunch yet? Because unfortunately as they start getting a little bit older and aged they start forgetting to do things like that And then their health starts to deteriorate quite significantly So I was thinking all right, let's see if we can come up with a way of being able to navigate around through the house And have something that somebody could talk to And just having something that you talk to even if it's not all that complicated Like where is Uganda, you know, but have an answer and it turns out that people really feel good about that It makes them feel good that there's something there that's going to interact with them And do more than just simply bark at them or require to be taken out for a walk so It really was one of those things of hey This is a great opportunity to do a service bot That then also allows me to experiment with all these other slam algorithms and lidars and kind of bring it all together in one piece So Choosing the platform was an interesting problem. Uh, again, I remember I uh, I'm typically used to dealing with big robots 100 pound plus kind of robots powered with car batteries That's not going to work in my kitchen. So at least my wife won't let me use it in the kitchen. It does sort of fit, but She doesn't gets a little upset about it And so I uh looked at it and I go well, all right, and if I wanted to bring it to a conference Having a 120 pound robot and a sealed lead acid battery Trying to get that on the plane a little tough. It doesn't fit in the overhead TSA doesn't like it And then when they x-ray your luggage they end up tearing your luggage apart because they can't quite figure out what that thing is in luggage So I said, well, I wanted something small And I wanted something different. I built a lot of wheeled based robots So I thought now let's do something that's track-based. That sounds fun So, uh, I found this particular model here This is from a company called A-O-O-K-Y-E kooky That should have set off an alarm bell in my head When you buy a robot from a kooky company We'll find out later exactly how that worked out But the thing is that these particular motors and the the motors are located back here So that's the actual drive wheel and all the rest of these are passive And the motor itself is a nine-volt motor, but it has a four and a half amp stall current And at four and a half amp stall current It means that you're you've immediately ruled out the l298 h-bridge type motor controllers Because those things, uh, I mean they're great motor controllers Six-volt motors. Yeah, you can do that typically two amp max draws But in this particular case these guys draw way too much current and I was concerned that it would toast an h-bridge So I said well, all right. We got to do something else Now, uh The plans for the controls I've used both raspberry pi and beagle bone black in the past and The last robot that I built was a six-wheel drive all-terrain robot Using red using beagle bone black and it was very successful But it didn't have a scanning lidar on it and it didn't have all these other sensors that I was going to have to put into it So I was concerned I just I'm guessing you know here. I'm in the design phase I don't really know how much data is going to be produced by this lidar because I don't have it yet And I uh, it was on order, but it hadn't arrived yet And as a consequence, I'm like, you know Beagle bone black would be great or beagle bone black wireless would be great because now I've got the wi-fi interface on it But it's like I really don't know how much horsepower this is going to take So I happened to have some raspberry pi 3s sitting around the house and I decided well, okay We'll build it on raspberry pi 3 In order for me to avoid the pwm jitter issue Even though you can run the the the real-time patch on it I wanted to make sure that I didn't get any problems with the motor controller. So I Decided that what we would do is we would use a high power motor controller This one happens to be from sparkfun It is built as an arduino shield Now when you buy it, it's unsoldered So some assembly required and if you don't feel comfortable with through hole soldering This is probably not the solution for you But the advantage of this particular motor controller is that it's a dual channel motor controller max current 30 amps sustained six excuse me sustained 14 amps And it can have a max voltage of up to 16 volts So the way brushed dc motors work if I have something that can handle up to 16 volts But I only put nine volts through it or less than everything's good. I don't have to worry about burning anything up so That particular motor controller. I think was like 60 dollars Which is relatively cheap for a motor controller in that general range The saber tooth motor controllers that can handle 80 80 amps Are like 125 a pop per channel So that's a little bit on the expensive side first robotics, of course has some really cheap ones at 45 dollars That'll handle up to 80 amps, but they're big And I didn't want to have to figure out how i'm going to get that on that little bitty robot Additionally it turns out there is an arduino Hat that's sold for this particular for the raspberry pi And it implements an arduino leonardo. So it's a 32 u4 at male part And it has the arduino pentouts. That's the thing that you see the kind of the blue and red things Those are arduino pentouts So if you put the arduino connector on this motor controller It should then just plug directly into the hat that then plugs into the raspberry pi That seemed like a really reasonable solution Considering the amount of space that I had on the robot and I said well, okay I don't want to have have multiple boards sitting out there The weird thing about this hat unfortunately Is that you would think that since it's plugged into the raspberry pi The raspberry pi would be able to talk directly to the arduino through the raspberry pi pens But you would be wrong You have to run a usb cable From the raspberry pi to the arduino shield And that's not intuitively obvious in any of the description that they have for the shield Of course, it's coming out of uh, I think it's uh df robot out of china and um Yeah, they they didn't say that you had to hook up the leonardo via a usb interface And You just think of course it's going to be able to talk directly to the uart coming off of the pens on the raspberry pi Nope, it doesn't go figure. All right So how am I going to power this thing? Because I wanted to be able to transport it without running into any Problems with our good friends at the tsa. I said, well, you know, let's go with uh, nickel metal hydride Nickel metal hydride doesn't have doesn't have any requirements for special charging circuits Unfortunately, they tend to be relatively large But uh, in this particular case, it's uh, about the three quarters of a stick of dynamite. Oh, no, you probably shouldn't say that with the tsi But no, it's it's it's a relatively large package, but it's a five amp battery So it can definitely power the unit for quite some time Now also, I found this really nifty breakout at spark fund, which is a usb micro connector And you just simply hook power and ground to it And now you've got a connection that just plugs into anything that uses a usb micro and now you can power it externally But of course, uh, the thing that you have to remember is that uh, beagle bones and raspberry pies and things of that sort require five volt And my battery was a nine point six volt battery and you can't just simply plug nine point six volts into a Uh, a development board like that. It'll toast it. So we had to go through a power transformer Well in my particular case, it turns out there's a really cheap way to do this And it's called a universal battery eliminator circuit and these things sell for about 10 bucks And they're made for rc aircraft They're made to actually power the receiver on rc aircraft. They're a buck transformer And uh, the particular one that i'm using here can handle up to 10 amps So it'll definitely handle whatever it is the raspberry pie drains, which is about two amps And then plus the lidar which is going to pull quite a bit of more power off of it as well So, uh, that was the basic model that we were using as far as the controls are concerned Choosing the lidar I have Four or five of the garmin lidar units that I've used in in past games and things of that sort And they're great, but again that whole I wanted to have a scanning lidar So just to make things more difficult for myself. I go, well, okay, I'm not I'm not going to use that I'm going to use a scanning lidar of some sort It turns out that the cheapest Scanning lidar that I could find was about 400 dollars and that's this guy here the rpi lidar a two unit This is from a company called slam tech It's 4 000 samples a second At 10 hertz and then 600 rpm is how fast the little bulb spins around It's also relatively low profile. So it's only about Say two inches high. So it's not very high That of course will come back to haunt me a little bit later when I start mounting everything Wiring up it took about, uh, I guess two days two and a half days to wire it and to actually construct the robot was Building the robot took me a better part of a day. Actually, I use slave. I mean student labor To build a robot. It's like, hey, would you like to build a robot? And they go, oh, we got nothing else to do right now because we're getting ready to put the robot in a bag So sure So, um, I allowed the students to build the actual robot Which was a mistake as it turned out, but uh, nonetheless, it was it was a good exercise for them And then I only had to redo part of it I then of course Can talk to the arduino using rs-232. It turns out the rpi lidar also uses rs-232 So I just needed to make sure that I had enough high speed serial ports of which the Raspberry Pi has only one To be able to communicate with everything. So it was usb to serial port adapter time And that seems to work. Okay. Uh, also I used anderson power poles no relation. Unfortunately. I really wish it was I wouldn't be here. I'd be retired someplace But anderson power poles these things are used in the ham radio community quite a bit They're just a quick connect disconnect kind of power And so I could easily what I did was I cut the tamia connector off of the Battery and then put anderson power poles on it and then it just becomes a real quick connect disconnect So I don't have to worry about putting a switch in the circuit to shut the battery off Then so this is the finished robot I'm not sure if you can actually see all the the elements inside of this, of course We have the tracks We have the raspberry pi here We have the shield mounted on top. There's the You see the blue and red elements of the arduino Shield or the arduino hat and then the motor controller shield sits on top of that It turns out that this whole scaffolding Environment here did not exist. I had to fabricate that Yeah, it was a nuisance, but nonetheless I needed to get the lidar up above everything So I didn't have anything that was interfering with the scan And I also added one more thing to it on the front here. I added a usb camera Just so that I could see where the robot was in case I couldn't actually Physically see it. I could at least see where the robot thinks it is And then could take control of it control of course all being done using wi-fi Of course in this particular case I mounted a tux on the top of it, but he spends too fast and gets out of balance So I had to take it off Too bad Now software So it's taken me several days To build the robot and in terms of actual invested money here We're talking about $600 or so 700 actually probably cost us $700 And I don't it can't move yet I got to have some software on it So fortunately sparkfun had an example for the arduino Where it would at least be able to control the motor be able to turn the motor controllers on and off Of course, it wasn't set up to be controlled with rs 232. So I had to modify it in order to be able to do that And it also turned out that for the lidar slam tech the company that sells the that sells the lidar Actually has an sdk available for it that works for c and c plus plus And of course if you want to use uh java you can use j and i or if you want to use python You can use the python c call. So um that all works there But um even though they have an sdk It was going to require quite a bit of development time in order to be able to spend the lidar up and then start sampling it Figure out what the measurements were. Yes, there are some documents that describe all of this But it's fairly complicated to interface to one of these things So as I was looking around I actually found an existence of the uh of some slam code for a robot operating system for ross and it turns out that the The slam tech rpi lidar happens to have a package that works in ross So it's like oh cool. I'll use ross as the way of being able to at least interface to the lidar and uh handle it that way unfortunately, uh ross is a fairly complicated package So uh ultimately I want to use ross for the entire robot But to get started. I'm just going to use ross to do the measurements on the lidar Uh ross is available for raspbian. It's also available for ubuntu So they have pre-built packages for both of those versions of the operating system That's a hell of a lot easier to work with that than it is to try and build ross from source That I didn't want to start messing around with uh, you know cross compiling and everything So it was uh, just again the path of least resistance here Now there were some references to the rpi lidar an earlier version of it on hackaday.io not hackaday.com Unfortunately, it was for an older version of ross that was not ford portable So I needed to go poke around a little bit further to try and find A version that would install correctly and work with the lidar Ross does support a published subscribe model So if you're familiar with how published subscribe models work, basically you have a data source You publish the data somebody else subscribes to it and they get the data on a refresh rate Fortunately, there's a wiki for the rpi lidar in ross. So there were pieces Took a little bit of research in order to figure out where all the pieces were and then to try and put them all together into one place Uh, so I used it turned out that most of the documentation for using ross Uh tends to be very ubuntu centric So, uh, the path of least resistance on the raspberry pi was to use ubuntu mate 1604 My preference would have been raspbian, but uh, it was just like, okay, how am I going to get this thing working before the show? So I wanted to bring it here Ross of course is uh, again only being used for the lidar at this point But uh, once I finish with all of this, there'll be a github repository And if you send me an email, I'll be happy to send you the link on it when everything's said and done So that you can take a look at how it was put together So I got it up and running and I got ross scanning and I have a it's uh, you can see it here These are basically uh desks and tables and things of that sort inside of the room So every one of these red lines represents some obstacle that's in the field of view Uh, it turns out that's a person and that's a person too And it's kind of cool as they walk through because there's this line of pixels that move through the screen as the thing updates Uh, so it's kind of a cool solution. But of course, this is just the first step This is just the distance measurement part This hasn't integrated it into the actual slam algorithm yet And I don't have the imu up and running for it yet So I can't start doing mapping inside of the house at this point. But again, a lot of the pieces are there So current status wireless communications works actually I did this Uh, a couple of years ago and had a robot here at ELC That was being driven with the xbox controllers through linux. So it was uh, I basically was running the raspberry Excuse me the bigel bone black on the robot It was the server my Controller with the xbox controller was the client I would connect over using standard sockets And then I would start sending joystick commands across the link and I could drive the robot around which is kind of cool But it means that I could then do some code reuse. Of course, that's always important, right? And pull all that information about using the controls getting the joystick working I mean, these are all little things, but they're nagging things when you say, okay Well, how does a how does an xbox controller work? You know, what events does it generate in linux when you push the button or when you move the joystick because there's analog portions There's digital portions. How does all that work? So that is already out in my github repository So if you're interested in that you can go out there and check that But right now I don't have the imu integrated yet because I'm thinking that I'm going to convert this all over to ross I need to be able to get an imu that ross understands I don't want to have to write a new set of drivers just to be able to use ross Uh, I uh as I had all this put together. I'm ready to start testing and I find out that one of the motors is borked Uh dead locked up. It will not move No matter how much force you put into it. It is dead in the water Um power doesn't do it, you know, you can't even turn it with a pair of pliers I disassembled the gearbox and Thought I had it all put back together again and when I put it back together and it wasn't any better so I've contacted the kooky people and uh, the kooky people are now arguing with me about, uh, this motor control about the motor And I go dude, it didn't work. I've only had it a week. It didn't work. Well, you know, people have problems with power And this is like, okay, dude, I've been building robots for eight years I understand which is the red and which is the black and where you put the positive and where you put the negative Uh, okay, but so that's the argument that I'm going through right now In the meantime, I'm I'm fed up enough with them that I've said, you know what, okay, I'm going to get a different robot So that's a new one that's on order That one's I think a hundred and thirty dollars or something like that But that one has four Sets of motors on it. So I'm going to have to use a different motor controller and do things a little bit differently on it, but nonetheless I I will hope that I'll be able to get a replacement motor from the kooky people But if not, then I'm going to have to punt that particular effort Take everything transplanted over to the other robot and try it again Which is not my desired situation, but that's the way it goes in the robot business So our summary here The first attempt at using a lidar, uh, this has been a really interesting exercise It required me to learn a lot about what lidars are and how they work and what they do It, uh, I found, you know a reasonably inexpensive one, uh at four hundred dollars That's definitely not something that most people would think is inexpensive, but for lidars it is It turns out that linux can handle most of the services with no problem The only issue that we have with linux and this type of robotic application Is the issue associated with the motor controllers themselves and making sure that we can guarantee that pwm signal coming into the motor controllers So in that case, uh, you know, if I'd had the beagle bone blue at the time Well, it still doesn't handle the power. So I would have to still do something else differently with that with that but, um That definitely is, uh, kind of a work in progress and I hope to have everything operational again as soon as I can get the replacement motor Uh, within the next couple of weeks or so and then we'll transfer translate it over to ross It turns out, uh, arduino. I mean the the linux drone project has a lot of cool stuff Ross has a lot of cool stuff. So there's several different frameworks that you can use to actually control this The real driving factor for me in this case with ross was the fact that the rpi layer already had an interface a package for ross That I could then just instantiate and start pulling data off of it So software development Probably at least in other two three weeks where the software development and I should have it all working At least mechanically if I can get the replacement motor out of them or end up buying new motors I don't know what I'm going to have to do on that But uh, you know the whole interface to the tracks and everything is going to be Totally screwed up if I have to get different motors. So I don't know. I'm going to have to machine something I'm not looking forward to that a 3d print. I could always 3d print it. I guess right So looking forward to getting the unit back online and uh, it's been an interesting exercise But I hope you got something out of this. We got just a couple minutes for questions and then we have to move on anybody You're all still asleep. I understand. Yes Yes, I have Now those are industrial lidars and they're usually, you know, a hundred thousand dollars for one of those things They are extremely good in terms of their resolution But um that the issue is because of the way the lidar is put together It's tough to get it into orbit Because the boost, uh, you really have to do some significant packing and and in protection of the lidar unit itself Measurement. Yeah, it was just for measurement Yep. Yeah Ah So time of flight cameras are typically focused on focus, right? Uh, they're they're typically, uh, built specifically for camera Pictures for being able to do imaging. So that's very much like a flash lidar also does the same general ideas They're to produce an image of something Time of flight is pretty much time of flight. So whether you're using, you know, ultrasonic or infrared or light lasers Whatever it is, it's still the same basic concept. So it's just a question of how do you power it? Is it eye safe? You know, what are the constraints that are associated with it? What's its range its distance and all that sort of stuff? So there really isn't a whole lot of difference when you're starting when you're talking about time of flight Now in terms of your particular implementation I have to see which camera it was to be able to tell you if how what what it was how it was different how it was similar Yep anything else Yeah, it's uh, let's see here. I think I have it here Yep, there is There I am So I appreciate your time. Hope you had something out of this. Thank you very much. Talk to you later