 So we heard the narrow connect here in Vancouver and hi, so who are you hi? I'm Tom Uli from UC San Diego and Renaissance robotics, so We're looking here at the famous MIP right so this is called edgy MIP So this is an educational version of a toy that we came to market with with while we I'm called the MIP Which was a highly successful toy. I'm consumer toy fun to play with that a fun place back And this version a student can assemble from a kit I can model the dynamics on Which is of course unstable segue like dynamics falls over unless you have the right kind of feedback Then learn how to program a little computer on it in order to stabilize it And then once you figure out how to connect the cyber to the physical you can then extend this and build a lot of fun things So right here Dylan our connect you you talking about A new version of it that has 96 board support. Yes, exactly So we're working on version 2 of This now and so as you see here, we've developed a Universal mount and so now the edgy MIP in this new version Can fit anything in the 96 boards format and here we have the dragon board 410 C based on Qualcomm chips And of course, there's a rich family of boards and the 96 boards ecosystem built largely on Cell phone technology so cutting-edge processors Ten times more powerful than the crate who was a quote that I like to use Which is the fastest super computer in the world when when I was an undergraduate and now people have 10 cray-tos running at about a Lot on these modern processors and so there's a lot that can be done So it's exciting time to be getting into robotics So this one is a beagle. Yes, and so the original product that we came to to market with you can buy them on renaissance press renaissance robotics, right on so the original Kit that we developed was built around the Beagle bone blue And the Beagle bone blue was developed in collaboration between our lab at UCSD And the good folks at Beagle board org And so it essentially fused together all of the key components of the Beagle bone black Which is a highly popular credit card sized Linux computer together with all of the the power electronics and H bridges and breakouts for motors and encoders and all of the individual buses together with IMU's And the battery management ICs all together on a single board and it integrates a Sip so octavo systems put together the satara processor that was at the the core of the beagle bone black Together with some memory and some power management ICs into a very small footprint Allowing us enough room to put all of this other Functionality together on a single board. We're only using a very small fraction of that capability in a mip Of course, we're taking two motors and two encoders onto this board but Once you learn how to to do this you can Extend to much more complicated vehicles, which is what we use this for is to introduce how to do motor control and feedback And what's the advantage of having something much more powerful? Potentially so it's exciting to be moving into the 96 boards family because now we can use processors that are custom designed for Advanced cell phones in order to pull in 4k video and have built-in DSPs and GPUs and so we can I do on I on the vehicle so edge computing if you like I'm we can Do object recognition and we can do face detection and we can really do all of the advanced things that a That in a modern cell phone can do the idea is to integrate that into our small robotic systems And again, this is just a test platform that we're showing here You'll learn how to use these things here and then you can extend to larger systems from there a for instance that does is kind of cool so Robot like this can fall off the edge of a table I think playing at the dinner table is an interest or a restaurant on the table is an interesting place to play So one of my goals is to do automatic edge detection So the vehicle can run around on the table, but it can stay away from edges and keep from falling off the table So that sort of thing requires vision processing So as it moves around it can recognize the boundary of this play area And it could also be interesting to have access to all these new AI Newel networking accelerators. I think in the future We're gonna see that there will be two types of toys those that Recognize your face when you come in and can greet you by name and those that don't and so really I'm using some of these modern cell phone technology in robotic toys And so they can interact with you and even detect your mood and and play with you on is going to be a very interesting aspect of How toys evolve in the future because this this toy I mean educational toy right here Mm-hmm When I see it move like this it makes me think that maybe it has some kind of feelings or something Yeah, you want to you want to have the next you know like the smart speaker kind of into like the next kind of AI thing Where it is suddenly you'll give them some kind of soul or what do you call it? Yeah, so that's why I think the the while we toy Mip was a breakout success when it first came out a few years ago Was the dynamics of the the motion make it feel organic makes it feel alive Just from the fact that if you give it a push it has to adjust to stay upright on gives it a sense of life liveliness that you don't get in just a statically stable toy sitting on the ground and so coupling that together and then with the advanced features of vision recognition and to facial recognition and Also communication voice communication and even detection of Maybe how you might be feeling that day if you're feeling direct or if you're feeling shy or sad and be able to Interact with the emotion. I mean that's one of the exciting things that we're able to explore now with with AI algorithms being able to really detect mood and Understand context and that's really exciting direction to go. So This this the M MIP stands for yes mobile inverted pendulum So it's a technical word for what's going on in a little segue like system. So it's kind of like the segue Concept yes, and and but how does this this kind of algorithm work on all these different boards? Yeah, so the core algorithm here is is Straightforward and so in the undergraduate controls curriculum at UCSD on the capstone class that kind of tops that all off is a embedded control and robotics class where undergraduates each buy this kit on for about a hundred and thirty dollars For everything that you see balancing here puts it together and and understands how to Close an inner loop and an outer loop essentially the inner loop keeps it stabilized around a Nominal angle and then the outer loop adjusts that angle to keep the thing from running forward or backwards And so the inner loop and the outer loop are essentially PID controllers so straightforward controllers. We tweak them to be a little better and then the whole system is then put together by coordinating these inner loops and outer loops Together to control the whole vehicle and then you have other loops going on simultaneously that take care of other Items you might have remote control from your phone driving the thing around giving information much more slowly as to reference Waypoints to to move to and so the the algorithms the control algorithms are in our Undergraduate controls curriculum at UCSD as I said a number of other schools have picked up and begun using this platform So at UCSD the original course that we developed you program the little Linux board there the big one blue in C and so other popular software packages Include Ross the robot operating system and so is Lewis Whitcomb at Johns Hopkins University developed class around the Using Ross to control a system like this Mauricio de Oliveira developed a class using Python to control the same vehicle and So what's nice is to be part of a much larger community where you have friends and colleagues at multiple universities Expressing their own creative vision of what you can do with small boards and small platforms that are affordable for system stone for student Stone and Developing new educational curriculum around it and so that's kind of nice thing about this meeting is here We're developing we're meeting a lot of people who are developing around this 96 boards format and so what I'd Am working on the the folks here at this meeting with is this next generation board and again We're making universal mount here And so we can mount anything from a beagle bone like we have done before To a raspberry pi so you notice this is a raspberry pi and this is The dragon board 410 C and the 96 boards format and so what we're working on is the daughter boards for these other processors of the Raspberry pi format and the 96 boards format and also a standalone single board Based on an arm cortex M3 so when you want to Prototype one of the challenges that we've seen in my lab Is that if you develop initial lab prototype on a low-cost? Beagle bone or a raspberry pi and then you either want to scale up to something much more powerful that incorporates 4k vision or you want to scale down To something that you can embed in a low-cost toy you often have to start from scratch And so the vision here is that we're developing a family of Boards and so we can be more platform agnostic We can do our initial lab prototypes on some low-cost Community boards that a lot of people use and then as we scale up to some more powerful system Or we scale down to a low-cost system which might be initially harder to program But can be something that eventually can go into a much more low-cost toy We can use that initial code base that we develop on our initial lab prototypes and be able to port them over because our hardware Will be on the the same hardware on on the daughter boards mounted on these other Platforms and the software So all of these that I have on the table here are Linux boards The next in the line will be the the low-cost board that will be on a Arm Cortex M3 or M4 that Is a lightweight target that can't handle running Linux? And so it's running a real-time operating system or an Arto's so we're developing a Library that's portable across this entire range of boards from these Linux boards To these more advanced Linux boards and all the way down to a lightweight target running an Arto's and that's running the robot control library Developed by my PhD student James Strossen and we're With that library now is well tuned for the the Beaglebone And so what we're going to do is work towards porting those across this entire family And so you get both hardware and software portability as you develop prototypes So I like to say from from prototypes to products And so we can streamline that whole workflow and based on my experience The the challenge of getting from the initial lab prototype to the production prototype That's what we're trying to streamline through this process so What if let's say there's another 96 board that comes out that has a super powerful CPU and all this new Networking and yeah, and stuff the good thing about this kind of balancing for example And the segues is that they kind of adjust to any kind of weight, right? So yeah, plug in another one on and then load the software and it will just work the motors and everything is So so a couple of aspects to that so so first of all working in a standards compliant way like this And so Beaglebone Raspberry Pi and the whole family of 96 boards are three very common standards And so if you and especially talking about the 96 boards Standard there are many many many boards that are in this format and more to come And so if you work in a standards compliant way across these boards, then as these new boards come out You can take your Your daughter board off of your 410C and put it on the new Dragon board Whatever it is that's that's coming out on and then you can use all of its more advanced features The other thing to mention Related to your comment about changing of mass distribution. We are working on a new set of adapters that you could for instance mount a Weight that you can put on top of it. So the idea is that you could Be balancing something like this And then say what happens if the mass suddenly changes So whenever I show these things at a conference somebody always asks have you developed the The robot that can get a beer from the kitchen for me And so it's always said as something of a joke But there's there's an interesting question there And so the first question if it's a balancing robot is yes You're changing the mass distribution significantly when you put a beer on top of this thing And so a certain type of control algorithm is needed one which is adaptive that recognizes the change in overall mass and mass distribution and adapts the control system accordingly in order to keep it from balancing now that its weight distribution has changed And so we're developing on a an extension a number of different extensions But you know one of the the first extensions that we're developing is simply something which is a beer can holder for the top of this Thing on and so you can put a mass on that and then you can tune your control algorithms And so they can adapt for moving mass on and off of this thing because the dream is as they are The questions are in the conferences and stuff like that Is to have the the mip bring you the beer right right and so if that dream is to be realized That's a fun dream if that dream is to be realized We're gonna have to have in addition to a good beer carrying robot We're gonna have to have the custom fridge that might be some adaptation of the of this sort of vending machine that you see in Commercial sites, but at the home scale so something that in the doorway on you can load in a bunch of 12 ounce cans and then you can select Using your Alexa say Alexa get me a beer Alexa triggers the refrigerator to Pull out whichever your favorite beer flavor is and it comes down through through a shoot down to the The bottom and the vehicle goes over and grabs it from that shoot and and then drives it outside to the living room So I think that's actually doable. I don't think that's that far off The idea of vending machines that can manage keeping drinks cold and selecting one Is the well-established technology? Building robots that can balance as well-established and Certainly being able to carry a load of that size is not a problem at all So I think that's near term if anybody is interested in pursuing that please Come to UCSD and we can work on that together. I can't imagine there's you have a lot of students already that are very interested in your In your projects here, but if you also have the beer that might be even more even more so yeah So I want to sign up so when I would be including the beer, right? There you go free beer So at UCSD one of the ways I Managed that that strong interest on is we have a an organized academic program And so as we go through this this this program that there's a class in the fall quarter senior year Called embedded control and robotics where you actually go through this exercise of learning how to multi-thread So write a program with multiple different threads in Linux or now in an artist and to Connect a discrete time rocker controller with a continuous time unstable system And coordinate all of the different things in order to get the communication between the threads in order to put all this together Once you get through that then I welcome those undergraduates and to do undergraduate research or to stay for a master's degree and take it to the next level so One of my very talented students Put a Put a spinning PCB on top of one of these units and It had a row of LEDs I closely spaced LEDs across it and if you spin it faster than about 40 Hertz Then your eyes blur it together. It's called the persistence of vision effect POV And so for instance if you Make all of those lights yellow and you turn them on for 300 degrees and off for 60 degrees and on and off Then it would make for instance a Pac-Man. So you can make other other characters And so for instance on that that group of students The idea was to have a bunch of MIPS running around in a little maze with little Persistence of vision heads that are spinning around so looking from above you could have characters chasing around each other in a maze Just like in a video game So that's one sort of thing that you can do you can of course print shells. So one of my students was was challenged to Print out 3d print on some some very compelling shells for the thing and he designed a an Easter Island head So, you know the the heads that that are facing the ocean on Easter Island So he built a head that fits on this and so we had those running around We've also built a version that is telepresence and so it has speakers and microphones and and Screen on it so they can run around and you can have a conversation with somebody remotely So yeah, there's a lot of cool spin-off projects that come out of this and It's really fun to watch what what students come up with with their creativity and these students are building the robots for the future Yes, they all get get out of this eventually get jobs in the robotics some of them or yes and in the toy industry and and that all the people trying to enter the the smart home market and so there's Like I said an incredible growth of the possibility now that we have low-cost cell phone technology As I said 10 kray twos In your pocket Now is the technology that drives these little systems running around and so a lot of interesting stuff can be done So now is a great time to get into robotics one of the interesting applications Is elder elder care? So something that can go around and and help out your grandparents in their home Keep them out of an assisted living community for five more years and keep them at home They want to be there, but sometimes they need help during the day something that can Help them around the home pick up their keys or Make sure that they haven't fallen and call for help if they have monitor things make sure they take their medicines And so I'm using robotics Both for fun for kids, but also for care So to have a companion that is lifelike and fuzzy But also can do practical things around the house is a very interesting place to be you can also use Robots for security for for monitoring Things so just an incredible number of applications Precision agriculture is another place where we can use them to evaluate every plant On a day-by-day basis to see what to sign kind of water and fertilizer and pesticide They they might need individually plant by plant across a field and then both Planting and harvesting so soft grippers made from a 3d printable Soft material like thermoplastic polyurethane that can pick strawberries I mean, they're just an incredible number of applications where I think we're going to be able to use robotics. And so yes, this is Really a actively growing field and a lot of interesting things to do and on the screen you were showing a PCB Yes, so I wanted to give an idea of this So this is on the daughter board that we're building. This is the raspberry pi version and so as we Work towards getting the raspberry pi and the the boards in the 96 boards format being able to Drive Small robotic systems in a platform agnostic way to give us portability from from the low-cost consumer boards to the higher-end boards or Or the the lower-cost consumer toys on this I think is going to be our tool to Make that porting and so this is a board that is going to bring up to 12 amps of power onto the board through a little xd 30 connector on the bottom And then we'll have JST breakouts for eight motors at up to 2.5 amps a piece and breakouts for eight encoders all the standard buses and 10 servos or ESCs of course, we'll have an IMU on here and a barometer and a lot of power conditioning to handle 12 volts from a two-cell or three-cell or four-cell lipo on and Sorry 12 amps two-cell three-cell or four-cell lipo so six volts up to 18 volts So we're regulating all that power as it comes on board and distributing appropriately sending a little bit of power down to the microprocessor and shipping the rest of the power off to the The motors the servos and the other components that you'd be connecting to And so there'll be a small microprocessor on there Controlling all of the supplemental features on here and again the idea of this board Is to have a version for the Raspberry Pi a version for the 96 boards version for the Bigel bone and a stand alone Based on a low-cost processor and so you can pour to cross these whole families on and then once you Have designed the prototype then you can take our schematic dial down to only the components that you need for whatever product you're taking to market and then you can Accelerate the idea from the the prototype to the to the product that you actually make so and So that adds a whole bunch of more kind of Sensibilities to the to the robot. Yes, and so I'm here again. We're only driving two Power motors and reading to quadrature encoders and low speed the idea here is that we have And we can drive eight high power motors up to 2.5 amps 18 volts so we're doing 100 watts of energy going into these these motors and You can even double these up and so you can put You can put Into each motor sorry Six to 18 volts at 2.5 amps you double them up. You can go five amps. So that's up to 60 watts of power in each of these motors And so you can build a large format 3d printer or a large pick-and-place machine So this can be used directly embedded into products But you can also use this to prototype for those products and then scale down and getting rid of the Other things that you don't need if you're scaling up to a large system You can also build a quadcopter with with this sort of thing or a multi rotor with with more rotors So the idea is to build a multi-function board that is an easy starting point and is portable across platforms Because this is the awesome myth, but you've done a whole bunch of other stuff, right? Yeah And so so actually the the base of this thing on is Is actually taken from the assembly line while we was gracious enough to Let us buy a few of these off the assembly line. So this is toy grade And by that I mean something pretty amazing that a two-year-old can have a temper tantrum with this wheel base And it still works and that I speak from experience my two-year-old Zachary had Emotional moments where this thing went flying across the room and hit the wall and the mip still worked and so if you just Take a motor and gearbox and bolt a wheel to it and then you throw it in your backpack and you take it out Some percentage of time that that motor shaft is going to bend is just not going to work But this thing has a load bearing in the output shaft and has very strong has the right amount of flexibility And so this is a really robust base And so we started from that and we use this as a teaching mechanism but the idea of this Edumit platform is once you get it working and understand it Take it apart and do something else with it. So you use this as a tool to learn how to connect the cyber to the physical and to to write to motors using PWMs and to read what the motors are doing using quadrature encoders to communicate to an IMU over I2C bus or an SPI bus and then to Really build out to the system from there to connect to as many other Motors or servos or ESC's that you have in your system in order to build an arm and extend from there And it's only $50 without the board Yes, the the kit itself is $50 at Renaissance Robotics and then the board is 78 so for $128 Yeah, so these are going to be a different price, but again, we're going to have much more Capable board so this version 2 of this Of this line is going to be able to drive like I said pulling up to 12 amps onto the board We're going to be able to drive high-power motors in order to build large-form at 3d printers pick-and-place machines Robots that have many actuators on them maybe a an arm to pick something up and so So really we're trying to scale up now to give a next level of capability in this this new generation And you working with stair-climbing? Yes, let me I show a couple of pictures on I'm just giving some ideas of directions We're going so you mentioned the stair-climbing one that I have a picture of that here so the stair-climbing oops, let me Increase the size on that So the stair-climbing robot is has indicated here So this is a project being led by Daniel Yang And so this is a mip two wheels, but it has a third motor And so the body can lift itself up this leg in the middle And so it can end over in I like to call the inverse slinky maneuver And so it segues up to a stairway plants its leg pulls its body up the leg leans over onto the step Self uprights onto that step and then does the same maneuver up the additional steps in order to climb up the stairs So we're working on what's the minimalist machine that can be wheel driven Which is which is fast and robust to make in an engineering setting as opposed to a legged locomotion Which is much more complicated and difficult interesting But a a different problem So we're working on wheeled robots and the minimalist wheeled robot that can climb stairs Efficiently and quickly and so the idea that's the idea behind this stair-climbing robot again a segue like motion driving around quickly on two wheels and then doing a stair-climbing maneuver using a third motor in order to do the lifting and the idea of this is it would be Possible to use this as a throwbot on so one of the the robots that I'm really impressed with Is the recon scout so it's a a robot that is used by the the military and security forces Something that you can throw over a brick wall or toss down the stairs And it's essentially two wheels on a toilet paper to tube variable and so the folks at recon robotics have done a Have a great job on that And so the idea of this is to add one motor to that sort of design And so in addition to being a tube with two wheels and an antenna We have the lift mechanism through the center of this thing And so we can not just fall downstairs, but we can climb upstairs. So stairs are integral in the human environment And being able to negotiate stairs with something which is robust and lightweight and small Is the name of the game and that's what we're pursuing with this effort and You've you've also had like a tank kind of that can yes And so we've worked on balancing robots if you look up the switchblade robot I'll not bring it up here But if you look up the switchblade robot on on YouTube, you'll find something developed by Nick Morozovsky which was one of our early designs that I Is a Treaded sort of vehicle where you can drive the treads around the arms and independently you can drive the arms relative to the body and so switchblade was the first treaded robot that could get up and stand on its toes Effectively and then it could flip the body up And so we use feedback control in order to extend the reach of a small vehicle And so you take a look at the switchblade blade robot and you'll see something like that And there have been spin-offs of of that general idea in recent years as well So the switchblade design is one. Let me mention one further Further design that we're working at so of course everybody's interested in drones. We have our own take on the drone idea and so This is a project that's also been led by by James Trousen And so this project is a hexacopter. So a lot of people fly quadcopters And so the idea of a quadcopter is by turning So you have two propellers that's been clockwise two propellers that's been counterclockwise So you have Four rotors there two of them are mounted in the front two of them are mounted in the back And so by independently controlling each of the rotors through what's called a mixing matrix You can have independent control of pitch roll yaw and lift So you have control of four things in a quadcopter and But you're moving in a six degree of freedom World and so the two things that you're missing is direct side force So you can't move like this in a quadcopter to move sideways You have to roll and then left to move forward You have to pitch and then left and so in a quadcopter you have those six degrees of Freedom and individual control of four of those and so you have to put them together in order to maneuver Now what's interesting is to make quadcopters small And to put a little camera on the bottom quad copters are often used for flying selfie sticks so look at me in my environment surfing or skiing or in the park with my dog or whatever it is on and The challenge is that if you're maneuvering to hold position You have to have a little gimbal by keeping the camera on the target And so you're adding complexity by putting that mechanical gimbal on there And so instead what we do with this hexcopter design on is we take the rotors and we change their orientation So we have six rotors and so we've worked through a mathematical algorithm to optimize their orientations And so we now have individual control of all six degrees of freedom. So in addition to pitch roll yaw and lift we have independent control of Direct side force in the two sideways direction. So now even if it's windy outside We can maintain our station while keeping the camera on targets because we can Directly control the vehicle in all directions that it might be get might get blown off. And so This is an interesting design that idea was originally introduced by sci-fi works on a design that they They proposed on Kickstarter. It actually never came to market. But then the the company reinvented itself sometime later but we have taken that design and Optimized the rotor angle. So we came up with different angles than they did that gives some more efficient control of each of the Directions of the degrees of freedom. And so a plan here is that we're starting with this line of vehicles that are just minimalist vehicles to learn how to connect small computers to To wheels to drive around and balance But we're working towards making a line of educational vehicles and this will be one in that line of learning how to do Six degree of freedom control with it with the X copter and this could potentially also have different boards Yes, and so we're working on a design for this So this universal mount that we developed you might want to blow on on that so that can accept a Beagle bone a raspberry pi or anything in the 96 boards format and so the way we've accomplished that is by putting little washers Underneath the mount holes that we were using and Designing the shape of this thing appropriately so it doesn't interfere with the the components on the bottom sides of the boards And so this little mount We are using Both on This little two-wheeled edging a vehicle, but we will also be using it in the other line And we'll put the design files for that little universal mount on on the web And so if anybody else wants to use this universal mount to develop their own vehicles We're happy to share that because this you open to Grow the community. Yes, exactly around the world. Yeah other using this stuff the better It takes it to grow this stuff. Yeah to get involved. Yes You can just buy these on the internet. Yeah, so so we can we can we can buy these kits the edging up kits are easily found on the web and As we developed these version two so we can port across all of these platforms and these additional Prototypes that are that are round these we'd like to make those Available nearly at cost to the community and so we can we can see that the good ideas that people have in the education setting and then Growing from there and doing cool things with them. So this is very exciting, right? Thank you the very exciting future for the robotics Yeah, I mean I think I mean we just happen to be at the right place at the right time I mean this cell phone technology really spinning out into things that that we can actually Embed into vehicles that can do useful things around the home and the farm for security purposes It's it's it's really interesting and there's there's a lot of fun that can be had with this technology a lot of useful things that can be done to make a better safer and Compassionally, you know work with others work with autistic children work with elderly to keep them in their home I mean there are a lot of things that we can use technology for the good now And so this sort of educational projects are really designed to help bring Students who are interested in doing this To the cutting-edge in technology