 Yeah, how are we doing now? Great. Thanks for having me. Like my colleague, Luca, that just spoke previously. I'm from the Open Source Robotics Foundation slash Open Robotics. We have an office here in Singapore, as Luca said, as well as all around the world. So I'm going to talk about an Open Source Robotics middleware framework for health care. I'll get into exactly what we mean by that. There's lots of applications of robotics in health care. The first thing that you think of might be surgery robots, which is great. But there's also many other applications of just logistics and large hospitals are a giant operation happening, and just everything needs to move everywhere else at all times. So I'll talk more about that. Of course, you can't talk very long in robotics without saying we're hiring. We are hiring from many different roles, ranging from robotics software engineers to DevOps to web to build tests that are just a whole software chain that needs to happen. And we're always looking for great people interested in this fun stuff. So let me back up a little and talk about the motivation of why we're doing this big project, what exactly we're trying to do, and how we're going to do it. So when we talk about hospitals here in Asia in general and Singapore in specific, these are very large buildings in large cities, talking like 1,000 to 2,000 bed hospitals. So there's a ton of people involved in the situation. There might be thousands of people at any given time inside the building, all of whom need to be fed, taken care of, given supplies they need. Inputs and outputs all process appropriately. So there's a lot of opportunities for automation to help make these processes run more economically so they can scale to the anticipated needs of the future. In general, there are many different vendors of robots that sell into the health care robotic space. So for any particular size of supply that you're trying to move around, there might be a few choices. But it's very rare for a vendor to be able to offer the full range of supply motions. So whether that's from just little, small medicine bottles all the way up to like a giant cart or giant pallet full of meals for an entire hospital ward, there's a wide range of volumes and shapes and sizes and masses that need to be moved around the facility. And again, we're not even talking about surgery robots here. We're talking purely about logistics or cleaning robots, maybe, or just anything that's not surgical. So just you can kind of block that part of the domain and think more of just sort of facility management, facility processing helping. So we're going to build these systems. We're starting with open source robotics projects that Luca mentioned called ROS, the robot operating system, as well as Gazebo, which is a simulation framework. And we're going to use those then to build up to hospital scale simulation and multi-fleet management. And I'll get into through the next 25 minutes or so what we mean and a little more specific about that. So at Open Robotics, we're the stewards of two large open source projects, the robot operating system or ROS. We've been working on that in public now for about 13 years or so. The first, I think, official version had a number on it maybe in 2007 or 2008. And ancestors go back quite a bit further than that. The domains are on the lower left, ROS.org. And then there's a new and improved, everything better, ROS2.org also happening in parallel. That project works hand in hand with a simulation framework called Gazebo. The URLs are on the lower right there, Gazebosim.org. And then the next generation, IgnitionRobotics.org. And to summarize these big projects in one slide here, what we try to do is to break up the robot software problem into a bunch of small sub-problems. So imagine, like Luca was showing previously in that open source camera projects, you have the device driver node, which maybe can get the raw data from the imagers and then publish it in a standard form of a message that is a kind of abstracted image. And then you can have further nodes that listen to that message, do some more processing, pass on more messages and such. So when you build a robot software system rather than building one gigantic robot binary, instead you end up launching lots of little small binaries that find each other at runtime. So that's the dynamic computation graph you see represented by those colored bubbles in the middle there. In simulation is invaluable in robotics because these systems cost a ton of money. So the humanoid robot on the bottom there, you can see both in the renderings and in the real-world photo, that's a real NASA robot. I don't even know how much it costs. There's not for many of them, so sometimes you don't even really put a price tag on them. If it falls over, it's very bad. So typically it's all like an umbilical, it hangs from the ceiling, so it can't really fall on its face. But yet you want to simulate it going up and downstairs and doing all sorts of interesting postures and things you are not willing to try on the real robot until you're really convinced that it's going to work out. And so that's where it becomes extremely useful to have a high-fidelity simulator. Also, in this particular example, that's a simulated Mars landing site in a simulated Mars lander. Obviously those don't exist. So when you're trying to simulate difficult worlds or extremely expensive worlds, you can use the simulation framework to save a ton of time and money. In the hospital, this is all relevant because hospital facilities, of course, are critical in national infrastructure wherever where they are. So you can't just run arbitrary experiments in them. If you want to see what happens with running different numbers of robots for delivery or different sequencing of lifts or automatic doors, how do you queue for a door, how do you queue for the lifts? Those are experiments that you can't really run in an operational hospital. So instead what we can do is we can create models of these facilities and then run all of our algorithms and test them in this simulation. So to do that, we have been constructing all sorts of simulation assets. So there are all sorts of different roles of people in hospitals, of course, when you go. There's all sorts of different staff, different visitors, different patients with different mobility aids there to help them as well as all sorts of hospital equipment. It's amazing when you start walking around large hospitals, how many things there are that are basically in the shape of a thing on wheels that you push around. It's kind of astonishing. Everything from any kind of diagnostic equipment to basically lots of different shapes of computers on wheels that they push around. COW called a cow. There's cows everywhere getting pushed around. Sanitation carts, that's just a few of them that we've modeled in the picture there. But there's, again, anything that's sort of the shape of a cart that you can push through a doorway, they've got it. So we're modeling all these things so we can simulate them and simulate the motions of the algorithms that are gonna drive their robots around. The mechanical infrastructure, by which we mean doors and lifts are a key part of this operation because large urban hospitals are essentially stacked vertically. So it's very common for a large urban hospital to be 10 levels, several levels below ground, many levels above ground. Where most of the infrastructure to run the facility and to provide for the patient's needs is in the, oftentimes, the basement levels. So things like the central kitchen, the central surgical supply sterilization areas, the laundry, imagine how much laundry is produced in a large hospital, right? So we're running all those linens through, getting them all sanitized and ready to go back up. That's all in the basement of the building and then all that stuff needs to be distributed up vertically so where all the patient wards are usually above ground for airflow and light and all that stuff. So the vertical transportation provided by the elevators are really the kind of what dictate the throughput of this entire operation. So getting the simulation of the elevator cabins or lift cabins, correct, is a key part here. Modeling the network and topology of the lifts. In many of these hospitals that are quite sophisticated in their use of robotics, they have certain banks of lifts or only for robots. And then, so you can schedule those totally for the robot operations and there's other banks of elevators that are used for patients, other banks used for visitors, other banks for staff. It's very complex and a typical hospital will have 30 or more lifts just in the complex. As you can imagine, once the facility is built, it's approximately infinitely expensive to add more lifts. So basically getting the most throughput possible out of the current lifts is required in order to keep introducing more robots. So here's a few of the common healthcare robots today that you go if you go to public hospitals here in Singapore, you'll see some or all of these. The Panasonic Hospi robots they use, Swiss log meal carts. Essentially the robot, but which I mean the kind of second one from the left here, it's a cart full of food that sits on top of a little robot that goes underneath that cart and lifts it up a few centimeters and it can drive it to where it needs to go. So odds are, if you're warded at a hospital here in Singapore, your lunch, your breakfast or dinner, all comes on these Swiss log robots. The essential kitchen is in the basement. They drive them around to the service lifts and they bring the cart up to the level where you are. They drop it near the nurse area in the ward and the nurses will then hand deliver the trays of food out of that cart. So you can imagine if that cart has the food for 40 people, it's very heavy. So it's the robots to get a ton done. Every breakfast, lunch, and dinner, they have to distribute these carts to all the different wards across, let's say seven levels of patients and then bring them back two hours later to have all the dishes washed and ready to go again. So now I'm gonna talk a little bit about what do we do with the simulation model that we've created. And then the goal is to allow multiple robot fleets. So this simulation snapshot will be, this is like what would be great to happen in the future, just as kind of a cartoon of you have three different robots from three different vendors, but they're sharing the same space and they're operating reasonably around people. So what we're trying to do is to have it so that you can mix and match these robots. And the question of course is why we're doing this is just making everything more complex. And the kind of easy single slide answer is that robot, it's a rapidly developing field. Every year, if you go to a robotics trade show, there's new applications. There's new providers of robots. There's new robots of course themselves. Today we do a lot of delivery in the field. So you see lots of like meal delivery, linen delivery, medication delivery, case files in a hospital, all sorts of stuff moving up and down around the facility. That's so delivery is very common today. But I think we can all agree there'll be new applications in the future. Things that we can maybe see coming on the horizon are like more and more of automated cleaning robots. So imagine like the Roomba vacuum, but like, you know, way more sophisticated with more, maybe like the ultraviolet light so it can spray around to do the sanitation, sanitize rooms, you know, it can scrub floors, mop floors, what have you. So cleaning is certainly coming. Security robots, you know, patrolling around when the facility is supposed to be closed or you know, not supposed to be someone there or if there is supposed to be someone there or whatever. So all these applications are coming. And then of course there's many that we probably can't even imagine right now that will be out in a few years. So what we want to have are software systems that allow new applications to come into the field and just deploy them without having to kind of sit there and replan and reschedule all the machinery. So if you look at that picture, that's a picture of Penang, Malaysia, we took that picture a few months ago just looking down from one of the big towers there. You can see there's an incredibly complex field or incredibly complex kind of like flow of traffic there. There's many different roadways that are all kind of coming and going in different directions and it all flows fine. And that's because there's a well-understood rules of the road that people follow mostly. And as long as most of the vehicles follow most of the rules, most of the time, traffic can flow. And when you introduce the N plus one vehicle, it just merges into this flow and eventually it goes where it needs to go. And so our goal is can we figure out something similar for robotics. So this is the situation today. This is in one of the hospitals here in Singapore. The hospital that are really impressive, they run essentially one robot fleet. And so this is a picture of like on their upper right there, the meal carts. So this is taking the, it takes the food out of the central kitchen in the basement, distributes it up to the wards. And then like an hour later, it brings back the dirty dishes and there's a giant facility that washes all the dishes. So it's impressive. These are large robots. Those are very heavy carts full of food for 40, 50 people. It's actually kind of amazing and almost hypnotizing to stand there and watch these giant things move around. It's very well orchestrated. It's very impressive. So the single vendor fleets, this is really impressive to see this today. What we are trying to push though is this multi-vendor operations in the same space. So if you look at how things are kind of where it goes today, it's like the fleets don't talk to each other. So vendor A doesn't talk to vendor B. And so then you have to either keep them separated in time, like you have the meal robots all go and then once meal robots are back on their chargers then you can bring out the other like kind of miscellaneous delivery robots. That's one solution. You can also just sort of hope for the best and kind of just send them through. They all of course have onboard collision avoidance sensors of many different modalities and so they never collide. It's just they kind of do this kind of funny thing. Sometimes we call it robot dance. They just sort of look at each other and like one of them backs up and kind of like it's sort of hilarious to watch. And eventually it'll all kind of, the traffic all flows through the lobby usually. So it's not optimal. I think we can say that pretty safely. And so wouldn't it be great if you could have a joint planner that is planning for both fleets or at least can kind of be like a traffic police. Like, you know, you stop, you go, that kind of thing. And finally, I'd mentioned it a little bit before but the lift integration is key, especially in these hospitals that have so much vertical motion happening. And essentially it's a little bit hilarious to think about but in some of these facilities the reality is that there aren't enough service lifts to keep dedicating new lifts to new vendors. In other words, so if we have two vendors of robots and you've allocated two banks of service lifts to those two vendors you actually can't bring on another robot vendor because you don't have any more lifts to dedicate to them because lifts are essentially infinitely expensive to add to an existing building. So what we're looking to do is to help multiplex these lifts so that you can have different vendors share the same lift, which is not current practice. The goal then is to more specifically say these multi-fleet conventions for robot operations. So couldn't we come up with something like a cross-vendor lane network, for example, in these large buildings? So if you're vendor N plus one you sort of download the road map just like if you have a self-driving car system and you're joining the public road you download the public road map and then you're not just gonna drive anywhere you're gonna follow the existing traffic conventions. So this is a part of the project we've been working on is to develop some conventions like this that we can push out as a vendor neutral. So our organization doesn't make robots we don't have any robots to sell so we're not trying to sort of push one over the other. What we're trying to do is instead is to generalize these ideas of train traffic motion how do you call a lift, how do you open door that kind of thing. So to do that we've been creating open source tools the URLs there on the upper left this one is called traffic editor so the idea is we can get the floor plans from the facilities. So that's our an office kind of layout the lower right there and you can see we've annotated some robot lanes on there and then the upper right is another facility where we've taken their blueprints we've traced the walls, the doors and stuff and then added where we think the robot traffic lanes should be and you can see there's different colors in there to imply that different fleets might have different routes through the same space because they might be doing different tasks like bedside delivery as opposed to surgical supply delivery or what have you. An interesting thing this is kind of an interesting journey we've been going down is how much control do we really get over these robots? So these are all safety certified systems they're operating in the hospital so it's not like you can just go and casually run some new software on these robots they're basically completely welded shut but instead what you can talk to is a fleet manager API and most of the vendors provide some of the semblance of an API where you can talk to some over some protocol maybe it's REST maybe it's XML, RPC what have you to issue various tasks or to get various informations from them. So as one example maybe some fleets only will tell you where the robots are but they won't ever let you issue commands to the robots or maybe other fleets will let you see where the robots are and you can temporarily pause a robot where it'll just stop and then you can say resume your task and it'll keep going again and maybe some other fleets will give you full control where you can like give it a path to follow and it'll follow the path as long as there's nothing in its way and so the hope is that if we can have a system which can deal with those various levels of control then we'd be able to mix and match and essentially do what a traffic police officer does when they stand in the middle of an intersection and just sort of give hand signals to drivers to help the traffic clear out. So of course the more control we have over these fleets the more potential there is for improvement. So there's kind of a give and take here of if the reality is just that over one particular fleet of robots we don't get any control we can just read where they are then okay that's just reality but we would then suggest that they have other fleets in the system that we can have more and more control over. So to do this we're creating this vendor neutral system that the URL is up there in GitHub, RMF for Robotics Middleware Framework this is the core scheduling system. So you can imagine the user interfaces in this case it'd be like the surgical supply staff making orders or requests for the new supplies for that day's operations or the nursing staff wanting an item delivered to the award. So that goes through a dispatcher or a task planner which then talks to a negotiation process between the different fleet adapters and a unified schedule that can able to predict where the robots are going, how much time they're gonna delay waiting for this lift or all that kind of thing. Try to push that all into as grand unified schedule of the facility so that the shared space can all be allocated in a reasonable fashion. And one thing that's kind of interesting about all these robots is that all of them will yield to the people in the hospital. So if you get in the way of a meal cart robot it just stops, it's kind of funny then it'll very politely say like please move aside I'm trying to make a delivery. So then you move away and it just kind of keeps on its way. So essentially the task that your prediction of the task completion time it can always be delayed because maybe someone like left the box in the corridor or something and the robot is not willing to go around the box it'll just stop there. It'll then ask the box to move aside over and over and over it's really almost comical. It'll just like park in front of a box asking the box to move aside. Eventually it sends a page to one of the operations staff and then they'll come and move the box out of the way and the robot can keep going. So essentially any of these tasks can be delayed arbitrarily but they'll never go faster than you think they will. So it's a bit of an interesting planning problem. Robots all have a speed limit on them. I think around here it's regulated to like 0.7 meters per second or something and that's it'll never go faster than that. It can only be delayed essentially. And when a robot's delayed you can imagine that does interesting things for the schedule where now you have to re-plan that you know this robot is got there sooner than we thought. So here's a little cartoon to try to show what's gonna happen. So let's say that there are three different fleets of robots they're all trying to use the same corridor or this more likely in reality the same lift lobby because all the past tend to kind of converge in the lobby for the lift where they're gonna we're gonna wait for the lift car to come. So it's interesting because the robots even though they all are kind of the same they're all sort of boxes on wheels. They're all implemented differently inside. And so each fleet doesn't necessarily know what the other one is capable of. One might be a very tall heavy robot where it has to move very slowly and turn very slowly. Another one might be in a more of like a cylinder shape where it can kind of turn on the dime much faster and be able to maneuver more easily. So each fleet essentially is just kind of generates what it would love to do in an empty world if there were no other robot fleets around. So that would be great but typically they're gonna interfere in some sort. So then now that they can see what the other fleets are trying to do then now you have a game, right? If you know what would I do if you do that and what would you do if I do that and that game continues as deep as you have fleets and then at the end of it you can now have a score and now a third party judge can be the kind of arbiter between all those plans and those plans that are determining of other plans as it all rattles out. And finally we can pick the winner, essentially the winning plan that requires the least sort of joint delay on everyone's desires through the space. So that's what to do with fleets. It's also interesting to consider different classes of robots. So there are many robots that do not have a fleet manager shipped with them. So let's say, you know, research projects, low cost, like rapid prototyping projects or even a standalone proof of concept or something like that, where it's not necessarily a totally shrink wrapped product that comes like with a fleet manager and technicians to install it and make it work great. And so for those we're having a fully open source fleet manager which can plug in at the same level of these proprietary vendor fleet managers. So those hook into the common internal robot software like ROS for example, that is used on many research robots. And then it's essentially a peer of these proprietary fleet managers so you can have these mixed fleets of, you know, fully shrink wrapped robots as well as kind of more R&D or lower cost platform. Okay, so that's a lot of talking. Let's do some demos, it's kind of fun. So I will just first show what the simulation looks like. It'll come up in a little bit. So here's for example, one of the test facilities in one of the buildings that we work in. This is designed to be kind of a mixture of a ward. So here's like a couple of people is like a four person hospital ward here as well as to simulate some of the storage rooms and storage facilities. So like this is what it looks like when you go to the surgical supply store room through these little like lots and lots of shelving units with sterilized instruments that are in there in storage. There's usually patients in transit. If you spend much time in a hospital you see people moving around in various ways. This over here is intended to be another kind of surgical store. Let's see here, what else is there? Oh yeah, and of course there's always tons of people waiting in various parts of the building. So what we can do is now I'll show the traffic manager after I first introduce you to the robots. So this is a common robot that you can buy off the shelf. It's a nice robot, maybe a company called MR. It's one of the common industrial sort of you wanna move a thing type robots. This will move your thing. You can put stuff on it or you can have accessories for it that make the top lift up and down to lift other carts that it rolls underneath. And then these are lower cost platforms intended for rapid prototyping here called Magni. And let's see, so now we'll show what happens when the grand planner all runs. So let's see here, let's go somewhere else. Right, let's tricky to type one handed. Okay, let's run this demo. So it's gonna go very fast. I'll play it a few times because one of the funny things with large robots because they're speed limited to like 0.7 meters per second. If you watch them in real time, it's like watching grass grow, right? It's just like that's so boring. Anyone who thinks robots are taking over the world hasn't ever seen real robots because they're so boring. They will do it extremely slowly. You know, it's fine, you're fine. Everyone's fine. So let's see, so we've sped the world up eight times here. So now this is showing that same world but with robots moving around. Okay, let me stop this for a second because there's a lot going on. On the right side, you have the path network from the robots that was a screenshot from that traffic editor before. So here we have a purple fleet. There's a blue kind of cyan fleet and orange fleet. So there's three kinds of robots here. We're actually only running two now but they're gonna only move around just like railroad tracks on those paths. At any given time, the central planning entity is trying to predict the future of where these robots are going to go. So there's a green arrow that you'll see and that green path is the prediction if you roll out the trajectory in the future. Like where does it think the robots are gonna be as time goes forward? At any given time, there's these little purple kind of dots and that's where the robots last reported position was. So we're simulating all this in gazebo but we've run it in reality as well in the same space. So the one, the big white robot is gonna go from this simulated storage facility here to this simulated storage facility here. It's just gonna go back and forth. These little robots here, which are simulated like a bedside delivery robot, you can deliver any items to people or whatever. So those are gonna be kind of just going everywhere. And the idea is we've set this up on purpose to be almost pathologically difficult because all these paths are intersecting on purpose. So the central entity is then gonna be sort of stopping particular robots from getting close to each other. So as you see them go back and forth, they will stop and wait for each other as the predicted paths intersect over time. You'll see some of them stop like this. There'll be a situation here. Whoop, that one moved out of the way. It's sort of this like ballet as it goes on. And you can imagine this gets even more complex as we have more levels to it and more robots to this situation. So it's a little hard to see what would have happened because we're not showing what would have happened if you didn't have the central planner telling people to stop and start. But so you have to sort of believe me when I say that it's not good, it's just a train wreck. It's not a train wreck. It's more like a traffic jam. Let's say on the road outside no one follows the rules and no one follows the traffic signals or the stop signs and you just like, just giant gridlock. That's what can happen with these robots. And the goal is to make this thing scalable so that when you wanna add n plus one or n plus two robots in the future, it's no big deal. And so you can imagine a future world in a few years or who knows when when there'll be 10 times as many robots as there are now where this type of a system will become increasingly important to prevent these traffic jams from happening. So I'll let this thing play. It's kind of fun to watch. So again, it's almost like the software is running really well when there's no obvious traffic jams happening, which is a little bit, you're trying to prove a negative, but it's okay. You can imagine what would happen if we just had the giant gridlock full of robots here. Okay, so that's the end of my prepared talk here. These are all open source projects. There's some URLs and GitHub there as well as the kind of foundational projects Ross and Gazebo, those are the websites that those have. And we're always fully welcome contributions from everyone from all domains. It's a fun wide open space to play in. So thanks for your time.