 So, welcome to my talk, Evolving ROSS for Safety's Critical Systems. My name's Tully Foote. I work at Open Robotics, and we're the, more on the company, but we're the driving force behind ROSS and Gazebo and a bunch of other open source software for robots. Just to give me a baseline here, how many of you are familiar with ROSS? Okay, how many of you use ROSS? Cool. How many of you have gone through a process of certification for any standards bodies like the FDA or the FAA? Okay, cool. So, I'll give you a little bit of background about me. I, in grad, college and grad school, worked on autonomous cars for the DARPA Grand Challenge. It was really awesome. Back in the early 2000s, we were driving around in the desert and then city-type environments, strapping sensors on cars, and hopefully not crashing them into things. After I graduated, I moved on to out to Silicon Valley and started to work at Willow Garage. The main platform that we worked on there was the PR2, a humanoid robot. It had two humanoid arms, but wheeled base, and it could drive around. After many years at Willow Garage, I moved on to Open Robotics, and I've been there ever since, working on core developer for ROSS and doing lots of community outreach, etc. I was also pointed out that I was involved in the Turtle Bot early on. It's been a great platform for teaching people about robotics and how to get involved. So, talking a little bit about Open Robotics, it was established in 2020, sorry, 2012. We're just coming up on our 10-year anniversary party next week. It's going to be a lot of fun because we were established in 2012, but our first employees started in 2012, yeah. I started in 2013 at Open Robotics. It's privately held. The parent company is the Open Source Robotics Foundation. It's a non-profit, 501C3 in California. We have a global team of engineers throughout the world. Our headquarters are in Silicon Valley, where we have the majority. We also have a non-trivial contingent in Boston, spread throughout Europe, Japan, and then we have a second office in Singapore as well. And we're driving force behind much of the robotics software that's out there. The two projects that I mentioned a little bit earlier, we have ROSS and Gazebo. ROSS is the world's most widely used robotics SDK. It's open source with a liberal license for the majority of it. We support the greater community making their own releases of software, and we'll help them redistribute it and they can put their own choice of license on it, but we highly encourage. We were originally BSD and now we're recommending Apache 2 to keep it permissive and let people continue to use it on their robots in their applications. Now, in addition to ROSS, another major project that we use and both use and develop is Gazebo. It's an open source physics-based simulation for robotics. Basically, at every time step, it's doing F equals MA and solving the physics of the system. This looks a lot like game engines, but we work really hard to focus this onto realism as opposed to looking good and having good gameplay. There are many projects that can use Gazebo as a testing platform, and our goal is that the software that will run on the real robot can run on the simulation without any changes. We're not at the fidelity level that you can tune the robots necessarily in the simulation and expect it to run in the real world, but if it runs in the real world, we want to have it run in the simulation. This way you can run continuous integration, testing, et cetera. Both ROSS and Gazebo are designed in following the open source ethos of having small modular pieces that you can put together and pull apart as you want. Gazebo has a modular physics engine. We actually support three different physics engines. We can change out the rendering engine. Say if you want to use the latest NVIDIA hardware with the NVIDIA rendering engines that they provide, you can swap that in and we can make that happen because we're open source through and through down to the core. So just a couple of statistics. Our users, our downloads are something in the couple hundred thousand per month, millions per year, and we're growing. We've been growing since the beginning of the project in 2008, 2009 exponentially, and it's kind of amazing. I was looking it back at some of my slides from 2013, and the numbers were much smaller. We have 19% of our users in the US. It's our biggest market. China, the numbers there are large, but there's a lot of challenges of getting through the firewall. Open source projects still have problems with visibility into there. And all of these numbers are modular, the fact that we are an open source project and we promote people using mirrors and proxies and everything. So these are numbers that are actually hitting our main servers here in the US. We don't know anything about what people are doing on their mirrors beyond that. Just to give you a little bit of background history on ROS, we started off ROS at Willow Garage and we wanted to be open source and what do open source projects do? They go and they release on SourceForge. So that's what we did. We created several repositories on SourceForge and that's where we started using Subversion back in the day. In 2009, we released our first paper on ROS at ICRA, it was a workshop paper. That paper is now approaching 10,000 citations. I think it's going to cross sometime later in the year. And in 2012, we started ROSCon. This is a conference like this. We modeled it on this or PyCon to be a developer conference, not an academic conference where we really want people to come together, talk about how they're using it and learn from each other. We got several milestones here and I want to point out just this spring, our latest one we got published in the journal Science Robotics with a talk about ROS2, which I'll get into a little bit later. So at Open Robotics in ROS, we put together a lot of effort into understanding where the robotics industry is going, how do we facilitate it. Our goal is just to make sure that open robotics products are as useful as possible to the community. And we have a large suite of tools. The original vision was to provide the lamp stack for robotics. So if you think of what helped kickstart the internet industry is having the lamp stack there, so you can just take it off the shelf, have your idea, put pieces together and deploy it. And so our goal is to have those same things to provide for robotics applications. So if you have an idea of what you want to do with a robot, you buy some parts, you grab some open source software, you glue them together and build yourself an application, hopefully overnight. We have some good test cases, case studies of robots like Tali, which is shown here. They were able to get to a minimum viable product in something like 18 months. When, if they went and looked at all the open source software they used, it would have been more like 22 years. That would have allowed one small project with three employees to get really far really quickly. And talking a little bit more about ROS, core services for robots that I've been talking about, we'd like to break it down into four major subsystems. There's the plumbing. This is basically the middleware, the messaging. It's how things get from A to B. The infrastructure that we help deploy to help people deploy things to the ecosystem. But then on top of that plumbing, there's also tools. We have a 3D visualizer. You can grab it. You can create your own plugins for your own data types, but all the default data types have their own plugins already and can just render. There's capabilities that build on top of that. Two of our flagship capabilities that are integrated are the navigation stack. So if you want to have a mobile robot driving around, the navigation stack is there. You can take it off the shelf, customize it, parameterize it for your robot, and it should drive. Similarly, if you want to do manipulation, there's the move it library. And it's a whole group of packages for arm manipulation. So if you want to do pick and place or any of those things, there's a lot of capabilities and move it that you can just pick up and use. And all of these go into the ecosystem where we package something like 6,000 different packages for the greater Ross community at Open Robotics. And we distribute those on our servers, use our mirrors at OSU-OSL, and make that available. And this has become a really great collaboration technique for people. Once they release it and make binaries available, that you can just sudo app, get install, it's much easier for people to collaborate because you just have the binaries, pull them down, and you don't have to worry about it. I've actually seen people like you're closely collaborating with people, even like next office, next door. I fix a bug and tell them it's pushed to source. And they say, well, when's it going to be in binaries? I say next week. And they're like, don't worry, I'll wait. It's just that much easier. So at Open Robotics, we do consulting to keep the lights on. We're actually up to 50 people. And through that, we're always supporting companies that are using Ross and, well, as government agencies, et cetera. Companies come to us to support them to make their product happen. And what we do is we focus on finding the things that are pre-competitive for them and not their differentiator so that they can come to us, work with us to build up that core infrastructure and then get their product to market while they retain their IP separately. And we work hard to make sure that everything that we do supports the open ecosystem, if not exactly fully open of what we're doing ourselves. Quick slide. Thank you to many of our sponsors, et cetera. So now I want to talk a little bit about where we were and where we're going next. So Ross is used everywhere. Land, sea, air, space. It's really awesome. Just to give you some examples, we have Apex, COBOL, lots of people doing autonomous driving, indoor robots, logistics robots, all on the ground. Call out some specific ones. In agriculture, we've got a couple. And in material handling, that is a great large robotics market. Amazon and Kiva have sort of set the tone, but there's a lot of people that want to do that outside of the Amazon Kiva ecosystem, especially since Amazon wants to keep that mostly internally. But what I want to point out about many of these things, and these ones I'm calling out are mostly looking back at people who have been using Ross One. Ross One was developed at Willa Garage. And our original target market was the magic grad student in the lab who knows and can figure out how to make the demo work. And we wanted to very specifically enable them to build robot demos. And that was a very effective thing. We got into the grad students. We got into robotics research labs throughout the world. And we've actually reached pretty good saturation on the research community for robotics. But the research community is a very small part of the overall robotics community. And so in 2013, we held a workshop called the Ross For Products Workshop. This was motivated by some of our collaborators at NASA as well as sponsored by Bosch Research in Palo Alto. And they saw that there was a gap between Ross being used for research and Ross being used for products. The very good example of this is Bosch Research in Palo Alto built an autonomous lawnmower using Ross. They built the hardware, put Ross on top, did everything. They had a fully functioning demonstration of an autonomous lawnmower that would mow lawns. And they took it to Bosch Corporate in Germany. And Bosch Corporate says, great idea. We love it. And throughout all their work. Because the product managers at Bosch in Germany said, this is unproven software. We don't know where it's coming from. It's works and research. We're going to start from the ground up and build it from scratch. Because we want to do that on this lawnmower. And that is how product companies work. And they were like, oh, that extra modularity you get with the research that's really valuable for research. One of the things in their original paradigm writing Ross was that we won't wrap your main. And this was that you can do whatever you want. You can write your main however you want. And we will enable you to do any very powerful things. But in this workshop, we actually learned that not wrapping your main while very powerful is not what product managers want to hear. Product managers want to hear that you will follow a very strict template. It will do all these things, A, B, C, and D. And that engineer one can pick it up. Engineer one can write it. Engineer two can pick it up. And engineer N five years later will follow the same procedures and policies. And so this is one of the things that we learned from this Bosch for Products workshop is that we really need to constrain what people are doing actually a little bit more. We also identified that we wanted to be able to target embedded platforms. Cost is a huge thing for products. Research doesn't care as much. If you're buying one off, it doesn't matter too much. We also pushed for multi-robot, real-time, and cross-platform. Back in the day, we were purely Linux. So with those, we came back and reformulated Ross 2. And Ross 2, we basically started from the ground up. We picked a new middleware. And we kept all the standard modularity and have a migration path for people to move from Ross 1 to Ross 2. But we really wanted to focus it on being able to support those companies going to product. And one of the ones I'm really happy to say is now, if you buy a Roomba, an i3 or i5 or higher, it's running Ross 2 under the hood inside. And this is something they haven't said publicly. But you won't find it anywhere in any of their documentation or marketing, et cetera. And this is actually exactly what we identified. And Ross is an enabling technology. And they just use it on the hood because it's more efficient for them to develop and use in their product. They see the lifetime of it, et cetera, coming together. And they're going to keep using it. And so now we've actually been able to get into the commercial market this way. And it's validating that our redesign of Ross to Ross 2 has actually hit the commercial market. And the really cool thing is actually the Create 3, which is their educational platform, has a Ross 2 interface. When you plug into the robot, you just talk Ross 2. And so they've actually able to expose that out to the educational research market. But it's just not in the, and it's under the hood in the commercial product. The other one that's picked up Ross 2 and run with it in a really great way is apex.ai. They've taken Ross 2 and modified it for their purposes, again, leveraging the permissive license. And they have actually created a platform for autonomous driving. And they're even looking beyond autonomous driving at just general autonomy ground platforms. And this is really cool. They've taken Ross 2. And they've actually pushed it through to get ISO 26262 approval as well as ASIL level D approval in Europe. This is a huge achievement. And part of what we were looking at for doing in Ross 2 is be able to support people to do this. And it's been really empowering for us to see this. And now we want to sort, now that Apex is demonstrated that this is possible, we're now looking at more how we can go in that direction. So coming back to what I was talking about different domains, I'm going to skip C and now focus on air and space because we've had a long history of using Ross in space. In 2014, Robonaut 2 actually got launched to the space station and was running Ross. It's actually interesting the Ross went up. You can see this Robonaut has some funky legs. Ross went up in the update that brought the legs. Originally, it was just a torso on a pedestal. But they had proprietary software that was driving the platform. And the company that was making the proprietary software didn't go under, but they weren't making money on that product, so they just discontinued it. And so NASA actually had to come in and do a full software update on it in space with Ross inside of it. It was definitely not an OTA update, though. But continuing that, jumping forward to 2019, Astro-B is actually on the space station as well. And it's running Ross. These are small little cubes that float around in the space station. And their goal is to be able to do servicing and monitoring, basically, of the space station. They're kind of cool. The early versions of these had little CO2 thrusters. But these ones actually now have just little fans. Because you're inside of the space capsule, you can take advantage of the fact that there's air there and just have little fans that blow air to the side. But you're weightless, so you can float around. It's fun to test that on the ground, though. So we've had this long story. And Robonaut 2 from 2014, I believe they're still using the same version of Ross from 2014. Things don't upgrade in space very often. When we look at the space, but also in the air, I don't know if you're familiar, but with upstairs on the west end of the wing is the PX-4 developer summit, which is going on right now. This is a longstanding open source autopilot project. And they, in their research platforms and commercial platforms, are now integrating Ross. And they've been doing that for a decade. Elroy Air is another cool example we just found out about this. It's one of the Silicon Valley autonomous flying vehicle startups. They're focused on doing cargo drones. So you can see they have that sort of tray on the ground. You fill that up with stuff. The plane will drive over it, pick it up, fly to the destination and put it down and go pick up the next tray. But the really cool thing is that they've been able to pick up Ross, too, and use it. And they're using it for all their ground operations. The navigation stack that I mentioned earlier, they're actually able to just pick that up, parameterize it for this airplane and drive it around on the taxiways and runways. And it's a cool way to reuse those capabilities. And the other thing that PX-4 has integrated well with Ross is it's designed to be also modular, et cetera. So we're actually working with the PX-4 community to have systems that are hybrid with both a backseat computer that's running Ross and Linux and the main autopilot that's running on a dedicated microcontroller. And the PX-4 community and Elroy Air, et cetera, can all just pick up the pieces of robotics that are available from the rest of the community, things like pick and place, machine vision, object recognition, scene recognition. So going back to space, we've been working closely with NASA on various projects. One of them is Viper. I forget what the exact acronym means, but it's looking for volatile chemicals on the moon. This is a system that's going to be launching in a couple of years. And when they say volatile chemicals, they basically mean water. And we're working closely with them and using Ross on their ground systems for now. As well, we're using Gazebo. Our simulator is being used heavily with them. There's been a couple of talks on that. And we're also integrating with CFS, which is something flight system from NASA, as well as YAMS, which is the mission control system that this group at NASA is choosing to use, as well as Verve, which is another NASA framework. For missions, then, to be on the vehicle and in flight, there's a whole other level. And it needs to be space qualified. And we've been working with NASA on this one project, but we really want to take it to the next level. And so we have a new initiative that we just announced last year called Space Ross. And basically, we want to take portions of Ross and get them space qualified so that NASA and others can take it out to space. And we're doing this in collaboration. We've kicked this off in collaboration with NASA and Blue Origin. We want to basically be able to have all these reusable modular components in the open source ecosystem available for you or whoever's doing development of tools to pick up and use in their space applications. And once we have that and we have this modularity and we get it to be flight proven through NASA, it will become much, much easier for doing this. So as I said, we want it to be qualification ready. So that means that we need to start thinking about how to deal with DO-178C and NPR-7150.2. A lot of this that we're doing right now is to do things like static code analyzers, linters, et cetera, to bring the software quality up. We've been doing a lot of this in Ross 2 already. But now we're actually going in and saying, what do these standards require? How can we make sure they're there? How can we make a dashboard to convince people that it's actually there? One of the challenges we've seen with open source software that I'm sure many of you understand is that it's there and it's high quality. But you need to be able to communicate that clearly. So we're working with these teams to do that, as well as when you want to get to space qualified, you need to start dealing with things like memory allocation, exceptions, making sure that you're not doing any of those at the wrong time. So the other thing we're going to need to do is also provide standardized modules that are useful in space. It turns out that nobody else needs free space docking except for spaceships. But those are modules that can be moderately easily, software modules can be moderately easily written, standardized, and then shared between projects. So what have we been going on now? We have set up a GitHub organization. You can go there, test it out. Continuous integration, we have Docker scripts that are Docker containers where you can test out the stuff that we're focused on right now. The linters and code quality scanners that we're looking at to integrate are many of the ones that NASA uses internally. Most of them are open source. So we're setting up the open source tool to just integrate that into our ROS2 development cycle so that as much as possible, we want all of these qualification and certification processes to be integrated into the larger ROS2 ecosystem. Even if we don't push all of the packages to that level, we make those tools available. One of the ones we've been doing recently is we've been pushing hard to make sure the serif output for static analyzers is universally available in all our different linters and analyzers. And this is actually something that for a long time, we've had different linters and analyzers, but they dump out in different formats that are not compatible. And so you can't actually compare apples to apples. In addition to what's going on, we're also, I mentioned, going and removing dynamic memory allocation where necessary and figuring out how to make sure that that doesn't happen during flight, improving our logging systems, and the dashboard I mentioned for reviewing the serif output so that we can pull everything together and make sure that it's all working together. And so the next steps that we're looking for is more people, more collaborators. We want to build this as an open ecosystem. And so as we start stepping through this NASA Blue Origin, we're also looking for other people who would be interested in joining us and going through this process, picking off pieces and building upon this as well. I mentioned the GitHub organization. Here's a link to Space Ross. Please go check it out there. We'd love to have more contributions, et cetera. Well, actually, I'm running a little head schedule. Does anyone have questions at the moment? Yeah, so the question is, what in Ross 2 helps with the certification process? And a lot of it is that we've worked hard to build from the ground up with intention for going towards certification. Obviously, one of the big challenges we have is that you can't certify software libraries. You have to certify applications and the whole product. And so what our goal is, is that we're gonna be working through getting the requirements in there, getting the traceability in there. Our goal is to actually set up a full quality management system at Open Robotics to help support the generation of safety documents that then you as a company could then pick up and use to support your application for certification. We've also worked really hard to make sure to keep our dependencies much lighter weight. The modularity is really good for being able to strip down. We have something like 1,000 packages in the main release. But if you say, I just need publish and subscribe, you can probably bring it down to like 15 packages. And we're gonna work hard to get those core 10, 15, 20 packages up to the certification level and then you use those in safety critical systems and not the other ones. But I think a big part of this, our push for this is that we want to make sure to maintain compatibility with the standard open source system, especially wire compatibility. So that if you have a system that you're pushing towards certification, you're running the certified code on it, it'll be wire compatible with the uncertified system, which means that you can break out your developer laptop and just bring up the 3D visualizer and look at what is going on on my system. And so we don't, with that sort of separation, we don't need to worry about certifying the 3D visualizer because that's not gonna be running on a production system. That's not gonna be in the safety critical loop. And so we can have this nice system where developers and researchers can run really deep and really fast and then we pick the part that's gonna be safety critical and make sure to clear that and push that through the process. And the other one is depending on the standards, I'm gonna get into this for shortly, different components can be certified at different levels and with our modularity, you can really focus in on that. What is Moveit? Sure, so Moveit is a motion planning library, I think is the best high level title for it. And it is a, basically if you wanna, it's designed to be able to do motion planning for things like robot arms. It can actually be used for full body control but the most common place is say you have a robot arm, it's bolted to a table, you've got some sensors that are detecting the world in front of you and you wanna pick up the can of soda and not knock over the jar next to it. It will let you, it's a bunch of type libraries and tools that help you do scene recognition, detecting obstacles, detect objects that you wanna pick up, do collision-free path planning to get there, grasp it and pick it up and then like attach, we attach the object for path planning to the gripper and then plan another path to the final destination holding the object that's collision-free. And so it's a framework for doing that sort of operations, those sort of operations. So actually that's back to the question is, talk a little bit more about real-time. One of the aspects that we built into ROS2 is we built it with the idea that people will want to use it in real-time and real-time applications and that is where you need to do things like not have dynamic memory allocation at runtime. So we made sure that we can do things like you can swap the allocators out and change those inside of the core implementation. We know that depending on which standards you're using, using things like the STL is verboten. And so people like Apex have actually replaced standard STD star with their own implementations that are guaranteed to be real-time and we've worked hard to make sure that we have the modularity to be able to swap that out. 99% of the people that aren't in a safety critical system are gonna want to use the standard tools that come with their system. But we have that modularity designed in because we were thinking about this ahead of time to be able to swap out those components, to be able to support real-time, et cetera. Cool. Anything else? So I'm happy to say that we have a new initiative. We've had the Apex pushed on autonomous cars. We have Space ROS going forward with Space and Aerospace. And we've had a bunch of outreach based on those previous efforts to us from robotics medical device companies. Most of them actually surgical companies because they're the highest level of complexity. And with that high level of complexity, they can actually get to the point that these general purpose tools from the robotics ecosystem are valuable to them. Being able to bring those in and integrate those into their systems is very valuable. And so we're starting a new initiative to start pushing ROS to be able to go through IEC 620304, which is the standard that FDA will want us to standardize on. We're also talking to a couple of European customers and there's gonna be, I think, we're also likely gonna push for CE certification as well. But focusing on the US side, FDA, the IEC 620304, again, the nice thing I mentioned this earlier is that we have different levels of certification and you need to have level A certification for tools that are used to support your certification process. But you either need level B or level C for things that are gonna go run on device. So our runtime libraries need to be certified up to the highest levels. But things like our build tools and our simulations don't need to actually be certified up to that full level. They need to be certified to the level that we know that if they fail, we'll know they failed and it won't cause any damage. So the simulation, if your CI fails, it's not gonna hurt somebody. Whereas if the arm fails that's holding the power drill next to a person, that's bad. So we need to make sure that that is safety critical and will not fail. And as I mentioned, we also have the ability to have all the developer tools, they don't even need to be certified because they won't be on the system or used to prove that the system is safe. Again, what we're working hard to do is make sure that everything in this actually will be open source. We wanna push back as much as possible. There are things that we may need to have modularity or switch out, things like swapping out STL. We don't wanna do that for the mainline developers, but we wanna have that code in the mainline so that someone who wants to could just change that pound of fine and swap them out sort of thing. I mentioned we're looking to set up a quality management system to help us get there, improve our requirements and traceability functionality, and also of course testing and coverage. We're gonna need to have basically 100% testing on anything that is inside of it. And then we will be able to produce safety documents and safety justification documents that we can then provide to companies that want to leverage it and integrate it into medical devices. The business model we're using, obviously we're open source, we love open source, but there's a lot of the overhead in this administrative stuff to create those safety documents, et cetera. And so we're looking to have all of the core software be open source and available. But if you, and you're welcome to go generate the safety documents, safety justification documents yourself, but we're gonna be looking to create those as a product that we can then sell to companies as well as provide support for those systems. So it's an open development process. We'd love to have everyone, anyone get involved. We're also looking for partners, companies that are looking for using ROS and some of these applications that wanna be certified. We're looking to do a collaborative effort where if you contribute up front, you get discounts on the safety documentation. And we have some anchor partners, but we're always looking for more, so. If you shouldn't please read out to me and thanks for listening. Any more questions? Yeah. So the question is what operating systems are we looking to target on the microsite? So we've ended interested about QNX, RTEMs is likely there. Basically anything that has POSIX interface we can probably leverage. It's mostly gonna depend on where people building the products actually want us to go. We're already cross platform with Linux, Mac and Windows. So picking up various platforms shouldn't be too, too bad, especially for the reduced set. Like we're not gonna bring all the GUI tools to your real-time OS, but we expect we can target most of them. I mean preempt RT I think should basically just work. We run on generic Linux, so if you're using the patch, people definitely do use the preempt RT patch already. Any other questions? Micro ROS, the question is what's the status of Micro ROS? It is still active. Micro ROS is basically, we're building on top of DDS as our default middleware transport. And Micro ROS is taking that and using the DDS XRCE spec, which is extremely resource constrained environment, AKA a microcontroller, and it will connect over a serial port. And so Micro ROS runs DDS XRCE under the hood and can let you run a ROS type API on your microcontroller that will then talk, looking like it's published and describing, and that'll get forwarded out over the serial port onto the main network, so that the other end is opaque to the other end, whether or not you're a microcontroller listening over a serial link, or you're just listening on a UDP port. And that's part of how we're integrating with the PX4 team. PX4 is running on the microcontroller, and so we have that serial link that then allows them to talk. But they've gotten fed up with the speed of serial, and they're actually just integrating ethernet port. And they're actually gonna be, I believe running Micro ROS over an ethernet port because you can get up to the gigabit speeds. Okay? Thanks again for listening. I'll be around hanging out if anyone has any questions individually after it as well.