 Hi, this is your host subtle Bhartya today. We have with us Adam the boss be VP of simulation and robotics at Robotech AI Adam is great to have you on the show. Thank you It's great to be here before we get started and talk more about open 3d engine Which is going to be the theme of today's topic. First of all, let's talk about Robotech.ai What do you folks do? I mean, it's in the name. So it's robotics and artificial intelligence What we mostly do we work with software. So Working on the robotics stack around perception and reasoning And we focus a lot on validation tools and chiefly simulation so how to Do R&D with your robots how to validate that they work correctly How to continuously improve and also assess productivity of your robotic use cases with the Benefits of simulation. Excellent. Thank you. And you folks are also associated with Linux foundations Open 3d engine. They have their own foundation focused on open 3d engine talk a bit about that 3d engine talk about the label of your association and When we talk about you know talk about the engine because sometimes when we talk about these the 3d engine most people think about just gaming But the fact is that it goes beyond that as you also talk of modeling and also Soon Apple will come up with the vision pro which means that you know VR 3d rendering a lot of things are happening in this space. So let's talk about number one open 3d engine Your association with the foundation open 3d engine Is the the chief project under the open 3d foundation? is For us a base platform for our simulations that we create for customers We focus on robot operating system, which is a framework a standard framework for robotics And we are making sure that open 3d engine is a very attractive solution for robotics developers who want to use ROS so robot operating system in their projects and We are a member company of the Linux foundation open 3d foundation and our focus is in the special interest group Simulation which I also chair. So as you mentioned game engines What we typically see as game engines and other examples being unity and unreal They are just More in more general terms 3d content creation tools to build simulated virtual worlds. So they are Often equally applicable for simulations. What kind of industries actually if you look at it almost an automotive robots You're I mean everybody's but what kind of industries because I just want to understand or explain to our viewers The scope of these 3d engines. They are not just limited to a niche. So robotics is a very broad field For example, as you mentioned automotive Includes robotics or self-driving cars This is a type of a robot Similarly, we see automation in fields such as mining where we have trucks or construction vehicles. So very heavy Vehicles very are typical in comparing to classical robots That also need to be Simulated because they are building operations automation of such sites Other areas include agriculture. Again, we are talking about larger areas and Machines But then when we move to things like warehouse automation, we are talking about mostly fleets of smaller robots that carry loads between different points and Then we get other fields such as of course healthcare customer robots such as automated vacuum vacuums And and there is many fields like that many application types such as inspection and maintenance where we have both airborne robots drones and ground robots Including robotic dogs Used for this kind of tasks So this creates a very diverse field it's Full of challenges and different applications, of course when we want to Provide tooling for them such as simulation have different challenges. There are different Key points to to address For example, one of the great benefits of our game engines Is the performance of simulating for simulating large areas multiple robots This is especially Clear when we compare these engines with classical Simulators that came from academia and are not coping with such tasks Let's just specifically since we are talking to someone from Robotech talk about the Scope usefulness of you know, open 3d engine of just 3d in general for robots robotics How does because robots when we look at this is something physical, but we are talking about rendering what we want to do when we simulate robots Is we want to take the software stack of the actual robot? Which we want to validate which we want to develop. That's the subject of our work, right and We want to keep it but replace the robot's body and the robots environment with virtual ones So that includes also providing sensor informations from this virtual world that look very much like real data Here of course one can consider things like photo realism of the engine So how good does the data look does it look like the real world can it simulate? similar effects of the sensor such as camera lens flare and distortions All kinds of noise that that you see in the real data. So these are very important aspects and What's important in simulation is of course you can Go very deep into realism. So what's the? Physical reality between the thing you want to simulate but from engineering point of view It's also very important to only simulate what is really needed to Validate that the robot works. Can we also talk a bit our distinction? And of course, you know apologies if I am going out of the scope of the description is that when we do talk about you know simulation Like for example, most of us don't know the internal working of you know, the modern robots that we talk about we Most we can think of the movie terminator back in those days are we look at Tesla Which is called so you can call Tesla also robots on you know, we'll because it is making a lot of decisions You know just it's our legs like Boston Dynamics. It has wheels So one is that the way we simulate how robots may look like and one could be the view that The word robots see because they have to make a lot of real-time decisions so when we look at these three engines is it for The internal workings of robots or is more about how people perceive them There is an argument to be made for each of course the primary function of the simulation is typically to validate The robot so it's important what the robot sees the There can be kind of user view in the simulation so that people understand what's happening was being simulated that things Work properly, but it's mostly about the sensor data. So cameras that are mounted on the robot It's LiDARs. So laser sensors and it's GNSS so global navigation positioning system such as GPS In all these sensor data that goes into the robot's reasoning system and allows it to make decision to perform work In this simulated environment much like it will in the real one, but also for humans If you are a company that wants to present For example to your customer how your robots will be deployed in their facility It's very useful to capture it in into a visual story. And of course then you can show The entire view of you know of the operation much like a game You would present to someone and they can really understand how this deployment will look like they can view it from different angles, you know and See the productivity gains through this Engine as a visualization tool. No, let's talk about, you know The technologies that you folks use, you know for this simulation and why you chose open 3d engine Why you chose to also become a member of Linux Foundation through this project? That's a very good question. Thank you So we have a field of simulators. So there are many simulators It's often called a simulator zoo The space so there is a lot of tooling to choose from and they have different strengths and weaknesses The type of use cases for robots that we as a company specialize in involve large areas a Lot of throughput in sensor data. So high fidelity sensor Automotive is a good example LiDARs and cameras produce a lot of data So for our company Performance was crucial. We wanted to simulate large worlds high fidelity sensors and multiple robots at once a Lot of classical simulators classic robotic simulators Instantly go out of Consideration because of that they just can't cope with this kind of requirements but game engines They were optimized for this kind of applications right for large worlds multiple characters telling good visual stories. So This this was the area that where we looked we started with unity 3d and Built our automotive simulations and mining simulations there, but we quickly found out, you know, there are limitations to that approach There is quite some technical details on why These engines weren't a great match Part of that is C sharp versus C++ which we prefer for robotic stacks due to robot operating system The technology choice the second part of course was the limited control on the in the engine code As well because it's not open source, right? We can't Do everything that we would like to we are limited to working with modules with assets building this this kind of scripting And so open 3d engine as an open source fully open source solution C++ engine Was a perfect match for us to build the kind of solution that we really wanted for our use cases to keep our customers happy And what is the reason for joining the Linux foundation? You can just consume their project. It's an open source community much commu much of it. This is community driven. So being able to take part in this community discussions in in the process of open source Content creation and choosing what kind of implementation to go with so that it's good for many companies so many users ensures that you know someone can take Over the parts of this code and contribute which for us, of course Is it's a kind of free value that we are getting and more and more companies are building their? Proprietary solutions on open source because of that There is a lot of value to be had from the community from other companies. It's like pooling resources Which allows you to compete with the biggest players there that build their own close source tags So to fully embrace this approach you need to engage within community and becoming a member of course grants you a certain privileges as well and You can be a part of Where the engine is going. Can you also talk about? How are you contributing to this project and foundation? We are Contributing quite a lot. So our main contribution is Enabling robotic simulation in open 3d engine We built So-called Ross to jam so robot operating system jam Jam is a kind of module that extends the engine. It's a highly modular engine so You can use it with a limited set of modules and you can add your functionalities to this kind of system so we built a Jam that includes all the necessities for a robotic simulation It's not just a connector in terms of to communicate with this robotic stack, but it provides a full kit to Implement sensors such as cameras, LiDARs, IMUs all this kind of the most popular robot sensors and also control methods, so Robot operating system typically has standard interfaces to make the robot go somewhere, right? We Implement not only the interfacing, but also the components in open 3d engine that realize that movement and it's both for the kind of like vacuum robots and Vehicle type of robots. It's a different steering, right? So we have built this Simulation gem Well, even a set of simulation gems because there's more than just that that allows you to Really quickly start with open 3d engine as a simulator Without that, you know, you would have to build all this yourself. So I think it's quite sizable contribution and We are continuing Work there and building on it further I'll go back to Robotech for a bit before we go back to talking about the foundation and the open-source product is number one is that When we talk about this simulation, can you also talk about the importance and rule of data? APIs and machine learning Absolutely. So one of the chief uses of simulation and increasingly so because of how Embodied AI is Rising as a topic with the new language models and people are looking for their applications it's only more important to Be able to provide a lot of data for the robots learning training data So that these robots can encounter plenty of situations and learn how to deal with them and this is a field Which is called synthetic data. So producing synthetic data from the simulation typically in large volumes much more than you could produce in reality because of you know Limitations of the real world you have to transport the robot. You have to actually You are limited by the number of robots that you physically have well in simulation you can parallelize you can Run many and once you can run faster than real-time if needed so that synthetic data is immensely attractive for the field of machine learning and Increasingly so not only visual data But the importance of additional modalities such as point cloud data For example a tactile so the sense of touch and recording this kind of data and using it to Build better AI models for robotics is crucial. So Of course having Good sensors so sensor simulation. This is the source of our data It's crucial because we need to capture what's important in This Sensor how they operate or what kind of data they produce to be close enough to reality So that it's useful for learning and then Being able to run the simulations at scale So integrations with all kind of scaling tools cloud solutions Being able to run headless and so and so on These are integrations that we also Already use our building and will continue building in in this year This is increasingly important as a company. We are embracing this wave of embodied AI. Can you also talk about? What kind of business model what kind of you know like because we talked a lot about the project that a technology side of Let's talk about the business side of Rope Tech and also How opens would because sometimes there's a misconception that open source versus corporate or business But the fact is that first of all open source is kind of driving the word today Number two is that if you look at Linux foundation organization with how it businesses open source will not succeed It will not be sustainable. So talk about that. Yes, that's a very good question. And I think that it's important especially for sales people who are kind of grew up or got experience a while ago To update themselves on the value of open source business models right now We as a company embrace it fully. So we We participate in several open source initiative Another one another example is Otoware foundation, which aims to build an open source Rost based Stack for self-driving cars. So that's an example of another initiative That we are part of Through this cooperation We not only built a kind of network of customers Where you know, there's always something custom that needs to be done But these are just services. We also built our products based on this open source engines open source modules where we have Open open core approach for some but we are also transitioning to providing services We foresee like like you mentioned the synthetic data Simulation at scale this will be using Increasingly using and they already are in automotive for example, this is already prevalent approach to use cloud computing To run many many scenarios to you know to increase the amount of simulation hours that you can Take out for you know for every kind of Software that you produce So so we'll be Operating also increasingly in terms of selling services But right now we already have our IP so our proprietary modules which extend what is Available in open source for example for sensors It's mostly about noise about simulating them in a more realistic manner While in open source, we have kind of perfect sensors, which are very useful that they are like they have interfaces for developers to build on But then the noise models are very specific to Devices often so we keep them as parts of IP Let's go back to and talk about the foundation of project a bit. We are in 2024 now Just share kind of the things that are in your pipeline the kind of road map that when we look at this project So for open 3d engine We are building three base platforms We are building a platform for mining For agriculture and for warehouse logistics. These are specific types of assets that are needed like 3d models and randomization tools for example to be able to You know run a lot of simulation in a useful way So we create these three platforms As products aimed at different application fields mining agriculture and warehouse logistics So that's already on the way and we Build functionalities for generative AI Such as large language models. We want these large language models to help our customers quickly create contents in the simulation such as Variants of the same environment For example, if you would like your robot to be able to clean Cantina what you would like to ensure is that it tries to do it for a million different containers with different layouts and different kind of problems and Generating these environments not just randomly but in a smart way Requires understanding of what the problem is This is an example of application of LLMs for simulation. We have several more Not going to talk about each of them But this is a really powerful wave of new technology that we want to Incorporate in into our products Adam Thank you so much for taking time out today and talk about normally the company But also this foundation that project. Thank you so much for all those great insights and I would love to chat with you again Thank you. Thank you. It's been a pleasure and also. Thank you for listening to the audience