 Hello everyone It seems I'm on so I can even say you completely turn me on My name is Petr Strakos. I'm from IT for innovations National Supercomputing Center in the Czech Republic We are part of Technical University of Ostrava and the topic I would like to present today is about trains or more specifically about Simulating the environment around the train and it's part of a bigger project which We participate on with private company Experta and this project was also supported by technology Agency of Czech Republic and before I move on I will also Express credits to all my colleagues which cooperate on this project specifically to get the set yeah, Marta. Yeah, Roche Stand up B. M. Elena Esco who did mainly the implementation work also to Milan yeah Roche for his support in area of rendering and also To the guys from Experta company namely Tomasz Kulig and Petri Allen for they work on 3D assets and for the valuable ideas Regarding the design of the whole thing as I said the topic of this talk or specifically the software simulator is part of a bigger project and It can be said that this project has three main activities first one is Development of working sample of a rail rail vehicle that would be able to capture Data during its journey second task is creation of an obstacle detection system That could be then used in a real train and the third but Last and but not least is a development of software simulator That would be capable of generating training and validation data That could be then used for the development of an obstacle detection system to have an idea about How it might look at the end You can see the video But let's move on The talk is about the software simulator we have been developing so as for this task We focused mainly on Several things we were about to create a Software that will be capable of generating virtual data from Railway track Such data Should be then used to support the creation of an obstacle detection system that would then serve in the real train We should allow within the simulator to simulate different critical scenarios under different conditions and We should also ensure quite high variability of the outputs But on the other hand Try to limit the complexity of of the simulator So what was our goal basically our goal was to create a 3d environment of a rail rail track as a virtual replica of a real track The outputs from the such simulator Should be of high quality and we should allow to simulate various sensors Similar to those that are being used on that working sample, which I was mentioning at the beginning So what was the tool or what specific tool? We were deciding at the beginning of The development of the simulator of course we were Deciding or considering Some game engines Like engine or game engine number one and number two you can substitute based on your preference we were also Thinking about using blender for the whole whole development and We stick with blender. We decided to use blender for the whole task and Mainly because we already had some past experience with blender. We did some development. We also cooperate it with Blender Institute and We have most importantly supercomputer available for generating the valuable outputs and We also have developed we call it rendering as a service. Basically, it's Modified version of a cycles renderer, which we can effectively Use on the whole cluster and to the rendering using cluster So that was The main motivation to use blender and now to the job basically What's inside? What's the design of the simulator? Since it was supposed to Provide kind of virtual replica of some specific Part of the terrain in the Czech Republic. So we try to use as much Digital data available as possible So in the simulator we are using ultimate three rasters We are also using vegetation rasters from open-street maps to populate the vegetation on the terrain and as for the Path of the tracks and all the roads. We are also using geogson again from open-street maps another important part of the simulator Word to mention Are the 3d assets? Thanks to Very talented modeler. We have in the team We have I have to say really nice assets available Those assets are very specific because we are in let's say rail rail Problem, so it was necessary to create basically all of them As for the vegetation We cooperate with polygonic and we have very nice asset as well regarding the vegetation For the user it is important to have enough ability to Decide what should appear in the scene What should be this let's say the final scenario and that is being or that is going to be generated? So we have created Such a simple system that operates with jason file and basically it captures all the high-level information about the scene So this is At the input so basically we use that information from the jason file that tells the simulator What kind of track is going to be generated? What's going to be the data used for? Generating the train and so on and so on what's going to be for example the Exact time and date that is going to be generated and this data are then Interconnected with or this information is that then interconnected with the real data and This is then Processed within the add-on Which we have created Refueling utilize python API. We have created our own UI for the whole simulator and we process all the input data to reach the final goal of Generating the virtual replica of some piece of the track and As I said we mainly aim to get Specific outputs from the simulator there. They are mainly Image-based so we use cycles as a renderer to get the outputs Here you can see the outputs. We are trying to generate and here I put these labels to Basically distinguish or tell you that those specific outputs are for the pot third the process on the way after Rendering all the necessary buffers which we've worked with so basically the lighter The GPS position and infrared camera are further processed based on the inputs from the cycles render Here is short showcase Of How we populate or how we can work with a simulator you can even see the scale of the generated piece of the track It's about four times four kilometers and in this showcase You can see how we work or let user work with the vegetation the vegetation is being populated using geometry nodes We use also some kind of optimization prior to rendering we do call the vegetation. So basically What is not necessary to? Populate in the scene is not being populated to reduce the load And user also can manually enhance the scene by some manual modifications Another important aspect Possibility for the user to add the objects We basically distinguish between two types of objects either static or We used to call them dynamic, but it would be more precise to call them tweakable objects Here you can see the showcase how you can easily populate with the static object similar goes to the dynamic objects And after you populate the scene based on your preference, you can then save the whole scenario in the JSON file So next time you use the simulator you can generate the whole scene again based on the description from the JSON file Here are already some showcases from the simulator We are able to generate different Seasons Mainly because of different vegetation assets So, yeah, I think you all can recognize spring summer autumn and winter We are still working on it, but I think it looks quite nice as the first approach Okay We are also able to Generate the night scenes or the scene and simulate the nighttime in the scene From left to right top to bottom You can see the we call it classical RGB output camera output then ground through very important for Training of the detection system then the depth map and the last one is LiDAR another very important aspect of the simulator or what was required in the simulator Was the ability to interact with different kind of objects? Which is very typical when you Drive the train on the rails. It's interaction with the cars. So here is Scenario with the cars and another important possibility is to Generate the scene with some people in it and their interaction in front of the train So again same kind of outputs camera output ground through depth image and Generated LiDAR signal Yeah, we were playing a bit with this and here is the comparison between the real footage and the simulated one Let's say some intermediate approach. It's not Completely the same but we try to do the best and As I said at the beginning The outputs of the simulator should serve as either training data or validation data for The obstacle detection system. So here you can see from those three different scenes application of the detection system on Simulated data. So what is important? That the detection system is able to detect the most important objects on the track such as cars people and What was surprising for us even that horse during that night scene and because this Kind of Simulation of quite large piece of the terrain Is precious on one side that you can basically create virtual replica of Real terrain, but on the other hand, it's not that flexible for example when you Want to really quickly create much smaller part of the terrain and Let's say create kind of weird stuff on it so we decided to Create also a smaller or in terms of the size of the terrain smaller version of the simulator Which is completely based on geometry nodes and what you are about to see is Kind of promo video of this tool. So it's with the sound so enjoy