 Could even watch it in VR or so So let's have a look at how that algorithm can be implemented so the basic setting is you've got some virtual viewing position and You've got an image plane and Then you've got this data here in this specific case It is it is an MRI or a CT data set So you've got stacked images that you'd like to display Which which get that way for me uniform grid and what you want to do is you want to project that data to the image plane So what do you do? You are shoot primary rays. You should not only one ray, but you should are lots of rays That is for each pixel of the image plane And in this very specific case what you now do is you intersect each primary ray with the Bounding box of the data set see if it intersects if it doesn't just fine Then you will sign a background color or dump do something simple And if it does what you do is in this specific case you perform a ray marching algorithm to do piecewise integration And eventually determine the color through that pixel. Okay, so different setting Similar type of algorithm. This is on and walk through through an archaeological data set and the scientist wants to Recorded this on the desktop, but typically the scientist would perform this walk through in virtual reality Sorry Okay, where was I what was I so typically the scientist would Perform this walk through in virtual reality. I recorded it on the desktop so that you can better view it here So what what happens here is the scientist performs a walk through through an archaeological data set That was reconstructed with a 3d modeling software blender or so not sure and Then he wants to stop at a position and say okay. I want a beauty shot here I know head tracking will stop but give me a beauty shop. So Typical algorithm that we would use here is path tracing which is as you may know Implemented in blender or in Maya or in 3ds max but for illustration purposes The this type of algorithms quite similar once again what you do is you shoot lots of rays through the image plane and once again you want to determine a final pixel color and once again you have lots of inherent parallelism over over image pixels but the Algorithm is completely different because what you now do is perform some Monte Carlo simulation are sample some fancy reflection distribution functions Flip coins to get random numbers and so completely different algorithm, but same basic setting and Then what is possible is variations on this theme? So we have the same algorithm say path tracing, but we want to have another primitive one that software does not Custom software does not provide Cylinders or freeforms surfaces or so depending on our application and This is where a visionary comes in visionary implements a whole bunch of Algorithms in a generic fashion just like boost or the STL does for more For more general purpose so does visionary for Yeah for ray tracing related kinds of problems visionary has a strong focus on our real-time algorithms so this means that the The intrinsic algorithms that the implement are highly optimized, but it is not a necessity and also use visionary to write offline rendering to To generate those nice images, you know for Maya or Blender Um Yeah, let's have a look at one of the most basic concepts of visionary It's it's the scheduler and kernel concept We talked or I presented to you on one of the slides before commonalities of the algorithms and similarity in dissimilarities of the algorithm And the square scheduler and kernel concept tackles exactly that Schedulers are used to generate primary rays. Um, you configure your scheduler with a virtual camera position and Then the scheduler will do will generate lots of primary rays and depending on the hardware that the scheduler targets. This will be Hopefully highly efficient And in most cases in in parallel so and Then went when generating primary rays the scheduler will perform an algorithm some type of algorithm which will in most cases return a color or Maybe some depth or something like that but such a result and then the scheduler will write the color to and Something we call render target, which is an output buffer which can be displayed To a to a can can be did right directly displayed to the frame buffer or a kid which can be written to a file or so And then on the other hand we have kernels, which are our meta algorithms That is we have a whole bunch of algorithms that visionary implements such as texture access higher-order texture access Ray acceleration structure traversal ray primitive intersection and then the user can construct meter algorithms we call Sorry That are called at curls that get yeah exactly and the we call these meter algorithms kernels And they are executed by the schedulers and the schedulers write the final result so this is So I mean you have your target hardware GP GPUs x86 CPUs or so and then you write a Scheduler then or you have a scheduler for that type of hardware and This is great because then immediately all the algorithms that you have already implemented in in the course before Are immediately parallelized or optimized for that hardware and so this is great when I started to work at the Institute for all five years ago Okay, I will keep it short Okay, when I started to work at the Institute a few years ago, um, I I should have it was my job to parallelize direct volume rendering and then I parallelized it with a GP GPU program and And yeah, then a few years later. It was my job to parallelize a path tracing algorithm and and Then I had to do all the parallelization once again Which was I mean if I had could have written a revisionary kernel that day It would be it would have been great because then I would simply just have retargeted the algorithm and some used an existing scheduler So let me give you Tiny examples so that you That you have an idea of how this works as I said in visionary. Everything is a it's a template so so is the Most basic class in a ray tracing application array And so it's basically just a an origin position in the direction of vector and by Yeah, it's probably better I'm not sure maybe it's So and as I said everything is a template and by instantiating rays with templates What we say here is that we want to traverse a single race in ray tracing a real-time ray tracing and what a Possible optimization is that you were traversed multiple rays at once to optimize the data path and In that case by just saying for instance by instantiating this template with a SIMD type For instance a float 4 or float 8 then you would for free get our packet traversal I'm simple sketch is a scheduler One of the schedules I talked about before and simple skates the most basic scheduler. They kept that you can imagine It's just traverses the image plane and scan lines It has it is not parallelized, but this is great for debugging because there are no parallel threats or so involved And so and then we create an instance of the scheduler by passing it the virtual camera position and the aforementioned render target and then we write a most simple Ray tracing colonel the ray tracing colonel in this case is a simple C++ functional Gets its first parameter A ray the primary ray that is created by the by the scheduler and returns a color A more advanced colonel which in this case in here do some type of traversal with the ray Because this is the most basic colonel one can imagine it returns only the color white and that's it I have to hurry a bit, so I think that I will skip Few slides, so there are lots of built-in colonels and schedulers Highly optimized and some are colonels implement and the building colonels implement some of the mostly used algorithms But in general we expect our users to implement their own kernels, so they have this bunch of Intrinsic algorithms such as texture access Ray bounding volume hierarchy traversal and so on as and all those templates and we expect that most users will write their own kernels Yeah, there are a few built-in kernels that that deal with Give that mostly deal with surfaces. This is not a not a necessity. We have seen the volume rendering kernel from before I Think that I want to not go into so much detail with it with the following slide because it is Yep, in general those kernels can be retargeted our competitors typically have said say Only support triangles or so because it's of course an optimization You have your ray tracer and then you have lots of rays that take lots of fancy paths And then you have in your innermost loop in the ray tracer when you write a classroom ray tracer So you have lots of v-table lookups that determine is this a sphere Is this a triangle is this a busy a patch or so or is this some Primitive that was loaded from a dynamic object or so and this is of course a nightmare from an optimization standpoint so What most of of the other libraries that are the ray tracing do is to say okay We support only triangles and some without wanting to go into the further details We have means where we on the one hand perform the only triangle task very efficient no Dynamic branching and innermost loops but on the other hand we have compile time mechanisms that can help you to to Have multiple types of types of primitives custom primitives and intersect them with rays in a very efficient way For the details I'd like to refer you to the examples that come with visionary if you clone the visionary repository You will get all those examples. There's volume rendering in there There is an example to target for to have simple ray tracing with Multiple primitives and there are different lots of other targets examples then if you clone visionary you will also get a Pluckin for the virtual reality rendering software open cover which is also open source You can also download it from github and you can try if you've got an Oculus Rift and I think you have to compile it with windows so Drop me a line if you'd like to do so Because it's I think there are people that compiled open cover with windows, but most people don't and Yeah, this is the end of my talk and I'm happy to take your questions So we take two questions. Thank you very much Questions for Stefan. Yes Yeah, so you talked in detail about scheduler being adapted to the specific platform you were using like CUDA or FPGA I was wondering how do you tackle that for the kernels? Do they have specific implementations for every platform or can you somehow cross compile? Kernels that were written once to be able to compile onto an FPGA on CUDA on SSA Intel Yeah, we expect expect that a kernel is written in C++ 11. So we require a Surrounding framework that is at least compatible with C++ 11 For CUDA, this is this is the case other GPUs we target we want to target with us sickle the upcoming standard or the already specified standard which has no actual implementation and If you want to target SSE then of course you can do so because the Intel compiler or GCC Implements the C++ 11 standard, but you can't write your kernels in open CL also FPGAs are a different thing and we are working on this. There is from company Silencs, they've got a tool you can write a C++ 11 code in a high-level synthesis language which is complicated and will probably not work out the way we expect it because there are Parting to an FPGA is a completely different beast compared to a GP GPU So I'm leaving this out by saying okay kernels are C++ 11 code C++ 11 plus Maybe in a few years don't know right at the time it is C++ 11 and if you if you can stick with this then We can do with those get that we've scheduled a support C++ 11, then it's just fine So one more question if there is No for now, so Stephanie you will be around no so ask questions when you find them in the hallways. Thank you very much