 Hello, thank you. My name is Sebastian König. I'm here with my colleague Simeon Konsendorf We are from a small studio in Leipzig called blend FX. We are five people like three partners and we have two employees and We do a couple of things which I'm going to show you just very briefly So we do architectural visualization product visualization For example doors is a thing for us. I don't know why we are rendering lots of doors for a company and floors as well If we are not rendering doors and floors, we are doing a little bit of CG stuff There should be more in the video. I'm not sure why it's stopped. Maybe I can press next There is again So you can enjoy this again, maybe after that there Will be more stuff to see If not, I will just continue Telling you what we do. So this is like some CG stuff. We also do visual effects Which is what you hear for short films. So a little bit of compositing We also do Animations for TV and sometimes also cinema, but the other thing we do a lot is VR and This is what this talk is going to be about So VR is something that we really like especially mobile VR for cultural applications We've been here two years ago I think we had a little talk about one app that we did rochlitz VR Which is about a castle in Germany and they have a room which looks like that It's like very old and and rotten because they they wanted to show that the rooms in the castle go through different time periods and we are we have been reconstructing these time periods and People in the museum can go there and sit in a chair inside that room and then put on their glasses and see how the room looked like We like to do that a lot because we think it makes sense. It's not like just gaming It's it's for culture. It's people learn something from it So we really like doing that stuff So we were really happy when we got an email in March or February 2018 from from a producer and she wrote hey, I know your rochlitz project, which I like a lot since over a year I have been working on this project about Hotel Astoria. I just got the confirmation that I'm allowed to film there before it will be destroyed I would like to produce a VR app about it similar to rochlitz. I even got funding for a story development. Are you interested? Well, you have to know that the hotel Astoria is One of the most famous hotels in Leipzig is what it was built. It was built in 19 and 15 Especially in the GDR period it was a very famous place Fancy place like in in Leipzig in eastern Germany during the GDR period There were not many fancy places. So this was the thing Businessmen and celebrities from the west could then go there meet locals It had a nightclub in the basement, which was especially fancy there It might not look like much but back then this was really like wow so people go there especially the people from the West and meet with the locals there were also prostitutes allowed there But those prostitutes were working for the secret state police Because that was a thing there. There was a lot of surveillance in that nightclub So you have this fancy place, which was a little bit naughty as well But then always the subtle threat about surveillance and the secret state police So a super interesting place which has been abandoned since Yeah, since 20 years it has been empty and rotting along It was like the holy grail for geocaches in Leipzig or like adventures who like to go in lost places But it's really hard to get into it because it's very well guarded and so Yeah, are we interested? Yeah So We were happy to hear that Alina the film producer was inviting us to to visit the the place And at that time when she wrote us the email that was the stage of story development for for her She had been doing research about the hotel for a long time before and now she got funding for story development from the MDM the middle-deutsche median further on the mid German media fund and this funding goes in three steps so first there's story development and with that you can then apply for prototype development and If you finish a prototype where you can show the funding board that how your app or your product will look like Then you can apply for the next and last stage where we which will be a product development And that's where all the money will be so We were really happy that they were inviting us to collaborate with them Especially since they would do the paperwork Because applying for funding involves a lot of bureaucracy and paperwork. So yeah, we were lucky that this would be done by Alina Syuronek and Falk Schuster So from Huck films, that's the film company and they are they are the producers And they are doing two things the one thing is an animated documentary where they will animate and like a collage of interviews and snippets and the other thing will be the virtual reality app and both of these will Will not tell like a straight storyline or let one person the story of one person But they want to capture like the essence of the hotel and the how it was like being in that time period and Experiencing that nightclub So it's storytelling through story fragments. So to say so our job was to build an app that will Transform you and bring you to that place, but where we can also explore and watch different elements from that time so March 2018 that was the stage of story development and The production couldn't start yet, but we got the chance to visit the Astoria. So I will hand over to see me on now so I'm simian and As you heard we are very infected by the idea to create the Astoria in the 80s and We have the chance to visit this place, but only for some hours. So we have to be prepared So we have to rent a good camera. We have to prepare because there will be no light there's no power source in this room and We have to wear helmets and this is the day 14 days later after the email where we wear the blender branded helmet and Yes, we got a chance to visit this place and here's a little video snippet where you see Some parts of this day That's the moment we entered the nightclub and as you can see there's nothing more in You see the paintings on the walls, but no tables and yes No carpet nothing a very rotten place and Then we started and preparing our equipment here You see the special lamps we built these are in battery-powered flood lights floodlights pointing downwards to a big paper lamp and this this this big point lamps as you know Give us the chance to not have hard shadows in the whole area It's very soft lightened area and that's it's especially important to take and photos without shadows to use them as textures later words and most important the textures have been on the walls they are from a painter who made them in the 80s and Yes, folk take care of these and to preserve these pictures for our app and also for the future And at this point we gathered all the information we could from this room But still there was no money for us We have to wait until the next six months if there's a positive message from the MDM saying Okay, Alina and convinced us about the story now start the prototype and in end of 2018 we get a message or a phone call from Alina saying Okay, they liked the idea of reconstructing the Astoria. Let's start this prototype so in October we got the Message that we could continue working on it and then we did a little bit of planning and then in winter like November December 2018 we started building that thing so when I was there I took precise measurements with an industry standard note-taking device So I could have all the reference and heights and everything so I could start Reconstructing the room not the entire hotel of course just the nightclub and the bar I had an old ground plan that I could use as a reference and I Used that to to build the bar Also, we prepared the textures in Lightroom before to the especially the paintings to clear clear it up a little bit So that we could then project them on the wall with blender texture painting And you might wonder why we didn't just use photogrammetry, which is like a thing You can do that press a button sort of this laser scanning and then you have the entire room But yeah, then you also have just a very crappy Model with lots of vertices which you have to clean up in the end and since we wanted to Reconstruct the the building like it was Like back then like when it was still intact We had to remodel everything anyway, so we could just as well build it right from scratch And the room was so easy to build that it was quite straightforward anyway Also, we want to go to a game engine with with the architecture So you have to have clean geometry then so you have to model it anyway And of course we did it in 2.80. I mean what's good enough for this spring team will be good enough for us I assumed and I wasn't mistaken because everything was quite fine Is it stable I mean in winter that was not even better stage. Yeah, it was it was alright there were some problems but It went well and also we wanted to be future proof because we we knew that this project might go on for a long time so We want to have all the good stuff from 2.80 of course and it's not like Blender is unstable or more unstable than than other software In fact the blender 2.80 in alpha was still quite stable working not many crashes there I'm working and reconstructing the room was was kind of a detective game for us So we there were not that many references and we didn't really know how these Lamps for instance looked like exactly or what's that thing the pattern on that thing there? We didn't know so we we had to to dive in some references also that Wooden thing in the middle was totally unclear to us. What is it? It doesn't really fit there, but it was a thing where you put glasses and stuff on and we had to dive into some private photo albums from the Former personnel to really figure out how that stuff looked like Also Alina had been doing lots of research and they talked to the former personnel and some of them had some Props and some things there from the hotel so which you could pick to take pictures of this weird snail cutlery or or glasses So yeah, we built we reconstructed the room and all the assets with blender 2.8 ad and that it went well In November December and January there were some issues. I I couldn't really I Didn't really know how to use the collections and do the hiding hide from viewport hide render boss So I was lost There were also some bugs which made me like struggle a bit But thanks thanks to the developers and the fabricator side those issues were solved quite quickly So yeah, I really started to love blender 2.8 a lot especially the collections Like in this hotel room where we have all these assets and details doing this with layers in 2.7 You would be like a nightmare. So Thanks for the collections and in the 2.8 One thing that we are missing Sadly still is a queer rectangular rendering in in Evie. I mean Evie is nice But we have to output Stereo images a rectangular and we didn't have that also not spherical stereo rendering which is very important for us So maybe maybe in the future with Evie, but since until then we will use cycles Now just briefly about our 3d workflow For VR so the app will be for a mobile device So we can't have like that many that much geometry so we are pre rendering everything into 3d 360 stereo panoramas like that obviously not in an agliff but in top-bottom format and We of course we want to be able to exchange some elements. It should be an interactive app after all So what we do is to have like first our a queer rectangular camera and then we use a perspective camera With a very narrow field of view And then we can just render a snippet from that and within that area that this camera is rendering We can then change objects and render that so One thing that is important if you do that don't forget spherical stereo. It's important to have a seamless match but that allows us to have like a Small like interactive elements people drinking beer and we can just blend that in with the panorama in the game engine And this is the the interactive part that now Simion will talk about So now we have these beautiful So renderings panorama renderings out from blender, but then there is currently not able to create and virtual reality apps with interactivity so I choose unity as my game engine and you see on this side on your left side This is the something like the editor view the scene view and on the right side You see the game view what what the user would see And I don't start with the spherical renderings because they will appear later what at first I also import a low-poly or a Base mesh model from the hotel so that I have a reference in unity. We are to place the sound objects or other objects Basically when I have or at first when I start with the base model I start with the VR device and search for and the best positions where we want to render So I go through the room or a transport through the room and say this is this is a good position to See the pictures on a wall or to sit on the table I gave this positions to blender back and then we render the positions and this looks like this actually, I disabled now the The whole mesh and you see that the camera position is just inside on a sphere But some objects are real-time or the spatial audio Sources and you see there's the the ball above that the tiny ball. This is the disco ball and he's rotating so he's This is a real-time object. So everything that should be moving or which you should act and interact These are still 3d objects with a sphere When I move the position From one point to another point. I also move the camera and the sphere though. I always work in absolute space Same for the position of Interactive objects on a table. I take the table from blender important unity Then I can place the interactive objects on the table and in on runtime I remove the table just or disable the table and it gets in front of the sphere like you see here Actually, it's just floating objects in front of a sphere, but from the user perspective It looks like they are lying on the table You have to cheat a little bit like the shadows on the for the cigarettes I just use a blob shadow, which is a transparent transparent shadow But this gives you the illusion that it's really lying on a on the ground Let's see this in action This is an original receipt from we got from the employees And I copy the rotation of the controller that you can investigate the objects Yes, we just don't want to have only in 3d objects We also want to have a testimonial something like a testimonial in our app We want to add a character and not a game character. It wasn't fit We want to have a real person which worked at the Astoria which are still alive and This one this person mr. C was a bartender or bar mixer in the old times and he worked for many years in Astoria and That as we knew that there is a person and he want to work with us we had to prepare for this and This there's a recording from from mr. C in front of a green screen So we ordered the green screen you see behind him We ordered the lights and make many testing and the most important part is this this stereoscopic camera and This that's something special about because normally you would say okay take two diesel are or something similar and Make a stereoscopic Recording but because our eyes are in a certain distance of 0.3 centimeters These cameras must be very narrow Okay, it could be achieved with some lenses, but we have a very very wide field of view I want the person standing one meter in front of me That means a field of view of over 90 degrees and the lenses lenses wide angle lenses are very big We choose these cameras. These are from yee 4k plus is the model We printed 3d printed our own rig and these cameras have great features like 4,000 by 3,000 resolution and very wide angle and we can play back via in pysons script It was a good choice to take these and this is the output the camera standing one meter just in front of the person Which is Falk and he was my test person and Yes, we have this really I would compare it like we built cameras as They were our eyes. Yes, we have to be very Near to the reality and also the camera height where we film must be exact the same camera height Then we render the images then you get the perfect Illusion like the rendering Sebastian described we we make a perspective cut out from reality and This is you have to do some compositing like king and then There are not good and video codecs with alpha support which are running on mobile devices So we have to build in the key in the video file and the rest makes the shader So we you have color left eye color right eye and mask left eye mask right eye and the shader and combines this for for unity and Yes, when we prepared for some weeks to for this special day We invited mr. C to come to us in our office and we did the recording and in the middle. There's our Technical most important part the tiny setup for the camera and we had a great day and he also mixes us a cocktail He mixes 20 years ago called grüne Wiese Which is a green cocktail which is funny in front of a green screen, but because it was in front of him It was just a masking job And that's what it's look like in unity. You see we are showing a plane in front of a sphere Sounds very simple, but the illusion is like this If you not look to him he he makes some noise Good evening. I'm standing behind you Thank you now the part of the Yes, we made an app the people liked and they could interact with it and what's next? Yeah, so once we finished the The prototype Alina went to the funding board and showed it to them And then they had a little discussion and then you have to wait for a few weeks and then we finally got the message Yes, you will get the funding so We can now produce like The thing like the product which is super great Something that we didn't mention yet though is that we are producing for a special device Which is the the Oculus go which is a very nice device. It's it looks good. It feels good. The interaction is great So everything is really nice except for Oculus like they we need a kiosk mode Where you don't have to what the user doesn't have to interact with the Oculus interface So we just want to start our app and then you will never leave it you would just you can of course leave the and get rid of the HMD but You don't have to interact with the Oculus interface Sadly, they don't listen. So if you listen now, please kiosk mode is super important for us Anyway, we might switch to a different device if they don't listen to us so We got more money now, which is super great. We can now start working We have a little bit over half a year of time to to produce more Rooms like the bar is missing and then another part of the bar is missing We want to add more details, of course more interactivity and during that phase of working half a year, maybe more blender features would be nice. Maybe we get some Like for instance spherical stereo in Eevee would be great someone's listening or We are in blender. I mean we are all in blender, of course, but Like virtual reality in blender would be super great and apparently it's happening. So Julian Eisel is working on it today We had a little meeting about the future of VR and blender. So there is a team working on virtual reality interaction right inside blender, which for us would be of course amazing because we don't want to always go to unity and Have to interact with that game engine would be nice to to stay right inside blender to to do all these things that you have to do to to plan your scene and Place your objects So, thank you. Thank you, but no, no so there is there is a project going on and My Simion might want to talk a little bit more about that. We still have like 29 seconds. Oh really? Yeah But just fun 25 minutes That's the sound from Venus mixed reality starting at VR session This is demo I did for Julian Eisel where I test the open XR branch Which is currently only a special you have to compile it yourself But as Sebastian Sebastian said it was so interesting to meet these people like Yeah, Julian Eisel and Dalai Linto and William and the Ubisoft guys and I'm really looking forward to this because we need it stable and usable. We leave the material preview We need a controller input for transformations And the option to disable position tracking but we talked to them about all these features and I'm really Convinced that we are getting the best VR tool with blender in the next month and years and really they have a great Vision and this will be awesome Yes So no, thank you. Thank you Just to wrap it all up so what we are producing is like a something like a digital time-travel documentary I would call it so it all started with one Polaroid one one photo that someone back there in 1986 took and then through lots of effort and Interviews and research it ended up in in this like this future VR app, which we are really Happy to produce. It's been a lot of fun working with blender 2.8 It's going to be even more fun once we get the VR tools in blender The producers are nice Alina and folk are really nice to work with so that's it's a great project and Part of that was also like going to that place like back in time in a basement which in an abandoned building Super interesting and we even got to have like the last drink in the nightclub of the hotel Astoria. So Thank you for listening. Thank you