 Boom, what's up everyone? Welcome to Simulation. I'm your host Alan Sokian. We are still at COFES, the Congress on the Future of Engineering Software, for our second annual partnership with them. We are now speaking with Elizabeth Barron. Hello. Hello. It's so great to be here Alan. Thanks for coming on to our show. Yeah, for sure. We really appreciate it. We're very excited to talk to Elizabeth. She is 31 years at Ford. She is immersive visionary champion of the world. I love it. I'm very excited to talk to you about this. Her newest company, Immersionary, is JuFresh 2019 started and we'll be talking about that and the importance of this 3D storytelling, the user experience of that and the awareness expansions that we can get from that and the purpose of it. All this good stuff. Before we get there Elizabeth, teach us about your journey. How did you get to where you are today? How did you even get excited about engineering software? Isn't that crazy? So I started my journey really with the natural love of the physical world. So I love optics, light, like the way light travels, photography, all that good stuff and I also love math and so I just so I got a degree in math computer science and believe it or not got a job at Ford Motor Company which you wouldn't think having a computer science degree. I work at an automotive but I started writing CAD code and because I loved geometry and all that good stuff it led me into that world. So all my time at Ford I never really worked on a car. I was always working on tech that supported the people that were developing those cool vehicles and so I worked in this paradigm for like lots of years writing CAD code and then migrated into more of like the process and how you would use the tech and then started looking at the whole entire vehicle like chunking it together and working with the team of people to develop some ways of making all the digital data make sense to visualize and then from there I started thinking about virtual reality and how cool it is. Yeah take us take us through some of the the first times for you of doing the getting excited about optics, getting excited about math, getting excited about writing CAD codes, seeing all the data that comes from that, making it applicable in the design and engineering process. Tell us about how all of that works because most of us get into vehicles we've completely forgotten about the the last hundred years that has went into the slow but sure process of making the vehicles more effective more more safe with all these fun oops moments. Yeah so there is a lot that goes into making a vehicle a lot of analytics a lot of thought about the the beautiful curves of surfaces on the exterior there's a lot that goes into your experience as a customer like how are you going to enjoy being in cabin and what are you going to do and so the cool thing about VR and then AR the whole immersive paradigm is we can represent all of that together and so there really isn't a medium that I know of that represents all of that stuff puts it in context and then puts you as the customer at the heart of how you see all this data so there's analytics for you know structures like mechanical things there's packaging things like zones for like the way those parts move around and you have to make sure you have clearance and then there's all kinds of stuff about found findability ergo reachability all that kind of stuff and it's all together in this immersive environment but back in the day when I first started doing virtual reality at Ford it wasn't really ready for prime time so we could have 60,000 polygons in our entire scene what does that explain that so that is basically hardly any data oh sure sure right right so like in a whole scene 60,000 polygons so just to frame that in today's pixels kind of well like a little 3d environment so yeah so they're like triangles that yeah like tri strips and you know a mesh so a mesh that contains 60,000 shapes yeah but the thing that's crazy about that is now we have millions oh yeah tens of millions like 80 million you know so the difference is is stark and the tech when I first started working on it was really not ready for prime time but I could see the potential in what it had to offer and I'm always very thankful that Ford let me do it even though you know when you think about what I would do to you is I would say okay Alan I'm gonna strap all the stuff on your body and from head to toe oh back here over here give me this let me feel that and it was very physically invasive to get you in that environment back in the day and now you can literally 30 seconds get immersed in a scene and explore an entire vehicle and look at it in context and it's it's totally great yeah I love how you speak about the how the exponential technology curve affected the 60,000 polygons to tens of millions of them because the resolution the amount of data that you get from that is just so much greater and your ability to actually have your awareness expanded to new ways of thinking from that higher resolution because we're such visual creatures so so then take us through like when you're you know when you're first starting and when even at the middle and towards this end of this you know of the journey with with Ford and with these with immersive experiences what were some of these most important use cases because it seems like it can be used for anywhere from you know the actual manufacturing and to be able to have employees be able to visually see how they're assembling vehicles but all the way to like you were saying the customer the customer is able to see themselves in the vehicle where they're going to what was the word you used for how they interact with the vehicle yeah just the the Ergo or Ergo yeah the Ergo the ergonomics and how quickly they can identify how to do certain things right right so that's a big question so basically the way I approached bringing immersive tech to Ford was case by case and what would happen at the beginning is I would find a case that somebody needed to solve so they needed to know like that the findability and reachability of radio controls for a vehicle and so I'd say I can set that up for you and you can come look at it and so then we would tackle that and then that would be a use case that people would say okay that's good I understand in the beginning they would say that was very useful but I really don't want to do that again because you're really violating my space putting all this stuff on me but as technology marched on we were able to go you know in a move the tech so that it was less obtrusive and less like violating your personal space and then just more value so between the less feeling like a cyborg and then more content in it but we are it's so really and so the good thing was is that as that progression happened we were able to take the tech from these singular use cases and start blending them so then you could do Ergo studies along with design because there could be an instance and they're very often is that a design is really beautiful but it's not functional so how do you have each of those people the artists and the user experience person and the engineer that designs the underlying components how do you get them all together so that they all understand each other's perspective through immersion it is a great way of getting people together when they don't normally understand each other's perspective because an engineer comes in with a spreadsheet and then an artist comes in with a drawing or an animation of you know a model and then an ergonomics person comes in with anthropometry data on people and reach zones and their their language that they normally speak is all different but through immersion you get in you look around you see in context and you understand so the the answer to your question was like we started attacking case by case and then ended up with blending multiple functions together in an immersive environment and then you could really understand each other's perspective so the real value of the tech is as a communication tool I like to say for people that have unlike minds so then how would the tool of immersion experiences be able to help the artist ergonomist and the engineer communicate so they all are seeing the same thing which is the vehicle which is the thing that the customer takes delivery of and that makes sense to everybody so no matter what your perspective is the final result of your work is the vehicle you know in the case of Ford right and so they all understand what that would look like they they intimately are familiar with the final product and so if you represent all of that data properly in context put it together then it makes sense to everybody that's in the room and and more and the thing also about the tech is you can loop in people from other locations and have them see through your eyes and then understand your context for them somewhere around the globe and so then the one of the main one of the main ways of the importance of taking these unlike minds and bring them together is that when they're this is pre-final product there in the process of deciding where some of these functionalities go that type of stuff that they're able to all be in this immersion area experience together where they're looking at like a simulation of the final product right and then determining where it needs to go for optimal functionality exactly exactly they can determine where it can go and then they can cheat the natural world so I love that about the tech is that so okay I'm looking at this vehicle and it looks good but what happens at night what does it look like does it still look good if I put like mercury vapor lights versus you know fluorescent light model on it what does that do to the color how does that you know play with reflections if I am in a cloudy day situation versus a bright light situation what does that do if it's different times of year and the suns in different locations so there's all kinds of lighting things you can do and like you think about the reflections on your side glass of your like instrument cluster on the glass can simulate all of those things in real time and get all this data that you really couldn't get unless you literally were at that time at that you know those weather conditions existed or that's good stuff yeah so and then think about that with options option a option b I mean that the ability to like switch out geometry switch out where you are what you're seeing you know what types of analytics change the the display so that instead of looking at the beautiful sheet metal you're looking at lines that show the the surface continuity so now you're getting analytics then you go back to what it looks like then you switch paint colors and you know you know the natural world is the way you describe it which is very interesting that you can really quickly make these iterations in the design without even needing to and you can also test functionality as so of what it looks like when it's at night so you're seeing the function how functional is it still at night with a certain this is this is cool what is this our immersion experiences also used in both the the very early training process of someone in the manufacturing at Ford is it also used for them to better understand how the vehicle is being assembled is it used at the end of the process for the customer to be able to immerse themselves obviously physically in the vehicle but also virtually potentially in all in the ways that they can use all the components of the car yep absolutely so what's cool is you can actually get immersed in a sketch something you a 2d sketch just surround yourself with it wrap it on a sphere and get in it and look around and then start it volumetrically understanding your flat sketch really early before you've actually even created geometry you can do that and then use that paradigm all the way through the whole process so like very early on when you're ideating and creating like volumes and spaces and thinking about like the ways in which people might interact in your vehicle space that it's just a perfect way to kind of lay out the shapes and and form around the occupants of the vehicle and with so much going on with autonomous vehicles there's a lot more use cases we might be in a situation where like we are right now looking at each other and sitting on something that looks like a nice sofa like this but taking a ride so how do we interact in that environment which is different than the occupants all facing forward you know so there's a lot of you know goodness in trying to understand really early on how those things are gonna go and then the four chairs pointed towards each other exactly yeah and then movable maybe they move like this and maybe they go like this way and you know to your left is the Museum of Natural History exactly I mean there's and then what do you do with all of the space around you this the surfaces can they now have more than one function can they have some smarts in them or you know like I don't know what they're thinking of but there's a lot of possibility for how the surfaces in the interior of the vehicle get get used now so there's yeah for computers right or displays or you know all kinds of stuff and and and then also if you think about so that's really really early on when you're ideating and trying to understand what what potentially you might want to develop and then carry that through bed mode yeah exactly hit the button and recline give myself a massage you know that sounds good just free some lavender relaxing jazz yeah and then fast forward that through to the end of the process for like vehicles that are closer to being produced and then we can this is I just think this is amazing we could take the whole manufacturing process from beginning to end and come out with the tolerances the build conditions for every part to every other part in the vehicle and we could take that read it in and show it to you without any pre-work done to the immersive environment and then and then you could pick any components and then we could show you with you know the six sigma capability how capable the manufacturing process would be for the vehicle that you're looking at and what is the six sigma capability so so most so every car company I'm sure designs their products around this way of determining like how accurate you are so a sigma like three sigma and six sigma it's like a away right right okay and so it's basically like how capable are you so six sigma is like the most like the most capable in the process performing the absolute best right possibly right yeah and so you know and so you want to get to six sigma capability that way the vehicles making it like a hundred thousand miles and the ratings are off the charts right and the in the tolerances like the build quality between the components is tight and where it should be so it's as designed not as produced so are they the same are they different you know it's that kind of thing so we you want to be very capable in your manufacturing process and you want it to look really good so if you can take all of that data and you can actually watch like one component to another move yeah and then they might move this way they might move flush like up and down they might move back and forth but you can actually take any components like dump all that data in apply it to the model and then and then pick things and watch them move and understand how capable your manufacturing process is that is really cool because it applies every like the whole manufacturing process going through and and like this get attaches to this attaches to this and as that happens there's locators there are other things that determine there's jigs and fixtures that measure where you're at all of that is considered in the end you get this spreadsheet that tells you how capable you are but we can take that spreadsheet and apply it to the geometry and show it to you in an immersive environment so you see literally what the customer would take delivery of not what was designed and at the peak capability of the vehicle or not right if you learning experience yes well what if you run that analysis and you find that you're not optimum now what do you do so now you have a chance to react because you haven't produced the vehicle yet you have only predicted what will be but the thing about the tech about the immersive space is it may something that is the highest tolerance may or may not look good so because we're there are you know lighting like tricks with lighting shadows something might be hidden in shadow and never be a problem but in the past it might have been flagged as a problem because it didn't meet the tolerance band that we thought but it actually it's okay and then other things may meet the tolerance guidelines and be spot on and then you look at it and it looks off compared to the other things that are around yeah so then there's a functionality and aesthetic that is hopefully an optimal but there are sometimes trade-offs and these need to be analyzed exactly interesting now okay what were some of the other because this slowly led you into wanting to start immersion area your own company what were some of these other really aha moments for you that you were like okay we I really need to take my skills and start applying it around the world and helping people around the world with this process so a couple things one is the value of collaboration in immersive environments the insight that you get when you're together with other people and studying the thing that you're producing is unlike any other way of understanding and communicating and the value of all of the work that I did when I was at Ford with a team of really great people by the way nobody does anything by themselves but the the value of that and the reason why I was able to get the company to invest in it was about collaboration it was about how people work together how they share how they can be in the same location around the world so you could be in Germany and I could be in Australia and we could be in the same immersive environment at the same time that's really powerful location agnostic yes exactly so so that's one thing another thing is I developed most of the immersive tech and without being able to see it so I was stereo blind and so I couldn't see in 3d so I was in this 3d immersive space and developing it and part of the reason why I got into this tech was I knew I was stereo blind and I studied like what does that mean why can't I see whatever one else can see and what's different about what I see compared to everybody else yeah and I really enjoyed providing immersive experiences for people and having them react even though I couldn't see it so when somebody would get in and say this is so cool oh my gosh I would I would know that I was on to something yeah but in 20 juicy I would see it's like if you took your iPad and you walked around the world that's what I saw so kind of like flat but dimensional because it you could move through it so like a TV but you know you're directing it yourself so just a flat so like you would have no dimension to you yeah so but if I turn and I look at you it just be like having the iPad yeah turning and looking but is it like is it really locked in that way like for for for all of your sight oh yeah everything because my eyes were in different planes and so to form a stereoscopic image your eyes are I mean they're separated for a reason they converge on a point and then your brain determines how far away that is and but mine were like this so I could never converge on a point so I had to use this side or that eye but I couldn't use them together because they they literally were physically incapable of coming together to converge ever on anything and so in are there advantages to this as well no I don't think there's any advantage it's I was waiting for your head aches it was oh yeah no no it gave me headaches it was I had to concentrate really hard and like where my eyes were so that they didn't appear weird to people so I had spent a lot of time like you know moving my eyes to acceptable place but I had surgery in 2011 and 2012 that gave me stereo whoa yeah that thank goodness for science exactly that is why I wear glasses now because it so basically the surgery was taking my eyeballs out of my head the actual eyeballs like sucking them out and then disconnecting them like keeping the optic nerve of course but like just disconnecting them moving my eye and then sewing it back together and then putting it back in my head and so yeah and that crazy and so it had people have it you know it's very rare when like two to three percent of the population have it at birth and that was me people acquire it through aging that's quite a lot of people yeah yeah it is so they people are out there they they can't see stereo and you know but I was the moment for you when you fight when you woke up after the surgery and you were like this is crazy it was instant it was instant and so what I did before the surgery my my doctor was like you know Elizabeth you're pretty old I'm not sure we can teach you this old dog a new trick you know you might we might do the surgery and you might still have mono vision I'm like well that's okay let's I'm can we try so he what he did was he gave me these glasses that were like thick prisms and they took my eyes and they adjusted so that I could form a stereo image and I went into my car and I looked at the Ford logo and my steering wheel like popped out at me yeah and I just burst into tears and I'm like oh my gosh I'm calling my husband like take some time calm down don't drive but it was it was awesome but having that experience taught me the value of immersion in a way deeper sense than I ever understood and what it taught me interesting was that aesthetics matter and little lights and shadows matter and it is part I I see because I didn't have it for so long and then I had it once I had it I got back into the immersive environments and they didn't look right were before they actually look better and I'm like what you know what why do I think it's not as good and it was all of the detail that I was now getting about the lights the way light moves in stereo and the way it emanates and and so it led me to do real-time ray tracing and develop that concept and in getting in an immersive environment and then you know being able to slightly move your head and then see how light is playing because those sometimes those are illusions but those illusions are part of our reality and yeah and so between the there's so much shading right now that's happening on your face exactly and it's very important right it's nuance it's very beautiful that we've taken this long period of evolution to be able to see that nuance and all facial expression all lights and shadow effects and this is a crucial for immersion or experiences exactly exactly but yet they're usually absent and just until recently the ray tracing is kind of coming into its own but back in 2014 I was just like determined to make it happen and so we at Ford they that was another investment thing and they they invested in some tech and we were able to do real-time ray tracing in 4k so that was an industry first and it just showed the power of what it really should look like and so your question why you know why did I feel I needed to go out on my own I think that the time is now to develop standards for how we work how we realize like cross-functional collaborative communication in immersive environments and I think that the technology has come into its own and with innovations and AI using AI for ray tracing like doing that's called denoising which means like kind of getting rid of the clutter in the environment and kind of making some assumptions about what the images should look like that is really really cool tech and so it's taking it from large scale supercomputers and making it more useful and then in enterprise that's that's what they do they communicate they talk they share they they make something and then if you make something then you really want to see it before you make it and then see it properly as the customer would see it before they take delivery of it these this your push for for these new standards in immersion experiences is really cool to be able to now unpack with you what are some of the the standards you're talking about location agnosticism so people from around the world can collaborate in the same immersive environment what does the the version control look like what does the engineer how do engineering simulations make their way into these environments how does that actually turn into you know final products and data AI cloud all these types of things that are involved in this process what does what about your ideal standards for immersive environments so the immersive environments specifically for enterprise yes really require discipline and that the the thing about immersive tech is you might see a really cool demo but it's a one-off and so is it part of how the enterprise works maybe maybe now but is a really good application of immersive tech works how the enterprise functions it works with how they produce their data it works with how they like their milestones and how they do their process like for the development of their tech it works with the the multitudes of data that they produce so some is analytics so like CAE computer engineering work like stress and load and weight cost information like all of this other ancillary data that's different than the CAD data that is produced to engineer the parts there's a lot of data yes and if it all is they every enterprise knows their process really well they have this well-oiled machine to get to the final product but where a good implementation of immersion takes that data and lays a foundation and then feeds the immersive environment at different points in the process so that you're working how the enterprise works and you're not retrofitting the VR tech to the process so in other words if you start breaking that chain it adds time it adds cost and nobody wants to do it so they might do it once but they're not going to do it all the time so you really need to fit into how the enterprise functions and then provide them the capability to assess their data in the way that they are used to assessing it not in the way that the software has like whatever choices that your rendering software has that's not good enough it's you don't want that you want what they do so what would be examples of the way that enterprises would want to use the immersion air experiences in with the way that they could continue to use the data in the way that they're used to how would how would these use cases look like for the new enterprises that you'll be working with so one would be really about manufacturing so taking the manufacturing process like I talked about earlier and applying it to the design and then showing what that those build conditions would look like that's one another one would be if if you're designing a product and you have to rely on somebody's like somebody operating that your product you need to know that the the vision zones are good and that they can you know under reach the things that they need to and that's analytical information that is developed by other softwares but you would actually need to see the the vision zones kind of come into your immersive environment to make sure that that is you can see and you can reach and you can understand so that's more analytical information in that environment you might have quality data that represents like some potential damage that might happen so you know every every company wants to make sure that their quality is really high that their products stand up and some of the things that are could be damaged you might never perceive like a slight bending in a part you know that so you can represent that in the immersive environment so that's analytical information that actually morphs the geometry puts like a maybe a slight bend or a dimple in something and then you can look at it and say I can't even see that I've ray traced it I've looked at it from up close and far away I've looked at it day and night and it looks fine so it's really getting all of this data from all of these different sources into the model and then showing it as it would look if a customer was interfacing with it okay so then this is like the the merc the mercenary experiences are at this entire process of the product life cycle management they come throughout the whole process and then and then our our would you say that the what is now the cutting edge and where is it going with the virtual reality or augmented reality how is Australia person and Germany person in the same environment doing all the things that you're listing through what tech yeah through mostly VR so and you know wearing a headset being in a virtual space and then having locations known of where you are in the environment so even though you're in Australia we would know where you were with respect to this couch yes and I'm in Germany and you know where I am with respect and so I would see your headset floating in space and you would see mine and we could collaborate yeah I think a lot of the innovations in this space are really going to be through storytelling and providing an actionable way of you to experience the world so we're we have now a lot of potential if we do the hard work and we have a platform and a pipeline of getting a lot of the different information into those models like we just talked about but what's new is getting data in so that you can control your destiny so you can have an experience with the product so if you're in a plane and you sit down in your seat and you're in you know you're blessed with first-class you can kind of recline your seat and turn your display on and reach for this and that and all of the things around you are actionable so you can do them you can interact with them the way you want to not the way somebody told you to or not in a way that is a one-off because you have this like game control in your hand and you're like pressing this button doing that instead you just make things happen and then you so so it's taking the physical world putting it together with the virtual world but the physical world is actually just a representation in very like minimal form of what you're studying and then but when you're in the immersed in it it appears to you that you are fully in the physical world and then you can start interacting so doing a lot with actionable haptics that are passive so that you can actually feel and touch and then kind of control your destiny tell a story really get it that use user experience and then that includes integration with like 3d printing you know like a smart 3d printed objects that actually have microcontrollers embedded in them so they function in the virtual world and then do more with like where you place those things and how they operate and then the other big thing I would say that's coming is really like training models in AI so you're looking at things and then while you're assessing your product you're learning yeah and then while you're assessing your product somebody you know somebody the cloud is in context presenting you with relevant information yeah about about how the functionality or about the static and all these other types of data points that are really important and you're there with another person from another country looking at it and talking about what you're learning about what you're designing and engineering even pre-manufacturing so you don't make oops is saving lots of money this way this is like this is kind of like the dollars and cents side of things that companies that's why there's a lot of investment into the space because they know there's a lot of oops is that costs lots of money and save on that yeah interesting I like the I like the idea of storytelling a lot that that is one of the it always takes me to this like big history perspective on civilization a lot of a lot of I think of the roots of of the malevolence is that happen is because we potentially did not do a good job at explaining you know how we birthed from the cosmos and how it's such a it's such a you know such a blissful ecstatic experience that we get to even live on this earth together and so if we if we could do our best to make these immersive experiences that get people's awareness from birth to understand that this is such a whole experience for all of us including a mother earth and all of its biosphere that that that is those these are the immersive experiences that that I'm also as well as all the manufacturing design and yeah yeah there's like yeah there's some other really powerful immersive experiences that very excited for as well Elizabeth any other thoughts that you think we may have missed that you think are important to cover so on the experience immersive experiences in the end we are all emotional creatures that use our minds eye to figure out what we see in our world so we we our brain stitches together images and it makes things appear to us for our reality but our brains are always involved in what we see because our our visual acuity is actually like this two degree field of view and then our brain is putting together all the things we see so the reason why I bring that up is because our brains are involved in how we see the world it is that means that we have the capability for insight and we even though we see things we may look at a spreadsheet and like look at data and think that's just data when you see it applied it's a different set of data it's the same set of data but it appears different to us because now it's in context and it it has meaning and all of the things that go with being part of you know a person of earth here they all kind of come out and that's what I think the value of immersion is it allows you to see it in context use your insight and and really form use your mind's eye and understand something completely and it gives you such a better connection to your end customer whoever that is that you wouldn't have in any other way yeah yeah that's so so important and the looking at it on a you know on a like a 70 year timeline just going from such a flat pixelated 2d surface to now to now the full 3d indistinguishable virtual realities that that and augmented realities that that is that is so profoundly it's it's a too much different experience for our brains to engage with them exactly 70 years ago right yeah we were yeah and there's so much promise coming with AR I'm really excited yeah where that's going to yeah I hope it has a lot of especially focus and potential on all of the retraining that that we need for the automation AI robotics age yes yes there's so many jobs opening up that we need to retrain for these of these new jobs yes yes those with this has been such a pleasure yeah thank you very much I appreciate you asking thank you yeah thank you you're really pushing forward some really important topics and it's been such pleasure thank you thanks everyone for tuning in we greatly appreciate you we'd love to hear your thoughts on the comments below let us know what you're thinking go and share these conversations about these immersionary experiences the importance of them with your communities with your friends your families on the internet go and share support the artists and entrepreneurs that you believe in support co-fes support Elizabeth all their links are below also support simulation our links are below as well and go and build the future everyone manifest your dreams into the world we love you very much thank you for tuning in and we will see you soon bye