 Cool welcome back My name is Patrick Pennyfather and to begin with I'd like to do just a small experiment So I need a couple of people to help me because I've developed this new application using the go which senses It's it's it's basically it senses where objects are in the room and If you don't believe me then I'll demonstrate. I just need there's a there's a couple of little things that I need First of all it it's always helpful when a friend And colleague calls you for emotional support So if we can engage the friend to call any moment that would be great Cool anytime But it Yes, thank you Bernard and do we do we have the speakerphone activated? Cool. Can we have the voice activated on the speakerphone? Okay, cool. So let's bring let's you might have to adjust the volume on the phone and just turn it up a little bit So that's gonna be key key key here Today I'm gonna talk about the new mr And I'm not gonna reveal the acronym just yet So let's demonstrate a little bit of the new mr And this is all based on an experience I had last summer with a lot of amazing people on an amazing project that was completely out of scope and Yes, Julianne it failed, but in a great great way So let's begin. Hi, we're here So young goes you're gonna guide me a little bit In the virtual world that are also Simultaneously in the physical world and thus making this a mixed reality experience Where forward back forward Success, thanks to the technology. Thanks to young goes and thanks to prop master rizadi Yeah, feel free So originally When rizadi asked me to present today. I started to think of the reflection of a mixed reality event That I co-organized and produced why am I holding two mics? Last summer and this is part of my reflection bear with me and Luckily, none of the students that were working on the project at the time at the master of digital media program are here But I think you'll all resonate with this especially being theater creators Who take care of the staging? Here we go It was entitled why didn't anyone pick up a broom? The perils of staging a public mixed reality event and I wrote Perseverance Compassion Those were my three first words and I'll repeat those today on the surface It's not really fair to expect students to anticipate certain actions to be undertaken prior to them understanding what may come This may be true for a team of grad students developing augmented reality applications for a location specific event On a mundane level it may also be as simple as sweeping the floor In a 3,000 square foot hangar While we may tout that one of the learning points a Project-based learning is self-regulation or managing your own learning and being motivated and doing your own thing and This by the way in the research Is what employers expect recent grads to manage Once they are transitioned into their companies What they think is important to learn might not always be what they learn How do we mentor students to anticipate the unexpected? Those parts of a production that are not spelled out now I say this in the context of a mixed reality event because in mixed reality events We do not know What to expect as much as we try to design those experiences beforehand And there are amongst us many people who are engaged in that act in mixed reality events Bernard as an example and his team at see it They focus a lot of time on the pre The during and the post experience of VR particularly in public spaces So today I will give sweeping remarks about mixed reality and those who are involved in the delicate Art of stage craft will resonate with a lot of these Sweeping remarks that I'm about to present So here's my revised title. I thought to extend it a little bit just to give you more emphasis and a Couple of things when you present you should never read your slides, so I will So sweeping remarks about the co-construction of event that by all measures was composed of so many different types of physical and real-world interactions with the focus on Considering the audience user experience before adjourn after as to the thusly named that makes no sense a carnival of mixed reality Even though I also had to call it the fun palace because a colleague was the one who initially dragged me into it with a calming words It's just a harmless event won't take much time. And so today publicly I Create a new acronym for mixed reality events and in fact I get rid of the word mix and What I make this stand for today is managing multiple realities and this is what I'm going to talk about and In talking about that. I will relate that back to my initial impulse, which is how do we create in Each of us whether we're students or professionals for many years. How do we create that? capacity to think ahead With the intention of making our experiences awesome For audiences for users whatever we choose to call this hybrid as was mentioned earlier by Ian so For those of you who don't know the carnival of mixed reality was this but before there was this there was this and There were many versions of this and this Samantha are you in the room? How many versions did we have of this probably? 15 20 different floor plans that kept evolving over time because everything did devolve and this was a three-month process No, hang on. What am I saying? Six week process. Yeah, that's closer in order to get to this point So before there was this There were some mistakes made There was this and there was this photo shoot in order to record people and there were these first iterations of what would be considered to go into the people Before there was this app Which was an interactive point inside of curiosity booth in which actors would roam around the space to gather people into and Secretly have a one-on-one engagement with an with an actor Before there was this or this Hector He's here. He Created this application. There was this We had to put it somewhere and this room was a storage room there was also Laura's hard work to create the window through which the audience member could actually see the actor and Laura, are you here? I Think she left temporarily, but Yeah, there were asbestos signs everywhere and Before there was that there was also this this was a little friend I think it had a name Rachel. Do you remember the name that was? Happening around the time for our little visitor Benji. Yeah, this was our Benji friend and Benji visited and this was three days before so yes, you can imagine me going, you know, I could have just let it go and Literally brushed it under a carpet But my concern was for people coming to this event in which a small creature might have alarmed them or remnants of so these little things when you're trying to Co-produce an event in a space that's not really used to the rigor of what those in theater Are used to calling stage craft we had a grid that I'll talk about in a moment and that grid Had a certain weight-bearing so before there was this work by Julianne and small stage and Lisa's there, too Hi before There were there was all this work to set it up Originally it wasn't going to be in a geodesic dome, but we changed our mind We also started to play with Different lighting scenarios. This was a photo shoot. This was part of a of a larger scale Event that small stage was engaged in as artists is read in residence at the Center for Digital Media So before there was all that glamour there was this before there was this construction There was Julianne and I going like boom because she thought Wouldn't it be awesome to hang Tyvek off the ceiling and you can all imagine it, right? It's like yeah awesome, but I just about freaked because There was never any Weight-bearing done on the grid so Connor where are you so you know I mean you know right? It's a risk So anticipating that we have to think about oh well. What if something falls? That would be bad UX Before where there was this iPad stands With a triggered augmented reality marker triggering a brain there was the idea that we actually Had to have some types of devices for people who didn't want to download an application on their phone Because that's also an assumption that we make as developers. Oh, yeah I for sure everyone's going to download my app for sure. No actually, so Part of thinking about mixed reality events is to consider all these aspects in the design and in the pre-production process So again, these are just beginning photos of it. They they were eventually repainted black they were secured it with sandbags and We had to test and went through a process of testing distance because we wanted to allow people to go in front of it Oh, right people are going to go in front of the camera. What does that mean? We have to put the marker way up high So we decided to use this piece of scaffolding that's been present in it and occupying our hangar for the longest period of time And then we began a rigorous testing process that testing process in also involved other colleagues so Claudia is in one of these photos and You can see that she is looking at her own brain, which I believe is actually that size We're still not sure how it fits within her head But you know we will talk about that another time But this is actually a real brain scan. And so the only way that would have happened is If we had had Claudia as part of the process, so is she a contributor to our mixed reality event? Yes, are we looking for partnerships? Yes, that's also part of the pre-production process is to is to look around To seek who else might be willing to participate in whatever capacity The students also did a our applications that were in different locations Around the space and these also had to be storyboarded. So we storyboarded almost every possible user experience that occurred in the space Before there was this a work by Katerina Stepanova and John Denway Stewart students of Bernard and this is called gel and it involves reading brainwave data with two people simultaneously and they are affecting the oscillation rate of two jellyfish and They're essentially. Oh, yeah, right. It's it's synchronized breathing, right? Yeah breathing. Thanks Bernard. Glad you're here Doesn't make me nervous at all And so they try they try to synchronize their breaths so that these two jellyfish eventually end up in synchrony well before there was that there was there was a lot of shit that happened the the hard drive not working and One of my favorite shots of the process to share with you is before all of that There was John at the back of the room soldering the video card, right? Now for those of us who are used to theater you might be thinking yeah, well, whatever. That's what we do But it's not usually what we do when we're in technology development there's a lot of assumptions that things are just going to work and Yankos can attest to that when he and his team Including Marcos who's here Co-constructed a an immensely Overscoped and awesome VR theater for SIGGRAPH they had no technical problems, right? So again another work by the same team from see it under Bernard's guidance was the body remixer and the body remixer consisted of Sensing people who were not in VR along with sensing people who were in VR creating really a true mixed reality experience complimenting that virtual and physical aspects of emanations of the human body and but before all that there was Katerina who was user testing this for at least a week before in the space and Eventually because you can see here. These are not the lighting conditions Connor Eventually you had to do it in the lighting conditions because that's one of the things that we learn about augmented reality is without the lighting Condition you will not have augmented reality if it's too dark of a space it will not trigger anything So all of these things have to had to be tested. We also had a piano bar. Oh my god. That's what it was called It was essentially a Triggered unity-based experience Allowing people who didn't not know how to play the piano to come up to it and the sounds were not of the piano at all They were synthetic sounds and it was part of a very I still don't understand the user journey, but anyway, it was fascinating the two developers were on it I'll be before that all happened Thank God we had Julianne in the room at the time when we decided we need to have another screen and lo and behold We have Leo here Getting up on the ladder and yes, he was insured And and he was putting that together and Julianne was helping out And so we had this whole collaborative process that was part of this mixed reality event So again the reason I'm saying all this is that we Were from the beginning Attempting to focus on the user experience on the audience experience on that hybrid person that comes in who for the most part May or may not engage. That's the risk that we take Reasons for non-engagement include I don't want to be seen doing this publicly for sure to That lineup is too big so Working with Sam who's here. Yay, and Rachel and others We had to focus on the lineup experience. How are we going to manage this and you'll see in in the video that comes We taped everything down I mean we used strategies of theater in order to help us stage a mixed reality event. We had lighting we had Four speakers with spatial audio. We had a total of I think combining with headphones We had 17 different sources of sound happening simultaneously in the room. We had to consider that we had to consider levels throughout we had a lighting team with Gardner from Simon Fraser University and she was leading the team to help us design that all of those things Huge things in order to enhance the audience experience now it might You might think in theater. I'm gonna ask Connor to put them on the spot Because I like putting Connor on the spot. I don't know why but who would have that job of taking care of all of that thinking oh, okay Yeah, we have to think of the entire aspect of a production in order to enhance the audience experience Anyone come to mind? Would it be a technical director? Would it be a producer? Is it a combined role? Is it the director as well? It's a tricky question. What do you think? Yeah, yeah, so Yeah, what a surprise. It's the entire team. Yeah, okay Yeah Awesome, so so you can see right? Just like there's so just as there's disagreement in that there's also disagreement in what the hell makes reality is so Part of moving into that and understanding how we define it is going to inform how we design the audience experience and so huge so here's one here's another And importantly Where's yours? because Depending on who you work with we all have that perception of what makes reality is and what it could be and so that makes it Incredibly challenging to design our audience experiences for But the important thing is to have that intention and that's why I come back to the caring aspect of this presentation The important thing is that ultimately we care about our audience We don't make assumptions as well of the audience in these types of events in the events that Ian and Justine presented to us awesome events Really hard to think of all the aspects of the audience experience, right because it's outdoors or it's it's site-specific Kendra also lots of experience there Julianne lots of experience there. Oh my god, there's all these parameters that we have to consider and my Value if anything with this presentation is to encourage us to do that even more and In saying that I also reflect on the practices of Emerging technology developers who are obsessed with user experience now It's not the right term for our audiences, right? So I've written about this too and and it is a malleable. Are they audience are the user? Oh, I'm watching someone in VR. Wait a minute. Who am I right now? I'm in VR. Am I completely immersed in VR? Am I partly in reality? What if I step out and suddenly I hear someone talking behind me? Am I still in immersed in VR? All of these questions come up. What about augmented reality? What about shared augmented reality experiences because that happened a lot as you'll see in the video coming up people sharing augmented reality Experiences with other people with their own devices. What about that user experience? Do we consider that when we design our lovelies so now after all of the This is the backstage of what we did. I'll play you the video Let the credits roll In order for us to see how many people were involved. Oh my god and thanks to Yangos and colleague Andreas for creating that video for us awesome You can see all the people involved. This is not really that surprising for us in theater, right? We you know, we we tend to know like there's tons of people involved But in mixed reality public events There's so many variables there's so many things to consider that we have to be In in a way and and one takeaway from this event is we have to be really really Considerate of all the people that are involved and be able to say. Oh my god We are also creating a user experience for contributors so with that and with this idea of caring and that theme of caring and Those are some of my takeaways and that's what I wanted to share with all of you today. So thank you