 of the 2020 Open Simulator Community Conference. In this session, we are happy to introduce a session called Developing and Evaluating Virtual Reality Simulations in Resource Scarce Settings. Our speaker is Rachel Umoran. Dr. Rachel Umoran is an Associate Professor in the Department of Pediatrics at the University of Washington, where she is the Director of the Neonatal Education and Simulation-Based Training Program and Neonatal Telemedicine Lead. She is also an Adjunct Professor in the University of Washington Department of Global Health. Her research focuses on improving neonatal outcomes globally through simulation-based education and global health partnerships. She has published and presented internationally on virtual simulations for health professional education. Please check out the website found at conference.opensimulator.org for speaker bios, details of the sessions, and the full schedule of events. Now, this session is being live-streamed and recorded, so if you have questions or comments during the session, you may send tweets to at OpenSimCC with the hashtag OSCC20. Well, welcome, everyone. Let's begin the session. Thank you so much there for that wonderful introduction, and I would say it's a highlight of my year to be at the OpenSimulator Community Conference once again. I love to look out and see friends and also new faces in the audience, and so when I was here yesterday, I saw there were quite a few new people, and so, well, I'll add in a few slides that highlight the journey that I've taken in Second Life and OpenSim. So, but first a little bit about me. I work at the University of Washington and some lovely Seattle, Washington. Yeah, I would wish you all opportunity to visit, but I know if you're international, that might not happen anytime soon, but I'm really fortunate to work in a very resource-available location, but we're gonna be talking more about resource scare settings and how we can potentially start to make the simulations that we all find so engaging available to people around the world. So this is the beginning of a conversation. I'm hoping that it will be a conversation that we will take up over time and for those who are developers in the audience, as I highlight some of the challenges, maybe there are gonna be ways to address them, because as we make these simulations available to more people around the world, they're also gonna be available to more people here in the US. Not everyone has a broadband internet connection and I think that's been highlighted even more so because of COVID. So I started this journey back in 2012 on the Global Health Island in Second Life. You wouldn't believe it, but it's still there. Every once in a while, I get a ping from someone who took a survey there and I was like, oh, it's still there. That's fantastic. But I ran out of space pretty quickly and I may believe to open some with the help of the five groups and members are here in the audience. Shout out to Nova, I see Nova there. So and the focus was creating educational sense for global health partners in high resource settings. So as we alluded to earlier, we had the Africa Traveler Center and I had East Africa and West Africa Traveler. These Sims are still out there. They're hosted by Vibe and they're available if you're interested in taking a look at them. I also used the Moses Grid at one point. Some of you might be prior Moses folks. But three years ago, I returned to that question of how virtual simulations be used with learners who are in resource care settings around the world. And the first question to ask as a developer is whether the environment itself is suitable. Do we have the kinds of ingredients that we need to make a simulation successful? And in many resource scare settings, there are lack of resources for physical simulation and thinking about the mannequins and the way that we train healthcare workers. Smartphones and laptops are widely available and internet connectivity is really improving and increasing over time. And that again has been a lesson learned this year with COVID that many collaborators who are located in resource care settings are able to connect usually through Zoom or Teams, but it is working. And if they're able to do that, then I believe they can do it with them. So back there, the next question to ask is, is there an interest in virtual training? And the answer is absolutely yes. I mean, when have you ever given a survey and you got an overwhelming response? 97% of those instructors and learners that we surveyed would use online simulation. And so if that's not a call to action, I don't know what is. But we needed a team. And as you know, to build anything successfully, you need, you know, you can't do it alone. And so we formed a team of collaborators. We had folks from the UK, from the US, from Nigeria, from Kenya, and within the US here, Indiana University, the American Academy of Pediatrics. The goal was to see if we could build the very first virtual simulation that was really designed for healthcare workers and focused on saving newborn lives. So one of the things, and so that's kind of my use case, but the goal here is to just communicate some broad lessons that we learned. And one of them was that we needed to plan for end user input and testing during the development. We wanted to be sure that this wasn't something that was baked in and just brought to a scarce setting to either succeed or fail. We wanted to target our design. We wanted to ensure that we had those critical features outlined and the ones that were nice to have, you know, jettisoned if we needed to. So because sometimes you start out with a vision that's really too grand for where the learner is at. And, you know, that's where instructional designers are so valuable, you need to be able to pare down and really figure out what the learner is gonna be able to do and what they're not gonna be able to do. So during our end user testing is where we discovered the adjustments that we needed to make. We needed clear instructions. We needed fewer teleport opportunities, I'll say. And more sound effects. It was amazing, you know, the kinds of things that seem to shape up as being super important. And so of course this testing is critical. One thing that we found out was that we could do it remotely. You know, could use Zoom, screen share and, you know, capture the experience of the learner and identify where they were struggling. So we partnered with local IT expertise that's still important for both development and deployment. And you can see here the testing that was happening. We were in this instance using a Google Cardboard simulation. So they were testing with one of our end users. And so the case in point, we used this approach to create a virtual reality simulation for helping babies breathe, which is a low resource neonatal resuscitation training program and it's used in 80 countries around the world. And that's, so we were able to successfully create it, but then the next step was how do we evaluate it? Does it really work? And so I want to encourage us from the standpoint of educators, please once it's developed, please don't feel like you're done. That's the time to think about how are we going to determine if this actually works for our learners. If it does what we said it to do, you know, the objectives that we set out to achieve, does that simulation actually achieve them? And there are different approaches and levels of evaluation, you know, for quantitative research, you can do a simple pre-post comparison all the way up to a randomized control trial like we did. But, you know, sometimes it's, you can do a crossover design, it can be a historical control. It's easy once you have your question in mind to think about, you know, how can I answer this question in a way that's feasible? Because some tests, field testing can be quite resource intensive. And so we had research assistants, we had technical support, but a lot of times you might not have that. However, you know, it was quite well received. As you can see, you know, the participant feedback was quite good. And so we were fortunate, but I think a lot of it is that it was and laid that foundation and making sure that end users were involved throughout the development cycle. The other thing I wanted to say is that qualitative research for those qualitative researchers in the audience can be so useful, you know, both in developments and in evaluation, just meeting your learner where they're at and figuring out what is going to be most helpful for them and what worked and what didn't work. I mean, there are certainly some learners for which virtual simulations are just not a workable modality. And, you know, they would far prefer, you know, in-person and hands-on. But then there are others that just really embraced the convenience and, you know, the objectiveness and the fact that, you know, they were able to take what they learned and translate it to practice. So finally, a really important aspect for successful implementation is the cost to the end users. Really tracking that cost intentionally through cost analysis as you go through the process will help you scale. So from an educational perspective, scaling might involve integration into curricula or use for continuing medical education. You might partner with industry and, you know, figuring out, you know, the process for getting what you've created into the hands of more and more end users. You might put it on, you know, make it available for free as our simulations are. But ultimately, there's a cost to you as a developer that I think is always at the top of your line. But ultimately, what does the end user need to be able to access your simulation? And do they have what they need to be able to do that? Always remember that. So that was here because I really wanted this to be a conversation to think about that many people around the world may not have access to the wonderful content that's been presented in this conference and how we can together make it possible for them. So I'm going to stop here and see what questions you have or what ideas and suggestions you have to share. Thank you for that wonderful session. Let's see, are there any questions from the audience? And I'm scrolling up, I saw someone ask about Western Australia. So they might have been asking about the African simulation or when you said worldwide, when it was being used around the world, you said what, 80 countries or something? Right, so for helping babies breathe, it is used in 80 countries around the world. It's a curricula, I did not design the curriculum itself, but the approach to resuscitating newborns for birth attendants has been used in many, many low resource settings. Now Lisa Laxton asks, how can we get more industry interest in this research or access to some of the information about it or use of it? You know, I think industry interest is driven by consumers. The more people seem to be interested and engaged, and the more they're asking for these types of experiences, the more interested industry will be. You know, there are some folks here in the audience who are part of industry and so I'm curious to know from them what they think industry wants or needs to have in order to become engaged. You know, I remember when the River City Project started what 2002 or 2004 from Harvard and how they adapted that and started offering classes or simulations, I should say, learning exercises to their physicians because I attended one of their briefings in 2005 and they said students only have access to five cadavers. They don't have access to real resources and even the mannequin simulations are still a one-off processor, not quite the same thing. And so one of the advantages they felt of virtual simulations is that you could replicate it many times and you could measure different kinds of characteristics. Do you have a comment on that, Rachel? Most certainly, I think that is one of the huge advantages of virtual simulation. The fact that you made this, you alluded to this earlier, the mannequin based simulations, using a mannequin is very resource intensive. You know, you can only have a certain learner to instructor ratio, you know, that takes time to get through a scenario and then to do a thoughtful debrief of that scenario. And you're limited by space constraints, you're limited by geographic constraints, learners can all get together. Right now you're limited by COVID, many sim centers are closed currently, including ours. And so learners are being pushed online or simulation is just not available. That's kind of the other option, but it's unacceptable because we know that the simulation learners are taking care of their patients more safely and when I say learners, I mean, not just the pre-service individuals who are training to become doctors and nurses, I'm talking about actual doctors and nurses who are training to take care of COVID patients or training to take care of, you know, more routine cases. And they use simulation all the time, but this year that has not been the case because of COVID. So one of the ways that we can address this need is by virtual simulations where they can engage with each other, they can engage with, you know, patient case or scenario and it can be replicated over and over without the necessity of having an instructor there. So there are multiple approaches that can be taken. I think open simulator is just, is one of the ones that could be super successful if we can get it into the hands of more people. You know, that reminds me before we went on you and I were talking about not just the two African villages that you designed the simulation for and ran on exploring disease and how to think about living conditions. You mentioned that in the future you might be interested in designing yet another simulation using open simulator. Do you have any thoughts on that? Yeah, I mean, I'm gonna just put it out there. Who's interested in a COVID simulation? You bet. Because the tools are all there. I mean, basically we had, as Lear mentioned, simulation was all scripted so that if you got close to a disease causing vector, whatever that was, you would catch it and over time you would get sicker and sicker unless you could get more healthy. So, you know, please drop a note for me if you're interested. I think we could resurrect this and make it a really cool COVID set. I suspect that that could happen or if you wanna do one, you know, feel free to take the idea and run with it. I think the more people are aware of the danger of COVID and how, you know, they could potentially keep themselves healthy and keep their family healthy, the better off will be globally. That's right. You know, I remember going through it with Barbara Truman and I was feeling fine but she must have touched something because she'd get sicker and sicker and I felt so dismayed. You know, I knew it was just a simulation and I'm an experienced gamer, right? But I kept looking in the manual. See, we're wearing a manual. We're carrying a backpack. We're looking in what test equipment and microscopes and I am not what we call qualified by any means, okay? And then we're talking to 25 different bots spread over two regions trying to figure out how can we help her? I remember thinking, this is very compelling because the sense of urgency, she's dying on me and I can't seem to solve this problem and I'm not used to being powerless, right? So I thought that was certainly not just for empathy but also a realization of how real these diseases are and how compelling they are for communities in crisis. Absolutely. And just a shout out to Barbara and if she's listening or able to follow along but she was just one of my inspirations as I got into second life and so, yeah. Thank you, Rachel and thank you for a wonderful presentation.