 And welcome to the 4 p.m. to 4 30 p.m. session of the 2017 Open Simulator Community Conference. As a reminder to our in-world and web audience, you can view the full conference schedule at conference.opensimulator.org. And please tweet your questions or comments to OpenSimCC with hashtag PoundOSCC17. In this session, we are happy to introduce a terrific session called an Educator's Approach to Developing Usable Prototypes for Serious Games in Open Simulator. Our speakers today are Evelyn Gossett and Rachel Oomerin. Evelyn Gossett is an assistant clinical professor of nursing at Indiana University Northwest School of Nursing in Gary, Indiana. Her interests are in using online and virtual environments for interprofessional education. Rachel Oomerin is an assistant professor of pediatrics at the University of Washington School of Medicine. And she's the director of immersive learning for the UW neonatal education and simulation based training program. Her research interests are improving neonatal outcomes through team science and measuring the educational outcomes of partnerships with global public health. She has published and presented internationally on using virtual environments for health professional education. Welcome, Evelyn and Rachel. Let's begin the session. Thank you so much, Barbara. It's so great to be here with you all and with my co-presenter, Evelyn. We've been collaborators in using virtual worlds for healthcare education since 2012. And I want to especially thank any attendees who are in the east on the east coast or in Europe because I know it's quite late for you. So for our in-world listeners, please feel free to ask questions in the chat as we go along and we'll answer them all at the end. And we'll also leave our contact information at booth 20 and expo zone 3 where you may meet us after this talk as well. So our objectives today are to assess our approach to the development of serious games in open simulator really from an educator's perspective. So you'll find that we have an institutional lens and a focus on healthcare. So we'll be discussing early prototypes and how they've evolved along with our usability testing approaches over the years. And we'll end us on tips for success. And we'd really welcome those of you who have been in this field for a while along with us to share tips that have worked for you. Initially, our focus in developing the various learning activities in the virtual worlds is to develop clear goals and objectives. And in considering those goals and objectives, we always ask ourselves the questions, why would a virtual learning environment work better than a lecture or some other way of teaching on the particular topic that we have chosen? And so our first example is Global Health Traveler. We developed this in 2013. And the objective here was to represent the challenges of working in a global health setting to orient medical students traveling abroad. It was initially built on a homestead region next to Indiana University Island, the second life, also shown on that slide, and was actually an upgrade from a small skybox above the main island, which really we should have called Phase Zero. We were very glad to upgrade this particular simulation to our Phase Two project, which Lear was referring to earlier. And I think many of you have had the opportunity of visiting. We were able to collaborate with survivors of educators and use their open simulator installation to expand Global Health Traveler from a small homestead region to a full island with eight learning activities that were based in East Africa region. We were able to, in addition, collaborate with the Moses Group on their grid and create an activity that was quite similar, but represented the West African perspective the next year in 2014. So this activity continues to be used by nursing students at Indiana University. And we're happy to have that activity. The students love it. Our next project took us to create the SPH Places, which is a challenge for the students in relation to public health and providing care. So the focus at SPH Places is about the students learning about social determinants of health. And we continue to build on that scenario by adding training for my senior nursing students on nursing leadership and management in 2016. We began with SPH Places in 2014, but in 2016 we added the training for nursing leadership with a focus on conflict management strategies. The nursing students in this that are participating in this challenge are taking nursing management and leadership. And so we learned a lot of valuable lessons in building the various scenarios that we've shown you. This slide explains a focus and case study on our trans and illustrates our transition to games in the e-learning platform. Here we use the agency for healthcare research and quality, team steps, strategies and tools to enhance performance and patient safety, to teach the nursing students and medical students about teamwork, cooperation and collaboration. And that was the first part. The second part of adding the team steps to our virtual worlds. This slide we began to focus more on the team steps communication strategies. And here in the second part of this phase we added open SIM bots to increase the engagement and interaction for the students. And as Lear mentioned earlier, we gained a lot of experience of using bots across the two different platforms. So second life and open SIM, but we found that the open SIM bots are a lot more flexible and adaptable to various uses. And so we started using open SIM exclusively for that. However, what we did still lack in using the, in the phases that we've described so far are is a real inefficiency in data collection. So we were not able to collect data efficiently from the users and we were not able to monitor and track users as much as we would have liked once we moved to open SIM. So we were fortunate to receive some funding to program a multiplayer Unity 3D environment which allowed us to use the browser interface and track responses more efficiently. And we were to present this version as an international meeting on simulation and healthcare and win an award. But just in 2016, the graph up there shows the number of users and scenarios that were accessed as we were able to also track the choices that users made throughout the scenario. And then moving on, we're currently in another phase of development with a focus on healthcare providers who are working in the University of Washington, neonatal care unit. And we're using again a multiplayer environment which is out of unity with scenarios that can be accessed in world or by learning management systems. So I think what we're trying to illustrate here is that prototyping can take different phases and translations to different platforms. But we always return to open SIM. I feel like we keep coming back each time, you know, we want to test out a new idea or develop a new program. So we couldn't have done this without a large team, had expertise in medicine, pediatrics, nursing, social work, and many other areas. Barbara here is our expert in instructional design. And so I was just saying it's so lovely to have her on the platform here with us today, as well as in multimedia and graphics and then research administration. And over time, we were able to standardize our virtual scenario development process. So we have six stages, which range from identifying and discussing learning objectives and narrowing them down for each scenario, all the way through writing questions and including a debrief, which encourages reflection and promotes our focus on adult learning. And additional lessons that we've learned over the years include that you should not limit your options for communication or collaboration. We began using the second life voice capabilities. And as you know, that's not so stable. So we quickly transitioned to using Skype. And later, we've begun to use Zoom. So we've used various virtual world environments, the learning management browser based the second life in the open sem. And as Dr. Marin mentioned, we used Google analytics for analysis. And we've used Google platform, Sakai on course, Canvas learning management system to present basic information to the students in there for their participation in the virtual environments. And this year, we started formal usability testing for all our scenarios. So our approach comes to using open ended questions that are directed at selected users and usability tests, such as the one click test, where we give the user a task and then ask them questions about their actions as they try to accomplish that task. And the other one that we are using more of now is the second test for the user gives their initial impression of a representative image. So this is one way that we're trying to improve the design of our platform in order to ensure that by the time we roll it out, the more larger groups of students, it's easy for them to use and acceptable for them. So talking about collecting data, and this is one of the key things that has been the focus, at least for our team, is to really make sure that what we do is meeting educational objectives and is acceptable for our learners. IRA protocols for educational studies are generally exempt. They're very easy to get. And within virtual worlds, as you all know, and much more easily now than in the past, it's easy to collect data on what the user is doing within the simulation, as well as through surveys outside the system. So thinking about that as you design is really important. And then we really want to put out a call to share your work. We talked about quite a bit in the last day and a half on how to go over the world. It's impossible to do that without putting out, you know, in public forums, wonderful work that we're all doing. So present an abstract, look for forums where you can actually publish some of the work that you're doing. And we know that funders look for evidence of prior success in the process of deciding where they're going to put their money. So research and development funding, as we know, is so crucial. And it supports faculty time, it supports servers, it supports design and development, research assistance, lots of things. So always be on the lookout for internal grants. Private foundations are relatively easy to get funding from if you tie your activity to something that is really, truly interesting to them. And then add on to research and quality improvement projects, which require an educational component. This is such a really good way if you can't get a standalone grant to add on a piece to a grant that is already existing or is being submitted. And finally we'd like to tell you some how we've managed to deal with potential pitfalls and barriers to working in the virtual world. Of course, time management, managing faculty and student schedules and matching schedules across time zones and busy lives and such has been a challenge. We've used various means of social communication to work around those barriers such as texting, cell phones and world meetings and such. A second barrier or pitfall is team cohesion, but we learned very early, especially working with the TeamSTEPPS strategies, we learned to work as a team. And the important component there is to listen to every team member's perspective and come to a consensus and arrive at your best solution. The third challenge that we have faced has been technical difficulties. As you know, the virtual world will lag. We've had voice challenges, which we were able to overcome through using Skype and Zoom. And many of the students have older systems and then we also have challenges with some of the students use Mac, some of the students use PC, and so that's been an issue. As Dr. Umarin mentioned, lack of funding has been an issue for us over the years to deal with this challenge. In the beginning, we did our own work, our own building, and as we received some funding, we were able to get assistance. And our final challenge or pitfall or barrier has been lack of institutional support. In the beginning, many around us thought we were playing games, and that may be true, we are playing games, but we're playing serious games. Yes, and indeed we have fun while we do it. So with this, I think another main tip for success is have fun. You know, the only way that you can persist along a certain track is if it's something that you enjoy doing. And I think we're all here because we enjoy being in the virtual environments and creating and building and using them for various purposes. So that is the first tip for success. Other tips are pay attention to your learners and their needs. We have over the years really, really simplified what we expect learners to do in our environment because many of them really don't have much exposure to games of any kind. And virtual worlds have their own challenges for them. So we have gone down all the ways to just log in and click on things. You don't need to know anything else. From the standpoint of instructors, we have occasionally some instructors that are very familiar with virtual environments, but the vast majority are not. And so again, we've adapted our approach to having those instructors just review reports that are submitted by learners from their activities within the virtual environment. So we don't necessarily expect the instructors to go in to virtual worlds, collaborate, collaborate, collaborate. Can't say that enough. You know, the world is hard enough as it is to face it with a team. And it'll be a lot easier be scholarly. And that can be even as, you know, just in how you test out your initial skills. You know, as we talked about with the usability testing, it doesn't have to be publishing a paper, but it certainly could be. And it's a way that you document your work and your progress. Taking an incremental approach is something that we've also learned over time. It's easy to want to do everything all at once, but that takes time and takes funding, you know, do a small activity, do an initial test, see how it goes, see whether your learners like it, add on as you go along. And that's what people are able to do successfully. The last tip is to talk with your IT department early and often. This is your instructional design group. Whoever manages the servers for your institution, you just never know where you might be able to get some help or someone to after your team from that group. So that's the, I think all that we have time for today, but we're really, really interested in chatting with you, answering any questions that you have. I know there are already some questions in the chat. Thank you very much for your time. Thank you very much, Rachel and Evelyn. We do have time for just a couple of quick questions. There was a question about following up with you on the OARs that you use with these different regions. If any of these, if anyone can participate in those or if they're open access. They are available. If you want to follow up with me, I have them all. And we currently have Dream World. Thank you for installation at our university. And so if you would like to access them there, it's available. And there is also an Indiana University installation that is still up and running at Evelyn University. Fantastic. Yes. And I know that another question surrounded the usability and you mentioned, I know in your data collection process, because you had a research design that you used as you would make these innovations for the students and you would reach out and you would have other faculty and other students get involved. But you had formal data collection processes that you used as part of the study, but then you also had other usability. Can you just elaborate on that a little bit, or what you would prefer or recommend others to do? So what we had done consistently over the years is establish some form of data collection. Currently what we do with our existing simulations is that we have surveys before and after. And we also have data collection within the simulation. But one thing that we wanted to pre-empt as we got a little more experience with this is those negative comments that always come up. You can't please everyone, right? But nonetheless, we wanted to be sure that our simulations were usable before we wrote them out. And so we started doing some formal usability testing just very recently on our new simulation with a handful of students from the same class and group that would, or level of learning that would be using the actual bills. And a lot of good information and perspectives, you know, things that you wouldn't expect to be a challenge for a challenge. And so we were able to make modifications before rolling them out. But I would highly recommend that. Okay, great. Well, thank you again, Rachel and Evelyn for a terrific presentation. And for our audience, I want to let you know that the next presentation will be right here. It's called Virtual Worlds Database Connect with the Metaverse. It will begin at 4.30 p.m. So we're going to get off the stage right away. And we also encourage you to visit the poster expo. Their booth is booth 20 in Expo Region 3. And don't forget to visit the other hypergrid tour areas and the crowdfunder booths as well. But thank you again, everyone, for attending our session. And we will see you at the next one.