 Welcome back to Virtual Abilities 2019 Mental Health Symposium. My name is Malay Freski, Dr. Amy Cross in Real Life, and my research involves the transference of identity between user and avatar. I've been a resident of Second Life since 2007 and founded the Four Bridges Project in 2008. It's my pleasure to introduce Dr. Kevin Holloway. Dr. Holloway is the Director of Training and Education at the Center for Deployment Psychology. The Center for Deployment Psychology, CDP, at the Uniform Services University of the Health Sciences in Bethesda, Maryland in the United States, has piloted virtual world-based training programs for mental health providers serving members of the military, both veterans and those in service, and their family members. A licensed clinical psychologist, he's particularly interested in technology solutions, including using virtual worlds to improve access to and quality of professional training. The title of his talk is Virtual World Training for Mental Health Providers. He will discuss CDP's synchronous and asynchronous Second Life Training models and environments and preliminary outcomes data, data regarding effectiveness. Audience, please hold your questions and comments until Dr. Holloway opens the floor for interaction. And welcome, Dr. Holloway. The floor is yours. Thank you so much for that introduction. Hello, everyone. As she said, my name is Dr. Kevin Holloway. I'm a psychologist and the Director of Training and Education at the Center for Deployment Psychology. CDP is part of the Uniform Services University of the Health Sciences in Bethesda, Maryland. And our primary mission is to train mental health providers in the Department of Defense in evidence-based psychotherapies, or EBPs, for mental health issues, relevant to military combat deployment. Additionally, we also train civilian mental health providers who are likely to treat service members or veterans in topics such as military culture, military deployment cycle, common clinical presentations that service members may have, as well as evidence-based psychotherapies. At first, I need to start with my standard disclaimer, and that is that all of the views expressed today are my own. They do not reflect the views of the Uniform Services University of the Health Sciences, the Department of Defense, or the U.S. government. Basically, what that means is that while the university thinks highly enough of me to hire me to do presentations and workshops, they don't think highly enough of me to stand behind anything I say. So all of the brilliant things that I say belong to the government, and anything dumb I say belongs to me. For almost 18 years, the United States has been in a continual state of war. Over 2.8 million service members have deployed in service of Operation Enduring Freedom, or OEF in Afghanistan, or Operation Iraqi Freedom, OIF, in Iraq. And over half of those have deployed more than once. Those without saying that serving in a combat zone is dangerous, even life-threatening, and significantly stressful. In this image, you can see a striker AFV armored personnel carrier lying on its side after surviving a deeply buried, improvised explosive device, or IED, blast on April 15, 2007, Iraq. The mental health consequences of serving in combat can be tremendous. Around 14 to 19 percent of all service members returning from deployment report significant symptoms of post-traumatic stress disorder. And similarly, around 14 to 19 percent of returning service members can be diagnosed with major depressive disorder. Nearly half report significant sleep problems upon their return, including insomnia. All of these conditions increase the risk of service member and veteran suicide, to which we currently lose approximately 20 every day. And the risks appear to increase with repeated deployments. The good news is that we have several evidence-based psychotherapies or EBPs that address these serious mental health issues. For example, prolonged exposure therapy, or PE, and cognitive processing therapy, CPT, for PTSD, both have years and years of research supporting their efficacy and effectiveness. Cognitive behavior therapy for insomnia, CBTI, and cognitive behavior therapy for depression, CBTB, both have demonstrated to be highly effective treatment that result in better outcomes and even then medication therapy. Cognitive therapy for suicide prevention has shown significant ability to reduce the risk of suicide. But for some reason, a significant portion of DOD mental health providers do not offer any of these interventions. It is a similar story in the VA, where the recent EBP rollout program had as their primary goal to make sure that at least one therapist in every VA mental health clinic was adequately trained to provide at least one EBP. And the situation among civilian mental health providers is even less encouraging. There seems to be a significant lag between research support for clinical best practices to implementation of these interventions at the clinical level. But what accounts for this chasm between a number of studies looking into this problem have identified several barriers to implementation. First and foremost seems to be a lack of adequate training. Many providers do not have access to adequate training opportunities. Which typically come in the form of multi-day face-to-face training workshops with didactic demonstration and role play with expert feedback elements. This is true sometimes because they work in clinics or areas where such workshops are not offered, or sometimes due to clinic issues such as lack of funding, time away from the clinic, large caseloads, and lack of leadership support. And it is important to note that while helpful, merely reading a therapist guide is not considered adequate training. Even when adequate training opportunities are available, post-training consultation may not be. Consultation seems to be essential as providers newly implement EBPs relying on the experience of experts who can offer support and feedback, brection, and problem solving. Sometimes this lack of consultation, coupled with an inadequate understanding of the theories behind these interventions, results in providers making modifications to treatment protocols which are unsupported and inconsistent with theory, and therefore likely to result in poorer treatment outcomes. Additionally, with overworked mental health providers with large caseloads, there is often a lack of motivation to do something new as doing something new requires extra energy, focus, that may be beyond their available resources. And finally, there may be insufficient institutional support to implement evidence-based psychotherapies because of a perception that it may require a lot of investment of resources. Time and funding while not appreciating the cost savings that come from treating patients effectively and helping them to recover more quickly. Since standing up in 2006, CDP has primarily provided training and implementation support in the form of traditional face-to-face EBP training workshops. These tend to be two-day long training events, face-to-face in a physical venue. This allows for the instructor to have eyes on the participants to see how the information is landing on them and to engage in observation and feedback during participant role-play exercises of the skills being taught. Typically, after these workshops, free consultation is provided to our participants in the form of weekly or bi-weekly group telephone calls. And for many years, this has been a very effective way of disseminating evidence-based psychotherapies. However, there are a number of limitations that are related to this model as well. First of all, we have to bring these workshops to centralized locations, meaning that instructors travel to venues that are accessible to large groups of participants. This means, however, that providers in remote areas or in smaller military installations will not normally be able to access these workshops without significant travel and cost. Graveling requires additional days out of their clinics to be able to participate, which means less opportunity to treat patients. As many of you already know, many physical face-to-face training workshops also have limited opportunities for discussion, meaning that while the presenter is presenting, participants are expected to sit quietly and passively absorb the information presented, but are discouraged from interacting with other learners except during prescribed role-play exercises or scheduled question and answer segments. Event scheduling can also be a significant drawback in that workshops may only come to a particular area once a year or even once ever. If the offered training workshop does not fall on days that a provider can realistically take away from the clinic, they will miss it. After all, usually it is not realistic that a whole clinic shut down for training, as someone has to cover for walk-in emerging cases. In 2013, CDP, with our partners, 2B3D Studios, started building a virtual world training venue in Second Life. And while building this training venue, we took input from all of our expert trainers about what kind of features they would like to see in an ideal training environment. Their recommendations included some basic things like online registration, being able to display all of the media that we use in our training workshops, like PowerPoint slides, demonstration videos, distributing handouts, and automated attendance taking. But they also included ideas for tools to streamline assigning role-play groups, automatic teleporting to breakout rooms, streamlined communications between presenters and attendees, and automatic nameplates at auditorium seating. Additionally, we tried to anticipate any technology barriers or steep learning curves our attendees might face and tried to develop tools to help ease newbies into the Second Life space so that they could focus their attention on the workshop and learning, rather than being frustrated by the Second Life platform. By May 2014, we offered our first two-day EBP workshop in Second Life. We had no idea whether mental health providers would consider attending a workshop in Second Life as a legitimate way to engage in continuing education training, or even if they would feel technologically savvy enough to try it. But when registration went live, it sold out within 24 hours. So we knew there was a demand for attending live, high-quality training online. Since that very first two-day workshop in Second Life, we have conducted 54 two-day training workshops in Second Life across five different evidence-based psychotherapies, with a total of 1,562 mental health provider attendees. Approximately 20% of these have attended more than one training workshop. Registration is open to any mental health provider, whether DOD or civilian, and 13-and-a-half regular continuing education credits are granted for completion of each workshop. I want to show you all a demonstration video at our Virtual Education Center, which will include examples of some of the venue tools and features, and hopefully can give you a sense of what our workshops in Second Life look like. So I'm going to pull up a screen and display a video here demonstrating our VEC. If you're able to view videos in World, go ahead and click on the screen when it appears. That will zoom your camera in so you'll be able to see the video. If you're not able to see videos in World, I'll also provide a URL in chat that you can click to view the video in your own web browser window. Once the video is concluded, please tap the Escape key if you're viewing the videos in World, which will zoom your camera back out. Or if you're watching the video in your own web browser window, once the video concludes, close that window and return back to the Second Life venue here. If you can type Y in the chat when you're finished, that will let me know that that most are done. Give me a minute to bring my video up here. Apologies, that is the wrong video. Let's try this one instead. Apologies, that is the wrong video. So you're going to have them walk through in detail like we talked about earlier as they did the timeline and told you their narrative. You're going to do that again as part of the imagery portion of the treatment. A couple of things to remember. First of all, you're going to ask the patient to close their eyes and describe in the present tense the sequence of events. I think the best way to really kind of see this is for us to show you guys a video of what this is going to look like. So at this time, I'd like to share a video of our therapist, Dr. Greg Brown, who you've seen previously, and he's going to be engaging our patient, Sergeant Badwell, in the first two relapse prevention exercises. So I want you guys to pay attention to the video screen. We'll watch this, and then if you have any questions, we'll talk about it after the video. So today, Carl, I'd like to go over the relapse prevention task that we talked a little bit about last week. And if you recall, we went over the skills you learned during treatment. And so what I'd like to do today is kind of like a dress rehearsal for when you get upset, even think about killing yourself and how to handle that. Really great discussion on protective factors, you guys. I love hearing your thoughts on that. So great job. So now that we've covered the suicide risk assessment section, we're going to do a practice exercise. So I want you to think about everything we just covered, talking about what you want to cover in a thorough suicide risk assessment. And we're going to focus on one area of that. So if you could just take a minute, pull out handout seven and eight. And when you guys have those, I want you to just slap me some lies. Perfect. You guys have both of those. Wonderful. So first of all, just to kind of point out, if you look at handout seven, this is really an overview of handout number eight. You're going to use handout eight when we do the breakout exercises. So this is what the handout looks like. So what we're going to do at this time just so that people are clear, I'm going to give you a brief explanation and then I'm going to have Maria give you a little more information. We're going to break into groups of three to practice doing this focused assessment of suicide intent. And what we're going to do is in those triads, we're going to have you have one person be the assessor. One's going to be the patient and the other person's going to observe. And we're going to do a 15 minute practice exercise and then we'll have you guys switch roles. So we're going to do this twice for a total of 30 minutes. I do want to point out that when you get to your breakout room, there will be a timer in there and it will do a countdown for 15 minutes. All right. Well, like Dr. French mentioned, you're going to be sent into a break room. You're going to get into a group of three. Once you get to your break room, make sure that you only your microphone by clicking on the speak option at the bottom toolbar so that you can participate in your role play. All right. So at this time, we are going to get ready to teleport you. So go ahead and click on the option stand, which is at the bottom toolbar of your screen. Right above profile. Great. Thank you. Now I'm going to go ahead and teleport you to your break room in about three seconds. Hey guys, so I guess I'll be the therapist for the first go round. That's not okay to you. Okay. All right. I'll be the patient. Okay. Great. All right. So I'm going to pull out my hand out here and we're just kind of going through the handout. Is that basically what we're doing? Okay. All right. So, you know, we've been talking about how it's been a rough time for you lately, Kelly. And, you know, first thing I wanted to ask you about is, you know, have you been actually thinking about killing yourself? Have you had, you know, thoughts about suicide lately? I have. I think about it a lot actually. Okay. When you think about it, can you tell me exactly what you've been thinking? Well, I just have a lot of downtime and sometimes if I don't have, if I'm not busy, if I don't have anything to do, I just kind of sit there and I think, you know, I don't really have a good reason to be here. I don't. A hypothetical way that you, you might do it. That's all right. Yeah, that's right. So I want to intervene for just one moment just to take an opportunity to point out a teaching moment and you were doing a really great job, Andy, and you were doing a really great job at being a really realistic patient, Kelly. So thank you for that. But Andy, I think this was a great example where really when you're looking at starting to assess the plan, you're really looking at two different potential methods. So oftentimes we kind of get stuck in what's your plan and she was talking about maybe looking at means, but then she started talking about overdose. So I think this is really just a great example to highlight that sometimes we need to assess for multiple methods and you were doing a really good job of just kind of following that. If you're talking Kevin, we're not hearing you. Thank you for that. I didn't mute my mic during the videos. Apologies. I'll go back and restart the beginning of this slide. We've learned a lot about improving the learning experience for attendees over these years. I just want to mention a few of them, though most of you here probably have already learned many of these three variants. First, we tried to create the virtual space to be familiar and intuitive. While it can be fun to create whimsical, fantasy or atypical venues, we thought it would be important for our participants. They already be a little unsure about the professionalism of mental health training, what appears to be a video game. Be a space that feels familiar and that it looks like a place where one might attend training physical world. And one where they intuitively know what is expected of them, where to go and what to do. Additionally, to facilitate acceptance of the legitimacy of our second-life annual workshops, we have an unwritten rule for our instructors and staff. That when we host an externally facing event, all of our avatars are human. When we have internal meetings, our staff gets very creative with their avatar self-expression, which is encouraged. But we want to minimize any psychological barriers to our attendees seeing our venue as a credible learning environment. We also learned that it is very important to provide good tech support from before registration all the way through the workshop. This starts with a good information page on our website about what is involved in a second-life workshop. Minimum system requirements, minimum hardware needed. For example, we require that participants use a headset with a microphone so that they can participate in role-play exercises. How to set up a second-life account, videos that demonstrate basic second-life skills, and a frequently asked questions section. This information page can be found at the link in chat there. It's also on the slide, I think, deploymentpsych.org slash virtual provider training second life. When registering for a workshop, attendees are required to sign up for a pre-workshop open house where they'll briefly meet up with one of our tech support staff to make sure that they can log in, find the venue, hear voice-over IP audio, and can be heard through their mic. They will also get a quick basic lesson on second-life skills they'll need to succeed, such as how to navigate, how to sit, reading dialogue boxes, and accepting experts. We also began every two-day workshop with a brief 15-minute tech orientation to review the function of the heads-up display, how to participate in audience polling, downloading handouts, muting, unmuting mics, and some of our shorthand words, such as hot mic for people with an accidentally open microphone, or slap me some Ys to refer to a quick yes or no question. At least one of our tech support staff are available throughout the workshop to assist with any technology issues so instructors can focus on the workshop, and attendees can get back to learning as quickly as possible. The participants heads-up display HUD. Its main purpose is to assist newbies with accessing more advanced second-life features such as relevant gestures, changing camera angles, and participating in audience polls. The participants HUD also helps presenters manage the breakout group tool, which makes group assignments, assigns breakout rooms, and automatically teleports participants to breakout rooms and tracks their location on the presenter's marauders map shown here. Another thing we've learned over time is to include two instructors for every two-day workshop. Together, they assist each other in didactic presentation. While one is actively presenting, the other one is watching the nearby chat, responding to questions, and aging with the learners. To learn that it can be very distracting for instructors to teach and track the chat simultaneously, we share that responsibility and played roles throughout the workshop. Additionally, we learn to spend time training our instructors on best practices for teaching a world, which includes frequently checking in with participants, asking for responses, and engaging them in interactions throughout to help them gauge the mood of the audience and motivate greater participants. And of course, being charismatic presenters. It's difficult to listen to a boring lecture joining on and on in person, let alone in a virtual world workshop. It's much more interesting than engaging to listen to a presenter who likes what they do. Responses from our participants have been almost uniformly positive, with a few notable exceptions. Here are a few selections from participant responses to our post-workshop evaluation. Quote, I just love this training, extremely interactive. Great opportunities to ask questions. I feel like I participated more in this format than I might in face-to-face workshops. Another writes, really great training. Second life makes the training accessible to providers anywhere and still allows for an interactive training experience. Very helpful. A few more. Easily one of the best trainings I've ever been in and I am a trainer. And at first I was reluctant but found it a very positive experience. And just so you don't think I'm only including overly positive feedback, this response from a participant who writes, keeping up with the chats and the speaker was difficult. I stopped attending to the chat so I could better focus, but I worry then I wasn't as active as I could have been. Which is fantastic feedback and helps us to calibrate the right amount of interaction in the chat window to facilitate learning without being distracting. Results from our program evaluation efforts have been very encouraging. We evaluate pre-post-workshop knowledge gain for all of our workshops, whether virtual or physical. Regardless of learning environment, knowledge gain is significant for all of our EBP workshops. Comparing virtual and physical venue workshops, participants in our second-life workshops demonstrated higher pre-post learning gain. Higher self-reported readiness to utilize the treatment or EBP and reported similar attendee satisfaction between physical and virtual venues. These results, while perhaps a little unexpected at first, make sense given a few observations. First, participants in our second-life venues seem to engage in role-play exercises more readily than participants in our physical venue workshops. This is partly due to participants often attending second-life workshops without friends or colleagues. And even if they do attend with people they know, we randomly assign them to break out role-play exercises. In our physical venue workshops, it's not uncommon for instructors to report overhearing more planning for lunch, or overhearing discussions of the latest office politics between friends or colleagues than legitimate participation in role-play exercises. But in our second-life venue, there's more of a focus on the task at hand. Participants know that instructors can teleport into their breakout room at any time, potentially catching them off task, unlike face-to-face venues where participants can see us coming and then reorient to the role-play task before we get there. Additionally, there seems to be a feeling of obligation not to interfere with the learning experience of an unknown other versus a friend. We've also observed a phenomenon we like to call synergistic learning. In physical venue workshops, it is considered rude and intrusive for participants to talk or chat during the presentation, sometimes even avoiding asking questions for fear of interrupting the flow of the presentation or waiting to ask a question until later, when they've either forgotten the question or the topic of conversation has moved on to something else. In our virtual workshops, a social expectation of participation quickly develops where participants are asking questions as they have them, and other participants jump in to contribute to the answer. Discussion among learners is invited and encouraged, enriching the learning experience for all. It would not be an honest academic if I did not also acknowledge the role of self-selection bias in contributing to these program evaluation findings, meaning that people who do not feel adequately prepared or competent in a virtual environment to consider signing up are probably not signing up for our second-life workshops, perhaps leaving us only with those who are primed to do well in such an environment. While I acknowledge that possibility, we hope in the future to be able to conduct research in which participants are randomly assigned to learning conditions. We can evaluate the direct impact of virtual worlds environments on learning compared to physical environments. Despite these impressive results regarding EVP dissemination, we still have some work to do regarding implementation of these skills. Based on surveys sent to our traditional physical workshop participants, we find that about half reported never accessing post-workshop consultation, either from CDP experts or from others. Only about half reported implementing the EVP with even one patient since their training. About half of those reported using the full EVP protocol with fidelity, while many reported utilizing modifications to the protocols which were inconsistent with the underlying theory. While these data are not drawn from participants in our second-life workshops, it still suggests a gap in training which virtual worlds learning could assist. So as this is the most exciting part to me, it is one thing to replicate or simulate traditional classroom learning models in a virtual world space. As I mentioned earlier, we have some data to suggest that our second-life workshops are more effective at knowledge gain and confidence building than physical world workshops. But as most of you are already aware, virtual worlds platforms provide many more learning opportunities than static lectures. Instead of the limitations of passive learning in a classroom model, experiential learning opportunities may allow for a richer, more meaningful learning experience. Instead of learning by listening, this is learning by doing. Our latest efforts at CDP are in leveraging gaming motivation and all of the affordances of virtual worlds to improve learning outcomes. The goal is to facilitate experiential learning, wherein the learner is immersed in a learning environment and learns through their experience in the context of the environment. Learning is enhanced with repetition and review, which is much more accessible in a gaming environment where replay is an option. And finally, experiential learning engages learners not just in a cognitive academic exercise, but one in which emotion or the feelings of the experience is also engaged. For example, we have learned that mental health providers are much more motivated to learn and master new clinical skills if they have a positive personal experience with those skills, rather than just learning research data supporting those skills. Like all humans, therapists make emotion-driven decisions more than data-driven decisions. But let's engage their emotions, too. Our efforts were also informed by the essential work of Reeves and Reid in their book, Total Engagement, in which they identify and discuss the 10 key gaming elements to facilitate engagement in a gaming environment. These key elements can be applied to serious gaming situations to improve learning engagement, retention, and persistence. They are 1. Self-representation by an avatar, 2. 3D immersive environments, 3. Narrative context, 4. Instant feedback, 5. Reputations, ranks and levels, 6. Marketplace and economies, 7. Competition with rules that are explicit and enforced, 8. Teams, 9. Parallel communication systems, and 10. Time parameters. As you can already tell, Second Life and other virtual world's platforms already answer several of these key elements, and we endeavor to include as many of these in our gaming learning designs as much as possible. The product of our efforts is two experiential learning environments in Second Life, the Virtual PTSD Learning Center and the SNUZIUM. The Virtual PTSD Learning Center utilizes a big game small museum format, meaning that introductory and supportive information regarding diagnostic criteria for PTSD, assessment tools, evidence-based therapies, and underlying theory is provided in a museum-like format with displays and small interactive elements. But the bulk of the learning occurs in a large role-play-like game called Operation Avatar, Virtual Allegory of Trauma and Recovery. This museum, on the other hand, utilizes a big museum small game format while teaching about normal sleep regulation, sleep stages, sleep disruptions, the development of sleep problems, clinical assessment of sleep problems, and treatment interventions, all in a series of interactive museum displays and mini-games. The culmination of this experience is in the Virtual Sleep Clinic feature, where participants apply all of the knowledge they've gained throughout the museum in a simulated treatment interview game. I'd like to show you some demonstration videos of the PTSD Learning Center, Operation Avatar, and the SNUZIUM. Now we want to look at case-controlled studies. This is where we would have a group of people who are receiving the treatment. Honestly, it's easier if my wife isn't with me. It's not easy to go alone either, but when I have to look out for her safety, it's much more stressful. If I go alone, it's a bit less difficult, say a 60. This next video describes Operation Avatar Game, and is displayed in the Virtual PTSD Learning Center to encourage visitors to play. In this feature-length experience, you'll have the opportunity to meet with noted PTSD expert Dr. Kenneth Obi to learn more about PTSD and how it affects the lives of those who experience trauma. Gain the benefit of his knowledge and experience by participating in a private, virtual consultation with Dr. Obi himself. Operation Avatar allows you to experience life as a virtual patient named John, as he experiences a trauma and develops some trauma-related symptoms. You will follow John over the course of three important days in his life, the day he experiences a trauma, a day in which he experiences symptoms of PTSD, and a day after he has received treatment. You'll have the opportunity to walk in John's boots and experience things from his perspective. Expect to spend approximately 20 to 30 minutes exploring each day of John's experience. Unlike consultation and training you may have experienced in the past, Dr. Obi has some unorthodox and imaginative ways of teaching. Get ready for a wholly unique, immersive experience. You'll wish all your consultation was like this. When you're ready, you can find Dr. Obi waiting for you in any of the clinical offices in the hallway to your right. Go right in and sit down and begin this unusual and highly engaging experience. We've only launched them in November and they're open to the public now. We've learned quite a bit from this endeavor as well. First, this was a massive collaborative project between psychology subject matter experts and technology experts. It was essential to the success and validity of these environments that both sets of experts contribute. The level of trust developed in each other's expertise was essential and we are greatly appreciative to our 2B3D partners. Second, we learned that while some information display is necessary to contribute to learning, we couldn't and shouldn't just post 2-dimensional information in a 3-dimensional world. Web pages do that very well without the need for a virtual world space, so we learned to not just make boxes with pretty pictures on them. Instead, create an experience, create a world. Third, utilizing gaming throughout, even multiple mini-games enhances learning and engagement. It also provides a learner multiple opportunities to try out various approaches and outcomes with replayability, an important element to their learning. Fourth, we learned that field of dreams isn't true. If you build it, they will come as the biggest lie. Instead, create a reason for the visitors to be there. Create shared experiences that facilitate interactions. Incorporate virtual learning environments into other experiences your learners are already accessing. For example, we've started incorporating the storyline of Operation Avatar into our PTSD didactic workshop materials as a case study woven throughout the course to illustrate concepts and put a compelling story to the data. Then we invite our participants to visit the virtual environment to augment their learning and elaborate on the concepts they've learned. And finally, validate, validate, validate. It is vital that we continually evaluate and validate the learning happening in these environments, ensuring the utility of these learning experiences, and being able to demonstrate to the naysayers who assume that play cannot be useful, professional or valid, that these efforts are worthy of our time and can result in deeper, more powerful learning. Thank you so much for your attendance and attention. I appreciate that there is a community of practice for those who believe in the power of virtual worlds to enhance learning, and I'm so happy to be able to share it with you all. I invite you to contact me with questions at my e-mail address on the slide, dayhollowayatdeploymentpsych.org. Additionally, I'm happy to answer questions that you have now in the time remaining before we take you on a field trip. Dr. Holler, thank you very much. I loved your comment about fields of dreams. I loved your statement about the importance of evaluation. We do have time for questions, and we have a big one from Luke. I expected as much. Good. That's good. Luke is a good question. Are your people first designed to these virtual EDPs? Did you get any input from the groups of people who might most tend to appreciate virtual environments over physical ones, such as, for example, autistic individuals? These groups could probably give you full descriptions of why digital interfaces and interactions can be even more effective than face-to-face EDP in many situations, and how to identify these digital-based strengths to enhance your project. She personally has identified 30 factors about Second Life that provide her, as a person with autism, with better quality and accessibility of social interaction, learning, living, and healing than the physical world. She could not attend a physical EDP, but she could in Second Life. Did you ask those types of users? We asked lots of different types of users that might come into the space. I can't tell you if any of them in particular were dealing with Aspergers or autism, particularly because these environments are aimed at mental health providers rather than the endpoint user. We were mostly focused on accessibility for providers who provide mental health services and how their learning would be impacted by stuff in the virtual environment. When we first, for example, started doing our live instructor-led training workshops, we were using the exact same materials, the exact same content, the exact same everything that we would offer in our face-to-face venues, and doing all of that, like we showed the very first video there, doing that all in our virtual venue. To be honest, that wasn't necessarily so much to make sure we were engaging the learners in the same way. We assumed that would happen. Part of that was to try to ease the level of acceptability for our CE provider, our CE sponsor. They were worried that learning in a virtual environment could not be the same as learning in a physical environment. So we had to try to help them feel more comfortable with the idea that they would put their name on granting continuing education credits for learning that it was happening in the environment. But I will say this, too, is that throughout all of the development, both for these projects and also some projects I really got to do when I was at the National Center for Telehealth and Technology, we developed a PTSD environment there that was aimed at the end user to try to help them to understand what PTSD was, how it might look in their own life, how might it manifest in their day-to-day activities and kind of secondarily for family members and perhaps friends. And we spent an awful lot of time with service members who had a PTSD diagnosis and kind of what their experience was like. What does it look like when they're having a flashback? What does it feel like? Or what are some of the examples of cognitive confusion they've encountered? How does that get in the way of them doing what they want to do? We brought several service members with PTSD through that environment to give us feedback and critique it and tell us about how it was similar to their experience or how it wasn't so that we could make adjustments and adequately represent what it was like for them. One of our aims was to try to help people and in this, in the current project, help providers understand what it might feel like to have PTSD. They haven't had that opportunity. But we really did ask for the input of folks with PTSD. Now, at the same time, folks with PTSD also may have other comorbid conditions and so we got some input about many things including comorbid depression and substance use and even obsessive-compulsive disorder, things like that. So we did try to get a wide variety of representation on the folks we were checking in with. Like I said, I can't tell you if any of them also had autism or Asperger's on board, but I can't answer that part of the question. And Raleigh has a question. She says, that was absolutely incredible and so helpful for all of us working in virtual environments. She's a researcher herself. She found in a small experiment that she did in parenting classes delivery, the participants were more likely to share difficult information and or to ask difficult questions because they felt anonymous behind their avatars. Do you find that's true in your experience? I do. And perhaps not so much in our experience working with mental health providers themselves directly because there is less anonymity there. We do ask them for their real-life names because we need to be able to connect their attendance and participation with getting the continuing education credits. But prior to these projects, I have worked in some projects where we've provided counseling and therapy in Second Life to patients. And so one in particular we are doing a pilot study where we were offering prolonged exposure therapy to clients with PTSD in a Second Life clinic. And there were lots of things we had to take into consideration around privacy and HIPAA and confidentiality and ethics. And one of the ethical principles that we had is that the provider, which was actually me, I had to be very identifiable. Like they needed to know my real-life name and they needed to have my license displayed in the virtual environment and we made it such that they could click on the license in the virtual environment and it would take them to, but at that time it was the state of Washington, the state of Washington's website where they could do credential validation. So on the one hand, in direction of therapist to patient, I wanted there to be no anonymity. I wanted them to know who I was and what my qualifications were. On the other hand, with the patient, we had a bit of anonymity built in. We, in fact, in that particular project had them use a standard avatar, so it wasn't connected back to their ownership for one. And then whenever I met with them, I never saw them face to face ever. We always met in Second Life. And I had one client in particular when he first started the therapy told me, this is stupid. I can't imagine why being in this cartoonish place is going to do anything for my PTSD. But at the end, when they experienced significant, significant symptom reduction, they were reflecting on it and they said, you know, one of the things that was most helpful for me is that as I was revisiting memories of some of the darkest experiences I've had, it helped that I felt like somebody wasn't staring me down. Like they expressed that if they felt like we were in the same room together and I was looking at them while they were revisiting these memories, it was much harder to access those and be engaged with them. But because we were connected through these avatars, and I could still monitor what was happening with them, perhaps not in the same way I would in a physical environment, but we could still be communicating. I could still be supporting them in fronting rather than avoiding these memories. They were able to do that better in their mind because I wasn't there physically staring at them. And so I mentioned this to somebody a little bit earlier. You can provide both this, you know, this perception of presence and being there with somebody while also providing at the same time almost like an appropriate level of distance that was necessary for that person. I don't know that that's necessary for every single person, but for at least a couple of people in that pilot study, they reported that that was something very useful for them. We're going to do a question from Kelly, and just in case Kevin doesn't know, Kelly is a nurse and nurse educator and she is a PhD. Excellent. So she's well qualified as a researcher as well. Great. Okay. She said, great presentation, Kevin, thank you. Would you agree that this is really clinical simulation, which we know is important and has better outcome for learning? Would you pitch it as clinical simulation when trying to engage people to take up the education and training? Also, did you find the students followed similar social norms that were expected, I think she means in a physical world classroom? Okay. Both great questions. You know, we didn't probably call it clinical simulation. Because I think among mental health providers, when they hear clinical simulation, they're thinking of like an automated avatar perhaps that would play the role of a patient that they could interact with and ask questions of and they would react as a patient would. And I think, you know, many have this image in their head of kind of those medical robots where you practice procedures on them. We didn't say it exactly that way, but the concept that you're bringing up is exactly right, right? That clinical simulation or being able to get in and practice skills is much more effective than passive didactic learning. Absolutely. And so I would absolutely agree with that. What was interesting though, too, with the folks that attend our live instructor-led workshops in the virtual space, there are some rules from a classroom that people kind of carry over. And there's others that they don't. Like, what's interesting to me is even just some of the basic social rules that we seem to follow in the physical space, right? Not being too close to somebody, giving somebody their personal space, a perimeter around them. Most of our learners in our virtual world environments provide the same thing. They apologize to one another if their avatars bump into each other. Even though there's been no physical contact, nobody's been injured. They're apologizing for bumping into each other or being in space. We've set up all of the chairs in our auditorium, too, just to click to sit. And so that inevitably leads to some people accidentally sitting on the laps of other people, which, interestingly enough, seems to cause a little bit of embarrassment and they feel like they've violated somebody's personal space, even though it's avatar to avatar. So I was especially excited about Nick's presentation and just thinking about how people identify with their avatar. What does it represent to them? Because we see some of that, too. But some of the rules that we don't see carry over is, again, I mentioned it in the presentation, the rule about not talking, not chatting, not asking your neighbor questions while the presentation is going on. In the physical space, you're just not supposed to do that, right? If you're talking during the presentation, you're distracting, you're being intrusive, you're causing problems, and so it's really frowned upon and the group as a whole tends to punish that and shape that behavior. In our Second Life workshops, the opposite is true, where it's almost considered rude not to participate in at least the text chat during the presentation, where you're contributing ideas and answering questions and pointing your fellow learners to other resources online and putting in links and chat. So that social norm develops pretty quickly in our workshops, even for people that have never been in Second Life before. And so it's an interesting dynamic to watch, people kind of taking on the role of both learner, but also being part of this community that's helping each other learn new things. I hope that answered the question. Yes, I was just commenting, Kevin, that, interestingly, when I was doing a similar simulation in Second Life, whether the students stood up and walked away and the rest of the students asked them where they were going, which of course... Yes. I mean, there was no difference to the ability to continue the conversation, but it was an interesting to see. Probably all of us have experienced lag in Second Life too, right? Where our avatar is not doing what we want it to do. It's not walking the right way or it's not sitting down when we want it to. And we end up walking into walls or floating somewhere. Absolutely. And so especially our newbie attendees have those experiences too. And at first, again, there's a lot of apologies, there's a lot of embarrassment. And yet we also try to communicate to them early on that, you know what, it's really okay, you're here for one. And that actually doesn't interfere with your ability to learn. Absolutely. Your avatar could be standing in a wall and you're still going to learn. But it's interesting how folks respond to it. And as you say, if you've not sat in someone's head in Second Life, you're probably never left. That's right. So right of passage, it seems, in Second Life. Absolutely. So you've got a question from Lauren Tone. Okay. He's asking, in virtual world environments, have you found rank to make a difference, even in anonymous forms? Do some people tend to pull rank? You know, that's a great question. And I'm kind of reflecting on that now. My initial reaction is that I haven't really seen that. And I'm not really sure why that is. I mean, one guess is that regardless of what rank somebody may have, whether it's a military rank or like Muka's suggesting, right, that there really are different levels of authority, especially in the medical community, right? Doctors have the white coat, the lab coat. The patients are known that they're kind of in a lower position of power. And yet, in the virtual world space, at least with our learners, I think everybody feels kind of equally new to the environment. And so there doesn't seem to be some rank pulling there. In fact, the only thing that I see is that people that are more experienced in Second Life or have come to other of our workshops before are so excited about the space, they're working to try to help elevate other people to their level, which is incredible. I do think, though, too, that the anonymity in some ways relieves people of the responsibility of the rank. And so, you know, whereas with high rank comes some power, but it also comes from responsibility. And a number of patients that I've had in the past, probably like in the physical world, that have high military rank, they almost feel like that's a bit of a burden when they're going to seek mental health care. They're worried that others will see them, either judge them or will feel like now they're no longer a trustworthy leader. Of course, as they engage in treatment and their symptoms are reducing and they're, you know, being able to reclaim parts of their lives that we're missing, that quickly goes away and they turn into an advocate. But early on, there's this sense of, I want to access these resources, but I'm afraid people will see me do this and that's a burden of the rank that I carry. And so, the anonymity in some ways relieves them of that responsibility where they can fully engage without worrying about how that affects other people around them or that it will kind of diminish their authority or their responsibility. Got two more related questions and then I think we need to let people go on the field trip. Okay, sounds good. Here's a question from the audience. What hurdles did you have in getting this project approved and funded and how did you overcome those hurdles? Oh man, how long did we have? Interestingly enough, when I first started working in Second Life in virtual worlds, it wasn't with CDP, it was with, like I said, the National Center for Telehealth and Technology. Now at the time I was in a department that was referred to as the Innovative Technology Applications Department. And so we were always looking for what isn't being done, what can we do, what are new ways to apply technology. And so that was part of our mandate. But at the same time, convincing the powers that be that it was worth funding this project versus a different innovative project was always kind of an interesting push. But when T2, as it was called, was first stood up, they interestingly had a lot of money allocated to them from Congress without a lot of direction. And so at times we often had more money to spend than we knew what to do with. And I know that sounds terrible and pretty shocking to anybody that's ever heard about government spending. But so we had a lot of projects funded that might have not gotten funded otherwise. But the benefit of that is that we got to engage in a process that I've seen work many times over now where if you're trying to convince somebody about the usefulness of doing something, first you do it and then you show them the results. And then you convince them that it's worth investing more in that. And we can't always do that, right? I mean it often requires first having the funding or the confidence of doing that pilot project. But in that circumstance we were able to try stuff out, spend some money that was allocated for innovative research. And then when we could show that people actually were learning this stuff and were engaging in it and accessing it, that increased some confidence around that. That was one thing. So once I got to CDP and brought with me that history and that background, they were already excited about the possibilities. Because when I was at T2 I'd actually collaborated with CDP about bringing some of the workshops into Second Life early on. When I got to CDP they were raring to go. The other group that we had to convince, I mentioned this earlier, was our CE sponsors. And they were just used to doing more of the same of what they've always done, right? And that was face-to-face physical venue workshops. Anything that was online was considered on-demand learning and came with a whole different set of requirements in order to get those approved for CE credits. And so they wanted to treat our live workshops in Second Life as if it was a point and click on-demand learning course. And it took a while to convince them that that's not what we were doing and that it really needed to be thought of more synonymous with our face-to-face physical venue live instructor led workshops than these point and click on-demand courses. And part of that actually required us going to their offices, doing demonstrations with them, actually created, we created avatars for some of them. And then, as part of the demonstration, had them drive their avatar, if you will, and have the experience hands-on of being in that space where they could really understand what it was that we were trying to do. And I'm still not convinced that all CE providers out there would see it the same way. We're lucky with the ones that we have that they were willing to take a chance. And like I said, we've been doing this now for five years and offering CE credits. These are officially sanctioned American Psychological Association, ACCME, you know, National Social Work CE credits. And they're treated the same as if they had come to a face-to-face physical venue workshop, which is fantastic. But part of it is just convincing people to take that cognitive leap to kind of get over that hump that this is worth doing and people can learn. Like I said, in the meantime, we've also been able to gather some data that demonstrates that people are actually learning here. This isn't just a fun thing to do. It is. But people are actually learning, and they're learning even better here than they do in a face-to-face venue. And that's convincing as well, seeing that people are getting something out of this. I was talking to Gentil, I think, just a couple of days ago, and I've had people confront me and say, well, how can you say that this is, you know, just as good as learning in a face-to-face venue? And before we had the data, we couldn't say that. But what I would tell them is, look, that's not the right question. The right question is, you know, is virtual world learning better than no learning at all? Which is really the real alternative. The folks that are coming to our Second Life workshops aren't the folks that can make it to our face-to-face workshops, because if they can make it to our face-to-face workshops, they're going there. Instead, hopefully we're bringing learning to people who would not otherwise be able to access it. So the real comparison is, what's it like to learn in a virtual space versus not learning at all? And that, I think, people can easily get on board with, that this is better than not having access to any training at all. But in the meantime, we've been able to find those data to do some research to show that people are learning in this space and they may be learning even better than in the face-to-face. Kevin, is that all your efficacy results published? We are writing those up right now. There are a couple of articles that have been published so far. Our data specifically with our Suicide Prevention Workshop are published. And I can get that reference for you all and send it to Gentle for a way to distribute that. But yeah, Sybil Maloney would be the first author on that study. That would be great. We can accept that into the transcript. We'll give you to see if I can find it right now and I'll send it to you if I got it right now. Otherwise, I'll have to send it to you later. It'll take us a while to get the transcript cleaned up. One last question. Zinnia asks, who has been an advocate outside of your team for this kind of experience? Gosh, you know, on the one hand, there's not a lot, but on the other hand, there's a ton of people, right? So folks like you all here, for example, Zinnia and I, for those of you who don't know, back in, I think it was 2006, Zinnia, we're part of the initial class in University of Washington in their virtual world certificate program. And learned from Ran Heinrichs and their group. They're all about building in Second Life and one or some of the things around that. So, you know, certainly Randy has been a big advocate and has been out there kind of for us. Zinnia is another one that does that. You know, as we make more relationships with people who are also doing some fabulous work in this space, who are doing really important work in learning, in treatment, in counseling. I met Brenna earlier today who was here doing some great work around addictions, counseling in Second Life. I think the more we can form relationships among each other who are doing work here, we can advocate for each other beyond this space, too. So, I mean, it's one of the reasons I'm so excited to be here today. It's nice to be among people who already get it, right, and who are already bought into the idea that Second Life is a legitimate place to be and to do important work, so. I think that to me that's where the advocacy comes from outside of our group is with other people who are already here and already get it. And again, any of you are right, he's preaching to the choir. I have to apologize. We really need to give him a chance to take us on the field trip. You don't want to miss that, and I know there's a lot of other questions. Kevin, do you want to maybe post your email address? Put it at that end, local? Yeah, and you're absolutely welcome to email me with questions. Yeah. So, yeah, let me give you that information then, too. So, I want to take you on a little bit of a virtual field trip to visit the PTSD Learning Center and this museum. In a moment, I'm going to rest teleporter objects into the space here. These environments are all put to the public access now and will continue to be open in the foreseeable future. So, if you don't see everything tonight before the next talk starts, that's really okay. You're welcome to drop a landmark and to come back at another time. It takes a bit of time to get through all of it, so you probably won't get done before the next talk. So, drop a landmark and please feel free to come back. I have slurls up on the slide here if you want to take a picture of those or keep them, but those are tiny URLs of the environments as well. In the PTSD Learning Center, you'll find the role-playing game operation avatar that we saw the video on on the lower level. Just a quick heads-up about this game. There's a simulation of a combat-related trauma as part of the game. It's not gratuitously intense, but there are representations of explosions, firefights, injuries, and blood. Please use your own discretion about whether this experience is right for you. For example, people who have similar traumas in their history and are still dealing with may find them to be triggering. Additionally, in order to represent some of the intensity of PTSD symptoms, some of the dialogue in the game includes swearing. So, again, I tell you this just to help you decide whether or not this is a good fit for you. I don't have any similar warnings for this museum. We welcome any feedback that you have about the experience. There's a guest book in this museum at the concierge desk that you're welcome to sign with comments. Come where you can send me an email. Here's my email address again. So with that, I'm going to res Teleporter objects right up here next to me. The red orb will take you to the PTSD Learning Center. Here we go. And the blue orb will take you to this museum. And like I said earlier, I've got teleporters that will bring you right back to the auditorium here. But if you just click on the glowing orb, it'll open the map for you. And then you just click on the Teleport button in the bottom right-hand corner to Teleport. I'll be in the environment to answer questions that you might have. These are self-exploratory environments. You're welcome to go at your own pace and in whatever direction you want. And that's why I picked a red pill blue pill. And please do come back on the hour for the next session.