 Good afternoon, everyone, and welcome to Celebrate Learning Week and CTLT's online course design showcase session on large scale online interdisciplinary workshop design. We're really all of you. We're really happy all of you can make it here today. And if everyone can hear me, feel free to send me a reaction, a thumbs up or an indication in chat that you can hear me. Yay. Awesome. My name is John Chang. I'm a learning designer at CTLT. I'm co facilitating this presentation with my colleague Caroline Boisene, who is a Curriculum Manager at UBC Health. I would also like to thank Alisa, Rachel, our event coordinators who are supporting us behind the scenes and Suki from CTLT. We'd also like to acknowledge and thank UBC Health, the Center for Excellence in Indigenous Health, and all the health program partners who help support and coordinate the integrated curriculum. And so first, I just wanted to acknowledge that our work takes place on the traditional ancestral and unceded territory of the Musqueam people that UBC is located on. I am situated on the traditional territories of the Coast Salish people. And we also want to acknowledge the territories and lands that our participants are located on since we're gathering online today. Please feel free to acknowledge the traditional territory that you are joining us from in the chat window. We're going to provide a link to native land, which will allow you to find which territory you might be situated on. So just a couple of housekeeping items for this session. If you have a question at any time, please feel free to write it in chat. We'll have time at the end of our session to answer those questions. The session is currently being recorded, so please feel free to turn off your microphones and camera. Do not wish to be recorded. So first, Caroline will outline a bit of the background behind the integrated curriculum and also what the pre-pandemic face-to-face delivery looked like at that time. I will then describe our strategy into how we shifted online last year when COVID-19 prompted everyone to pivot online. We will then summarize some of our lessons learned, what evaluations have shown, and then we'll go into a bit of a large group Jamboard session where we will ask you for some good ideas on how we move this forward. We'll also look into considerations for the upcoming delivery in 21-22, and we'll have time for some Q&A at the end. So, Caroline. Thank you, John, and welcome everyone. So I'll start by introducing the integrated curriculum, which is an interprofessional education program that we deliver at UBC Health. So for those of you who are not familiar with what interprofessional education might be, it's when students from two or more professions learn about from and with each other to enable effective collaboration and improve health outcomes. So really the purpose of the integrated curriculum is to support the development of interprofessional competencies. And we do that by delivering a series of online modules and face-to-face workshops on a number of topics such as ethics, indigenous cultural safety, health informatics, or professionalism. So in 2020, we had six topics. We will have five topics in 2021. Those online modules and face-to-face workshops are delivered to pre-licensure health professional students, mostly in their first two years of their program. So pre-licensure health professional students can actually be undergraduate for programs like dentistry, medicine, dietetics, or graduate programs in occupational therapy or physical therapy, for example. The workshops we deliver are usually in the fall and we deliver them during regular hours of learning. And we run each activity, let's say professionalism being an activity, we run them on two separate dates in the fall to give you an idea in 2020, 2021. So last year we had 15 health professional programs participate. And so that means that more than 2,000 students participated in one of our activities. And it's important to note that those activities are a required component of student programs. So it's not optional. It's really part of the curriculum. Next, please, John. So this is really to show a little bit of what the pre-pandemic, as we're calling it, design was. So that was in 2019. So as I explained, if you look at the left of this graph, we have a face-to-face activity. Let's call it number one. That would last two hours. Students do normally a bit of a preparatory work online before attending the face-to-face activity. And so for each activity, we have two dates. Date number one, for example, would be October 1st, and 500 students would participate on that day. And then on the second day, November 1st, another 500 participants would be on that day. And then for each of the dates, we would divide the students into rooms of about 40 students, which means that we would have 12 different rooms on campus with 40 students in each. And each group would be facilitated by one or two facilitators, and we would do that for each date. And so basically what we would do would be to multiply what you're seeing here by eight because we had eight activities. So in 2019, for example, we had more than 220 discrete workshops, so 220 rooms that we had to book on campus for an activity. So that's just to give you an idea of the scale of the program. Next, John, please. In terms of the workshop structure, all of them are quite similar in the sense that there is usually some preparatory online module for the activities that would be around half an hour to an hour. And then for the face-to-face workshop, there's some introduction. The facilitators usually deliver some didactic content. And then the students are divided into smaller groups of around six to seven students to have small group discussions, and then they come back in their groups of 40 students to have large group debriefs. And so most of the activities would be designed that way in terms of the learning management systems that we used. So all the online modules were housed on Moodle. And then we would use an event management system that we call the Passport to help student register for their small groups and also to track and monitor their participation to the different activities. So now I'll just pass it over to John to explain how we made the shift online last year for this program. All right. Thanks, Caroline. So this shift to online required really extensive planning, and this started as soon as we had an idea of the decision of what the fall semester would look like, which was that there was obviously not going to be a return to face-to-face activities. So we had a number of considerations that would inform our planning. So from the student's perspective, we wanted to ensure that any redesign would align with interdisciplinary, interprofessional learning outcomes, and that there would be relevant opportunities for what we call IPC or interprofessional collaboration. We also wanted to ensure that, you know, equity and inclusion in terms of the student experience and recognizing that there would be more self-directed, asynchronous learning that taking place, that engagement would be different than these face-to-face, than the original face-to-face decisions. And we also wanted to acknowledge the workload and stress, you know, especially around Zoom fatigue of this shift to online on top of everything that was happening for students. So from the facilitator's perspective, of course, we wanted to make this a very meaningful experience for them as well. And we're concerned over video conferencing training in general as Zoom, as most of you know, was quite new for many people. We didn't want our facilitators to become Zoom experts. We really wanted them to be focused on facilitating the content and the learning activities. There were considerations around over how their roles would be coordinated in all this. And then from a UBC health perspective, there were considerations over how the content was to be delivered. The coordination and administrative challenges of delivering this at such a scale, such as how we create optimal interdisciplinary mixes, how student attendance was to be tracked for academic rigor, how programs were consulted and brought into the virtual delivery approach, and how all these were considered within the constraints of the limited human resources that we had. And then from a UBC perspective, there were considerations about the technology resources and support that were available campus-wide, and also the capabilities and restrictions of video conferencing tools. So all of this were considerations that we noted while we were planning. So given these considerations, we also had to look at the challenges and opportunities of shifting online. So creating interdisciplinary mix was a particular challenge given the complexity of the different number of students from the 15 different health programs that were involved in the integrated curriculum. The challenge of creating pre-assigned breakouts versus random, given that UBC did not or does not require Zoom accounts for students. There's also the challenge of addressing technology disruptions. How are we going to provide a consistent student experience, particularly with varying bandwidths? And how did we deal with bandwidth issues? How do we deal with bandwidth issues? There is the challenge of administrative coordination, especially around limited human resources, and how do we use Moodle, given that it's a barrier to having a singular consistent experience, which is given that UBC Health is using Moodle, which is a different learning management system than Canvas, which is what most students are used to. But there are also opportunities to shifting online. So shifting a lot of that didactic content online. There's a flexibility of using technology asynchronously. And for synchronous activities, the fact that there were no physical classrooms meant less facilitators. There were less face-to-face sessions, which meant less physical resources, and also taking advantage of centralized UBC resources like LT Hub, AV Support, CTLT. And we were also running on a principle that maybe the best approach is probably the most simple one. So with these opportunities and challenges in mind, there were sort of three different approaches that were considered. And so the first one was replicating the entire face-to-face workshop online using completely synchronous sessions. And while there are benefits of completely synchronizing these workshops on virtually through Zoom, maybe the benefits were like perhaps a more immersive face-to-face interprofessional collaboration experience or IP. And the minimal challenges required to shift the workshop content and structure, there were some very real disadvantages such as a large risk of technology issues with the scale of the delivery we were planning. And also technology training for facilitators. The second approach we were looking at was shifting much of the dialectic workshop contact asynchronously and having larger synchronous workshops with small breakout interdisciplinary groups. And so with a blended asynchronous synchronous delivery, there were advantages such as, you know, there are still relevant opportunities for synchronous interprofessional collaboration, or maybe some lower risks with respect to major technology disruptions. And there was a low demand for our facilitators with respect to technology training on there. One of the disadvantages of this obviously is, and maybe with all of these approaches was there was a required significant, you know, amount of work that was required on the back end, on our end, to move this content online and redesign activities. And then the third approach in the last strategy that I would consider was having a completely asynchronous delivery. And so while asynchronous provides low risk in terms of technology disruptions, maybe low demands on our facilitators with respect to technology training and less administrative coordination. There were significant challenges in moving content online, designing meaningful and relevant and professional activities that may not have aligned with interprofessional outcomes. And also, there was a challenge a real disadvantage that interprofessional collaboration might be less valuable for students have done asynchronous. And so the decision was made to redesign these large interdisciplinary workshops with both asynchronous and synchronous activities. So asynchronously students would use Moodle, retain Moodle for self-regulated regulated learning, but they would also use it before a synchronous session for pre-reading, previewing video vignettes and submitting post workshop assignments. So synchronously students would participate in large scale facilitated Zoom workshops with breakout rooms of interdisciplinary groups. We actually performed a risk benefit analysis of randomly assigned breakout rooms versus pre-assigned breakout rooms for these ICs integrated curriculum sessions. And so for black cities and risk of pre-assigning interdisciplinary, interdisciplinary breakouts, we recommended randomly assigning students to breakout rooms. And for sessions where the disciplinary mix was particularly uneven, students from disciplines with small numbers were assigned to breakout rooms manually to ensure that they were spread out amongst rooms evenly. So this was a workshop wiki which we used, which would be shared with students, and this provided an accessible low bandwidth login free way for students to access workshop information content and activities which included an embedded bat padlet and video vignettes and I can share that with you here. This is just a sample of the wiki that we created and a breakdown of the schedule of the workshops, activities, and the video vignettes that were shared during that workshop. And also, I can also share the Moodle site that was used for the pre-reading and post-bark workshop assignment templates. So this was all sort of part of the whole workshop experience. And so the goal of this design was to provide a consistent inclusive experience for students with the least amount of technical barriers as possible for students, facilitators, and UDC health moderators. To decrease the number of synchronous sessions on Zoom, we shifted to activities to completely asynchronous. There was a redesign of activities for the synchronous session with small breakout rooms, the use of these padlets. We used the chat for questions, so really creating sort of that trade-appear engagement we were trying to. We were really seriously hoped for in terms of interprofessional learning. And we really tried to take advantage of UDC support resources, so IT, AV support, and all of these decisions were done in collaboration and endorsed by the health professional programs. And so the results of the 2020-2021 delivery is summarized in this graphic. There were six topic areas covered by 11 online modules. There were 77 Zoom sessions attended by about 2,000 students representing the 15 health programs. We had 98 facilitators and about 22 moderators who were supporting the workshops in the background. And this is a breakdown of sort of the students by program. Thank you, John. So in terms of sort of lessons learned from what we did last year, so from the UBC Health perspective, of course, having a partnership with CTLT was absolutely key to the success of this delivery. And we were very lucky, of course, that we partnered with John, who was working for UBC Health previously. So he had a really good understanding of the integrated curriculum and of the online modules. So I think that really facilitated the work. It was also very important, as John mentioned, that we had this agreement with purchasing programs on some key principles to guide this transition. And that would really help us make all the decisions throughout the process. We spent a lot of time really understanding the technology capabilities and limitations. And I know that, you know, faced with this situation last year, it was really tempting to just make decisions quickly and say, OK, we're going to use this and do this and that. But we really took the time to compare different tools, looking at Zoom versus collaborate, what were our needs, what would work better. And that really, really helped and sort of paid off afterwards because we I think we made the right decisions with the tools. Creating this moderator role was also very important. So we didn't want to have to train all the facilitators on the technology roles, because as you will have seen, we had 98 facilitators with different levels of comfort with technology. So we created that moderator role for staff basically at UBC Health and from the Center of Excellence in Indigenous Health. And so moderators were responsible for creating breakout rooms, launching polls, everything you have to do on Zoom. And that was a really important thing as well in the success of the delivery that enabled facilitators to only focus on facilitating the content. We had to pay attention to training the facilitators on changes to their role. So not that they had to worry a lot about the technology, but that they had to be reminded that they would be facilitating large group of students with, you know, up to 150 students. And so they had to understand that this would have they would have limited interaction with the students. So it was really important for us to make sure they were aware of those important changes. We paid a lot of attention to communication, communication with programs with students with facilitators. We tried to make sure that all the information we shared on processes and all the changes was as clear and transparent as possible. And I think that was also important for the success of this delivery. And compassion and patience. I think this was really key not only for us, but I think for everybody who was doing that work a year ago, really having compassion for all the programs, all the students who were going through this and compassion to ourselves really for trying to navigate this and to do the best we could with what we have. What we had. And so that was really important and a lot of patients just making sure that we were explaining things, you know, over and over again, sometimes to a number of programs just to make sure everyone had the information that they needed like to continue John. You're on mute, John. Sorry about that. I think that compassion really lent itself to also being us being really flexible in the approach and really trying to think of the most inclusive way we could create this or develop this experience for our students. We also wanted to really capitalize and leverage on the resources that were available at UBC so taking advantage of the enterprise software so zoom and CTLT help and every supports help and UBC IT support support. And we also wanted to, you know, thinking as much as you know, as sorry, being as asynchronous as possible with the content and maximizing the interprofessional synchronous learning experience as possible. So we'll just present some of the evaluation so we did evaluate I mean we evaluate the integrated curriculum every year but we did an evaluation last year with, I guess an eye on the online delivery. So, I mean, it was great to see that students ratings of the overall value of the integrated curriculum actually increased in 2020 compared to 2019 when it was done face to face. And also the value of individual activities increased, as you can see in this graph. So it's interesting to see on this graph that the two activities that we shifted completely asynchronously resilience and health informatics so those two activities didn't have a synchronous component didn't have an interprofessional component. I think this is where the increase was was the the higher. So that was interesting data for us for us to see. The students ratings of the overall delivery were actually very similar in 2019 and 2020. So I won't maybe read through all of this but you can see that the numbers were were pretty good in terms of technology use the matter of work of work, the role of the facilitators and the small group discussions. The facilitators also felt that the delivery format worked well for them in 2020. They felt that the students were attentive and responsive during the zoom sessions, they agreed that the technology used was appropriate 100%. And most of them agreed that the design of the sessions was appropriate to meet the learning objectives. Next please. Oh, just one one before. Yeah, this one. Thanks. So we found through the evaluation that the online delivery had important benefits to program students and facilitators. So for programs, as John mentioned, they had fewer facilitators to recruit because the zoom sessions had more students. So that was definitely appreciated by the programs. They could also increase their facilitator pool because they could recruit facilitators outside of Vancouver pretty much anywhere across the province. And they had sort of less communication to do about logistics because that's something we we really made sure to pay a lot of attention to through UBC health. And there was less logistics of having to communicate information to facilitators on where to go and what to do. For students we were really, really pleased to read that they found many discussions were still very rich, which was something we were a little bit worried about going on zoom with the large group but I think having a lot of time for students in smaller breakout rooms really helped with that. Less time wasted way finding because prior to 2020 they we had rooms book all over campus and it was really hard sometimes for the students to even know where they were going. And also it was the ability this year for students at distributed sites to participate so some of you might not be aware of this but the undergraduate medical program for example at UBC. They have students at four different sites across the province. And so it was difficult prior to 2020 to have them participate of course in our sessions and have a true interprofessional experience so having everything online made that possible. And for facilitators, same comments that less time wasted traveling and way finding on campus was really great. Less troubleshooting with technology in rooms which was always an issue every year where facilitators didn't know what to expect in the rooms that they were going to use for to facilitate. And so overall less stress for the facilitators. Next please. And so we've sort of identified a number of enablers that help to for effective program delivery. So as we've said, shifting didactic content to self directed online modules was was really key. We had a co facilitation model. So in each synchronous zoom session we had three facilitators, which was really helpful because it was the first time for all of them of course doing this online so they could really help each other, especially when some of them had to levels of varying level of comfort with them technology. So having the technology moderator role, as we've said was really important. And we were able to do sort of a more effective framing of the integrity curriculum, because we spend I think less time on logistics than previous years. And so we were really able to communicate a bit more and a bit better to programs about, you know, the purpose of the integrity curriculum, and make sure that it was really well understood by programs and students. We were able to do the facilitator training online. And this was something we would normally do face to face prior to 2020 and attendance was really high to those training, because we did it online. Some areas of improvement that we identified through the valuation was to create more time for breakout discussions. So even though we really sort of redesigned the synchronous zoom sessions to have most of the time spent in small breakout rooms students still felt that they could use more time. So that's something we'll be taking into consideration, increase consistent attention to interprofessional mix in the breakout rooms. As someone mentioned, we used random breakout rooms to create those small groups and so we rearranged them manually those breakout rooms to make sure they had a good balance of students from different programs but it was still not perfect. So, so we'll continue to work on that and then to create mechanisms to increase accountability for individual participation. It's mostly true for the activities that students do in their small breakout rooms that are self facilitated. And so we've heard from a number of students that some students could just be, you know, on mute and not have their video on and not really participate and only have their names for the group assignment so we're trying to think about ways to increase that accountability and maybe some of you when we have the discussion piece can can share some ideas that might have for us to do that. And so, yeah, what we're going to do next is ask you to think for a few, for a few minutes about these questions and it's not going to be a breakout room activity but it's, it's going to be a large group Jamboard and so I want you to take about five to seven minutes to think about these questions. What would you do differently, considering the design considerations challenges and opportunities. What would you do to read the redesign or rethink how we delivered the integrated curriculum. And so, we're going to insert this Jamboard link or Charlie has already just shared it, which will be used to post your sort of sticky note ideas about how we should really rethink this. And after about five to seven minutes will come back and discuss in a larger group debrief. And if you if somebody has already posted there, a really great idea and you, you like that idea I would simply just use this pen tool and do a check mark like that. So, we'll give about maybe five minutes five to seven minutes, and we'll call everyone back. I think maybe most people have had a time to contribute their thoughts. What do you think, Carol? Yeah. So maybe we can go through some of these. So obvious linking. I love it. So, how do you continue the discussions after the workshop I think that's actually a good opportunity to use maybe asynchronous and asynchronous platform, the asynchronous discussions on canvas. Maybe Carolyn can speak to that with respect to continuing. I'm actually noisy at the moment at my place so I hope the sound is okay. Can you hear me fine? Yes, you're perfect. Okay, great. Yeah, so we've definitely thought about continuing the conversations after the workshops I think one of the limitations for that is that we were given I guess by each program. What we call protective time to do those activities so we don't have unlimited time that we can ask students to be working on those activities. So that would be some sort of consideration that, you know, any work that we do after a workshop would have to be somehow considered part of the activity. So maybe we would have to reduce then the duration of the workshop itself of the pre-work. So it's just one limitation I guess. But as you were saying, John, that's something that we were not able to do maybe using Moodle in the past and that we might be able to do as we are transitioning everything to canvas this year. So having more functionalities to do that on canvas. Definitely a possibility for sure. Is there any advantage to using the Zoom breakout feature that now allows you to choose what room you go to? Would this make it more diverse or more work? That's a great question. I think it's to sort of, I think, Caroline can speak to this, but I think to maintain that interdisciplinary mix we wanted to ensure that students were placed into interdisciplinary groups and not simply choosing groups that their friends or people that they might know might be in. And so we really wanted to create sort of that interprofessional experience. But didn't you indicate that you randomly, you randomly assigned them? Or was someone behind the scenes manually putting them into groups? No, so they're randomly assigned. So we just tell Zoom, please create whatever, six groups. And so they're randomly assigned. But because we ask students to rename themselves, as you were saying, Donna, on the Jamboard. At the beginning of the workshop and indicate what program they're from, then the moderators could just go once the breakout rooms have been created, they can just go there and see, excuse me, if they see too many students from the same program in a room, they can just rearrange the rooms manually. So that's what we've done as moderators to try to make the breakout rooms more balanced. But I think, I mean, we didn't have that feature a year ago, right, to have people choose their own breakout room. That's something we could do and that students, we could tell them, okay, just have a look through all the breakout rooms. And if you don't see anyone from your profession, or you don't see a lot of people from your profession, please join that breakout room. That could work. But I think the limitation is the one that John just said that they might tend to go with their friends. And we, we rather, I guess, create different groups each time. So yeah, I was thinking if the groups are like a group on group two, and then you said to all you gave all the students a number and that they go to that group. So it's, but still a lot of work behind the scenes for the moderators to do that. I think. Yeah. Yeah, for sure. Because we're looking at groups of 100 students. So it's, it's a lot to do and we don't always have the time as moderators to do that, before the first breakout room has to start. What else? So I like this comment of someone saying wonder if there's a way to have the breakout rooms facilitated and do students like it when teachers randomly pop into breakout rooms. So we've heard that they don't like it. I mean, not so much that they don't like it, but it's sort of sometimes it really stops the dynamics in which they were in that we're having a good conversation and someone comes in and they have to explain, I guess, you know, what they were talking about. So we've heard that it wasn't really great. And the limitation for us is really that when we have a group of 100 100 students, that means 15 breakout rooms and if you have an activity that's for 20 minutes, basically the facilitators don't even have time to go in each breakout room. So that's not something that we've tended to do. So we try to, for each activity we ask students to actually define or assign a facilitator within the small group and assign a scribe so that really have to self facilitate. Is there a way to track which disciplines are interacting in the breakout room, the student participation pie chart you shared earlier seemed like there's a healthy mix of disciplines and so each session is designed to, I guess, hold each session is has been predetermined for, I guess, I believe a few disciplinary mixes. And so there is a way to know which disciplines, we already know which disciplines are participating in each workshop to determine who is participating in each breakout room as well would be a little bit more difficult if it was randomly and so we're trying to think of ways in how this could be achieved. And so for the fall, for this upcoming fall delivery we're actually considering, because we are moving this to canvas, using canvas groups and potentially creating interdisciplinary mixes before that on Excel spreadsheet and uploading those interdisciplinary groups into canvas that way. For interdisciplinary mixes in a workshop. I think that's still something where we're still trying to also figure out. Yeah, I mean, we'll probably do the same rate for the all the reasons that we've talked about. It will be easier in a way for moderators because we're will be having in 2021, slightly fewer students in the large zoom session so last year we had up to 200 students. We're trying to decrease that number to no more than 100 students in each zoom session so it will be easier for the moderators to rearrange that group those groups. But just to that point join I think, as I mentioned earlier, we have two dates for each of our activities. And on each of those dates. We have a group of programs that participate so we always know what program participate on what date. All the programs can be on either of those dates on day one we will have let's say OTPT and nursing students and they too will be only pharmacy dentistry and medical students so we know that initial disciplinary mix. And so then in the breakout rooms it's those programs that get mixed together for that particular day. There was also a common saying building better directions for the task in the breakout rooms, perhaps to increase the amount of individual involvement. So that's around accountability and there was another one create some individual assignments that students must prepare and submit before synchronous breakout room sessions. And another one in this I guess theme require the group to submit a summary after a synchronous breakout room sessions. I think those are all really great ideas we're definitely considering improving the the direction for the test that we give to students in the breakout rooms because that's we got comments from the students that they need to have very clear tasks so that we'll be paying some attention to you for this year. And we do require the group to submit a group assignments at the end of each synchronous breakout room session and that's how we track attendance. But I quite like this idea of creating some individual assignment that students must prepare and submit before. So I guess for us we've been using the assignment as a way to track attendance so that would be, we would still require to have something that they also submit after the workshop. So that would just be more work for the students. So lots of limitations and things but thanks for sharing those ideas with us that's, that's wonderful. If we have one more slide before we get to a Q&A. So really based on what we've presented and and the evaluation that we did. We made some decisions already of course for the delivery in the fall of 2021 and we're working on the planning right now. So we've made the decision to deliver the integrated curriculum fully online again. Although UBC has said that some classes will be face to face we felt that because of the nature of interprofessional learning and the fact that we are mixing students from so many different programs that we would just deliver online again. We are transitioning to Canvas as we mentioned which is a big project in itself that we've been thinking about for a number of years and we are finally doing. So this will have implications are our communication processes information we share with students programs and facilitators. We will not be using the week is anymore because we will just have the ability to have this information on canvas and have it visible to the students only on the day of the workshop. So really, we're seeing some great advantages of being on canvas for these kind of features. We are faced with some unique challenges with interprofessional group creation on canvas because canvas is not I guess designed for interdisciplinary learning. So it doesn't when you have a group of students from different programs enrolled in a given course it doesn't actually show you what program they're from. So that's a bit of a problem when you want to create interprofessional groups but we are we've been working with the LT hub and the data governance unit to find some solutions. So and that's something that maybe John at some point will want to present in another presentation because it's been interesting. We're making as I said also the decision to have students do an individual assignment or at least submit an individual assignment at the end of each workshop instead of a group assignment to increase accountability. And we're adjusting the length of some of the zoom sessions so last year we tried to make them as short as we could some of them were just one hour. And we heard from students that that was actually a little bit too short so we're moving back to one hour and a half for a number of them. We're having fewer students in the large zoom session as I said so closer to 80 to 100 students versus the 150 that we had last year. And we're continuing with the co facilitator model the technology moderator role and the online facilitator training as we heard those really worked. And I guess beyond 2021 will have to have of course conversation with our program partners to see what the delivery will look like. But what we're hearing is that we might go back face to face for some of the activities especially the ones that have more sensitive content, like indigenous cultural safety, some of the ethics curriculum, but that they would like to continue the online delivery for other activities so we anticipate that we will just have a blended approach and that we will probably never go back to having to book 220 rooms across campus which is not a bad thing. So I think this has really changed the way we're delivering this this program. With that we have about five more minutes of Q&A. If you have any questions feel free to type them into chat or turn on your mic. Um, just so fantastic you two well done well done well done really shines the light on on UBC and how innovative and committed we are to this. Um, thoughts about what other large university large scale universities are doing similar to this or are we kind of the most innovative and leading the way on this or are there any lessons we can learn from say U of T or McGill or Delhouse et cetera. Yeah that's that's a great question Donna and it's interesting because of course when we were doing this work a year ago I was trying to search the internet to see what other people are doing please help and of course we didn't know at that time. So since then I've heard of a couple of examples. And it's interesting to see that most universities have done something quite similar. So some of them have actually moved everything asynchronously so just made the decision not to have an interprofessional component or just moving online through discussion boards. But then other universities have done something quite similar to what we've done, moving, you know, the sessions on zoom, having slightly like bigger groups. And yeah that's that's what I've seen I don't have lots of example I've seen only a couple of institutions but I think that's something that would be really interesting to monitor and compare. Yeah, across Canada or even in us with the IP programs. Yeah and I haven't actually looked into the, the literature or the recent literature to see what's, what's being done, especially with interprofessional education, but it would be interesting to see what, what some of the larger universities are doing. Any other questions. I guess I'll ask again I kind of know the answers to these but I, it's always good to ask some more. Are we going back to or do we do any pre and post student survey around change like change in in them around competency acquisition. So from the, I know from the beginning of participating in the IC to the end. It's hard to isolate just this integrated curriculum because many programs are doing other forms of interprofessional engaged learning. So I know it's hard to isolate it but So, I guess just to for other people to have the context we did an evaluation so prior to pre pandemic in 2019 we did evaluate the interprofessional competencies of pre post so we asked students their perception of the interprofessional competencies before doing the integrated curriculum and after they had done the integrated curriculum. Although we did that as a retrospective pre post so they answer the question at the same time. We didn't get like the results we got. It was a slight increases like it improved slightly it wasn't really significant from the data that we got. I think as you're saying Donna what we're seeing is that of course because the integrated curriculum is delivered throughout two years of their programs. They are exposed to a number of other interprofessional experiences outside of the integrated curriculum. Mostly in their clinical placement so it's really hard as you know to isolate really where they are getting those competencies from. The direction that we're moving towards is to work with programs to encourage them and help them measure those interprofessional competencies throughout their programs like throughout the four year two years whatever it was including the activities of the IC. But I think from what we've done it wasn't really reliable data, what we got from that 2019 evaluation on interprofessional competencies. But again this is ongoing conversations and it's, especially with the programs it's not something that says UBC health we decide by ourselves and it's, it's an ongoing conversation with 15 partner programs. Any other questions before we wrap up our showcase today. Thanks Donna. Thanks all. If you have more questions feel free to email Caroline or I Caroline will be only with us for another couple of weeks she's unfortunately she's leaving us forever. She's going back to France. If you have any questions I'm sure both of us can can help answer any of your questions. I would like to plug a couple of celebrate learning events. There's course your celebrate learning week events calendar, which you can find here. I'll put that in the chat. And also, my colleague man well is co facilitating this workshop on blended learning with another colleague from CTLT, because right and Dr. Charlene black from the school of population and public health. And that is shaping our future blended learning from a different lens. And that can be found here. And my other colleague try to transition to that slide. My colleague Dr both on Kim is co facilitating this online course design showcase on single point rubrics with Dr. Alifa bandali from the Institute for gender race sexuality and social justice. If you would like to attend that event that showcase, which is similar to the format you find here almost. And I'm just going to copy that like sorry about that. And finally, there is one last showcase I want to highlight which is going to take place sometime around the end of June, June 29 I believe my colleague. Sook, Zhang is going to be hosting another showcase on team based learning. And so look out for that on CTLT on the CTLT events page. And we really want to thank everyone who's attended the session and. And one last thing is, please fill out the feedback form that Lisa has highlighted in the chat will give us some really good feedback on today's session. And I'd like to thank my co presenter Caroline for doing this presenting this with me was a real pleasure. Thank you so much, John.