 So, welcome everyone. My name is Judy Chan. I'm an Education Consultant with the Center for Teaching, Learning, and Technology, and I'm here today with Hailein. Hailein, would you like to introduce yourself? Okay, I'm Hailein, Hailein Chan. I'm a learning designer with CTLT, so I'm most involved in many projects working across campus in different faculties. To help faculty design courses, either online, blended, or face-to-face. Thank you, Judy. Welcome. I'm going to say this again. I am joining from my home in South Vancouver that is in the unseated ancestral and traditional land of people from four different nations, Muslim, Squamish, Staira too, and I would like to say, I'd like to call Salish people, the four people. Sorry, I'm just so mindful that a few of you have been in my other workshop this week. This is my time for me to reflect my relationship to home in my home area, just because around me in my neighborhood, there's no signage, there's nothing. Yeah, there's just like in South Vancouver, there's just no indigenous community and there's nothing around me. So I really want to use these opportunities to think about my relationship here as a parent, as a first-generation immigrant. What can I do to preserve the traditions, the history, to understand it first and then to preserve it and then hold my own children to understand our relationship with the land, with our everyday actions. So I invite you to also join me to think about that for your everyday life. So it's just a beautiful totem pole right next to the forestry buildings on campus at UBC Vancouver. Here is the agenda for the session. So we are going to review some online assessment principle. We will start with just assessment principle and then moving on to online. We will look at some UBC examples with the focus on authentic assessments. Authentic in this context means Highline is going to talk about authentic assessments, but some examples of what faculty members at UBC are doing and trying. And then we will get into some breakout group activities later and where you will share and chat among yourself because I think there's so much. For example, I haven't seen this Russian, the language example before, but perhaps some of you will have some ideas that can offer to Vita. So we will get into small group sharing and then we will also touch on ideas on how to promote and introduce academic integrity that Vita asked about. And at the end, we will share some quick, briefly introduce you to some technology and resources where you can find additional information. And then I'm going to pass this over to Highline because I am the host of this Zoom session. So Highline is going to tell me when to go move to the next slide so it may not be as smooth as it should be, but Highline will tell me when to advance. So Highline over to you. Okay, thank you, Judy. So, yeah, before we dive into some examples, let's review some online assessment principles. Before we present any content, actually, we would like to know what do you know, before we present any content. So we have some ideas, but we would like to gather some of your thoughts here. So now you could invite you to use the annotation tool again to share with us your some of your specific strategies you might currently using for your class. So we are aiming to get about 10 to 5 to 10 entries before we move on. Yeah, so how about use the annotation text tool as highlighted in the bar on the top there. Can you find your annotation tool? Yeah, it should be on the top. If you don't see that you might see a more option on the top of your navigation. I mean, the top there's a black bar there. And you can choose annotation and then choose text. Yeah, we give you a couple of more minutes here. We hope we have some entries here before we move on. Okay. Hey, I see some coming in here. So we see for some formative assessment. Some of you have some activities for discussions group presentations and assignments. Yeah, for for the summative strategies, there is exam, right. And we see low stake quizzes, followed by a later and exam, right. So these are, I suppose, is the formative type of the assessment. Yeah, you can resize your font or replace. Yeah. So thank you so much for making clear for meeting on the screen. Yeah. So, oh, some of you are thinking about simulation, stimulating the final exam for learning process progress test. Okay. Yay. All right. So quite some ideas over there. All right. Any more entries. So these are some of the, oh, I see that they're down at the bottom there scaffolded assignments, right. So that is probably a kind of milestone type of assignments. And then you target help students to find to do some final project over there. So that's a pretty good idea as well. Okay. So with that, we move on with some review of the principles probably here, right. So when thinking about online assessment, we would recommend consider the assessment as any means of the following, evaluating your student achievements, providing feedback, or even just moving your student forward in their learning process. So the assessment, as you, you put the some examples of there it can be formative and summative in online learning, probably the importance of having effective strategies that can help students to forward to their learning progress is more important, probably more important, because of the not because of the lack of the immediacy of there. So we would encourage putting more thoughts, thinking about some more formative type of assessment, like examples you provided over there some, some kind of, for instance, self check quizzes type, and even sometimes it might be just a peer commenting. And some self reports, some brainstorming activities, informal checks, those kind of activity, thinking about more, probably put more thoughts, especially if you're fully online course. You have a, well, limited, you know, live session, and you have a lot of, for instance, the asynchronous type of assessment activities, you might thinking more to provide some more formative type of assessment. Next, please. Yeah, so for online assessment, share actually the same principles as other teaching modes, or the assessment here, a kind of listed, whether they are online blended or even exclusively face to face can, can be guided by the same principles, in fact, as the following, actually, the first one is evaluating learning process. So this is a recommend moving away from evaluating your students performance to evaluating the process. The next one is a line assessment strategies. Basically, this one is a line assessment with learning outcomes and the teaching and learning activity. The next one is authentic assessment strategies. So basically that's a kind of thinking about some opportunities to apply real world ideas, use a variety of assessment methods. So this is a kind of reminder, not just midterm and final with very heavy weight. And the next is both a summative and formative. So here, try to see a balance between the summative and formative have some type of some degree of the separation for between the grades and feedback distributions for for your overall design of your course. Provide multiple opportunities this especially for online courses, probably thinking multiple opportunities for timely feedback on your students learning progress. So this timely feedback, it doesn't have to be, you know, always provided by instructors, it could be using some features, while have some self checked or set up a kind of a go hands in hand with the next reminder there is having some type of self and peer assessments. So that is that could provide enough opportunity for students to self and peer reflect their learning process. So that is for students actually is for them to reflect on their own work and for instructors is it could be a way for you to try to balance and manage your instructors workload actually, because actually for fully online course that you might find it's a pretty, you know, labor intensive type of engagement in there for both students and for you as well. Have transparent and clear expectations. This is especially for online class. Have to put ahead of time some clear grading and criteria and rules that those kinds of consideration. So these principles actually encourage authenticity and the transparency within the assessment practice. Next please. So I'm on time as what is authentic still not sure what authentic assessment strategy means. Okay, so next slide so we have all that. Thank you. So, yeah, when you're thinking about sense authentic assessment. So, so what we highlighted here are some features or examples of authentic assessment strategies. So, I'm just using a very simple one okay while I'm using case study for instance as an authentic assessment idea right to elaborate the features. You can thinking about. Okay, or you put down some knows some other authentic assessment ideas that you might have in your mind. While while we discuss. So, so if you if we thinking about because a case study is used across I mean disciplines and different type of courses. So when we're thinking about like case studies as authentic assessment ideas. So when we design that case study, then what we do is we, we pick or make the case based on a real world, while in situation, right, and we use real life situations to make a case. Then we try to include activities closely matching the real world tasks to help students wanting to make connection of their previous knowledge and skills and try to help them to solve some real world problem right and along the way we allow multiple pathways for the solution for a specific problem right it's kind of problem based on learning as well so it's an authentic assessment is many many times that you'll define and open ended. So it's kind of mirroring what is really people could encounter in their real life, and highly engaging opportunity that can help students thinking after the forester their higher order thinking skills. Right. So, so this is just a one idea so so what some ideas come to your mind. So, keep a note, and then when we have a break, break out rooms activities we would like you to share ideas with the participants here today. And then we would like to hear back from you later. Okay. Well, because I sorry for into this because this is really supposed to be a high lands part but because I think I recognize a couple of you here in the audience I've met you before. For example, in medicine. There's a lot of in, in, in addition to memorizing all the body parts and all the names of the different parts, they students need to learn how to apply that in a real low in a case situation where there's a patient with the different symptoms. Now they need to solve that problem. It's really real world real life situation. And I imagine in this course in Russian language that she might also ask students to translate a piece of instructional manuals such as how to mentor your students, or in translate a newspaper article into the Russian language. So, again, taking real life example, and helping them to make that connection from the individual vocabulary and grammar, the, the, and, and anatomy the body parts that the nervous system and the drugs and then putting them together to, to heal a patient to get a Russian piece of newspaper article. So those, that's what we, that's to me, I think these are some real authentic assessment here in front of us in the audience. Sorry, hi, now. Yeah, yeah, yeah. So, on top of that, for instance, some examples for forestry is that they pick up a half student to go to the field, a kind of go to the field school, and then try to analyze or kind of analyze the sites, how to manage the, the natural resources, those type of activity. Okay, so we have some examples there. Okay, the next slide is this one is on this slide. Here's a list of the most commonly used online assessment method. Here, have you used any of these strategies. Again, you are welcome to use the annotation tool again to show us what methods you are using or plan to use for your online class. So, so evidence informed assessment actually are key to improve the overall quality of the students learning. So there are many different ways of the effective assessment. So the most common ones. So on the left hand side, the most common ones, people usually use those most common ones on the right hand side will list the sum of the strategies that is collected from more experienced instructors. So, let's see what you are using right now I can see. So, so we can see online discussion is pretty popular used among you and online exams, AC written assignments, project groups and E portfolios. All right, so you touch all almost all of them, right, for the commonly used ones on the right hand side, we can see some of you have those new approaches, concept mapping, digital media project reflection of the reflective type of writing or generally, or even simulation activities. Okay, awesome. I thank you so much for sharing what you are currently doing. All right. How about, let's move on to the next. So with all, all this, any sets of the recommendations here, not all of them will be helpful in a given situation so you will have to make this decision, which one of them are the most appropriate for your course, maybe. So when you're thinking about the design of the assessment is like you might thinking about the context workload and availability of teaching TAs those kinds of practical issues. And when transitioning here, we listed some transitioning considerations of assessment when you, especially when you design online courses. So the first one is have front heavy preparation probably that's a reminder these are just reminders here. So, because when you design you're converting your face to face online, the first thing is you might want to reflect what to keep right what to remove or replace. And some activities that you designed on week my is a if you're face to face you might design them on on daily basis or a kind of like that. But when you move it online. Some activity might thinking about designing on weekly basis, especially for asynchronous activities. So this will change the overall design. So, so, so we'll need a kind of revise the detailed assessment descriptions that rubrics and supporting resources and actually emphasize that is that the procedures instructions should be clear for students so you can see these is going to take time. The next one is a course in the media organization. So this is about how when you design for the fully online course that how to organize the course that helps students streamline your activities and assessment and how to make the process clear, simple and actually is easy for a student to follow. The next one is the informal assessment opportunities as I just mentioned a little bit. So this may need to thinking the balance of how much to you want to have some breakdown activities or or formative assessment right need to avoid both too many or too less. So, so because actually we we hear feedback from students saying that students are really struggling and frustrated to track and follow so many small deadlines for so many online courses simultaneously they are taking right now. So, so that is something a reminder and to for you to thinking about communication with students. This is about assessment expectations to procedures and how you are going to communicate that so we might recommend use announcement right in what instead of using email. Maybe the more effective ways announcement discussion board have a general Q&A and if you have live sessions and live sessions could point on top of that so so help a kind of do not a kind of give a student a kind of leave them frustrated with so many channels of communication. So that's another reminder and adjustment adjustment time or opportunity. This basically is about whenever in using a new lab tool or application in your courses that you want to implement your assessment or activity ideas provide an opportunity for students to practice doing that ahead of time. So this might not just for the final or midterm so sometimes is for some big project you want to student to to accomplish and then if you use any new tools to give them some opportunities to practice that. So these are just some reminders as you trans going to convert your courses for online. Okay with that said I'm stopping here and try to see if you have a questions or not. Otherwise I will hand it over to Judy. Thank you. I'm staying on this slide for now. I know I'm supposed to move on but we are towards the end of the term now we are hearing more and more from students. One thing that they really want to know one suggestions that we are hearing is the students really need to I'm really sorry I'm really stuck at that part about accessible and easy to find instruction. Students are not able to find our instruction it seems it could be very logical to us. To us it's very easy to find but I am really thinking that our students are taking four, five, six courses. Each one of us faculty members are organizing and using Canvas slightly different. I know in some faculty in arts there is a recommended template that we should use but our students are taking courses in science, in arts, and then for systems, everywhere. They see different ways of organizing information and some students may be so comfortable with receiving the information in the announcement but they and therefore they're looking for that instruction in the announcement but if you decide not to do that they will be lost. So a lesson that I'm learning from our students is that we need to keep repeating in different ways of what we expect them to do. So not just one simple logical way it's logical to us but it's not logical to us. And then another thing is yeah multiple way of telling the students what are we doing. And then the other thing that students are asking is a routine. You will have something post on Monday at this time. They need to do something by Thursday and there's something that will be due on the weekend. They want that routine every week. They just want a routine. So if you're still finalizing your course design for term two I would like to ask you to remind you to think about these two latest lessons that I'm learning from our students right now. Okay keep going. So I would like to share some examples at UBC on how our faculty member brings the real world and real life to the classroom to the assessment strategies and also what they are doing with the online environment. The first example I'm going to use the example of online discussion. Many of us are familiar with the idea of having online discussion. But what is new in this course is again they are using slightly different in different courses in this four I think it's four history. In 502 it started with a thought provoking discussion. They've done the reading. They are supposed to have done the reading. But the faculty member is asking them something that is bizarre like this question that is debatable so the students need to debate and share their thoughts. And then for 501 is every day life in the local views of the policy there's something that's happening in the city. And then in the forest in the province and then they need to discuss it for everyday news. So both of them are real and asked for students to reflect on what they are seeing and apply them and talk about that with their colleagues, with their classmates. The technology being used is just the simple campus discussion board. One thing that the faculty members are doing to enhance academic integrity and also to make sure students are actually thinking. They don't just simply say, oh, I agree with you, Peter. Hey, Susan, I really like your point with those saying what they like about that. They just say, oh, I really like your idea. So students actually need to do a reflection, drive a post and they need to post it before they can look at everyone else discussion. Even though there is a chance of repeated idea, you know, after they drive the thing, they recognize that that same idea has been expressed five times by other students. But at least the students need to do that reflection first. In order to have meaningful discussion among the students, we recommend a smaller group. In these two classes, this group size is less than 30. I personally find it still to bake. I would recommend 8 to 10 students in each group, even though they are talking about the same thing. Having 8 to 10, you get to know everyone. You get to know who is at what and what is their perspective. So then you can have a real discussion. Even 8 is too many, but I'm just afraid that if we keep this small group size too small, it's too much monitoring to do. At the same time, what if a couple of students not contributing. So small group, so it's easy to have that meaningful discussion. So this is not something too unique. We are doing discussion, but just like small change of how we begin start that conversation. What we can do is to start with a very controversial idea, a very local idea. And so online discussion, we've done that. And in terms of rating, I'm also hearing ideas about it's not, of course, we don't want to just tell the student you need to post and then reply to each other three times, four times. Student is seeing it as busy work. They will just go in and say, I agree with you. I disagree with you and just try to make that minimum expectation. So I'm hearing faculty members are now trying to ask in the final exam, in the midterm, ask them to summarize what happened in the discussion thread. Ask them to reflect on the discussion topic again. So then they actually have to read that discussion so that they will draw on from that discussion and then put some final exam questions there at the end. So don't just expect your students to respond three times, agree four times that that's not going to work. Student do not like it. My next example here is again, nothing to Neil research paper research essay, but then now they just need to put the research paper online. You might want to start with you might even want to just start with in canvas. Student need to post something they need to be shared by your classmates, you can set up groups so then students will see a few others and comment on the research paper. You can have your own classroom block where or you can even have password protection. So only student in your class can go into the block to look at the research paper. If you're ready if you trust your students and it also also to protect students privacy. If your topic is not too controversial, then you may actually open this for everyone to look at so you're on regular class block. I'm saying it because I'm being careful because each one of us will be teaching something different and some of the topic that we are talking that we discussed here in Canada, maybe a illegal topic. You are in we are engaging our students to do something that is illegal in the country where they are joining us from. So, if you wanted to ask your students to pose something more controversial online, be pay attention to the privacy around the globe. Not just in Vancouver, but password protected block will be a useful place. I find that as soon as the student find out that the word they're going to contribute that they're going to submit is not just going to be read by TAs and yourself. The paper that they're going to submit will be viewed by the classmate will be viewed by past students in your course or future students in your course. The quality just go higher. And of course, if you can actually, if you can, right, I mean, again, talking about the privacy legality, if you can actually open this to the world, students quality work, they pay, they work harder because they care. They know that the parents, the friends will be reading the research paper. The future employee may also be reading the paper. This is something that they can put on the resume. So they work harder and we end up getting better quality research paper. So again, nothing too different. It's still just a research paper. But because it's online, then we may ask them to add more image, more link, but finding a way to ask the students to finding a way to ask the students to put it on in the real world. I'm thinking about, again, I'm thinking about the Russian course or any language course. I would really strongly suggest that you ask the students to contribute to the Russian version of the Wikipedia. In my course, I asked students to contribute to Wikipedia English version on the subject matter that we teach. So again, putting the work outside in the real environment. So two cases here. Holocaust artifact essay students are sharing what they find and put it online. Video game law is actually a little bit more controversial because it's a discussion students need to express the opinion. So this can get quite whiskey, but this is a law course. So I think I trust that the faculty member knows what he's doing. I, I, yeah, I think I would be really careful if I expect my students to express an opinion on a controversial matter for my for my students in my subject matter. And next example here will be come on. School. Yeah, here. People polio digital media against creating something that the students can use it elsewhere. Here is a website you we have the link in the PowerPoint back so you can get the link later. This is a web site that's developed by Christine on a real she's from digital media and art. She teaches art history and she also teach digital media. So this is a very appropriate format for her course. So what I really like about this is I use it myself sometime I allow my students to present their research paper let's say just keep it with research paper to present the research finding in different formats. It doesn't need to be as an academic traditional research paper. You can do a video, they can make a pamphlet, they can make a media guy info graphic pressy different way to present the learning again. Again, because I also teach in a course that is a elective course. I imagine that not every one of my students will become an academic in mind discipline. I really want students to be able to create something that they can use. And I also really want to recognize my students talent outside of what I'm teaching. I use in my course I used to have a cohort of students from film. I have a group of students on film, they, they must want to my class together. There's a group of 20, and then they, they, they come and they love that video production of the research paper. They, they put in so much effort as the sound and the music. And so I'm, I'm very pleased. So you recognize the talent, start with something small. Thank you for something. They love it. The list of credit, you know, like, at the end of the movie or the TV that goes up. My students who the student was involved in that media production, they, they got page and pages of credit. They even credit the music that they use on my I don't even know how to cite music myself. Interesting. So the student find that the instructor recognize that other talents outside of academia. So, So I'm reading the chat so I get a little side try that. So using different way to ask students to show their learning. And Christine, in her website, she provided work. Instruction, things that you need to know when you need to assign digital media assignment. So lots of resources there. I highly recommend recommend it to you for if you're considering this type of format, this way of gathering evidence of the learning. I mentioned this earlier wiki. We have two different type of wiki here we have the UBC own wiki and then there's also the Wikipedia. Similar to what I mentioned before students can pose the research finding on the wiki and UBC wiki we have a lot more flexibility. You can structure your wiki pages any way you want for your own course. And here in this conservation 200 course, they are looking into some issues in different part of the world, there's local locally applicable issues around the world. And then the students need to make suggestions on actions that they need to take to help with that conservation challenge. And I mentioned this before is open assessment encourage students to do well perform. And one thing that we recognize we've been using the UBC wiki for many years now but recently we find out that we should know, but we never pay attention to that. They lock into the UBC wiki is associated with the CWL and therefore the information that they contribute is also associated with the personal information. So, if we, if you are going to ask the students to talk about something more controversial and you worry about the report effect 10 years later, then we can also help the students to get an ADS if you want to ask them to do UBC wiki project Wikipedia. I tell the students to find username that is not associated with themselves. Chocolate lava, for example, just make up my name and then tell me in Canvas what the username is so I can mark them accordingly. One last one, I think this is really authentic now this one is really authentic, and I am surprised I just heard information this earlier this morning. This is community work. So students need to work with community members to solve a problem to help them investigate and issues that is happening in the community. We have courses and then in full system that the whole course everything is built on community work. And you can go to the website, the course webpage above LFS 250 or 350 line food dot UBC dot CA. The students also need to document the process how they meet with the community member what they learn from the first meeting. Identify the challenge and then the whole proposal, the whole research investigation. And then they with the with the consensus from the community member because sometime it's not there's private information that we need to respect with the consensus from the community members. Maybe sometime we'll have the final product, the essay, the infographic, the poster post online there too. So, so welcome to take a look. So in this this morning I was at a meeting. There's actually an increase needs request from UBC faculty and wanting to do more community work during COVID. I was a little surprised because I thought, oh, nobody wants to do it. We don't want to go out to the community. We may not want to interact with people as much as before because in the in the past students need to go out meet the community members. But I, I find out from the Center for Community Engaged Learning that there's actually a higher demand faculty members would like the students to perhaps at least virtually meet them. We assume to to to meet maybe maybe technology is helping us to reach out to them. We don't actually need to go there in person. Perhaps, but there's an increase needs, but this one takes a lot of work. If you really want your students to start to work in the community. They start thinking about it now and plan it for next September. I wouldn't recommend it for January. It's a lot of work. But students are, they are applying, they're making a difference in the community and, and that's okay. Focus on three questions. Thinking about how can you add some real world examples or ideas using that specific method and any challenge for implement your ideas using what type of tool you are thinking about implement that idea and how you might thinking about taking care of some academic integrity issues. Closing thoughts. We would like to say that assessment is a way is not the end of learning assessment should be used as a way to encourage students that they continue a growth. And, and encourage them to wanting to learn more. That's what I learned and I hope that that's the primary goal of assessment is helping them to do to learn more. And with our closing thought is we want to ask you what, what are you going to do next? Where can you find more information to prevent the cheating? Where can you find information to help you redesign your assessment? So then, like scaffolding, we heard about that from one of you earlier. What are you going to do next between now and, and the Christmas break? I hope that you will all have a break too. So feel free to put it in the chat or just have a reflection, put it in your notebook. What are you going to do next? Moving on while you are doing that reflection and making that determination of what you need to do next. We wanted to let you know all the resources, all the, there are some roots here and link to all the courses that we introduced. We're trying to link it to as close to the assessment as possible so that we don't need to think around too much. And feel free to share everything that you heard, you learned from each other, from us. This is me, this is with other colleagues. Hi, then, and I, we are happy and our colleagues are also happy to have one online consultation with you. There are times that we will pass you over, we will suggest you to talk to someone else just because I call myself a generalist in CTLT. We do have colleagues who is more specialized in different areas, so they will be better able to support you for your needs. So we might need to say, hey, we will introduce you to a different colleague, not that because we don't like you. So thank you, Antoine, for putting your next steps here. And there's also a feedback form. Hilene and I would love to hear what works for you, what didn't work for you. And then so we will continue to refine and modify our plans, our facilitation plan. And we will also like to know what type of workshop we should offer in spring, in January. I'd like to call it spring. And we will have drop-in session next week at CTLT. So you are welcome to bring your questions to our drop-in session. I think Highlands team will also be offering some more additional consultation sessions for you to drop in to ask questions during the first week of January. I hope that by this time you should all know that most classes. Being very mindful about who's in the audience, most classes will be starting in on January 11 instead of the 4th. So with that said, thank you very much for your participation. We will stick around for a couple more minutes in case you have any questions.