 Well, welcome everyone. My name is Carl Blyth. I'm the director of Coral that stands for the Center for Open Educational Resources and Language Learning. We are one of 16 National Foreign Language Resource Centers and we come to you from the University of Texas at Austin. And today we are hosting this, what we call our Hangouts or OER Hangouts or our Language Teaching Hangouts because they're very informal affairs. And we're really excited today because we have a topic of great interest all about online assessment. Our speaker today is Meredith Clark. Meredith is the consultant for Finance and World Languages at Region 10. That's the Region Service Center and that of course is in the Metroplex area, Dallas and Fort Worth. Meredith puts on lots of professional development events for teachers and also helps them work with different districts on their curriculum and make sure they're compliant and things like that. She has a lot of experience in giving these kinds of talks. I'm going to turn things over to her in just a second, but I do want to remind everybody to unmute. I know that as she's talking, she won't be hearing a lot of background noise. So make sure that that's muted. And you can write as ideas or thoughts come to you. Please go ahead and write your questions or your comments in the chat box. There is a chat box on the bottom. And we will collate them. We'll take a look at them and make sure that if you have a query that your question will be answered after Meredith, she'll finish. So she plans on about, I don't know, 20 minutes or so to talk. We'll have a lot of time for you to have a nice kind of Q&A. Okay. So again, thank you everybody for being here. And Meredith, you can take it away. Okay. Thank you, Carl. I really enjoy working with the great folks at Coral and they provide such valuable resources for so many languages and that are open. And this is just really so important. So please share what Coral is doing with your colleagues. And this is when teachers need resources more now than ever. It's really great to know that you can depend on Coral to not only learn, but get good resources that you can share and edit and adapt. So we believe in the mission of open educational resources. So thank you. Thank you. Thank you, Carl. So here's what we're going to do really quickly. The agenda is on the screen. So I am going to, I'll be talking for about 20 minutes. Okay, I do have my timer on. So when I say 20 minutes, I mean 20 minutes. Okay. And if you see a little camera on a slide, it would probably would be a good idea to take a picture. It's a suggestion. Okay. And after I talk, we're going to break out into zoom rooms and we're going to look at some sample assessment objectives. And then that's when we're really going to dig into, hmm, how would I administer this assessment online, or face to face, or in a hybrid teaching situation or a concurrent face to face and remote learning situation. Okay, and then we'll come back and we'll share and discuss and ask questions. All right. So that's that's the shakedown for this afternoon. Now let's quickly check in. Let's start our learning today with a goal. And here's our learning objective. It really has two parts. And I know this sounds a little elementary, but sometimes distilling down right like we're in a pandemic we got all this, you know, information coming at us all the time is, you know, especially if you're in K 12 you have to communicate with, you know, 180 parents plus the students and the administrators. So let's just take a deep breath and we're going to really just focus on the basics first what what am I going to assess in my language class, and how am I going to assess it. Okay, once we define that, then we will examine some of those assessment objectives, and consider how we're going to implement that. Okay, now everybody's situation for implementation is different. So my hope today is instead of me telling you what to do is that you can critically think about your situation, and find a solution that works for you and learn from what others are doing as well. Okay, so that's where we're going to spend the bulk of our time collaborating. Okay. All right, so let's answer that first question let's make sure that we're all on the same page. So the question is, what do I assess in my language classroom. Sometimes teachers spend hours searching for assessments online they're like they're you may feel desperate at some point, oh, I just need to find something I need to find something. Right, and so you spend a lot of time looking, but let's simplify a little bit and just ask ourselves the question what what should I assess. What do I want to assess in my language classroom. Okay, because you're going to find lots of stuff that's available out there that isn't necessarily going to help students communicate better. Okay, and so what we ultimately want to assess in the language classroom is student language performance. Okay, now this means, for example, for interpersonal that students are actually having unscripted conversations with other human beings. Okay, that's what we want to assess. Okay, if we decide that we're not going to do any of that during the pandemic, then we're really leaving out a huge part of communication, right. Interpretive tasks mean that kids are reading and listening to authentic texts and under and demonstrating that they can understand them. Okay, and then on the presentational side, students are reading, excuse me, they are writing and or speaking some sort of polished communication in order to communicate ideas to an audience. Not just the teacher, right, like writing five sense sentences using the verb of law is not a presentational task. Okay, because there's no real audience there. It's just the teacher and nobody really cares what the meaning is in those sentences. Okay, so what we really want to do is during the pandemic we were actually seeing our kids less sometimes right because it could take you 15 minutes to take attendance you're only on zoom with them for 30 minutes half your class is gone. So we've really got to make sure that what we're doing in class is communicative. Okay, and helping boil that down and getting rid of all of the other stuff maybe that we used to do to fill a 90 minute block. Okay, and really honing in on the essential is going to help you plan because you'll know exactly what to on what to focus. Okay, so this is what we want to assess for everybody here knows that language performance in the classroom is going to help students become proficient speakers. When they exit the program meaning that they can use the language outside of the classroom in authentic context, and the reason that we're doing this. I always like to talk about the goal of world language education is because we, our goal is to create more multilingual people. So, even though we're in a pandemic. And I know a lot of us are on survival mode and we just want to get something out there to the students, right, we still have an opportunity, even if it's less of an opportunity because the contact we have with our students is we still really want to make sure that that opportunity is about creating multilingual, not just getting kids to pass the class, right, and the more communicating they do in class the more engaged that they will be actually heard from some of the language supervisors up here, like in rock wall ISD who said you know what we changed our curriculum it's more communicative now. And the teachers who were teaching in a more communicative way had less of an issue, moving to online instruction and hybrid instruction. So, let's make more multilingual people in the United States we can do it. Okay, so just why don't you guys type in the chat just a little quick checking for understanding here. What should we assess in our language class. So if you have access to your chat and you're not driving. Okay, which I have seen before in professional development please do not PD and drive at the same time. Okay, let me know in the chat there what do I assess in my language class. I have about 30 seconds to a minute to type in your answer into the chat. Okay, so I'm communicative skills. Okay, seeing here language performance communication. Yes. Okay communication and I really like seeing three modes. Okay because speaking practice and writing practice aren't necessarily modes of communication. Right. So think about the difference between writing five sentences that no one else is going to see and a presentational creating an infographic that you're going to post online or responding to a social media post in the target language. There's a big difference between what we call writing and presentational mode. Okay, so. Yes, we want to assess those three modes of communication. Okay, now the next question is how are we going to do it. And so before the assessment comes the thinking this is the fun part. Okay, now, if we want to consider two different approaches to assessment here. Okay, I'm going to put two approaches on the screen and I'll give you some time to read them. This is approach from teacher a she's getting ready for her assessment. This is what she's thinking about before she designs it. Okay, let's take a look at teacher bees approach. Okay, so question is between teacher a and teacher be approach more purposefully plans to assess performance in all three modes of communication. Why don't you all let me know in the chat there, or I saw some very quick ones. Okay, now if you really think it's teacher a don't feel you've we've seen a lot of bees and this is the downside of using the chat to check for understanding because it's like all five people already put in be guess what I'm going to put in. If you think it might be a go ahead and put it in there. Feel free you could be the one person who has a different idea and we would like to hear about it. Okay, I'm seeing that most of you are saying here that teacher be teacher be is more purposefully thinking for the modes right and then Devon here is saying aids and survival mode. Yeah, and you know what some of us are in survival mode right now. And I, you know, as I just finished watching this great Korean series on Netflix is called it's okay to not be so we are not machines. Right, so there's no judgment here right but but we do want to acknowledge that what we're aiming for if we're really going to get kids more proficient in the target language, we be is a better option. Right. Okay. Now, so if the approach is off and planning for the assessment. Okay, this this reminds me when I flew to the island of Colabra from the mainland of Puerto Rico, right. Okay, if the approach is off even by one degree. We can end up in that in a bad spot. Right. So this is why I mean I'm just going to go back here for just a second. This is why the pre thinking assessment the pre planning how you approach the assessment mentally in your head matters almost just as much about more than the actual prompt that you give the students. Okay, so what we want to make sure that we do is we get we start this approach in the right way, and we end up with a an assessment prompt that could be administered online face to face or in a hybrid situation that gets kids communicating. Okay, so we're going to start with our backward planning framework. If we even sit down and think about planning the assessment, we have to ask ourselves what are the desired results. Okay, so take you 30 seconds what are the desired results I'm teaching level one that desired results are novice high to novice mid to Okay, then we have to ask ourselves what does that look like. So if we're doing a food unit and I need to get kids to novice mid to novice high. What does a novice mid to novice high student sound like in the interpersonal mode. What kind of things can they understand when they're reading and listening right what can they present. Okay, and that's how well can they perform language in those three modes, and then the very last piece is planning the learning experience and I include in assessment is learning experience. A lot of people when they think of backward design they say, start with the end mind plan the test and plan backwards, and I'm like, well, I'm not really sure you want to write the test first, I think you really want to think about this. Okay, and even if you just take 20 seconds to articulate this out loud or write it down on a piece of paper. Okay, you're going to make sure that that approach, right that the assessment is exactly where you want it to be to get communication. Okay. All right, so here's quickly what this looks like. Now I know some of you on the chat may not be from Texas, but these are how the desired results are defined for K 12 educators in Texas. Okay, and it's very similar for post secondary as well. Okay. And then the X, then we ask ourselves okay I'm planning an assessment for level, let's say it's level to this time. So how do I know. How do I know, well, if you're using national standards, you can look at the actual performance descriptors, and they're these are clickable links here so you can go back and click on this and open it up. And then a K 12 teacher in Texas, you'll use your teaks here. Okay, and if students can complete each one of these language functions listed in your standards, then you know that they're performing at the correct proficiency level, excuse me performance level because they're doing it in class. Okay, so I really like the Texas standards. It's very clear for what students should be able to do with the language, and at each desired proficiency level so this is column one is level one that's novice mid to novice high second column is level to second column and third is level three and fourth and so on. So you get a really nice articulated view of how students are expected to develop language. Okay. All right, so then the last thing after you know what the acceptable evidence looks like is to then plan your assessment. And I'm just going to put teacher bees assessment up here so you can see it again. And we can see here that teacher be is assessing all three modes of communication. Right. Okay, ask and tell others. That's a language function. Okay, for interpersonal mode, identify main idea in details from an authentic text interpretive mode, express their opinion about food on a social media post. Okay, all three modes of presentation was the last one there. Okay, and I know because I've done the second step of backward design that I know that for novice high to intermediate low range. I need students using simple sentences. Okay, if we're going to make it to intermediate low, they got to be using simple sentences. Okay, they need to be using that indicated vocabulary in order to communicate and enough grammar control language control or grammar rather. Okay, and communication strategies to keep the communication going and make sure that other people understand them. Okay, now as our non example. You can see teacher a. Okay, and you can tell here teacher a is really only planning for about one mode of communication. And it's down here where it says tell others what to eat. Okay, and that's, it's kind of an unnatural, like, quasi presentational speaking task. Like, I'm, you know, like, when in real life, do we tell people what to eat. And I might be unlikely to record a video and just say 10 sentences where I'm telling who somebody I don't know what to eat. Right. So, you know, when we look at the difference between a and B, we see that this is a lot more natural in terms of yeah humans that actually communicate this way. Okay, more likely to ask and tell other people what to eat. Maybe when we're standing in the kitchen together having a conversation. Okay. All right, so let's move on here. And stop and reflect. What what are your takeaways from that little snippet. Okay, so the question was the question that opened up this section of learning was, how do you plan for an assessment. How, how do we approach assessment. Okay, so how would you answer that let me know in the chat there we'll spend about 30 seconds to a minute, writing our answers down in the chat. The question again is, how do you plan an assessment. What is a good approach for planning and assessment. Okay, so we've got some mentions of backward design. Okay. And I think our language here matters. Okay, because when when we say when we when we don't talk about desired results, right, we can kind of get caught up in, I mean what is what is the end in right. What, what is that right but when we say oh the desired results are. Okay, so what are the desired results that's a there's a question I'll pull out for you guys what are the desired results. Think about a level one a level two a level three class what are the desired results. Okay, yes that they can communicate effectively. Yes. Okay, but more specifically let's really get down to the granular right so yeah level one novice mid to novice high right. Yeah, Deanna here is saying it depends on their level right so so when a parent comes up to you, or an administrator and says what you know what is my kid. I'm going to get out of this class, then we can say, by the end of Spanish 312 K, which I know we don't call it that anymore but right whatever even if it's a post secondary course, or a K 12 course, by the end of this course level the desired results are that your student develops proficiency or is able to perform if you can't have a proficiency test done is able to perform at the specified range, which for level three. It would be intermediate low to intermediate mid okay so saying that has power right actually specifying the level because if we if we totally miss that. Right, then our plane is one degree off. And if I don't know that I'm aiming for novice high or intermediate low, then do I just how do I grade the kids work. Good question right if I don't know that I'm assessing the students work based on their performance, how do I how do I grade it right and do the students know what novice mid novice high, all these things are. Okay, so those are also really important questions for assessment. Okay. I have a comment here from Deanna it's confusing to some students because in other classes they have points. Yeah, I mean, sometimes it's as simple as having a representation of that actual proficiency cone in the classroom, and say hey you know what you guys came in your intermediate low by the time you get out, we're aiming for intermediate mid. Here's what an intermediate student can intermediate mid student can do. Right. Okay, because if we're if our students aren't understanding that and we're not thinking about it before we plan the assessment. We may end up in the wrong spot, we may it will still be on the target somewhere but we might not be specifically in the center of the target. Okay. Thank you Deanna for your comment. Okay, so at this point, I'm going to stop talking. And let's look at some sample assessment objectives. Okay, so this is the part where we're going to break out into zoom rooms. And you see your little camera icon up here so I recommend that you take a picture of this slide, just because when I put you into zoom rooms. And go away, and you're going to say wait what are we supposed to do right because that's what your students do when you put them in zoom rooms and if you have a picture beforehand, then you just look at your phone and you say oh, I know what to do it says right here. Okay, so go ahead and take a picture. All right. And what what we're going to do is I'm going to show you another slide I also want you to take a picture of the next slide. I'm going to show you some sample assessment objectives and in your zoom room really three minutes or less folks really short through a short introduction and decide who will be the timekeeper, and who will be the note taker or the and or the reporter. Okay, the person with the notes needs to be the person who's going to report back to the group. Okay, and I want you to choose at least one sample assessment objective, which I'm going to show you on the next slide. Okay, decide which one you're going to do really quickly and then discuss as a group how you can administer the objective in a face to face hybrid and or remote teaching environment. Okay, and when I mean hybrid I mean concurrent or simultaneous face to face and remote students at the same time. Okay, you'll have 12 minutes to discuss and ideate and throw out certain situations. And then we'll be ready to report back to the group and then we'll take what you said and compare it to some kind of general best practices in just a second. Okay. All right, so I'm going to go ahead and show you that next slide which I would also like for you to take a picture of. Okay, here it is. These are your sample assessment objectives so you're going to meet in your zoom room. And then introduce yourselves. Pick a timekeeper pick a note taker and a reporter. And then pick which one of these assessment objectives you want to think about how you're going to plan out for those different teaching environments. Okay. And just as a visual because visuals help us. Okay, so you'll pick one of those objectives that you just saw previously that you now have a picture on your phone. Okay, and you're going to think, oh, how would we do this in these different teaching environments. Okay, so you can take a picture of the slide if you want to. Are there any questions about the instructions before we move you into zoom rooms. Hi everybody welcome back from your breakout room I was able to visit three rooms and hear what you all were talking about and the different environments in which you're currently teaching very interesting to hear what's going on with with with everybody. So let's go ahead and move forward. And if you had the role of reporter, what I would like for you to do is in the chat to let's, you know, let's look at this interpretive task. Let's look at this interpersonal task and this presentational task and if you had the role of reporter in your group, please share in the chat with us. So one you did, which one you discussed and any little golden bit of knowledge, right that you would like to share like for example one group said that they practice interpersonal by having the students call each other on the phone and that way they can do a socially focused conversation and not have to get close to each other and worry about not hearing each other so that would be something that's like hey that's really cool let's share that in the chat. Okay, so just tell us for your group, which assessment did you mainly talk about or which mode of communication did you mainly talk about. And what was your strategy for how to logistically administer that assessment. We'll take one minute to do this. We'll give our reporters some time to report out should have about 10 people reporting in the chat. Okay, thank you Devin for sharing. Okay interpretive we've got some folks are using Ed puzzle. Okay, to like you know that puzzle is really great for chunking, right chunking the video chunking that listening, yes. Very good. And Ed puzzle is also really good for asking students to infer the meaning of unknown words like a lot of times we forget to ask students to infer which is one of our functions and our ticks. Okay, so summarizing comprehension understanding all very important, but it's also very important that we're teaching them those inferencing skills as well. I did hear in some of the rooms that presentational and interpretive were quote unquote the easier ones to assess. So I'm curious in the chat here let me know what were some of your ideas for how to assess interpersonal. Okay, now some of them are coming in. Zoom breakout rooms, Mary and Russell is sharing right so you can do interpersonal conversation to human beings having a conversation with each other in the zoom room. And those zoom rooms can be recorded you can assign one student recording rights they can record that room and then when the teacher closes the zoom session the recording automatically downloads onto the students computer, and then they can upload the file into your Google classroom or your canvas or your blackboard or whatever you're using. Okay, so good old fashioned zoom rooms work great for interpersonal. Okay. Let's see creating scenarios for interpersonal like meeting somebody you're having a phone conversation good yeah great discussion here Mr. Pettis thank you so much for sharing. Good so it doesn't have to be super tricky. Right. For interpersonal. If you don't use zoom like I know there's some school districts out there they're like we will not use zoom. Then you could asynchronously do a Google meet or a Google hangout. It's tricky the kit you have to sign the kids up or the kids have to be responsible enough to schedule their own conversation but you can also do a screen castify of a Google meet. Okay, which is also a really good way for students to do this asynchronously. And so that brings us to our next topic here what what should you focus on when assessing asynchronously versus synchronously. Okay, so as you plan for your learning environment which could change at the drop of a hat right like my husband was told yesterday evening that no one's going to the office today. Or some of some of your students could be pulled out of class because of quarantine so you know this is a very important thing to consider not on top and in terms of the learning goal but on how you administer the assessment. Now that goal here is or the idea big idea is you don't always have to administer the assessment synchronously. Okay, now for interpersonal, it pretty much needs to be synchronously but it can happen at times outside of class. Okay, and you will know if this leads us to our next question here, you will know. There's a couple questions up here. Right, you will know you will be able to tell if students are reading, right, if they have to do an asynchronous interpersonal conversation it's going going to sound like they're reading. Okay, so, and a lot of the academic dishonesty this was a question that came in before it was submitted before the webinar. How do you prevent this. And I think you all are welcome to chime in in the chat here but number one students need to know the difference between artificial intelligence and Google translate and what the purpose of that is, and what proficiency is. Okay, and we're in a language classroom we're going for proficiency right when you end up in a country where you're in a place where you don't know the language and you need access to language fast. So artificial intelligence right so, and then also the way we grade students really is a factor in academic dishonesty. So if we're expecting their language to sound super polished and perfect. They are going to be more motivated to use Google translate and things like that. Right. So, I think, articulating those expectations for students ahead of time and helping them understand the difference in the tools that are available to them will help with that academic dishonesty. And then, this is just another question just feed for thought. What about in person and remote assessment should we keep in mind. And you all have listed so many really cool strategies and in the chat about how you are facilitating communication and assessing language performance. In this unique teaching environment, but I just wanted to share this link I found this article, very helpful and it really does address things like, you know what, we don't have to assess all the students at the same time. That's point number one I was like, Oh, yeah, maybe I have options here right. Second thing that this article talks about is that projects. I think projects like everybody design a poster project. I mean, performance based tasks there we go let me back up here performance based tasks are way easier to do remotely and then these hybrid teaching situations and if you're giving your students prompts like the ones that we saw earlier those three assessment objectives, then engagement will be up. It's not a huge summative assessment that's just going to crush them right so you know you could even think about eliminating your summative assessment and just assessing as you go to see if the kids can do those language functions. So we really have a lot more hopefully you have a lot more flexibility, some districts mandate you have to have a summative assessment. I think we can ask ourselves now what does that really need to look like, and it really should look like what they're doing in class, right, not this big huge thing. Okay. Yeah, good. Thank you Devin for sharing there in the chat. Yeah. Yeah, so we have we can be a little bit more creative. Okay. So I will leave you with this we have five minutes left and I know Carl is going to mention some things for you. I would just like for you to take 30 seconds and let me know in the chat. What did you learn today and how will you use it. Okay, a short little reflection on your part will really help you process your learning and help it stay with you after you're done this evening so if you wouldn't mind sharing in the chat. One second. What did you learn today and how will you use it. Yeah, I love this one, help you refocus yeah like let's go back to the basics. Right, we can go back to the basics and it might actually be less planning for you. Right sticking to that streamline planning. What are the desired results intermediate mid. Okay, that's what I'm going for. What does that sound like what does that look like. Okay, I know what this means that I know the language control the function. Okay, I got it got my cheeks I got my standards. Now how am I going to write the assessment prompt and I think planning in that streamline fashion for assessment probably will will help you with some time efficiency. I hope I hope it will. Okay, oh good someone learned about a new app. Great. Wonderful. So you could share that in your breakout rooms. Good. Great reminder and backwards design. Thank you guys so much. I'm going to read some more of these in the chat afterwards but at this time I would like to go ahead and thank Carl for this opportunity and pass it over to Carl. Okay, thank you so much Meredith and thank everybody for participating as you're writing a finishing up in the chat. I'm going to ask Sarah to put up the last slide because the last thing that we do is to ask you to fill out a really quick survey remember that our money comes from the federal government and they want to make sure that we're spending it wisely and we're actually doing what we say we're doing it's this takes really just a couple of seconds it's a very quick survey it's not long at all. Again thanks to Meredith for animating our discussion today for teaching us lots of tips tricks all kinds of ideas to think about in terms of online assessment it couldn't be more relevant. And I also want to remind people before you click on the link there that go that takes you to our survey. And that in an in our next OER hangout, which will be in a month, November 16 will be focused on all the cool stuff that we're doing with Spanish heritage we have a very active group. And so we'll have a very interesting talk ready for you on November 16. Okay, so thanks everybody. Click on the feedback link and let us know your thoughts, we appreciate you and until November 16 for the next OER Hangout. Hi folks. Thank you. Thanks everybody. Thanks Meredith.