 Okay, great. All right, hello. I am Susan Wu and I'm a consultant from the Edmonton Regional Learning Consortium. And I'm also representing the Alberta Regional Professional Development Consortia. We're very excited and thankful today to be joined by two leaders in educational assessment, Thomas Gusky and Wormeli for considerations for assessment and grading in a pandemic online world. If you follow along with this presentation on the screen, you'll see a bitly link underneath the title. I also had put it in the chat box previously. So if you'd like a copy of the slides, there are also associated readings that are recommended by Thomas and Rick that are posted at the very end of the presentation. And that way you can also have a copy of the questions that we're asking today. So just to start with some introductions and again, thank you, Tristine, for facilitating our technology. I know I'm very thankful to anyone who can help me navigate the technology. But I'll start with introductions. Thomas is a senior research scholar in the College of Education and Human Development at the University of Louisville and Professor Ermitus in the College of Education at the University of Kentucky. He has authored or edited 25 award-winning books and more than 250 book chapters, articles and professional papers on educational measurement, evaluation, assessment grading, and professional learning. Rick Wormeli is an experienced classroom and building educator who now writes professional articles, education books while training teachers, principals, superintendents, business organizations, school boards and parents in North America and globally. With an emphasis on up-to-date pedagogy, innovation and professionalism, Rick's work focuses on the topics of assessment, feedback, grading, differentiated instruction, leadership, literacy, motivation, cognitive science, linguistics, teacher-student relationships and racism and creativity. So that is quite a list of accomplishments and so we are very honored to have them both with us today. The COVID-19 pandemic has led to school closures globally and within Alberta. As educators, many of us are having to pivot very quickly and respond to the rapid changes required to support students learning from home and as well, school leaders need to support teachers with this transition. So just to give you some background of our preparation for this webinar this morning, a survey from each of the Alberta consortia was sent out to a district leader for each school jurisdiction across the province asking what are your top priorities for assessment and grading in a prolonged online learning environment. Then we took those responses and summarized the most critical questions posed to Tom and Rick who are generously sharing their insights with us today. So after we've addressed these 10 questions, there will be an opportunity for an open Q&A from today's participants. So you'll have an opportunity to type those in the chat box. So please feel free to type your comments in the chat box as we go through the 10 questions and given the large number of participants that we have today, we won't be able to address specific questions until the last half hour of our webinar from 11 to 1130. And once again, I'd like to thank Christine for helping facilitate with the session. Okay, so we will start with the first question and this one we will start with Tom. How do we ensure the validity of evaluating student work online and report card accuracy? So in other words, teachers are concerned with how do we know who's submitting the work and how accurate can we actually be with percentage marks for report cards? Okay, thanks Susan. I think that actually I'd like to start with sort of a little bit of preface. First of all, I want to wish a happy birthday to my good friend Rick. Rick is giving up time today on his birthday to share with us today. So thanks Rick and happy birthday to you. Have a great day today. Thank you very much, Tom. I was hoping to keep that under the radar, but thank you, it's a great day to be alive. A special day and we all should celebrate our birthdays. Thank you. Second of all, I wanted to preface this by saying that we are really keenly aware of the difficulties that we're experiencing today. The problems that this has created for schools and teachers and school leaders and the hardships that's actually created to for students and for their families. We recognize to the fantastic job that educators are doing to in the midst of all these difficulties to really try to ensure the best and highest quality education programs we can. We realize that this pandemic has imposed on us some very real restrictions, limitations. We can't do everything that we did before and we certainly can't do it in the same way. And so that means that we're going to have to establish some priorities. When we think about assessment and grading in particular, there are two major directions that kind of pull on us. On the one hand, we have this need to sort of document and quantify student learning. That there's this sort of pressure for accountability. We need to complete report cards. We need to fill in transcripts for graduating seniors now. We feel this need to calculate a class rank. We need to identify the students get in the top 10%. We need to distribute academic honors. And of course we have to think about naming a valedictorian. On the other hand, we have this other need that just to facilitate student learning. And we need to think about how we can do that under these very demanding and restricting conditions. How we can provide the best really learning opportunities for our students and really ensure that all of our students learn excellently those things that we count as most important somewhere. The problem that we face is that those two sort of things don't always coincide. And that means that in order for us to address these issues, we have to establish some priorities. Recognizing that when you focus on one, in some cases it's going to be sacrificing a bit of the other. So we sort of considered this and what we wanna focus on primarily during this time is our priority, which is really to think about how to facilitate student learning. And if that means sort of giving up some of the things we would have when we talked about accountability and quantification, then we're just gonna do that because we think this is maybe the most important. So these difficulties do present us with this sort of rare opportunity to make that priority clear. And so as our responses to these issues come about, please keep that in mind that our priority is going to be how do we really facilitate learning rather than really worry so much about the documenting quantifying achievement part. In terms of validity, the typical definition of validity is ensuring that the evidence that we gather clearly matches the learning goals we've established for students. And represents an accurate representation of students learning of those goals. So if this is really our focus, then we need to think about what evidence we can gather to provide students with the best feedback on their learning. If that means sacrificing a bit of the notion of security in it, that's okay. But we wanna stress to our students that our major purpose in gathering this information is to help you learn really well, that we need to get evidence from you to indicate to us how well you're learning these things, whether you've learned them as well as we want, whether there are things within your learning that need to be remedied or fixed and then allow that to direct us and our efforts to really accomplish that. So in terms of the validity issue, I think that a major issue is how can we guarantee that the evidence we're gathering from students provides us with an adequate basis for giving students adequate and appropriate feedback that allows us to guide their learning and then recommend any strategies or alternative techniques that might be able to help them remedy problems we might identify there. What you're thinking about that, Rick? Oh, I love everything you've said. I would just add that for some teachers, they lack a sense of our versatility in assessment prompt design. They think there's only one way to do something to assess it when really there's probably dozens of ways to do that. And if you rally truly around the evidence that you've identified with others, lots of different opportunities to assess legitimately in different formats, it's very hard to copy and cheat across multiple platforms, so to speak, or to have one person give you a little bit of assistance here, but then you've got to show it again, these other places. And I would suggest that teachers really began to use this time as a chance to kind of upgrade their sense of how to assess the same thing. But if you rally around the same evidence all the time, the format, the compliance of actually following through and using this vehicle becomes less important. And I cannot emphasize enough the need to teach developmentally. So it doesn't create a sense of panic. If I have the opportunity to give you an initial sense of where I am regarding this subject, but I know you've got my back, you're not gonna pin me down and hold that as a permanent registry of where I am in my proficiency, then it's less likely that I'm going to cheat, I'm going to copy because I know subsequently down the road, I can come to know it better. And then my teacher's just looking out for me. So the developmental nature of what we do, am I being appropriate or am I being over the top? Is there hope of a relearning and reassessment if I fail? And one of the things that I've discovered is, and many other people have as well, that our innate sense is curiosity to learn. And it's unnatural to try to mask that and to get away from that. So if we do, like Tom says, we're so focused on the accountability when really that's not the most important thing right now. Most important thing is learning, but also the compassion we extend to one another in a very difficult time. Parents don't have a lot of training on assisting their children with schoolwork. So to what degree do they help, do they pull back? They're gonna be intermittent and kind of stumbling a bit with that unless we give them some training along those lines. Plus the emotional cycles of the day and the equity access measures all really create that sense of, do you understand validity in general? Does your test accurately portray what you claim it portrays in student evidence? And then what are all the ways that we assure that validity in general? You can still apply those on the majority of assessments done online. But one of the later questions is coming up and I think I'm gonna be prepared to take the initial response on it, is how do you minimize the worry about copying and plagiarizing and other things? And I'll be glad to speak more specifically then. Yeah, let me add to that too. I think that's a really excellent point. Plus it also gives teachers the opportunity to engage students more in the assessment process. So if we're really clear about these learning goals, we can say to kids, how can you show me you've learned this really well? What would you help me see from you that shows me that you understand these concepts really well and you can do these things in a way that really makes sense? And so to the degree we can engage them in the process, it opens up the variation of ways that students can actually demonstrate their learning. Can I also just add Christine and Susan before you jump back in with the next one, that what Tom just said is one of the clearest, most effective ways to develop student self-efficacy. I'm not gonna develop this learned helplessness and make excuses about other things. I'm gonna own my learning when I get a chance to monitor, to self-evaluate where I am and my teacher provides those tools to me. Absent those tools, it's gonna be very easy for me to blame situations, excuses and to be overly dependent on external validation for a sense of self-worth. But I own my learning when I'm actually invited in as Tom just described. So wow, for so many reasons that's spot on. Yeah. Oh, Susan, your sound isn't coming through. Yeah, you're muted, Susan, for some reason. Sorry about that, yeah, sorry about that. I have dogs in the background, so I just wanted to make sure everyone heard me and not my dogs. So we'll move to the second question. How does school leaders support teachers' understanding and implementation of formative assessment tools that engage students in online learning? Tom, we'll start with you. Okay. I again think that the idea here is that the true meaning of a formative assessment is about. When Benjamin Bloom initiated these ideas in the 1960s, he stressed that too often students see assessments as the culmination of learning experiences that it's used to evaluate their learning and it's a one-shot do or die kind of affair. His whole work was to get us to look at that very differently and to see assessments as an integral part of the instructional process. Now, what we can do in an online format is limited somewhat because of the restricted nature of that format. But I think still it allows us opportunities to open up a world of ways for students to demonstrate their learning that we have not typically used because of the way we've organized classes up to this point. Now that we can open the door to different ways that students can demonstrate to give them some choice in how they can do it, along with our approval. A teacher should be willing to say to students, well, I like your ideas, but that's not quite what I was getting at. I don't think that it shows me really clearly the ways that you've learned. Can you come up with another idea of how you could demonstrate this to me? We can eliminate the notion of sort of test anxiety. We can move it into a way that it's seen as really a feedback device to help students learn and change their whole disposition toward the assessment process. So to allow teachers to think of it not just as, I have to give you something and here's the time I'm gonna post it and here's the time it has to be turned in, but to allow more flexibility in the learning format so that students can take their time going through these things, show teachers adequate and authentic evidence on what they've learned. And then if they have trouble with something, be able to view that in different ways or show that in different ways that allows the teacher the flexibility and the students the flexibility of accepting that as a valid and appropriate way of demonstrating learning. I would just throw in that, if we're trying to help leaders support teachers, one of the things that really helps is to experience it yourself. And so for the leader to work with the teacher and say, how did you become so competent as a teacher? Let's look at the process of how you grew in your teaching proficiency. Notice all the formative things we did along the way, not holding you to an earlier digression as the permanent report of who you are as a teacher. A lot of things were formative, you received critique and feedback and you grew and you improve. Don't we want those same things for our students? And then we extrapolate to almost every single profession of which I'm aware that's exactly how anybody becomes competent. So what I found is when teachers feel the same things we'd like them to share of course with their students, they're way more likely to do it with their students. And if we make the parallels between this is what happens in the working world and the professional world, and they see like all doing this, the competency-based, evidence-based, ethical reporting, assessment that's formative, emphasize more than really the summative, is actually the most preparatory and the most maturing for what's to come. Teachers are more likely to go with it. But you might have to, if you're a school leader, overtly list that out. How are we formative with one another in our profession? And how is it affecting and improving your capacity? And truly your sense of hope and meaning in what you do. If we just closed you off, you had one particular bad lesson, you would run out of momentum. You would not be motivated to show up the next day when all people do is document your deficiencies and label you thusly. So there's so much more to this. I kind of would suggest school leaders get in that side of it, the dynamics with the adults doing this and how they learn, not just say we're doing it to students, but hey, we're in this with you. And let me add to that. I think that's a great point, Rick, that I find some of the teachers that are adapting best to this are teachers who have actually been students in online courses. They've seen some of the hardships that have been created in that. I know I had the experience at my university where I in one semester taught a graduate class that was not only online, but it was asynchronous. So that I went through an entire semester with a graduate class of 26 students not having a single personal conversation with one of them during the whole time. It was probably my very worst teaching experience ever. And I hated it. And I found out that many of my students hated it just as much. And so to really think back on that experience and say, well, here's the aspects of teaching and learning that that took away, that really were enriching and vital and made it a vibrant experience has changed my disposition toward how I teach that. It's changed my students dispositions toward how they learn best in those kind of environments. So I think that to put yourself in the perspective of the students and how they understand this and see it, you know how it can be done poorly, but hopefully you have some ideas on how it can be done well too. Right. There's- Can I just riff off that for a second? It'll be a very short one. In my experience, and it might be yours too, Tom and Susan and Christine, teachers who've been trained in instructional or cognitive coaching, they have a lot of tools of descriptive feedback and helping the other come to the knowledge, the solution. And when you have that training and then you have been the recipient of instructional coaching, you see that value of formative assessment and the feedback that must come from it. So I might say things like, tell me more about that. How does that align with the voucher criteria for success? What would you need to change? Or I'm coaching you, not didactically bestowing upon you content is actually kind of the, you know, the new way of thinking about teaching too. This is a launching pad for your own self monitoring, but for your own progress, even beyond the teacher. So I would just throw in if school leaders are looking for very specific tools that teachers might use and that they might use with teachers themselves, that teachers can use with students, you might look at the world of instructional or cognitive coaching. Right, thank you. Well, it's definitely what really when you look at teachers as learners, it's no different than how we view students as learners. So the same principles that are good for children are definitely the same principles would apply for professional learning for adults. Okay, we'll move on to the next question and we'll start with you right this time. With diverse needs within a classroom, how does the teacher differentiate with online learning assessments? One of the things that I have found is again, I have sort of alluded to this before, is it teachers' creativity, innovation, divergent thinking tends to atrophy when you've been teaching the same thing over and over again. You just plain don't see, because you're kind of myopic, you know, you got this tunnel vision, you plain don't see the other ways there are to assess. And as I mentioned before, there really are dozens of ways to assess the exact same thing. For example, I might say here's one math problem, here's how five different children tried to solve it. Who did it correctly? How do you know? Who did it incorrectly? What would they be taught properly? Your analysis or critique of others' performance of the very thing on which you're being assessed is actually an expression of your own dexterity, your own depth and complexity of knowledge regarding that very thing. So that would be a wonderful assessment, not easily copied by the way, but you could also begin to think, okay, well, what advice might they give others? Could they create a physical model of a very abstract idea? All of those things where you're trying to think of all these different formats will always go back to evidence. So the idea that we sit there with one another are subject-like colleagues. And in this day and age, of course we can call other people if we're a very small school where there's only me in the building that teaches that thing and say, look, what evidence is really excellent when it comes to this? Now, if format was not an issue, I'm not literally teaching the test format. So I could do an interview. They could do an interpretive dance. They could do a poster. They could do a media presentation. They could have a podcast debate between two historical figures or two book characters or two inanimate math symbols or punctuation symbols like in this essay, who's more important? The colon or the semicolon? Go, talk. If they could change the verb somehow, argue against gravity, argue against pronouns. I dare you, go. These are unique assessment formats but always we would do this. If I use this prompt, what evidence is it eliciting? I think I'm assessing this. Am I? I might ask a colleague, can you have another pair of eyes looking at this? And then I start developing this repertoire, this versatility of assessment options. So online, I am overwhelmed with the number of assessment apps that are out there. Avail yourself of it. I'm doing this other webinar on that a little bit later this week. I'll send you that one slide on the wide variety of ways to give feedback and to assess students. But more importantly, rally around your evidence first and then develop the myriad of ways. Even if we're not online, teachers still struggle with this because they're so consumed with compliance. The student followed the format, not necessarily rallying around, am I getting an accurate picture of evidence? And if you become evidentiary, all these doors open for you that previously were closed. And I would invite you to do that. So it really depends on the situation. If I have a child who can avail himself or herself or themselves of TikTok and do some kind of rhythmic, really cool thing that does portray the content, I'll go for it. If they choose something that's so artistic that I can't really discern evidence, I will demand of them that they send me some kind of annotation, annotated note taking or bibliography or something where it says, right here in my musical concerto, I am demonstrating this and here's how it demonstrates that. I will not be an evidence archeologist. I won't go digging, I don't have time. So I'll expect them to give me kind of the key, the legend to the unique things they do. But I have found that eyes are bigger than their stomach. They actually choose things that are far more complex, interwoven, multifaceted than I would have them do. But I've also found that what they choose to do if I work with them to help figure out another way to express evidence, they own it. They find it more meaningful in what they do and they invest a lot more in it. They remember that material longer. A lot of teachers get caught up in sense-making and that's where they stop in their assessment but they really weren't out for meaning-making. So it might simply be, let's draw some metaphors and analogies between a favorite sport and this political science idea, this math idea, this grammar construct of some sort. That's lovely, you go for it. As long as I can discern the specific evidence between it's really irrelevant what format they use. So kind of wind this up and then turn over to Tom to address as well. One of the things we can realize is that we can do a lot of the same assessments and only a subset of the larger assessment really needs to be tailored for an individual need. Quite often kids can do everything but I might need one subset of the test where I do an individual video or a phone call real quick to make sure they can do it or whatever needs to happen. I also realize that so many kids can just take a picture of a very traditional assessment straight out what you would do in the classroom and send the picture to me. I kind of make it larger, I can see what they did and we're good. So it doesn't have to be really way out there but if it is out there and the kids are rallying around evidence it can point to where the voucher criteria is clearly expressed. We're good and it doesn't diminish the legitimacy of the final grade report one millimeter. Tom. Yeah, I think those are really great ideas, Rick. And the only thing that would add to it is that that same kind of differentiation can be carried over to the feedback the teachers offer their students. So one of the great advantages I think of moving to this online format is it's compelling teachers to pay more attention to things like screencasting. Where the student does send you that photograph or the sample of their work that a teacher can through a screencast program which you can download for free from the internet is able to go through that, post it online do sort of a follow these checks, move their cursor give them up to two minutes of feedback on that and then send it back to them based on the criteria the teacher used. Then the student just opens that listens to what the teacher said about it and gives that guidance their direction then for making improvements. And typically you can accomplish so much more so much more rapidly using a verbal kind of feedback through screencasting you can by trying to write notes on students papers or give them written directions. Screencasting is a wonderful, very versatile way of offering feedback to students that might differentiate the kind of feedback we offer to them in ways that can be purposeful and useful to them. Great, thank you. Okay, we'll move now to the next question. What is the role of formative and summative assessments in an online learning environment? So is it any different? Do you do the roles of formative and summative assessments change given a different platform? Rick, well, who do you? It really doesn't change. Next question. Sorry. Okay, how does she... No, I'll go back one. Okay. That was a bit of humor. Okay. You know, Dylan William, Damien Cooper, Ann Davies, Tom Shimmer, Myra Dreck, Ken O'Connor, Tom Gusky, everybody agrees that assessment isn't summative or formative necessarily. That instrument that you might use for a summative question could be used formatively. And the same thing used formatively could be used summatively. So basically, it's not the instrument, the format. It's when you use it and how you use the data from it, diagnostic or evaluative, for example. So the roles of formative, they're coming to know. They're gonna get lots of feedback. They're gonna be allowed to revise in light of that. It's very low stakes. It's high feedback, but low stakes. It's where you are right now in your learning that will subsequently give a new learning and lay evidence down the road could change at any point. Summative pretty much means post-learning after all the learning is said and done. So it's almost anticlimactic. The interesting thing is you could take anything that was originally declared summative and turn it into a formative to some degree. Most often the only reason we don't allow papers, final tests to be redone is because somebody sets that policy. Not necessarily because it was a part or serving the learning experience. It really doesn't. So what if I could give you feedback on a summative and it really helps you improve your learning? You go back and learn it better. Do I have some kind of obligation maybe? Not just an invitation, but truly it, kind of an ethical obligation to go back and give it another go. I would just suggest you probably do. So if you put lots of feedback on that summative and there's no chance to really operate with that and go back and improve, that's a really frustrating exercise. It's an exercise of what might have been and the loss that one has felt. And we're there to teach so that they learn. So can I clearly delineate with online assessments? These are formative. This is coming to know. And these are final declaration. Absolutely nothing changes in that regard, but we have to be mindful about the instructional diagnostic utility of formative assessment. And that has to be a place of hope, a safe place to wrestle, fall down, get critique, pick back up and constantly improve. That's how anybody becomes competent is through that formative assessment and descriptive feedback cycle over and over. Every again profession we mentioned before, that's how people become competent in the field, not by taking one summative test and that's the end of your journey. And then summative tends to be final judgment, final evaluation, but I hold open this radical idea that summatives can be turned into formatives as necessary. So I don't think too much changes, but if you don't already have that mindful utility or purpose of formative versus summative, that might be a place to start. Tom? Yeah, I think that's perfect. And I think that too, this brings us back to that distinction that I tried to draw when we first began. You know, the notion of quantifying or documenting, achievement versus really facilitating learning. If our purpose is really on the facilitating learning aspect, then this distinction between formative and summative becomes very blurred in my perspective, it's formative until you get it. And so as long as we take that as the feedback perspective to it, it makes this very clear, it makes it very direct of what we're going to do for students and provides an opportunity for them to use it in ways that really benefit them and allows teachers to accomplish what they really want to. So formative until you get it, when you get it, then it's summative. Great. Perfect. Well said. Thanks. Okay. So the next one, we'll start with Rick as well. How should formative assessment and grading be done to accommodate the developmental differences between primary students, so mainly kindergarten to grade three and older students, grades four to 12. So just to clarify that one, primary grades attempt of over-reliance using a combination of observations, conversations and products. Whereas with older students, their report card marks often reflect more products such as tests and projects. So as we see developmental differences throughout the grades, how can formative assessment be done to accommodate those differences? You wanna start with me? Yes. Okay. Remember to go back to your criteria for what makes for clear evidence. So one is frequency. They did it once, but can they really repeat that down the road? Can we add in more complexity? Can they apply it in different situations? So the intensity, the level of performance, but also was it seldom, was it rare, or was it really very consistent? I would suggest the conversations that might be used in primary are really in effect ultimately the product. I'm looking for evidence. And I think all of us are 24 seven as Tom or somebody has said in decades ago, assessment junkies. We're constantly looking at everything in terms of assessment. And while we might not be as formal about it in primary, it follows the exact same universal principles. There's no big difference. Yes, we overly rely on easy to grade formal structures at the upper grade levels. Totally get that. That's fine. We got a lot of those kids to go through and do those things. And we don't always have the kids all day long. In fact, very rarely do we have the kids all day long, except maybe an upper elementary or if we're in a multi-age classroom, perhaps in Northwest Territories where we have everybody from grade four through 12 in one class, because there's just not as many schools around. I totally understand that. But if I could look at kids' conversations in grades four through 12, and these were after the learning is done, they were declarations of where you are right now and whatever it is I was trying to assess, I might draw from that as well. It just depends where it occurs. But I have to go back to what constitutes evidence of being able to tie a shoe, knowing the alphabet, understanding a story as a beginning, middle and end, like in primary grades, understanding sounds and whatever it is. I know that in my work in Alberta, Saskatchewan and Matatoba and British Columbia in particular, we have phrases in there like in social studies understands or appreciates the contributions of indigenous peoples or First Nations peoples. And then my subject like colleagues, we would have to sit down and go, okay, seriously, what does that really mean at this grade level? And we would decide what constitutes a relative proficiency. And then we would do that. So if we're trying to accommodate the differences, we're gonna be mindful of what's evidentiary, but also the question said formative assessment versus grading. Grading is the final report after learning is done. It really doesn't have a lot to do with the formative assessment along the way, which as Tom reminds us is really the focus of what we're about here is that learning along the way. Grading is just how we document that for the larger province, for the ministry, for parents, for sorting and selecting kids for the next grade level or the subject or the program they'll get into. Those are very different things. And as long as we hold close and powerful, the formative assessment emphasis will probably serve our kids right. Tom? Yeah, I think that's absolutely right, Rick. And I would add that this is also related to the nature of decisions we're asking teachers to make. If we're expecting teachers to look at the seven and stay gather from the students and rate the level of the performance of students in our scale that has 101 levels of performance, then we are dooming them to failure. They're just not gonna be able to do it. But if we step back and we can say, can you make more holistic decisions about it? Kid, tell me, do they have it or don't? Can they show me this or not? Do they have this skill or don't they? And make those sort of yes, no decisions, fewer cut-offs, better accuracy, far more efficient for the student and for the teachers, then we're going to be on solid ground. This is why almost all the major college universities here in the States have seen how difficult this can be for faculty members in making these decisions and have gone to either a sort of pass-fail system or a pass-in-complete system, where we're saying, if you show me you've got it, we're gonna make getting it pretty rigorous, then we're going to pass you or give you a passing grade. And if you haven't, then at this time, we're just gonna consider that incomplete. And we're going to give you other options to show us at some later time you've gotten it. So if we can also adjust the way that we report on these things to that kind of compelling nature of the assessments, I think it'll provide us with the differentiation we're really looking for. Okay. Just to add one thing on there, one of those big tools is to negotiate that evidence for passing with other teachers that share the same subject, because passing to one might be not passing it to another. And that also might mean, because we're trying to finish out the year and distance learning in general takes a little bit longer to get through the same curriculum that we will have to identify what's really paramount. What are the most leveraging pivotal learner outcomes for the province or for R1 school division? And then we will fight to assess those, to teach those and to assess those, but to sit at a table virtually and say, well, I think this is important for passing. And somebody goes, well, I don't think that's necessary for passing. It's a scary thing. It's not a natural skill set. So it might be something where school leaders will need to teach teachers how to compromise and identify that. And that's again, intellectually very heavy lifting. And we might need to consult, we collaborate, consult, not consult. Although if you wanna consult, that's up to you. But consult with our subject associations who've already done that kind of hierarchy and that filtering. If you have limited time and resources, at least teach this and maybe let this go until you have more time to do that and let the grade represent the most indicative standards or learner outcomes, not necessarily all of them thrown together equally. I hope that helps. It definitely does. Thank you, Rick. Okay, we'll go to Tom to start answering with the next question. What advice do you have for teachers who don't feel competent in making professional judgments with learning outcomes in online classrooms? Well, in many ways, I think this takes us back to the idea that Rick just brought up, that it provides a framework for teachers to really move forward in their collaborative efforts to establish these sort of ideas and the criteria they're gonna use to judge student performance. I think that it's very difficult for a teacher working in isolation to be confident in decisions they're making about looking at the evidence students provide in making those professional judgments. But to the degree that teachers can take advantage of each other and work collaboratively to share in ways to talk about what their expectations for performance should be. Can we incorporate these in the criteria we set for students? Can we adequately communicate those to our students and to parents? And then can we consistently implement those across students and across classes in the similar ways among teachers? I think it really serve us to become a lot more professional in the decisions we make and also make those decisions more meaningful to the students. So I think that notion of collaboration and sort of the cooperation among teachers to share in those discussions, this is a really excellent format for really facilitating that and moving us in that direction. I would add this, even if we spent a year or two preparing teachers to move everything online, they would still lack confidence and be a little bit concerned about this or a lot. And we did it suddenly, almost overnight. To me, it's like a blink of an eye overnight when you were given maybe a week, maybe not even that to get things ready. So understand you're making it up and you're not alone. A lot of us are struggling with that very thing. Do I feel confident in doing this grand experiment suddenly poised upon me where I feel inadequate to the cost? And that's exactly what y'all feel. And we should start with that when we do that collaboration to admit, hey, I'm not sure. And then of course, and by parents and students to kind of give us feedback on that particular idea. You were judging me on my proficiency in my demonstrated evidence, but I feel like you didn't give it a good shake because I did this, but you claimed I didn't. I mean, that back and forth dynamic still really matters, but that we're in this and this emotional element of we're all a little bit unsure, should be acknowledged and embraced. And then we kind of hold hands figuratively and we step forward together. Yes. Yeah, and sometimes I... Yeah, to the degree that we really focus on the feedback aspects and improving learning, both versus the sort of quantitative for our documenting student achievement, it'll be all the better because we take that burden off of teachers that they're coming true to their true purpose and that is to really help the kids learn. And so the minor missteps we take on that process are easily overcome. It's just when we wanna make these really hard line, high stakes decisions about kids in an assumptive way, it becomes more problematic. And I'm starting to keep bouncing back and forth, Christine and Susan, but... Tom wrote a really cool letter to his Kentucky, University of Kentucky colleagues and so on and others advising people about how to handle this going forward. And others of us have been talking about it as well. And that is, are you really gonna stay beholden to an arbitrary timeline based on this? Just because June 1st or May 16th is this date officially in the calendar, doesn't mean we have to hold to it. So we have some students because of equity access issues, can't really learn the material because of whatever's happening in their family and access, but they come to learn it over the summer or into the fall even if we're extending that long. Could they go back and have a previous recording of a grade or proficiency overturned given the subsequent evidence and not hold to that arbitrary timeline in an act of compassion and simple equity and ethics. I think that the case should be made for that. So if we're really worried about, oh, I gotta judge this because I have to submit reports here. How about saying, look, here's what I know as of today. Given subsequent evidence down the road, we'll totally be glad to go back and change that. And every opportunity should be afforded every single child to be able to do that. Just offering that as something to consider as perspective. Great, thanks, Rick. Sometimes I almost wondered too though, is it a lack of confidence with their professional judgment or does it have more to do with their feelings of not knowing how to use the technology? So that's another question as well. Could it be both? Yeah, could think about. Okay, so onto our next question. We'll start with Tom. How can we collect evidence of learning that represents deep transfer learning? So just to give you some background, one of our current goals and of new and future curriculum is for our students to develop conceptual understanding and higher level thinking. Now, obviously that's desirable in all classrooms, but how do we collect evidence of that in an online learning environment of deep transfer learning? Yeah, and I think this is a critical issue with regard to the whole idea of assessment literacy. And do we know how to get that apart from the online environment? I mean, what evidence would you accept from a student to say that they have shown me that they do have this deep learning that they've transferred learning to a new and different context? If we start from that basis of saying what types of evidence will be accept and then finally turn to what can we do within the online environment to allow students to provide us with that evidence versus the other way around? I think too often today teachers are saying it's online, it really restricts me in what I can do. And they said that is the screen by which they filter all these different sorts of evidence to step back from that and say, no, what are the wide variety of sorts of evidence that students could provide to me that would show me they have developed this deep sort of transfer learning and then how can we adapt the online environment to provide that? So, and again, buzz, an excellent opportunity to engage students in the process. If we're clear with students about their learning goals that what we want them to be able to do the kind of skills we hope they will be able to represent to us how would they see displaying that to us? What did they think would really show their teacher that they can do these things in this kind of way? So I think first of all to think from an assessment literacy perspective about what ways that could be demonstrated to the teacher in a wide variety of contexts and then put the online as a sort of filter by which we look at those different sources and say, what can we do within this online environment that gets close to those kind of representations? Rick, what do you think? Oh, I agree with everything you've just said and it really kind of cycles back to things we refer to earlier in the session here. Even if there's no online thing it's a difficult thing to say what does it mean to transfer learning that is deep? There are some report card systems that I've seen where this says the student went above and beyond and as Tom has written so eloquently about and others have been addressing for quite some time it is very difficult to be consistent teacher to teacher to teacher about what that really means. For one, it means this, they did more of something which is just quantifying it more they did four of this rather than two so they get a higher grade where here I really drilled down and said here's what it means to show intellectual agility skillful versatility. So we begin to have conversations about well, is this just a formulaic response? If you give me this exact situation I can apply a formula and we're good so it looks like I'm proficient but what happens if I introduce novelty if I change the variable? For example, I might say I wanna see what you know about metabolism, diet and exercise and how those three things intersect. So here's a student's life his meal journal for two days or so and the sport that he's trying to get better at doing what kind of afterschool snack choices should he make that would most support his goals? But the next question after that would be well, what if I took this out or introduced this variable? How would your advice to the previous question change? Is that showing dexterity? I don't know what we would have to decide that with each other. So if a child can simply define osmosis versus diffusion that's really an assessment of recall. The child I have proven is a good echo an excellent parrot but that's insufficient for deep learning. So now how can I take what we know about osmosis and the difference with diffusion and really show a dexterity of flexible thinking about it of incorporating variables into things. This is an overt assessment literacy as Tom labels it so beautifully that teachers need to develop and many don't have that quite yet. Now you have wonderful assessment consortia in Alberta you have the Canadian assessment formative learning network you've got so many resources in Canada that the United States we were just deeply envious you have that and I would suggest you avail yourself of the natural easy access treasures you have your own backyard. Oh, absolutely we do. And so, and as you were talking it kind of made me think of web steps of knowledge. Sure, I could be a catalyst or starting point. Right, and just thinking about how we the type of evaluations that we give to students do they have those different levels built in with it? Okay, moving on then to the next question is for Rick as well. How do teachers make online assessments equitable when home conditions vary from full to little support for learning? So for example, I've talked to teachers where they said some students have access to technology where others don't and some students have one on one help from a parent and others have very little parental involvement. So Rick, we'll start with you. Well, just to kind of take a step back and preface is Tom wisely stated the very first response we've got to make sure we understand equitable as opposed to equal. And, you know, Rick LaVoy and Tom and many others and myself and others we've talked about that it doesn't mean the same. So in other words, if I could find another way that you know that stuff, I'm gonna go for it not hold you because I'm sort of behind a false sense of standardization protocols. I have to do the exact same thing otherwise it's not the same legitimate evidence when I'm really rallying on evidence not that you took a test. So this idea of I would do different things with different kids to get up to speed on that stuff. So I have been so inspired already in just the last two or three weeks with teachers who are finding unique ways to assess their kids. We've already outlined some of those here and I don't need to belabor those people can go back and revisit the Zoom session if they'd like but they're doing things like the teacher literally goes to a few of the kids homes. Now I know there's a wide space in between a lot of kids homes and Alberta totally get that because I've done work with teachers and Yellowknife and Grand Prairie and Lethbridge and Red Deer and Medicine Head and Mighty Peace River and Alberta and all over I get that. But to the degree that we can and I might be able to stand outside of the student's door and have an interview I'll do it or a phone call if I need to do that I will try my very best. I will also realize for some kids I'm simply gonna extend a deadline or just say here's where you are now but you can totally overturn it anytime you want like we just discussed. But I'm not gonna say oh well there's no way to assess this I'm gonna continue to think about it. It's kind of like I've been a fight for the sound assessment principle. And if I can identify that principle I have energy. I find courage and fortitude to fight for it even when the mechanics and logistics of doing it fail me right now. It doesn't mean I will pull away from that. So if my principle is I will do equitable expressions which means they might not be able to express it that way but they can do it that way. Maybe they only did a portion of it for right here right now because that's all because of dysfunctional family or access that we can do. I'm good with that. I will find and we'll do the other down the road but I won't pull away from the principle because I can't fathom logistics in the moment. And again we gave you some specific examples before but there's so many online that teachers are getting really good at that. I'd advise them to go there. Tom you wanna add anything? Well yeah Justin I would add only reckon that we find that too and I would share that like you I am so inspired by the diversity that I've seen among the teachers that are working and moving this direction. Here in Kentucky we have schools where there are a number of students who don't have internet access. And as a result the schools are developing these sort of paper packets of information that the kids can have. In some cases the parents come by the school and pick them up in another case they're actually delivered to homes. The teachers for those students are calling them and just having conversations with them on the telephone using that as a basis for the feedback that they offer. And many teachers have told me that they find this to be a really amazing way of getting a clear understanding from students about what they're getting and what they're not. I know of other teachers that have set up in their daily schedule they have one hour each day where they say you can call me. And so if you're having questions or concerns from two to three o'clock I'm gonna go by my phone you can call and we'll have a little chat about what's going on here. So I think that diversity providing for greater equitability within this and the opportunity for kids to be able to take advantage of that when they don't have the conditions that we would think would be available to everybody but just are not today. So I think that that diversity really is complimenting kind of things that teachers doing and it's truly inspiring. And not to overwhelm anybody just to make sure you're mindful short segments, short chunks is a way to survive the day. So don't try to teach everything you have to teach for the rest of this school year it's gonna be a slower process but if you can do the most pivotal and leveraging you're still serving the kids well and the whole system well. Okay, thank you. And one of the things that both of you kind of alluded to as well was that idea that we have to still maintain our connections with students with regular check-ins and whether with phone calls or online meetings and so that definitely is an important factor as well. Okay, we are getting close to 11 o'clock but we still have two questions left so I hope we don't mind we spill into the open chat time because these are important questions. This one is for Rick. What are concrete ways that teachers can address and prevent plagiarism and cheating in the online classroom? So I believe that both of you had kind of started to talk about this earlier. So Rick. Well, just I wanna offer this if anybody wants to use the email address I don't know if those were sent out or not but I'm fine with that. I'm glad to send you all these slides. I wrote an article on this. How do you minimize cheating, plagiarizing and so on? It's listed down below but I've been doing some webinars on it. And what I thought I would do is just highlight, I don't know, like seven or eight, very, very quickly out of the 30 or so that are there. But one is of course, we've already mentioned make your assessments very complex not easily cut and paste plagiarism off the internet or from a friend. So for example, I might say how did this premiere head of the economy during this historical era and compare that to another premiere did that? Well, you might be able to just cut and paste with somebody else said. So what if I said here are four economic principles rank them in order of importance to this premiere based on what you know about that premiere and economic policy and now change that ranking for this one and time of difference. You have to take a stance. It just increases that complexity just a little bit more. I could do a book character rank these strange unrelated objects in order of importance to Hermione Granger and time of the ranking for different for Ron Weasley two different characters from Harry Potter creating a character study and the new verb was rank. So this increase the complexity that the difficulty with assessing it. Second one is I'm going to teach you to develop my appropriate manner so you're not panic and you need to do the cheating or the copying. We've already talked about that. Third, move away from the reliance and need for external validation help kids self monitor. They'll own their learning and not need to do the cheating nearly as much. And then one I learned from a college professor very, very effective really confront kids to some degree and say if you do copy and you do claim somebody else's work is your own that's a form of lying. You are a liar and nobody wants to be called a liar and it really kind of pulls people back from ah, so I want that footprint. So I want that reputation of being a liar. I would also suggest that many, many reasons why people do copy cheat play drives it's an impulsive act and it's born from a lack of executive function development. If you develop executive function which is a last thing to come online right behind the forehead prefrontal cortex that's where your decision making moral and abstract reasoning, time management meeting deadlines, take your responsibility responding appropriately in a social interaction. All of that is like right there and it usually comes online between 18 to 25 for most adults. So we're talking about kids who are K-12 they're younger than that. So overtly teaching students about executive function and building those skills. I've got resources I'll send you in the slides if you'd like to do that. Also, training mom and dad don't forget on what is appropriate level of assistance and what's over the top. Then I have found so many kids do not have skill sets on note-taking, paraphrasing, summarizing and yeah, shameless plug about to happen. So hold on, cover your ears if you don't wanna hear it. I wrote a book on that from ASCD. So I will tell you about that but a lot of times kids simply don't have the skills to realize, oh, that was a verbatim comment that for somebody else that wasn't my own oh, I need to worry about that. And then I think it's cool, it's very effective analyze work that was copied versus work that was not. And so they kind of get inside the teacher's mind of what that's like and what the teacher is thinking is if I happen that teachers can't see it, don't forget multiple assessments on the most important learner outcomes. It's harder to cheat over multiple formats, multiple timelines, multiple dates and then please oh please in this difficult age and honor code. Some kind of statement of honor and integrity that the school is designed that all parents and child has to sign. And then if you need to do a two-tier verification like we do for a lot of websites where the child submits the work but you do a spot call, email on the spur of the moment some kind of synchronous thing might be video or not and say, hey, I just wanna ask you a couple of questions to make sure this work is truly yours. And it's a subset of the much larger thing because you don't have time to do that with every child that's still legitimate. And you can say, look, this is not it against if I do this with everybody it's not a personal thing. And then please show kids the assessments ahead of time. If they know exactly what they're getting into they're gonna be less likely to be panicked and worried about it and make those impulsive decisions. And then for long-term projects of any sort have them submit the little pieces along the way. It's hard to cheat on those things when you're doing that along the way. It's a way to check in and you kind of get that inkling that intuition already. And then whatever you do please oh please do reduce and retakes in such a way so they don't have to worry about that. But I now have a controversial final comment. Don't sweat it. Teachers are so worried about this and to be really honest that's not the most important thing right now. So the way I kind of look at that is compassion before curriculum, grace before accountability and the idea that parents are struggling with levels of assistance. The child runs through emotional cycles. There's so much inequity in what's going on. I'm not gonna have a teacher tantrum or a harsh scolding and I'm not gonna make the grade unrecoverable. And so many of those things we would normally do in some more traditional schools like suspension, you get an F for the whole course. You're not allowed to attend school functions. Those things have been pushed off to one side. They're no longer useful to us. And if you really get down to it a solid unrecoverable F is never a solid learning tool. It does not create moral fiber, respect for deadlines, the way most teachers think. In fact, Tom has written about that as well. Fs, Ds and zeros are terrible teachers and builders of self-discipline. We literally know how to build self-discipline and integrity and honor. And none of that stuff, none of it says use the grade book to do it for you. So if teachers don't have a background in how to develop in a student's self-efficacy and honor and integrity, then the over there were lying in the grade book to do that for them. And this is the prime moment to get to move us away from that. And I'd be glad to help with that journey as much as they can. Tom? Yeah, I think that's absolutely right, Rick. And it gets us back to sort of our purpose when we started today that if you can see an assessment as a measure of accountability, if it's used to judge them, to evaluate them, to rank them in some way, then you're providing the framework for plagiarism and cheating take place. On the other hand, if they see an assessment as a tool designed to help them learn to get information about what we can do to make sure they learn excellently and that's its major purpose and how we're going to use it, then the cheating and plagiarism do you know good. It's all for your benefit to give the most honest information you can have. So helping our teachers see that purpose and then communicate that clearly to our students. I think we'll move us in this direction of really removing those aspects of this whole sort of assessment process overall. And just to kind of reframe that, some teachers inadvertently develop a gotcha mindset and relationship. I've caught you making mistakes. I've documented it for all the stakeholders. Come back tomorrow for more of the same rejection. It'll be fun, y'all love it. And that's not what we're about. We're there to teach so they learn not to play gotcha. So you might want to do a little audit, a little self-assessment. Is everything I say a matter of gotcha documenting where you fall short or is everything I do moving the kids towards his own personal growth and cultivating those talents. Okay, thanks Rick. And we're moving now to our last question for Tom. What advice do you have for teachers to balance the workload of triangulating evidence of progress and being responsive to student learning online? So for example, feedback takes quite a bit of time when you don't have that face-to-face with students and also with the provincial mandates for the timeframe that is allotted for each divisional level. So for example, for K-3, they have five hours of learning per week. Four to six is five hours. Seven to nine is 10 hours. And then in high school, it's three hours of work per course per week. So if a student is enrolled in four courses, that would equate to 12 hours of learning. And so it's a big adjustment for parents, students, and teachers. So how do teachers balance that workload? Well, I think there are a couple of major issues here. The first, and Rick referred to this earlier, you never want to make a decision about a kid based on a single source of evidence. We need a multiple source of evidence because any single source of evidence can always be tainted, not only by the type of evidence it is, but the situation in which the student completed that evidence. So always think about collecting multiple source of evidence when you're making decisions about kids. Second is to recognize that there are multiple ways of demonstrating learning of any particular skill. That students, we might have an idea of how we best would communicate, but that may not align with the way the student best communicates. So to allow some flexibility and even student choice in the way they would demonstrate to us what they've learned and what they're able to do pretty well. And the third aspect, again, is moving again, primarily in the direction of feedback versus just documenting or quantifying. If it's a feedback purpose that we have in mind, really designed to understand what they've learned well and what we can do to help them learn better, then it opens up a world of possibilities here that we don't have to be restricted to anything. I know when I taught these classes online, I felt very hampered and restricted in the kind of assessment formats that were possible within the structure that we had available to us. And so out of my frustration, just turned it over to my students and said, I don't know how to do this. How could you show me that you've really learned these things well? And I found they were far more creative than I in coming up with different ideas. So I think to not get yourself trapped in a sort of one-way approach to think about multiple sorts of evidence, to think about different ways that this kind of learning could be communicated and then to allow students some choice, especially as they get older, with the teacher's guidance on how to demonstrate that well would move us very powerfully in this direction. And I would just quickly add a couple of thoughts. One is tiering and scaffolding. There are very specific books and articles and I'm sure webcasts and videos on how do I teach simultaneously multiple readiness levels in one class? So if I'm trying to get evidence of one thing, but I'm also trying to be responsive to the need to progress and move on, how do I teach those things simultaneously? So that's the world of differentiated instruction, responsive teaching and understanding, universal design for learning. These multiple ideas. Do I move on because this is not really foundational to what's to come? So I get Elsa about it and just let it go. Or do I say, no, this is so fundamental. I have to teach it. Or do I realize I can spiral back and re-teach it in part of a meaningful context down the road? It turns out a whole lot of what we teach is not exactly linear like you might think following this very lockstep thing. Humans like a lockstep maneuver because it feels certain. It feels logical and we can almost quantify that. But learning can be very messy. So the strategies for tiering, raising and lowering complexity levels, meeting the needs of a kid who doesn't get it, there's many lesson pullouts which you can do with a lot of technology. And then while the rest of the class is working as I'm moving forward, will be the exact same questions you had even before online distance learning. And employ some of those tricks when you do the online distance learning. Second quick response, I've gone over the Alberta recommendations. You can see I've marked it up a little bit here. And the most outstanding thing that comes to me is when we are secluded at home, and this is like really, really important, they have nothing in here in kindergarten through grade three and grades four through six and then seventh you're not pretty much all the way through with civics, fine and performing arts, PE. And those things are far more important when you're stuck at home and they bring richness and connection to all the other subjects. And there's no mention, it just says, you'll focus on language literacy and math, numeracy. Boom, that's it. What a missed opportunity, what an unwise move. I realized they're trying to survive and it was a compromise, I get it. But I implore teachers across Alberta to weave in fine and performing arts and we need civics today with so much that's going on in both the United States and Canada more than we've ever needed before. As we try to work with each other in this pandemic around the globe, I'm begging you to reconsider that to the degree you can. That's my little two cents. Okay, thank you. Let me add one thing to that. I think that's such an excellent point. I know here in Kentucky, we've really gone out of our way to provide directions for teachers in how to do this. And one of the things that was stressed in this was that if any teacher, you wanna begin, schools are typically beginning with some daily announcements. They have a minimum of two and a half hours of instruction, but also a minimum of one hour of physical activity every day. Yes. It's so easy to be complacent as you're sitting in front of your computer. I know my daughter has convinced me to wear this Fitbit. And if I don't take 250 steps every hour, my Fitbit buzzes. And so if I'm sitting here at the computer and I get involved in something, then my Fitbit buzzes and says, get up and move your butt a little bit. You know, don't sit there all day. And I think that to the degree we can build that in along with art and music and drama and dance. It can be so powerful for kids. Really provide parents with the kind of things that they might be able to engage in with their kids. It could be really helpful. And you could integrate absolute evidence of proficiency in all those things. Oh yes. So when I think about integrating some of the subjects, it opens doors. This is the time to be creative and to run with that. Okay, great. And so this we are now transitioning into the chat portion. I'm gonna start looking at the questions from 1103 a.m. this morning. And I apologize if I skipped over your question, but please type your question in the chat box and we'll try to get to as many of them today as we can. Susan, we've gathered some from the chat box for you already. So after you go through those ones that came in early. Okay. We've asked people about 20 minutes ago to re-enter questions. And we've got some of those for you when you're ready. Okay. Sounds good. So the first one I'll start with is from Amber. Considering the incorporation of the Canadian Assessment for Learning Networks suggestion of pass-pass with distinction, or should that be pass-fail for distinction for high school grades, we submit generic percentage grade to comply with Alberta Ed requirements, but still offer third alternative of actual earned grade, where teacher feels confident in authentic assessment of the full program of studies. And this works for classrooms and students who have made full transition to effective online learning. What would be the pros and cons of that? Again, the pros and cons go ahead. What was your grading scale? I'm sorry? Is it the pros and cons of the limited categories in the grading scale? I'm not sure. Or having the option of if we feel competent in the actual grade itself going with the grade instead of pass distinction, pass, and incomplete. Is that correct? I believe it's, she's talking about the, a third alternative where the actual earned grade where the teacher feels confident. So I would assume that is talking about what they would have received from the work that they've been doing throughout the semester of the full program of studies. So equity, equity, equity, baby. Yes, plus I would add, I mean, if professors at Harvard and MIT don't feel confident in making those kinds of distinctions in a grading scale that includes only five levels, A, B, C, D, and F, to say that we are confident in making those decisions in a grading scale that includes under one levels is a pretty confident statement. And I think in many cases it would be very difficult to defend. I mean, our good friend Dylan Woehm has recently looked at the assignment of the percentage grades and finds that the amount of variation among teachers with criteria specified is plus or minus six percentage points. Well, if we can't do it when we have clear criteria in a typical classroom environment, why would we ever be able to do it in an online environment? And it's really smart people and universities are saying, it's not fair for those equity issues you've discussed. Then I think we need to take that same kind of advice at our level too. I really like. And again, this is what we got from the folks and this is again the Canadian Assessment for Learning Network. They've done a great job of outlining some things that I think are sound, they're equitable and they're educationally defensible. I agree. And even at the end of that document, they say the equity issue must be addressed. And it says we recognize when we provide the opportunity for passing the distinction, it places equity of access and equity of learning in jeopardy. And some students will opt into deep and rich learning and some will opt out even worse, some will not have the option because access and so on to do this stuff. So those things have to be addressed. So if there is that other option, go with the grade, you might need to end up doing something like we will give you the grade for up through March 1st and we will specify this is a truncated curriculum based on where you are heading during that time. But even then it can be a little dicey with the equity issues. So if we do the pass with distinction, once again, you'll have the same issue. What does it mean with distinction? We have to clearly calibrate that with one another, otherwise it'll be all over the map. And the follow-up is, what does that mean for next year's placement or for honors or whatever comes of it? So if somebody has the letter grade but somebody else has passed fail, can we still look at them equitably as reasonable candidates for this advanced level, whatever it is. And the other point too, is that this pandemic is worldwide. This isn't something that only a school or a province or a state or a country is really experiencing. So even when they do this, like at Harvard University, they have a little asterisk beside the grade. And but anybody that knows a grade that comes from spring of 2020 is going to be done under different conditions that we've faced before. So that asterisk, Harvard calls it an emergency pass or emergency unsatisfactory. And so I think those distinctions will not only be prevalent, but they'll be clear. Everybody, these are unusual circumstances under which we're all dealing. And it'll be widely understood. And the teachers next year must be aware of the previous year's curriculum because they'll be submitting pieces. They can't assume they all have that. And that means we have to get out of our comfort zone. Just let me teach my one curriculum. I don't want to learn any other curriculum is no longer sufficient for the true education professional. So I'm gonna have to actually spend some time diving into the previous year's stuff so that I can integrate that with this year's stuff, at least for the really big boulders, the big powerful, most applicable to my curriculum learner outcomes. That is a brilliant statement, Rick. I think that our teachers across Alberta are going to need to shift their minds a little bit as they facilitate learning now through the rest of this year. The focus has to turn a little bit to their planning for September, particularly junior, senior, high. If I'm the physics 30 teacher receiving students who in physics 20 in the spring of 2020, I know they got five solid weeks of instruction and 15 weeks of online something. I've got to plan in September for revisiting the big items they need to know moving forward in a totally non punitive supportive way. There you go. I agree totally and I'm just gonna suggest it that maybe we try to find some funding for teachers at different grade levels to actually get together, some spend the time getting together, making sure they're up to speed on each other's curricula, if at all possible. Of course, teachers will end up doing this because they're conscientious people. And I know that funds are tight, but I'm wondering if there might be a charitable organization or a way to get some ministry money just for that. Hey, let's get people up to speed. Last little thing, and that is regarding this particular thing, I am pretty certain I'm willing to kind of put money down, so to speak, but I just wanna make sure people are clear. This is long term. So I don't think schools are gonna start again in the fall within the classroom. I think it will continue to be virtual until we get a widely distributed vaccine that's proven and that we can do tests for antibodies and bloodstreams and things. So I think people need to think about this in terms of how do we get through spring of 2021 during this unique setup, not just how do we finish the year? Okay, now that was maybe a little bit discouraging and the folks from Alberta, when you say more funding, there's a little bit of- Oh, I'm sure they probably blew up and said no way. No, we got a few challenges there. I do want to- I'm thinking creatively. Yeah, I do wanna bring one other question in that came from Rose, the Ministry of Education in BC has produced a planning guide for teachers regarding continuity of learning. This document provides the following recommendation, teachers will determine a final grade of students based on where completed to date and the assessment of participation in learning opportunities that will occur over the coming months. The question Rose puts is, what advice do you have for teachers to help best to assess participation in these current learning opportunities? And I know there's a little challenge there about assessing participation anytime. Wow. And what do you do with those students don't have access to internet, those students that are working from a paper form about about the school? Sure, we can document the amount of online time students spend but what relation does that have to their learning? Maybe a lot of what they're doing is work that's independent and away from the screen or away from the computer. I think there's so many problems with trying to do that in an equitable way, a way that's fair and representative that I really want folks to think about that deeply before making that a rule regulation and all teachers be expected to follow. Unfortunately that is a BC piece that those tend to drift across and back and forth across the border appreciate that lots of challenges for us. Susan back to you and those were the questions that we harvested that I had. Okay, well we've got six minutes left in this webinar. So if anyone else has any questions please put them in the chat box because we have Rick and Tom for an additional six minutes. Christine would there be any from the live stream? I've been looking on there as well. If I wonder if we could just maybe be so bold as to bluntly ask the question because it has been referenced several times and certainly there are opinions. The entire idea of final exams at any grade level in this current environment, traditional some form of a traditional form of a final exam, your thoughts. Well, let's go back to the issues that we began our discussion with and I would care there what's the purpose of that final exam? Why are we doing it and what purpose does it actually serve? If we address that issue first, I think that'll clarify not only the framework for it but what we do with the results. To just say that we do it because we've always had final exams. I don't think it's an adequate defense but to really think carefully about what is this going to provide us that we don't have now? What purpose would it serve? How is it gonna benefit students? Then once we address those questions then we can turn to do we give final exam to require final exams and how are we gonna use the results of that overall? I would add that the testimony for a teacher let alone a grade is what a child carries forward and can do independent of all assistance not what they did for a test and then forgot and never carried it forward. That's not real learning. The goal is that they actually carry it forward and they learn it. So if you're trying to see what they carried forward maybe you could do some form of final assessment if it follows the criteria that Tom has outlined there so wisely but you might be open to a variety of ways to do a final assessment. So instead of an EOC or end-of-course test or official exam that we now have a variety a menu of options for the ways to express and that we narrow that curriculum focus again to just the most paramount from the earlier part of the year. This learning since we have been sheltering in place is gonna be insufficient to really be demonstrative on some final tests down the road. So I'm suggesting you move away from using final exams but if you wanna have some final graduation by exhibition and it's a wide variety of ways of just these particular principal points that were pretty well taught up through March 1st or whenever your deadline is there might be some legitimacy as long as there's multiple formats and we've answered every single question regarding the equity, the access but also the emotional things that kids and families are going through especially if they're worried about the economy and members of their family that are already ill. It's just gonna be a completely distorted truth and grades are about accuracy. If you wanna maintain the integrity of a grade you'll do everything to increase accuracy and diminish distortion and a final exam might be very, very distorting for a subset or for a lot of kids. Yeah, and I would add to that too a lot depends on how you define final exams. Yeah, for example, I teach a class that is designed for advanced graduate students who are getting ready to prepare their doctoral dissertation and they're in that class we have three major segments having to do with the first three chapters of the dissertation. First is a sort of a description of the problem. Second is reviews of literature and then the third is a chapter on methodology how they would explore this problem and the way we set the class is that they spend a lot of time really clarifying their problem and writing that they turn it into me. I give them very explicit feedback and then they revise and then they turn it into second version. We do the same with review the literature and the same with the methodology. Their final exam is putting those three things together and submitting it to me as their dissertation proposal. So when you define a final exam in that way where students are given regular feedback on every step to get there they know what the expectations are they know what that performance is going to be at the very end and it serves a purpose as Rick said and maintaining and keeping that information in a useful and purposeful way then I can see it to be valuable. So if we define final exams in a manner like that I think that then we'll be on solid ground we'll serve the purposes where we want to be a learning tool rather than just an evaluation device. Okay, sounds good. We are drawing near an end to our webinar. I just would like to take the opportunity to thank everyone for joining us today for considerations for assessment and grading in a pandemic online world. And I'd like to say that it's been an absolute honor to have two of the most articulate experts in educational assessment engage with Alberta educators to assist us in our journey of supporting students learning from home. So on behalf of ARPDC a great big thank you to both Thomas Gueski and Rick Wormelli we definitely appreciate your insights and we are definitely planning to do some future webinars with Rick and Tom and so please look forward to professional learning with both Rick and Tom in the near future. And so you will be sent a survey for the participants to fill out for us. So we would appreciate if you took the time to do that. If you look at the presentation which I shared earlier with the Bitly link there are some recommended articles that Tom has made using the Bitly link AB assessment. So bit.ly forward slash AB assessment will get you access to this presentation so that you can get to all the articles if you did not receive them. So those of you watching on the live stream can also access them as well. And then I have another page that our recommendations from Rick. So it's now 11 30 and we are concluding our webinar. Thank you again for everyone for joining us and a great big thank you again to Tom and Rick. Thanks. Thank you Susan, Christine and Rick. Happy birthday, Rick. Enjoy your day. Thank you. I bought an apple pie. Can't wait to party tonight. Great. Thank you. Have a great day. Thanks. Thank you.