 at the University of Kentucky. He is the author and editor of 25 award-winning books and more than 250 book chapters, articles, professional papers on educational assessment evaluation, assessment grading and professional learning. And with that, I will turn it over to you, Tom, this morning. Good morning. Good morning, Susan. Thank you so much. And thanks to everybody who's taken time from their day to come and join us today. This is a kind of special time. This is Teacher Appreciation Week. So it's a great time to honor all those folks that are going out of their ways to do so much for our students now that are really, you know, working from home and as we are as teachers too. So thanks a lot for being a part of the day. And welcome to my home. This is, I'm coming to you from Lexington, Kentucky, where this past weekend, we had the virtual running of the Kentucky Derby. I know that seems sort of hard to imagine. They were able to take virtually films and recordings of the top Derby winners for the last, I guess, 50 years and ran them in a race together. Of course, the irony of it is that in 1973, a horse by the name of Secretariat, not only won the Derby, but set the track record there. So there wasn't really a lot of debate even running virtually who was going to win this race. And indeed, Secretariat did win. So all those people that bet on Secretariat beat off very well on Saturday. Today we're going to take on different issues from last time. We had the chance to be together just a few weeks ago and we talked about assessment issues. And now we're going to turn from that to sort of grading issues. And I wanted to share with you a series of slides that are part of that. So I hope you have access to those. Christine, can we go ahead and put up those slides? You're there for folks? You're muted. I'm sorry, thanks, Thomas. You mean your slides, right? Yes, my slides. Yes, sorry, I'm going to stop sharing and now you can share yours. Okay, let's go up by share screen. Boom. And this. Continue. Share. There you go. See the slides there? You can see it, yeah. Okay, good. Well, let's go forward here. Maybe. Oh, there we are. In terms of grading, there are two issues that we need to consider as part of our time together. First is sort of where we are now with the grading. And again, everybody knows that there have been documents that have come out from the province about what procedures schools should follow with restablishing the grades for students for this year. But this is really an excellent opportunity for us also to look ahead and see what we're doing with grading, what we've learned from this experience and if there are changes we might want to consider and weigh and then think about in incorporating in years to come. So with that in mind again, as we did before, I'd like to start by sharing with you some contact information for me. Here are several ways that you can contact me and I do hope that if other questions or concerns rise, you won't hesitate to get in touch with me at all. I do have a website where you can find articles of referring to today, copies of presentations, blogs on this and other topics. You may contact me through email. That's my email address here at the university. It's just gusky.uky.edu. Those of you who follow Twitter can follow me there, contact me through Twitter. It's just at TGUSKY. And that's my phone number here too. That is my cell phone and rings here at home. So if that's a more convenient way to get in touch with me and message me, you're certainly welcome to do that too. We did establish some learning targets of what we hope to accomplish during our time together. And these are the three things that we really wanted to turn our attention to. First, we wanna recognize the inequities that are really affecting learning from home and explore ways we can work around those. These, as you know, looking at provincial documents that have profound effect on what the provincial educators are suggesting we do with regard to grading for the coming year. And we'll be talking about some of those. We wanna understand the importance of articulating a purpose for grading and to ensure that all the practices aligned with that purpose that we set up. And then finally, we wanna be talking about the developing guidelines for ensuring that our practices yield grades that are accurate and meaningful, but also honest and equitable, so the purposes we want. So with that in mind, let's return again. We talked about this briefly to some of those factors that do contribute to inequity that we have to recognize and work around as we are thinking about grading issues with regard to our students. The first is that we know there's inequitable access to technology that our students throughout the province have various levels of technology available to them, not only access to internet and the hardware and software that's available. That's no different from us here. We have students who to gain access to internet are getting your parents to drive them to McDonald's or to a Starbucks and park in the parking lot so they can use the internet access because they don't have the internet access from home. And these kinds of things are affecting what we can expect students to do. And in terms of facts, the kind of things we'll be able to do with regard to grading. A second level of inequity is just access to school supplies and resources. We know that homes don't have all the necessary resources that students might need. These are the typical things we provide in school that may not be available to students at their homes. We need to keep that in mind as we think about structuring learning activities, gathering evidence on student learning and then evaluating, judging, grading that events. And then finally, we have inequitable adult support that we know that homes differ in terms of what adult support is gonna be available to students. That there are homes where the parents or guardians actually work outside the home. So they're not there to provide students with the kind of assistance they might need. There are many homes where the multiple children at different grade levels who have to share access to whatever technology is available at home. Parents and guardians often feel unprepared to help their students. We find this to be especially the case when students move on to the middle school and high school levels. In many homes, English isn't the first language. And then finally, we know that there are many homes where the parents or guardians are also teachers. And so the technologies might be further shared where the parents are providing instruction for their students while having to try to keep up with the learning programs of their children as well. Now, we know that in terms of grading, Alberta education has offered some grading guidelines. And here's some of the major ones that we need to keep in mind that will actually guide the decisions we make, especially for students that are in our classes now, what we need to do between now and the end of the academic year. The province has suggested that students will get final grades. That'll be a percentage grade or a letter grade depending on the grade level of the school decision making that's been taking place there. That all students who are on track for the progress will go to the next grade level. We have also indicated that students will work with high school students to complete their courses to the best of their ability, providing of course that the final mark and awarding the credits accordingly. They've given principles of latitude in what they can do here. They can award credit to students to progress to the next grade level if the student wasn't able to complete the course because of inadequate access to technology resources. And then there are also some high school diploma criteria that have been mentioned. Principles can award up to 15 credits to grade 12 students negatively impacted by school cancellations that schools can work with students to complete courses underway. And finally, that if students are unable to complete courses that would have led them to achieve the high school graduation. Principles can award credits to ensure that students graduate. So these are all things that our school leaders will be addressing during this month and until we finish the remaining part of the school year. The challenges that we face there are these. We need to follow these guidelines for the academic year and be mindful of what is fair and equitable. And I think that province is a really excellent job of making sure those things are well-known. But then we need to think about what we're going to do for the next year. Whether schools open or not on schedule is still something that's been being discussed. We're hoping that that's the case, but we want to make sure that we establish effective grading policy practices for the next year. So with that in mind, I have two questions for you. We'd like to take you to the chat groups and give you about two or three minutes to discuss these things in terms of grading. Knowing that the province has indicated to you some guidelines for what we're going to do for the remaining portion of the semester. But then looking ahead to what we're going to do next year and what this might lead to changes in our grading policy practices. What are the questions that you have with regard to grading? What are things that are most troublesome to you? And then what are your greatest grading challenges? What do you think are the biggest things that you need to face as school leaders in moving forward and making sure that the grades you offer students are fair and accurate and meaningful, but also equitable? So if we could, Christine, just give folks about three minutes within these chat groups to talk about these issues and maybe we'll come back and try to get a general survey of what some of the major issues and challenges are. Sure, Thomas. I will break people up into those rooms. So you'll have three to four participants in your room. And again, you'll see the option to join the room. If you have trouble seeing it, you need to exit the full screen to see it behind the window that you're in. So I've just opened up all those rooms. You should see the option to go into them now. And sorry, before you go, once the time is up, you'll see an option to come back into the main room. They're all going in there, Thomas. Again, please let me know if you have trouble getting in there. Thomas, you might be, I think, are you assigned to a room or do I need to? I think I'm going to break out group one. Okay, do you want to go in between the rooms? Okay. Because now he's going to have to reshare a screen. Yeah, he can reshare a screen. He went into breakout room one. You see, there's a few of you that weren't able to get in yet. Jen and Jody, please let me know if you don't see the option. I just, I'm sorry, I'm late. I just arrived. So it's okay for the breakout room. I'll just wait for everyone to return. Okay, sounds good. Thank you. I apologize for being late. Yeah, no problem. I just didn't want to leave you out in the cold if you wanted to get into that breakout. Okay, thanks so much. You're welcome. Hi, Thomas, we're all back in the main room. You're muted again, though, when you pop back in and you'll also need to reshare your screen. Oh goodness, what did I do here? Do you see the zoom window? Yeah, we can see that. Okay, so what slide are you seeing now, though? Three guidelines for effective grading policies and practice. Perfect. That's great. Now let me see if I can get that up. You might need to hover over the bottom of your screen and click the zoom icon. You want to reopen the zoom window and you should be able to see yours. You should be able to. Oh, I'm going to decide to just say meeting controls. Okay, how can I advance this? Okay, well, let's go ahead and start. So you're seeing the screen that says three guidelines for effective grading policies and practices. I want to make sure that I can advance those. Okay, let's start there. Okay. In the time we have available today, there are three sort of guidelines for effective grading practices that I want to share with you. And there's sort of background things that need to be the foundation for our work in this area. The very first, we can go to this. So what slide are you seeing now, Sydney? Same one. Oh, right, I need to find a way to. There you go. Yep, it changed. Okay. Yep. All right. The first thing we have to do is agree upon our purpose. Where schools make really terrible mistakes when they're changing grading is they go forward and they change the process or the mechanism of format used for grading. They work on how issues before they decide to why. And so I have two really sort of crucial questions for you here. But again, I'm lucky to take just some time to discuss quickly. And the first question is, why do we do grading in the first place? I mean, why do we assign grades to students work and why do we use report cards? And the second issue is, what evidence should be used to determine those grades? We find that there's lots of variation among teachers, especially in what evidence they count as part of the grade. So I'd like to just take two minutes, again, with your new chat group, to consider these two issues. Why do we give grades and what evidence should be included? Let's see if you can reach them in consensus in your group. We're gonna go and come back from that and I'm going to review some research studies that have looked at these two issues in particular and share with you what they found. Thomas, I'm just gonna, and how many minutes do you want me to give them? Just a few minutes, please. Okay, and I'm just gonna move you to another room so you're talking with different people. Or if you wanna stay in the room as well in this main room, that's fine too. If you can just join, totally okay. Stay here for right now. Okay, those breakouts are open and you should be able to join them now. I have a few of you that are unassigned so I'm gonna add you there. Everyone should have a room now. So why can't I get to... Do you maybe need to check your view options and exit the full screen? The view options are at the top, right. Let's see. Anything I can do, Thomas? No, I'm sort of, I've taken myself out of things completely now. So let's just go to, again, back, hmm. When you're looking at your screen, do you have the green box that's sort of around everything that's on your screen? Let's see, I have new share. Let's see, go. So are you still seeing the screen with the important questions on? No, now we see the presenter view where you have the thumbnails of all of your slides on the left-hand side. Oh, okay. So let me go, hmm. So you need this. How's that? Yes, now we can see just your slide and not the ones before and after. Okay. Important questions. Right. So I don't know how to change my view. Your view options are at the top right and it says view options and you have to click the dropdown so that you can change that if you want. I think it's probably defaulted to fit to your window. Yeah, right, we'll go with this. I can switch slides from here. Okay. Okay, are you okay for me to call them back now? Yep, yeah, we're good. Okay, so it just gives them the 60-second countdown and pushes them back in the room. I'll let you know when they're all back. Give it back? Most of them, there's just a few more. Yeah, they're all popping back in now. Oh, terrific. Well, thanks everybody for considering these issues. I wanted to share with you that the advantage we have as researchers of asked educators exactly these questions. And when they ask a question about purpose, they find their responses of most educators can generally be classified into one of these six broad categories. One of the reasons we grade is we're trying to communicate information about student achievement to parents and families. A second reason is to provide information to students themselves, especially for the purpose of the self-evaluation. Third, we use grades to select the identified group students into various educational programs. You have to have high grades to get into honors and advantage classes. You need decent grades just to be promoted from one grade level to the next. You clearly need high grades in a college university. And low grades are the first indication additional assessment may be necessary to determine a student has special needs. Number four, we give grades because they provide incentives. Now, a lot of people debate this idea, but the evidence is pretty clear. What's the first thing students ask when a teacher announces is gonna be a quiz or an assessment? Does it count? And if the teacher turned them and said, no, it doesn't count. Well, who studies for a test, it doesn't count. And so clearly it's some incentive value. Number five, we look at grades when we evaluate the effectiveness of various educational programs. We hope that the program's working well, grades are going up. And then finally, to show evidence of appropriate effort and responsibility on the part of students. Now, we could argue that any of these are really legitimate purposes, but what I often do is ask educators just to look at these and rank order them from the most to the least important. And what always amazes me is we don't agree. That I always find some people are ranking one of these as the most important and others as the least important. We are all over the place. And see, that's the first problem that leaders in grading reform have to confront. Because when we don't agree on purpose, it's very difficult for us to deal with methods of procedures that align with that purpose when we don't agree. That's the first challenge you have to take on. You have to get people to somehow come to some agreement about why we're doing it in the first place. For whom is the information intended? What information does it include? And what do we hope to desire results to be? Now, researchers have also, as educators about the source of evidence they use when determining grades. And this is a list they've generated. Now, it'd be very unusual to find any teacher that incorporated all of these. But these show the vast variety of source of evidence teachers use when determining grades. Now, again, I'll often ask teachers among these, just to give me account of how many they include. And what I find that it ranges from about two or three up to 10 and 12 among teachers in the same school. And see, that's the major problem we have to face. That when we think about this, one of the greatest concerns that parents and students have is the inconsistency in grading practices among teachers in the same school. But as leaders, to bring about that consistency, we must seek some sort of consensus with regard to the purpose. The means that we're really going to have to be sitting aside some time to sit down with our staff and go through these and reach some consensus about the purpose. It comes down to a process like this. It's actually called, we call this disagree and commit. Now you've probably heard this discussed as a management strategy that's used by these high level tech firms. It's used by Intel and Amazon uses disagree and commit where we bring people together with all these different ideas. And then we meet them, but we come to some consensus because we all have to have a consensus to go forward. But actually the roots of this go back much further. It was actually used in planning military strategies. This photograph, for example, is this team of military leaders that got together Montgomery and Eisenhower in particular to plan the D-Day inversion that ended World War II. And they went through exactly this process. All these brilliant minds came together. They had a variety of different ways they would go about it. But after three days, they came up with one plan that everybody was going to follow. And that's just what we have to do with regard to grading. And that means that you must develop a sort of purpose statement that you'd be willing to print on a report card. Now, I've been working with schools all around the world in doing this. And I'd like to share with you a purpose statement from one particular international school. This is the American School in Paris, Canada along with the United States, every place in the world that we have a diplomatic mission. We have schools for the children of those people serving diplomatic mission. And this is one for the American kids that are in Paris. And this is their purpose statement. And you can see in this statement, they've tried to communicate exactly what they hope to accomplish. That the primary purpose was to effectively communicate student achievement towards specific standards or we could call them outcomes in Alberta. At this point in time, the grades reflect what students knows and is able to do. Students receive separate feedback on evaluation of their learning habits which are not being included as part of the academic grade. Now. Sorry, just one second. I think you're not on the correct slide. We're not seeing that purpose statement. We're on parents and students. Greatest concern about grades is inconsistency. Oh, okay. Well, my slide changer is working differently from yours. Oh, this is slide of the Eisenhower in Montgomery with their disagree and commit group. And now we're here. This is the purpose statement for this American school in Paris. They've been working at grading reform for several years. And this was the purpose statement they developed. Now I'd like to draw your attention to one particular point in this purpose statement. At this point in time. Because you see by including that statement at this point in time, it really dictates to them what their practices are going to be. Because when you say at this point in time, the first thing you have to do is, you need to throw out averaging. Because what we're saying is, we don't care where kids were. We care a lot about where they are right now. And so we really want the grades that we assign, we put in report cards to reflect where the kids are at this point in time, not where they started, but where they are right now. And so this is a prime example of how your purpose statement can dictate to you some very important strategies with regard to how you go about the grading process. Now, a teacher's tasks in grading are basically these three things. They have to gather evidence on the student learning. What the kids have learned up to that point in time, they have to summarize that evidence based on their purpose and then communicate that summary to the students and to the families. If this is their task set, it leads me to the point number two that I wanted to share with you. And then when we start this process of that communication, we need to begin by tip-relating the very top level of student performance. One of the major dilemmas we're finding with grading reform is often, as we discussed with assessments, people start in the middle and then try to move up and down from that. But what we have to recognize that for many of our students and hopefully for most, their striving is going to be for that very top category, whatever that top category is set to be. And so when we start articulating our purpose for grading, that's where we need to begin. We need to start by actually talking about what that excellent level performance would look like in our classrooms. The key issue here is, you need to begin with a model of excellence. And then when you begin with the model of excellence, then you can talk about what the categories below that model of excellence start to look like. But this brings me to the next challenge that teachers are going to have to face. Because you see, if they start with that model of excellence, then the fewer grade categories you have below it, the more reliable or consistent the grades are going to be. And again, where we get into trouble with this inconsistency is by having lots of different categories for student performance. Here are some of the options that are available. So here are the four, five major kinds of grading systems that we see your grading system categories. We have percentage grades. We see that being used a lot with many of the schools in Alberta and other places around the world. You can step down from that to plus and minus letter grades, where we're given A plus, A minus, A, B plus, B, B minus and so forth. Letter grades, we have five categories and outcome based, where they're typically about three categories, exemplary proficient and not yet. And the satisfactory and complete. We can see these differ drastically. Not only how many levels there, but how many cut-offs teachers have to consider. With a percentage grade system, teachers have to be able to distinguish a hundred different cut-off levels. With plus and minus grades brings down to 12. With letter grades here at about four, outcome based takes down to about two to the standards or the satisfactory incomplete. Now the challenge with this last category and you know that many schools in the States have decided to move to that because our college universities have, is what you set for a level of satisfactory. Research evidence on this, most of it's been conducted at the college level and they find that when it's a pass-fail option, students would do whatever's necessary to pass but not a whole lot more. And so the key issue here is what do you set that pass to be? What do you set that satisfactory to be? And so like at my university, what they did is they looked back at what we expect students to maintain in order to stay in school. At the undergraduate level, you have to have a two O average to stay in school. A two O is equivalent to a C. So what we've set the satisfactory at is at the C level. For graduate classes, you must maintain a three O average to stay in school. So the satisfactory level for any graduate class is set at the B level. Those are all arbitrary decisions that you have to make but recognize that as you make those decisions, you're really compelling teachers to finally describe these different kind of categories that they're going to have. So if we think about what they're going to have to distinguish. Here are the decisions that teachers must make. If you have a percentage grade system, then you have to be able to tell me in pretty clear terms, what is the difference between 90 and an 89? If it's a plus and minus letter grade system, what's the difference in the A minus and the B plus? Can you make that clear? If you assign both of those grades, can you distinguish between the one and the other? Could two teachers look at the same body of evidence, make the same decision? If you move down to letter grade, you have to be able to distinguish the difference between the A and B. If it's a sort of outcome based model, then what's the difference between exemplary and proficient? And again, if it's a sort of two level system, what's the difference between satisfactory and incomplete? And the question we have to ask ourselves as leaders is, how consistently can two different teachers make these decisions? And if we have some doubts about it, then we have to come back to the criteria that we're using for assigning those grades and asking how clear are those criteria really going to be. That brings me to this next point that how consistent can two teachers make these decisions? That has to guide us as we move ahead in this process. So the third point, the one that I wanted to share with you, that makes a profound difference in the kind of things we do with grading, is that moving toward multiple grades describe multiple outcomes. This has kind of been the premise of an outcome based or standards based approach where we're breaking these down instead of a single grade for language arts, you actually get grades for different aspects of language arts. They might get a separate grade for reading, for writing, for listening, for speaking, for language skills, and we articulate the learning outcomes of one's students to meet within each of those. But to take that even to more broader terms, we have within learning three different types of criteria that teachers typically consider. We divided into these categories and we label them product, process, and progress. Product criteria relate to the sort of achievement or academic learning goals. It's what one's students learn and be able to do. Process goals are the behaviors in able learning. And so for example, class participation is a process goal. We want students to participate in class, but it doesn't really look like they're learning. Homework would fall under process. And then we have progress. And progress indicates not necessarily where they are, but how far they come. Now the important part of this is all three are important, but they are quite distinct. And where we get into trouble as educators is when we start combining elements from both of those. Now the problem has been pretty clear in saying that the grades we give kids should relate to that achievement overall, but oftentimes teachers find these process things important. Excuse me, here in the States, we know that there are actually school districts that have said they're going to grade in terms of participation. And we know how difficult that can be. When we think about these sort of product elements, we find that there are three different kinds that educators typically consider. And we divided into the categories of sort of learning enablers, the way of social-emotional learning aspects, and the way of compliance. Here is kind of a list of the elements we're talking about. Now, educators are increasingly being called upon to broaden the scope of the academic goals they're looking for, and including what we call the C's, the communication, the creativity, critical thinking, aspects like that. They clearly fall into the academic area. We have a whole list of learning enablers. And these are important. They're things that we want students to be able to engage in, but they don't necessarily lift learning per se. Then we have this wide range of social and emotional learning skills that are important for students to learn and acquire in school. And we're coming to realize that we really need to bring those to the tension of parents and families and helping students develop as well. You might have heard that a very large study was conducted here in the United States looking at the CEOs of what they call the Fortune 500 companies. These are the top financially successful companies within the US. And they went to their CEOs, their own multi-millionaires, highly successful people, and asked them what their college GPA was. What would you guess to be the college GPA, grade point average, of these multi-millionaires, their CEOs of international, multinational companies? It came out to be 2.9, a less than B average, which really says that there's something other than their academic prowess that's contributing to their success. And many of those things come down that social, emotional learning aspect. Now, the advantage you have is that Canadian schools are far ahead of schools here in the US is actually documenting those. And all they do is they just pull those out of the sort of the academic rate and report those separately. You know, for example, that in the province of Ontario, they've been doing this for about 15 years. And other school districts, even some there in Alberta have been doing this as well. I'd like to show you an example of report card that I kind of stole from a school district there in Canada that had been using some of these ideas. I've adapted it to suit my purposes, but just to give you an idea of what it looks like. And so here's an example of what the report card looked like. They include the teacher's photographs and report card. This is the high school report card. And then they have the achievement grade in this place that's a letter grade. And, you know, they there put all the evidence that the teachers used to do that, but then they have a separate grade for participation, homework, punctuality and effort. They have a narrative part of the report card that has two parts. There's a class part and then a student part. In the class part, the teacher records it only one time, but every student in the class gets those senses. So in this case, for example, you have this first period, and I've also included, as you can see, my dream team of teachers. But here we have the first period language arts class top of Miss Angelou. And you can see Miss Angelou has gone in, she said, this quarter we focus on poetry and different poetic forms. Students read both well, none less than poets and constructed their own poems. Now remember, she recorded that only one time and every student in the class got those senses. But then she adds, Chris actually participates in class discussions with several excellent poems, but needs to be more conscious about completing homework assignments on time. Now I look down over this, and this kid's clearly got a problem with homework. Teachers are consistently noting that. I mean, that's really good information for me to have if I'm that student's parent or guardian. But all that's pulled out. Now, in this particular school district, nobody told the teachers how to come up to the achievement grade. All they did was say, pull these things out, report them separately. Now, there's nothing says that these are the most important, sort of non-academic and non-cognitive skills you need to pull out either. There are, as we saw before, a whole host of things you might want to consider. But you can't do it all. So one of the critical tasks that leaders need to lead their teachers in doing is going through this list and picking out the four or five things they consider to be most important. There are some basic tasks you need to consider if you're going to report on these sort of non-cognitive competencies. The first is you have to limit the number no more than about four or five. Remember, you're asking teachers to keep track of these things, and they can keep track of 10 things. So you're going to have established some priorities of what they consider most important for students. Second, whatever you pick, you have to develop a clear rubric for it. If you're going to give kids a grade of a one, two, three, or four, like that report card I showed you did, then those have to be clearly specified. You see that in the report card that I showed you, the non-cognitive things are based on a four level rubric, four, three, two, and one. So the homework rubric is pretty simple. A four, all homework assignments turned in on time. Three, one or two assignments missing. A two, three to five assignments missing. A one, multiple assignments missing. So they pull that out of the achievement grade, recognizing it is important, but reporting it separately. Next thing is that you must be sure that we ensure developmental consistency in these things. That what cooperation looks like if you're in a kindergarten class is different from what it looks like if you're in a grade five class or a grade eight class or a grade 12 class. And so we need to be clear in the rubrics we develop, there's developmental consistency. And finally, we need to make sure that they are aligned with our purpose, that we really have these things aligned with that purpose statement. So if you remember, back to the purpose statement I showed you earlier, they've indicated it specifically on this report card what it is they want to accomplish. Where they say students receive separate feedback and evaluation on their learning habits. We've called these learning habits which will not be included as part of the academic achievement grade. You're pulling out. I suppose that I'm particularly sensitive to this issue because of a problem we had with my son. I have a son who's very bright and I like to hope he got that from me. But he was also very obstinate and he definitely got that from his mother. In high school, we fought this constant battle because he would say to me, he would say, you're an educator, right? I see that's right. He said, okay, homework. Homework is designed to provide you with extended practice, things you get in class, right? I'd say, yes, that's right. He said, okay, if I get it in class I can show the teacher I've got it. Why do I have to do the stupid homework? He said, if I go to class and all the teacher really does is goes over what's in the book and I can read the book and get it, why should I have to go to class? Here's a kid, he would not do the homework. He'd skip class, he wouldn't leave school. They just find him in the tech lab doing something he thought was more important. So in high school, he takes what is called advanced placement calculus. It is a calculus class for high school students that is designed to be more aligned with the kind of calculus they would get when they were in college. So advanced placement basically means college level. The first year, he took the exam for the A, B level of calculus. We're the first level college level class. You get a five, which is the highest grade you can get. The second year, he took the BC level, which is the highest level of calculus. Not only did he get a five, he was one of 38 kids in the United States who had a perfect score because he's in calculus because his teacher zinged him because he wouldn't do the homework. And we just fought this battle constantly. And I would say to him, Michael, you've got to understand, it's a game. He said, dad, that's my point. It's their game not mine and I don't wanna play. This kid, he graduates from high school with national merit scholarships. He was not in the top 25% of his graduating class because he refused to play the game. Now, you could convince me that my kid acted in irresponsible way. I will go along with you on that, but you can convince me that it deserves a C in calculus, not when he gets the highest grade possible on this external examination. That's not right. And so that's what we do. When we provide multiple grades, then we pull these things out and we put them separately. They're all important, but they all report different kinds of indicators. You'd be like going to the physician and the physician says, well, I'm going to measure your height. I'm gonna measure your weight. I'm going to measure your heart rate. I'm going to measure this. I'm gonna bind those all into a single grade that's gonna calculate your health. Well, that doesn't work. These are different measures and we have to do the same with kids. So pulling these things out and reporting them separately can provide valuable information about kids and valuable information about their learning that would then make the grading much more honest. So three basic issues here with regard to what's really important. The first of all is you do wanna establish the why before considering how. One of the first texts I hope will be initiated in this time we have between now and the end of the school year now that our grading practices are pretty much outlined for us is for us to be clear about why it is we're doing this. And to really establish a sort of purpose statement that we might wanna use for our school that we could then make clear to everybody about what it's about. Second, we wanna really begin articulating by defining that top level of performance and can we reach some consensus of what that top level really looks like. And if we can, then it's gonna make it easier to make the graduated levels below it a lot easier to identify. Among those questions, we're going to have to consider how many graduated levels can we reliably indicate? And even at my level, we've altered that just in recent years. At my university, we had a letter grade system, A, B, C, D and F. And that had been in place for, gosh, nearly a century. But in recent discussions, we decided that for college level work, grade of D just isn't really satisfactory anymore. D has always been the gentleman's F anyway, so we decided to get rid of it. And so we've gone to just a four level system, A, B, C and F, that you need to perform at least at the C level to get by. And you'll have to make those same decisions. And finally, to really look at reporting multiple outcomes, multiple grades to describe these different outcomes that we have in school. If we can accomplish this, then we can really make grading a lot more meaningful, a lot more accurate, a lot more honest, and provides grades that are a lot more equitable as well. Now, I'm sure in many ways, this is kind of a mind blowing experience and it might be a little bit like this. But what we'd like to do is take a little bit of time to go to our chat groups and see what questions or concerns you might have. Are there issues that this may be prompted, that you want to explore in greater depth? Do you want to know about studies that might have been conducted into these areas and what might be able to do that can form our decisions as we go forward? So we're going to take a portion of time to go to the chat groups. Christine, why don't we set these chat groups up for about five minutes at least? And then we'll have folks come back. If you could record any questions you have in the chat section, then we'll try to monitor those and we'll come back after that time and try to address as many of those that we have time to address. Thank you. Okay, your chat rooms are open. Good timing, Tom. So how was it, did this make sense, do you think? Folks? Yeah, I think so, because like what you were referring to, interestingly refers to the Alberta education competencies. Yes. But on our government websites, it explicitly states we aren't supposed to like report or grade the competencies, they're just meant to be embedded in the instructional practice. So why not report them? Well, I think we're not going to report on them but we can assign like a grade. Oh yeah, report them separately. Yeah, that's why we decided to go and the schools where I've been working going with the rubric, the letters, the numbers, but still report that. Like, I think parents should know if their kids aren't doing homework, but it doesn't mean you can't homework as part of the grade. Right, and I think some teachers might wonder how could I gauge the competencies through an online platform. So that piece is a little bit more difficult and challenging as well. Yeah, and I think that as I looked over the documents you sent, I think they're really thoughtful. I think they did have those equity issues forefront in making those decisions overall. That's right, but it's allowed for a lot of divergence of interpretation as well. That's right. Which is interesting because the communication is oftentimes like, I've received an email last time where this person was under the impression that what their district had decided was provincial and it wasn't, and it was only specific to their districts. So even just that understanding that different districts have adopted different assessments. The challenge of percentage grade, I think it's just a real dilemma. I mean, I understand that we don't want students to get below what they had when the school was closed, but how much are you going to bump it up? And what is that difference between the 89 and the 90? And can we really make that clear? That's a real challenge for folks. So interesting to see how they do that. Yeah, absolutely. So Thomas, I did collect some of the questions from the survey yesterday that all people are typing initially. I will start with those questions and then as the chat box fills up with the immediate questions and we'll start transitioning over to those. Okay. Okay. So what were some of those questions, Susan? One is, how do we ensure that what we learn about supporting students from home through the use of sound assessment practice does not get lost when students resume in whatever form in September? Yeah. Another one is, what would be appropriate comments for students who have not been given, who have not been given evidence of work and for assessment of students who are doing well with parents support. So I see that one as being really relevant because a lot of districts right now are trying to figure out the wording on the report card for comments. And especially in like division one where there is a lot of parental support. How do you, because I was thinking about this too when I saw that question, I guess it would have to do with like indicating that this child is able to do this with parental support. Kind of like similar to that statement at this point in time, right? Right. Is noting that. So another one with a mandate to assign final grades, how do we support teachers in taking the emphasis off grading and put it on feedback and assessment? And how do we encourage students to stay in the game when main June are already a struggle in the traditional classroom setting? And then I guess having that disconnect with not necessarily seeing them synchronously all the time online, how do we, how are we able to maintain that engagement? Okay. I've got a few more, but I just thought we'd probably start with those. Yeah, that's really good. Christine, how much time do we have left? Should I give them a minute now? Are you getting that question in the chat? There is a question from Pat Laura in the chat. She asked, what does the body of evidence look like for effort or SEL skills? I'm assuming the SEL is social emotional learning. Yeah. Okay. Christine, you'll let us know when everyone is back. Yeah. Okay. Oh, 30 seconds. I wanted to let you know we are, we are about to get some snow here in Kentucky again. We're forecast to get snow tomorrow night. Oh no. Yeah, it's such a shame because all the flowers are coming up and the trees are budding and there's a fear that it's going to really ruin the fruit crop in particular apples. Yes. What a crazy spring. Everyone's popped back in the room now. Okay, thanks. Okay. Well, welcome back. And this now starts the Q and A portion of our webinar today. So a while groups are populating your questions in the chat box. I will begin with some of the questions that emerged from the survey that was sent out to participants yesterday. So the first question I'm going to ask is how do we ensure that what we learn about supporting students from home through the use of sound assessment practice does not get lost when schools resume in whatever form in September? That's such a wonderful question, Susan. Thanks for the folks that submitted that. I do think that that's the one real positive thing that has come from all the dilemmas we struggle with during this very difficult time. And that is that it's made us much more sensitive to these issues of equity, recognizing that what a powerful influence the home environment can have on kids and what we can do really to help kids knowing that those differences exist. And I don't think that's going to be lost. I think that one of the things that this has shown is that we do have this influence on kids but they bring a lot with them to school and we need to be sensitive to those as we plan our lessons and as we engage students in learning activities. I think it will be a leadership challenge to make sure that these lessons stay forefront for us. And, but I think it's also made teachers much more sensitive to recognizing that for many kids, school is the highlight of their life. You know, it's a time when they can be removed from those other responsibilities at home, when they can come into environment where they can be successful. I think that it's also helped us recognize the powerful influence of that success that we want to as best as possible provide successful learning experiences for our students so that they feel competent and competent in learning situations. As I talk with teachers in different sessions that are engaged in these kind of processes, I'm truly struck by the increased sensitivity they have developed at this time. And I think that it's incumbent upon us as leaders to ensure that those things remain forefront in our minds as we go forward. I don't think that it's going to be easily forgotten. And I'm really confident that teachers will on their own, but I think as leaders, we need to keep pressing to make sure that that stays sort of on the forefront and really remain mindful of those things as we go forward. Okay, my next question also comes from the survey. Many of our school districts are now preparing to consider what type of report card comments to make. And so in this pandemic time, what would be appropriate comments for students who have not given evidence of work? And another one would also be what would be appropriate comments for students who are doing well, but with parental support. Right. So I'm assuming this person would be typically talking about division one in which the parents are working with their children quite closely. And so they're not sure how to report the progress because they know that the students are guided very much by their parents. Well, I can tell you that there's not been any research done on what kind of comments work best during a pandemic. It's kind of a new area for us. But I can share with you a little bit that might be helpful about what we know about grades versus comments and what seems to be positive with regard to comments. The research been done near comments indicate that the comments we offer students should do really four things. Number one, it's really important that the comments always start with something positive. That you want to begin any list of comments you make to students by saying something good. Here's what you've done well. Here's the things which succeed. And that'd be sort of your outcome-based kind of comments. Here are the things I've noticed that you're working on and you've done really excellent work in this area. Number two, then you can turn to things where they need to improve. Here are some areas that haven't been coming through and here are some areas where we need to put a little bit more time and effort. Number three, offer directions and now those improvements can be made. So when I say to them, you need to do better in this way, I need to add to that, here are some ways you could do it. Here's some things you could do to make this better and give students actually that guidance and direction. Oftentimes parents are not familiar enough with the area to provide that guidance and direction. So this third aspect becomes really critical. What guidance and direction could you offer students to make the improvements in the area? And then finally, number four, to express our confidence that they're going to be able to do it. That we believe that they can really learn these things well, that there's nothing beyond their capabilities and we're sure they're going to be able to make really adequate and important progress. So four things to keep in mind with regard to progress, comments. Number one, start with something positive. Number two, then identify the shortcomings or the areas where there's concern, areas where they need to make improvement. Number three, guidance and direction and making those improvements. And number four, confidence, those improvements can be made and they will meet with success. I think if we let those four things sort of guide us in making these comments to students, they'll be really productive and really helpful for the kids. Okay, we've got two questions from our chat box that are very similar coming from Dan and Darren. Dan says, what can we do to build greater consistency with colleagues across the province on the why? And Darren is asking in the event of further ongoing discussions to schools, how do we get better at helping teachers gather evidence of learning for assessment and reporting it to the online context? Yeah, well, I think that what we've found as we've gathered these purpose statements from schools is that you need to really begin on a school basis. Remember in the presentation I said, when we survey parents and students about grades, the first issue that always comes up for them is fairness. And it's that, you know, they wanna make sure that it's a fair estimate of it. We did a study several years ago where we actually asked teachers to think about their very best and their very worst experience with regard to grades. And these are all teachers who had gone through school and gone through college had their degrees. And every one of them was able to identify a good and a really bad experience with regard to grades. And when we ask about those bad experiences, over 90% of those all related to issues of fairness, it just wasn't fair. You know, the way the teacher determined the grade, the grade I got, the teacher didn't like me. He doesn't reflect what I learned at all in the class. And so that has to be paramount to us in what we really think about. Are the grades, the grading system really established fair? Are they transparent? Do students know exactly what we're looking for and how they can provide that? The second issue is the consistency. And consistency has to do with that when a child leaves one class and goes into another, in many schools, all the rules change. What counts as the part of the grade, what, how the grade's gonna be determined, all that changes. Now for some students, they see this as a huge game and they become strategists in the game. When they understand it well and how to play and some of them are very good. But for a lot of kids, it's a total mystery. I can remember in my very first year of teaching, I began as a middle school teacher. I taught grades seven and eight. And in the middle school where I taught, it was our practice to give final examinations to our eighth graders. We thought that was a good idea. We thought it was going to get them ready for what they would base when they were in high school. I remember the very first year doing this because I'd worked really hard in developing the exam and I thought I did a pretty good job. And as my students were coming in to take their exam was standing outside the classroom door, kind of greeting them as we were coming through. And down the hall came one of my very top students, Jessica. She had done so well the entire semester. And I was just confident she was gonna do really well on the final exam. And she came in and said, hi, Jess, how are you today? She said, fine, Mr. Gusky, how are you? I said, fine. I said, Jess, did you study for your exam? And she looked at me and said, no, not really. I was shocked. I said, Jess, I don't understand. This is a really important part of your grade. How could you not study for your exam? And she looked at me really quizzically and she said, well, Mr. Gusky, I worked it out. I only need a 50.2 for my A. I don't have to study it at 50.2. This eighth grader had worked it out to the tenth decimal place. What she needed to do to get her A in my class and she was surprised that I didn't get it. And I was shocked. And I thought, oh my goodness, what have I done to this brilliant young woman that to her school was not about learning, it was about getting a grade. And she had done her job. She was shocked that I didn't understand that that's what her job should be. So since that, it's really, really compelled me to think about this differently and to really try to understand that we need to be consistent. And the way to bring about the consistency is getting the agreement on a purpose statement. Then as we look at all these different practices and policies, we can ask, do they relate to our purpose statement? If we come up with a why first, why we're doing it, then the how it becomes a lot easier. But if you do that by school, then that's a workable group. It's a group that then has some ownership in it. And a group that is going to feel comfortable communicating that overall. You will go through that process of disagree and commit because not everybody will agree with you when you first start. But to come to some consensus about what we believe the purpose should be. Now it's also likely you're going to find that that purpose may differ depending on the grade level. So sometimes the purpose is a little bit different at elementary versus middle school versus high school. We find, for example, even the audience of the grades or the report card does differ as you move up a grade level. At the lowest elementary grades, the primary audience for any grade report is always the parents. But we're letting the kids know pretty much how well they're doing in class, but we're using this to communicate with parents. We move up to middle school and high school, then the students become an additional sort of audience or a stakeholder in the report card. So oftentimes it's to communicate information to parents and to students. And you get to my level. At my level, it's only the students. We often have parents call us and plead desperately for students' grades. And we have to say, we can't get them to you. After 18, then that's their information. And so I know you're investing thousands of dollars in your child's education, but the grades pretty much you're going to go to them. So I think that to see that consensus among the faculty in your building is the foundation for then providing consistency as they go from class to class. Okay, thanks, Thomas. Again, if you have a question, please type it into the chat box and we'll do our best to get to your question represented. The next question is with the students, I'm sorry, with the mandate to final grades, how do we support teachers in taking the emphasis off grading and to put it on feedback and assessment? Well, this was the issue we spent a lot of time discussing in our first session, that when you think about why we're gathering this information, it falls into these two categories. One category is the grading and accountability and the other category is the feedback and learning. And I think that what this current situation has done when the students are learning from home, it's really reinforced the notion that learning and feedback function has to be the most important. I also think that's why the provincial government wisely has given a lot of latitude in determining these grades, recognizing that it's tough to gather information, that we don't wanna use it to hurt students in any way at all, that we're going to establish sort of where they were when we closed schools and what March 15th, I think was the date. And then we're going to use what evidence we gathered to allow them to bring their grade up to demonstrate their learning. And we're going to provide feedback as they provide us with this information to help them improve. So wherever they started in March 15th, our orientation is not going down, but what we can help them do to bring that grade even up higher than what it was. Now we need to establish and be very transparent about how to do that. And then look for evidence that we would trust and verify that it's been done. My checkpoint in that is if two teachers are looking at the same body of evidence, would they make the same decision with regard to this kid? And if they can say, yes, I'm competent, that if two of my colleagues were sitting beside me, would they make the same decision with regard to this student that I'm making? Then you're on solid ground. So to really emphasize the feedback and learning functions and the grade just basically being a reflection of what you were able to accomplish through that and not a final checkpoint either. I think one of the other really important parts that's come out of what we're done with grading in this is that whatever grade we assign to students, we're going to special note that to indicate that it was done under these sort of difficult conditions, but it's also temporary. That no matter what timeline we set for this, and there are going to be timelines set, but there are always going to be opportunities for students to make that even better to improve it. So I think that that flexibility, that temporary nature of the grades allows us to emphasize the feedback aspects of it even more. Okay, thank you. Another question from our survey. Teachers are feeling overwhelmed at not knowing how to choose enduring concepts, ideas and skills, which are foundational and leveraged. Are there any strategies to help them pick out those essential learning outcomes? Yeah. There are several, I think important works that are being done in this area. The process is generally referred to as valuing, that as an expert in these subject areas, a teacher has to look at this and say, of all the things, all these outcomes that are here, what is the most important? What do I value the most? If my students were only going to get a few of these things, what would I consider the most critical, the most vital to their success? So you think about what's going to be necessary for success in coming levels? What skills are going to be built upon and extended as they move on? What skills are going to be foundational for what's necessary for learning at the very next level? Have those be your basis? They're professional judgments that you need to make, but you're in a really good position to do that as a teacher. So to be thinking about all the outcomes you have there, what are the ones that you consider to be most vital, most important for your students together? In most cases, we're finding that the ones that teachers choose to be most important tend to fall at the higher levels of sort of cognitive complexity. If you think of it in terms of like Bloom's taxonomy, where we started with the lower levels of just knowing and being able to recall and remember stuff, up through comprehension to application and transfer to analysis of the synthesis evaluation, the things that are most difficult to learn but are retained longer and most vital to learning tend to be those that fall at those higher cognitive levels. So it's not necessary that maybe they recall these things as well as it is to understand how to apply principles and the application to transfer that to new and different situations. So I think those three things I keep in mind as I try to choose, that number one, to think about what's going to be necessary for their success at upper levels. Number two, to think about what would be most vital for them in that subject area overall. And then number three, to focus on the things that are at the higher cognitive levels, that those are likely to be a little bit more difficult to learn and difficult to teach, but the ones that will retain for a longer period of time. Benjamin Bloom always used the example that if he offered a very complicated word problem doing with subtraction, sometime everybody would probably be able to get it right. But if you ask in a subtraction problem, which term is subterranean, which is a menu end, we might not know that. And so it's knowing those things were not necessary for you to be able to apply subtraction to problem solving situation. And that's the same kind of decision-making process you'll have to go through here. To think about what's most important for your students to learn, what will they carry over, what are they gonna build upon at upper grade levels that it really be vital for their learning? All right, and it sounds like if teachers are going to be focused on evaluating essential learning outcomes, they're really gonna have to be intentional with the instruction and scaffolding within their online lessons as well in order to achieve that. That's very true, Susan, nice point. Okay, all right. Another one would be what sort of advice do you have for moving teachers away from traditional models of evaluation so that they are able to be more creative in terms of how they gather observations and have conversations with students in addition to products? Okay. I like to do this by turning the question around to the teachers. And so when we have any particular outcome, instead of asking how would I measure it or what kind of question would I ask, what I ask teachers to think about it, what evidence would you accept to verify that a student learned this well? And are there different kinds of evidence that we might accept from students? So would it be acceptable for a student to be able to do a demonstration instead of having to write it out? Might they do a video chat? Might they be able to develop a play or build a project that might be able to demonstrate these things? If you start with your overall learning goal and that general question of what evidence would I accept to verify that a student had really learned this well? Again, going back to what we talked about earlier, start with your model of excellence. And really get away from this idea that there's only one way to demonstrate your learning in that excellent way, but that once we establish what that level is, then we can think about what evidence best reflects that level. And we can move ahead to the various formats that kind of evidence might be used and how it can be gathered. I'm muted, Susan. Sorry about that. Christine, I've just lost my ability to see the chat. Was there any other questions that came up in the chat? No. Okay, so I guess we will go to one more question from the survey. Oh, let's see here. Sorry, Susan, actually, there was one that I don't think we covered yet from Pat Lohr. The one about what does the body of evidence look like for effort or the social and emotional learning skills? Okay, that's a really important point. And I'll share with you a story on this. A few summers ago, at my university, we do summer institutes each year, or have until this year at least. And we typically do one on assessment. We do one on grading. We brought together a group of schools two summers ago that were looking to revise the reporting. And we talked about those three different kinds of learning criteria, the product, the process, and the progress. They were moving in that direction. They were going to identify the process elements they wanted to report separately. And one of those they chose was effort. And we said, fine, develop a rubric for it. And these were a group of educators who, elementary educators, who for about seven years have been giving an effort-grading report card. And they, as dedicated as they were, spent a day and a half and could not reach consensus on a rubric for effort. They found it difficult to distinguish at the elementary level the difference between the student who was sincerely trying versus the one who was just good at faking it. They also found it difficult to distinguish what was the challenge effort from what was the parent's effort. We knew that's always come into play, and in to play even more now that kids are learning from home. But I can remember, I tell the story of when I was in fifth grade, we had to do a science project. And my science project wasn't very good. I knew it wasn't very good. So when I turned it into my teacher, I said, here's my science project. I know it's not very good, but you have to understand. I told my mom to start early. She left to go into her last minute. By that time, all the good volcano kids were gone. So the kids know this. And so what they did is they dropped effort as one of the process skills they were going to report. They moved away from it. Now, there's some really important work that's being done at present time when we look at these social and emotional learning skills. There's a group together at the University of Chicago that's been established as center in looking into these and what most contributes to kids' academic success. There's another group that's been put together at Harvard University where they are looking into these things too, where they're struggling because there's some debate as to what should be considered truly success. A lot of the studies that have been done have looked at these particular skills, then the financial success of adults knowing in their life. And other people are pushing back from that and saying, well, is financial success the only criteria? Maybe we should looking at things like honor and integrity and empathy, things that might not contribute to financial wherewithal, but are truly important if we want to benefit our society and move ahead as a society. So it's prompted a whole new set of new questions about what is truly success in life and what leads to a really rewarding life, to happiness, to fulfillment for it overall. And can we relate any of those to people who have been able to reach those levels of success, not only just financially, but also by happiness and feeling a sense of being worthwhile, making a valuable contribution to the road overall. So it's an area that's right for research. We haven't really gotten there yet. We've been better at identifying these potential skills. Well, this that I showed you, for example, mostly came from those two organizations, the University of Chicago and Harvard University. There's another center that's established in California that's doing good work in this area too, but they're just beginning. So we kind of know what some of the schools are. Many of you might have followed Carol Drex's work with regard to a growth mindset. We've found that a category as well, but as far as establishing which ones are most important, which are most critical, we're not quite at the yet. We're just kind of investigating it, but looking in the right direction. Clearly though, it's important to recognize these things are different from just academic success. And so to the degree that we can identify what they are, establish some criteria for recognizing them when we see them, and then we can develop strategies for helping kids develop them. Okay, great. Thanks, Tom. All right, so I'm sure that after watching the seminar, you'll still have some further questions, but I just wanted to let you be aware that we have some more upcoming assessment webinars. We have Rick Ramelli tomorrow, who is talking about differentiated instruction, principles, and practicalities that hits upon the how do we differentiate for students? And then as well on May 13th, the piece about descriptive feedback techniques. So how do we provide students with that formative feedback that is what is ultimately going to propel them forward and move them along in their progress? And this Friday from ADLC, we have various ADLC presenters who are going to provide some tips and strategies for how to maintain academic integrity. So they're going to share their experience with being online educators full-time. Some of the tricks of the trade in terms of guaranteeing that the assessments that are coming in are indeed authentic. And Christine, if you could advance to the last slide. Again, we thank Tom Gusky so much for helping us, moving us forward with this conversation. All the participants will be receiving a recorded version of this webinar. So please feel free to share that video recording with others. And on behalf of the Alberta Regional Consortia, consisting of the Calgary Regional Consortium, the Central Alberta Regional Consortium, the Francophone Consortium, the Edmonton Regional Learning Consortium, the Learning Network, the Northern Regional Learning Consortium, and also the Southern Alberta Consortium. We thank you very much and thank you again to Christine Kuang for being our tech facilitator. And we welcome any questions. And you can direct those to Thomas directly at Gusky at UKY.edu. Thank you for joining us today. And we hope you enjoy the rest of your day. Thank you. Thanks, Christine. Thanks, Thomas. Guess you gotta go get your snow gear now. Yes, I know. Get the boots back out again. That's right. Well, good. I look forward to hearing from you. Thank you so much. It's been a great opportunity. Great. Thanks so much, Thomas. We'll chat later this week. Thank you. Take care. Bye.