 Welcome to this webinar. I'm Deb Rogge, and I will be your trainer. As you can see, we're going to be visiting about text-dependent analysis, analytic scoring training, and there are some materials you'll need for this training. If you would like to pause the webinar at this time and access the Google URL that's identified on this slide, you will need hard copies of the TDA rubric and of the cover sheet. Here are a couple of pictures that help you to identify what you need from this particular Google page. When you've done that, go ahead and click Resume, and we'll continue with the training. The purpose of this training is to give you a brief yet thorough training in the skill of scoring text dependent analysis student responses. The benefits of participating in this training is that students' scores, you'll be able to identify their current level of their proficiency in writing text-dependent analysis pieces. It's going to help guide your teacher instruction and also their learning of the students, and it's going to allow and encourage the use of TDA as an instructional strategy within your teaching. I bet you'll identify, get to the purpose and why of what we must do. He uses the three question words, what, how, and why, in order to best describe what's happening. He says that many times we know what we need to do and we know how to do it, but we never understand truly why we must do something. So the what of what we're going to do is that we're going to score student writing samples with accuracy and confidence. And how we're going to do that is that, first of all, you're going to participate in the scoring training and then you're going to apply that training to an actual scoring session, whether it be a large group or just within your own section of what you teach. And then why do you do that? Well, you want to make sure that all persons that are scoring have a common understanding and an application of the rubric because we want to make sure that the student scores are accurate and that the student scoring is a best reflection of what the student actually knows and is able to do. This is the Nebraska text dependent scoring rubric. When we're going to the TDA rubric, let's talk real quickly about what each of the, how the rubric reads. The scoring points are from lowest on the left at a one to highest of four on the right. And underneath each scoring point, there's a brief definition. Down the left side of the rubric are the three domains by which the student piece is analyzed and scored. With each domain moving from the lowest or the most weak performance to the strongest performance in rigor are the indicators of what students typically, you will see these types of performances within the piece of writing that the student has. The rubric weighting is that each one of the three domains are weighted equally. None is more important than the other. So each one contributes 33 and a third of the student's total score. There are some tools for your work. We just went over briefly the text dependent analysis scoring rubric. This is your main tool that you use in order to score student writing. And then you're going to use a pencil and coded scoring keys. Each one of you will use a pencil within your scoring. There are three to four of you in a scoring team or a scoring group. And each one of you will have a pencil. We do that so that there isn't any biasing that one person is using a different color or a different texture of writing your pencil. And then we also use coded scoring cards where rather than writing the numeral one, two, three, or four to indicate the performance, you use a letter code in order to disguise the performance of the student so that another score is not biased by how you judge the student's proficiency. This is a student cover page and every piece of student writing has this as a cover on it. We're gonna take a quick tour of this. There is either a sticker or a number or pre-populated in some way. There's a unique student and building ID that's there. Each table group, you have a little table tent with a number on it and you have to indicate whether you're table one through 15 or if you're table 10 or whatever you are. But you write that right in that particular area. Then we ask you, if you're the first person to pick this particular paper up to score it, please circle the color of the coded scoring card that you have. Rater two, you'll do that also as you notice on out to the right. These are the spaces where you will write your letter codes for each of the three domains, both for Rater one and Rater two. This just gives you a little bit more clear understanding. There's where you're going to be writing, only in the white spaces, not in the shaded spaces. This is overall the information that you as a score need to make sure is correct on the student cover page. Those that decode scores will be using the shaded areas and out to the far right, you'll know the composite score, the total number of points that a student received for each of the domains, what their total composite score and how that translates into the scale score. Also tools that you have are the table maps and they're unscored, scored once and scored twice. They're pretty self-explanatory, unscored means that these particular student papers have not been scored by any Rater, scored once means that one Rater has scored that particular paper and scored twice means that two Raters have scored that student sample. These are the cut scores for the performance levels. If a student's scale score is between zero and 39, they are declared below between 40 to 54, they're declared meets and if they're 55 to 70, then they are exceeds. There's a scoring process that we use that I'd like to describe to you right now. Each paper is scored by two Raters and those two Raters want to have an agreement. They want to have their scores either match, that means that each Rater gave the same level of performance to that student or they're adjacent, which means that the scoring performances are right next to each other. In other words, if I were using number scores, a match would be both scores gave the student a three and adjacent would mean that one score gave it a two and one score gave it a three. They're right next door to each other. If we have a situation that's called a negotiation, it's required for each domain if the scores are not in agreement. We want to make sure that we have matches or adjacent scores. Let's talk a little bit about the structure of the rubric. Notice that I have drawn a blue line between a scoring level of two and three. We affectionately call that the river. In other words, what happens here is that the river indicates that a three and a four are a stronger performance and a one or two are a weaker performance on our particular rubric. Please keep in mind that in the state of Nebraska, the state board a few years ago indicated that a level three or above is what a student in Nebraska needs to have in order to meet the standard. As you see within here, when we look on the stronger side, three is weaker than a four and two is stronger than a one and so on and so forth. So we have weaker and stronger all the way across in the performances and the identification of the indicators of each of the domains. Talk to you briefly about a match. Remember, both readers give the student sample the same score. Both in this case have given them a one. In this case, both readers and scores have given them a two, a three is both of them. And again, a four with both of them. These are the four scenarios for a match. I talked to you about adjacent scoring. This is an example of an adjacent scoring with a one and a two. This is for a two and a three and this is for a three and a four. No, I also referred to or mentioned about negotiations. This is a situation where the two readers are more than one scoring point apart. Notice that the two comes between the one and the three. And this would be the case where the negotiator, the floor walker would come and get these two people and ask them to negotiate. Now, a one and a three is a scenario that can happen. A two and a four can happen, but believe it or not, actually a one and a four can happen in a third read or a negotiations situation as well. Let's get a brief definition of negotiation. It's a process that invokes a specific protocol when reader ones and reader twos domain score is greater than one scoring point apart in one or more of the domains. Now, this right here is the protocol for negotiations. I want you to pause right here right now in the webinar and I want you to read through this particular slide step by step. Okay, now that you have read through this and had a chance to digest it a little bit, please notice that at number four and number five, after the two readers have given their validation for their scoring and the conversation is held between the two raiders, please understand that in step number five, each raider does have the opportunity to make a decision whether to score the paper at a higher or a lower. But what we're looking for in a negotiation is that we want to try, we want to have raider one and raider two come to an agreement which is either adjacent or matching. Now, if they don't, we will invoke step seven and that's as it just states down there, a third reader reads the paper and we double the third reader scores. Raider one and raider two scores are not even considered. So we know that every paper scored has both strengths and weaknesses. We've seen that when we looked at the rubric and we've talked about the fact that when we go into a negotiation situation or a third read situation, that that's where we really talk about the strengths and the weaknesses of that particular paper. Now, there is a table or a room setting protocol that we're going to use. You can pause it if you want to at this particular point and read through these 11 steps. But please remember that when you begin to do a scoring or you have someone come on site that's going to be facilitating a scoring, they will go through these 11 steps with you in order to get you ready to score so that your scores are both reliable and valid. There are some best practices for scoring that I really want to visit with you about. It's important that you take a paper from either the unscored or the scored pile. You need to read the paper completely from beginning to end. Don't skim it, don't scan it. Read it word for word before you make any decisions about anything. We want you to focus on analysis of text and the use of the evidence after that first read through that paper and determine what your scoring point is and indicate the scoring point on the front of the students using your code on the student cover page either under rate one or rate two. And then we want you to review that paper again and focus on the writing skills this time. Here you'll assign your score for each domain and write that letter as well. Now, remember it's very, very important. Don't write numerals one, two, three or four but enter the letter code that's associated with each of those scoring points for each domain. Once you've assigned your code, place the paper on the mat, either scored once or scored. And then your next task as you go on to the next paper is to take a paper from either the unscored or the scored once pile and repeat the whole process of steps one through five. Step seven is so very important. That's why the red letters are there. It's important to create a paper flow to ensure all papers are scored in a timely manner. You need to read from scored once and unscored in a balance because we want to make sure that all papers are scored and decoded and completed at about the same time so we can quickly get the information back to the teachers and students. Well, you've completed the online scoring training. I wish you well as you begin your scoring. Welcome to this webinar. I'm Deb Rogge and I will be your trainer. As you can see, we're going to be visiting about text-dependent analysis, analytic scoring training and there's some materials you'll need for this training. If you would like to pause the webinar at this time and access the Google URL that's identified on this slide, you will need hard copies of the TDA rubric and of the cover sheet. Here are a couple of pictures that help you to identify what you need from this particular Google page. When you've done that, go ahead and click resume and we'll continue with the training. The purpose of this training is to give you a brief, yet thorough training in the skill of scoring text-dependent analysis student responses. The benefits of participating in this training is that student scores, you'll be able to identify their current level of their proficiency in writing text-dependent analysis pieces. It's gonna help guide your teacher instruction and also their learning of the students and it's gonna allow and encourage the use of TDA as an instructional strategy within your teaching. I bet you're wondering why you need to participate in a scoring training for TDA. Well, it's very unique. TDA is in, I'm gonna be training. I suppose you're wondering why you must participate in scoring training. Simon Sinek of Apple has a very simple graphic that he uses, which he calls the golden circles in order to identify, get to the purpose and why of what we must do. He uses the three question words, what, how, and why in order to best describe what's happening. He says that many times we know what we need to do and we know how to do it, but we never understand truly why we must do something. So the what of what we're going to do is that we're gonna score student writing samples with accuracy and confidence. And how we're gonna do that is that first of all, you're gonna participate in the scoring training and then you're gonna apply that training to actual scoring session, whether it be a large group or just within your own section that what you teach. And then why do you do that? Well, you wanna make sure that all persons that are scoring have a common understanding and an application of the rubric because we wanna make sure that the student scores are accurate and that the student's scoring is a best reflection of what the student actually knows and is able to do. This is the Nebraska text dependent scoring rubric. When we're going to the TDA rubric, let's talk real quickly about what each of the, how the rubric reads. The scoring points are from lowest on the left at a one to highest of four on the right. And underneath each scoring point, there's a brief definition. Down the left side of the rubric are the three domains by which the student piece is analyzed and scored. With each domain moving from the lowest or the most weak performance to the strongest performance in rigor are the indicators of what students typically, you will see these types of performances within the piece of writing that the student has created. That is true for use of evidence as well as writing scales. Excuse me. The rubric weighting is that each one of the three domains are weighted equally. None is more important than the other. So each one contributes 33 and a third of the student's total score. There are some tools for your work. We just went over briefly the text dependent analysis scoring rubric. This is your main tool that you use in order to score student writing. And then you're going to use a pencil and coded scoring keys. Each one of you will use a pencil within your scoring. There are three to four of you in a scoring team or a scoring group. And each one of you will have a pencil. We do that so that there isn't any biasing that one person is using a different color or a different texture of writing your pencil. And then we also use coded scoring cards where rather than writing the numeral one, two, three or four, to indicate the performance to use a letter code in order to disguise the performance of the student so that another score is not biased by how you judge the student's proficiency. This is a student cover page and every piece of student writing has this as a cover on it. We're going to take a quick tour of this. There is either a sticker or a number or pre-populated in some way. There's a unique student and building ID that's there. Each table group, you have a little table tent with a number on it and you have to indicate whether you're table one through 15 or if you're table 10 or whatever you are. But you write that right in that particular area. Then we ask you, if you're the first person to pick this particular paper up to score it, please circle the color of the coded scoring card that you have. Rater two, you'll do that also as you notice on out to the right. These are the spaces where you will write your letter codes for each of the three domains, both for Rater one and Rater two. This just gives you a little bit more clear understanding. There's where you're going to be writing, only in the white spaces, not in the shaded spaces. This is overall the information that you as a score need to make sure is correct on the student cover page. Those that decode scores will be using the shaded areas and out to the far right, you'll know the composite score, the total number of points that a student received for each of the domains, what their total composite score and how that translates into the scale score. Also tools that you have are the table maps and they're unscored, scored once and scored twice. They're pretty self-explanatory, unscored means that these particular student papers have not been scored by any Rater, scored once means that one Rater has scored that particular paper and scored twice, means that two Raters have scored that student sample. These are the cut scores for the performance levels. If a student's scale score is between zero and 39, they are declared below between 40 to 54, they're declared meets and if they're 55 to 70, then they are exceeds. There's a scoring process that we use that I'd like to describe to you right now. Each paper is scored by two Raters and those two Raters want to have an agreement. They want to have their scores either match, that means that each reader gave the same level of performance to that student or they're adjacent, which means that the scoring performances are right next to each other. In other words, if I were using number scores, a match would be both scores gave the student a three and adjacent would mean that one score gave it a two and one score gave it a three. They're right next door to each other. If we have a situation that's called a negotiation, it's required for each domain if the scores are not in agreement. We want to make sure that we have matches or adjacent scores. Let's talk a little bit about the structure of the rubric. Notice that I have drawn a blue line between a scoring level of two and three. We affectionately call that the river. In other words, what happens here is that the river indicates that a three and a four are a stronger performance and a one or two are a weaker performance on our particular rubric. Please keep in mind that in the state of Nebraska, the state board a few years ago indicated that a level three or above is what a student in Nebraska needs to have in order to meet the standard. As you see within here, when we look on the stronger side, three is weaker than a four and two is stronger than a one and so on and so forth. So we have weaker and stronger all the way across in the performances and the identification of the indicators of each of the domains. Talk to you briefly about a match. Remember both readers give the student sample the same score, both in this case have given them a one. In this case, both readers and scores have given them a two, a three is both of them. And again, a four with both of them. These are the four scenarios for a match. I talked to you about adjacent scoring. This is an example of an adjacent scoring with a one and a two. This is for a two and a three and this is for a three and a four. Now I also refer to or mentioned about negotiations. This is a situation where the two readers are more than one scoring point apart. Notice that the two comes between the one and the three. And this would be the case where the negotiate, the floor walker would come and get these two people and ask them to negotiate. Now a one and a three is a scenario that can happen. A two and a four can happen, but believe it or not, actually a one and a four can happen in a third read or a negotiations situation as well. Let's get a brief definition of negotiation. It's a process that invokes a specific protocol when reader ones and reader twos domain score is greater than one scoring point apart in one or more of the domains. Now this right here is the protocol for negotiations. I want you to pause right here right now in the webinar and I want you to read through this particular slide. Step by step. Okay, now that you have read through this and had a chance to digest it a little bit, please notice that at number four and number five, after the two readers have given their validation for their scoring and the conversation is held between the two raiders, please understand that in step number five, each raider does have the opportunity to make a decision whether to score the paper at a higher or a lower. But what we're looking for in a negotiation is that we want to try, we want to have raider one and raider two come to an agreement which is either adjacent or matching. Now if they don't, we will invoke step seven and that's as it just states down there, a third reader reads the paper and we double the third reader scores. Raider one and raider two scores are not even considered. So we know that every paper scored has both strengths and weaknesses. We've seen that when we looked at the rubric and we've talked about the fact that when we go into a negotiation situation or a third read situation that that's where we really talk about the strengths and the weaknesses of that particular paper. Now, there is a table or a room setting protocol that we're going to use. You can pause it if you want to with this particular point and read through these 11 steps. But please remember that when you begin to do a scoring or you have someone come on site that's going to be facilitating a scoring, they will go through these 11 steps with you in order to get you ready to score so that your scores are both reliable and valid. There are some best practices for scoring that I really want to visit it with you about. It's important that you take a paper from either the unscored or the scored pile. You need to read the paper completely from beginning to end. Don't scan it, don't scan it. Read it word for word before you make any decisions about anything. We want you to focus on analysis of text and the use of the evidence after that first read through that paper and determine what your scoring point is and indicate the scoring point on the front of the students using your code on the student cover page, either under rate one or rate two. And then we want you to review that paper again. And focus on the writing skills this time. Here you'll assign your score for each domain and write that letter as well. Now, remember it's very, very important. Don't write numerals one, two, three, or four, but enter the letter code that's associated with each of those scoring points for each domain. Once you've assigned your code, place the paper on the mat, either scored once or scored. And then your next task as you go on to the next paper is to take a paper from either the unscored or the scored once pile and repeat the whole process of steps one through five. Step seven is so very important. That's why the red letters are there. It's important to create a paper flow to ensure all papers are scored in a timely manner. You need to read from scored once and unscored in a balance because we want to make sure that all papers are scored and decoded and completed at about the same time so we can quickly get the information back to the teachers and students. Well, you've completed the online scoring training. I wish you well as you begin your scoring. Best wishes and thank you for watching.