 OTAN, Outreach and Technical Assistance Network. Say thank you to Luke Philly for CASAS tech support. He's our expert on CASAS essay tests and I asked him to join us today. Also thank you to Cynthia King who is our in the room, room host assistant and I totally appreciate having both of you along for the ride today. So for the next hour we're going to be talking about the two new CASAS assessments. One is called essay and one is called language. The title of my presentation, let me go ahead and show my screen is four steps to improving writing skills and its ideas for implementing the CASAS essay and language tests. And the reason I talk about the ideas for implementing, it's not just your multiple choice type of test. What we found with the pilot agency is there's a whole lot of preparation that needs to happen in order to have things go successfully both for the proctors and the students and I'm going to be talking about a little bit about that. So just to start, we talk about writing and we talk about how important it is. I like to think of it as something that's not just a call to action but it's also persuasive. It can allow people to clarify a position or document an event. I think good writing is inspirational and probably the biggest challenge to agencies in helping students to be better writers is the amount of time that it takes for a teacher to score an essay. There's also a question about ambiguity or about the fairness of three different teachers all scoring the same essay. Do they all come up with the same sort of a rubric or a score? And we believe that what we've done with CASAS essay test is kind of the best of both worlds. We've used the artificial intelligence and the knowledge of teachers to train this autoscorer to do that work. And the great thing about it is it does the scoring in about four or five seconds. So sit with me for a few minutes, I'll give you a quick tour of what it's all about and share a little bit about the experiences that our pilot agencies have had. If I could just for a minute before I dive into that little video, if you could either in the chat or I don't, I think we've got enough people, it might be a little hard to raise hands and go around the room, but what is your role in adult ed? Because what I've found is when I talk about essay that administrators have a view of wouldn't this be great for a school? And sometimes the first impression from teachers is oh my gosh, not another assessment. And really what the benefit that we've found is that the students are the ones that really embrace this once they've gotten over the hump of taking that first dive into the deep water of an essay test online. Also looking for ideas if you currently offer writing classes, like a dedicated writing class or maybe it's bundled into part of your other curriculum as part of your reading, ABE rating, GED prep, that kind of thing. And there's lots of great comments coming here, lots of ESL teachers, beautiful. And I'll say that the essay test is geared for intermediate, high, and above. So if you have ESL students in that ability level currently, these tests are for them. Okay, now Andrew, I see you're teaching beginning level students. This would be probably too hard. I'll take you through a tour and show you a little bit about what the questions look like on the language test and what the essay prompt looks like. And I think we'll kind of agree that it takes a certain amount of a command of the language. I know when I took Spanish language and literature in school years ago, that for me to be able to write an essay in Spanish took a little bit of extra prep and ability in different kinds of ways. Okay, let me go ahead and queue up this video. I'm actually really excited about it. I can't talk and type at the same time, but here it is. An introduction to the CASAS essay test. To summarize the video, we believe that students will benefit from these different assessments and the preparation that you can deliver to them through your curriculum. It'll help your HSE candidates as they prepare for their HICED or GED exams. And it'll support students who are transitioning into different roles in employment or secondary education where writing skills are required. The video gave some examples about this. Now, the pilot sites did just exactly what I'm showing you here is they grouped the students by their goals and their abilities and they created different classes. Some needed like a short workshop. Others needed more of a boot camp for four to six weeks. And then others needed more of a semester-long writing assignment to practice with things. So the next few minutes, I'm going to introduce what I call the four steps to improving writing skills. And as you know, teachers, the writing skills have four basic components of assessing, preparing, assessing, instruction, and then repeating. And I like to think this is similar to what we do when we write, that we're thinking about things and then we compose and then we edit and then we reread it. And it really is a cyclical kind of a nature. So I'm going to go into these four different areas and talk a little bit in more detail. When I talk about preparation, first off, for the practice and teachers, we have documents, we have a lot of materials and that reading in preparation helps a lot. It includes screenshots and sample reports and also includes a rubric, which is very helpful so that you can see what the auto-scorer is going to be grading you on. Also, we found that when you take an online test, I mean, thinking back when I used to use a blue book and I had to write my essays by hand and have them scored, it's a little bit different in the world today because they're in an online word processor. Well, there's a few tricks to that. Do they know how to copy and paste? Do they realize that after a punctuation like a period or a comma, that there needs to be a space, right? Because the end of a sentence with a dot with the next word starting right next to it, that's going to be scored as a typo. So these kinds of essay test advisory notes are very helpful for teachers and for students to learn some of that insider ways to make things better when you're taking a test. Another thing that we created at CASAS is what we call the essay E-test sampler. Okay, the E-test sampler already exists. Most of you are already familiar with it, but we've added that component called essay. And teachers can use this in the classroom because it just runs in a browser. It's not actually connected to E-test. It's not going to do any scoring. But it gives the students 10 minutes to play inside the word processor, and you can kind of describe what it's going to look like. And that preparatory step, it really reduces a lot of anxiety. And that was the feedback that we got from our pilot agencies that the more preparation and the paint the picture for the students of what that is going to look like, it makes it so much better. So if they go into this tool and they click on it, it shows them what it looks like inside the essay test. Starts at the top where there's a timer up here, and this counts down from 10 minutes in the practice. In the real test, it would be 60 minutes. It gives them a prompt here. The prompt for just a sample is healthy eating habits are important, but often not followed. Why is this? What can be done about it? So that is the style of prompt that the CAS essay test uses. And then below it is a box where the students are going to type their essay in. You notice that the last word ingredients there has a little red underline underneath it? Right. That's the spell checker caught that. And we want the students to see that so they can right click on it and correct the word. Down at the bottom, there's a directions button that they can click on to get more information, like to remind them you're allowed to use scratch paper, no copying, pasting from other places. And actually that's prevented because in the CAS essay test environment, as you know, we use a secure browser. So they can't exit early or move things back and forth. All right. At the very end, when they're done, they click the blue arrow and that's it. All right. Another topic of preparation. And this one came out in the pilot study as well. We found that 120 students initially surveyed. We asked them a question, do you think this would have been easier on you if you had better typing skills? And 63% of them said, oh, yeah. We definitely could use some kind of a tool or a lab. So the labs and workshops that became popular at South Bay Adult School was for students to use something like typingclub.com. And they would take an initial test with it and get a baseline score. And then the counselors would set a goal of something like maybe an additional 10 or 15 words a minute or getting up to 25 words a minute to be able to type. And then they would have that typing certificate in their portfolio, right, which is good if you're trying to get a job. But I just shared that little piece of information that came up as part of the study. So those are the slides I have for preparation. Does anyone have anything they'd like to share or comment on? Or should I keep plowing into the next section, which is the actual writing assessments? Looking for hymns or comments? Okay. All right. Let's look. The second step, assessment. S.A. is looking at how writing flows. Is there a hand back here? Oh, I'm sorry. I can't see. Let me go ahead and stop sharing for a minute. Okay. Yes. Please. Can you share? Okay. Well, no, not to share with just a question. In the preparation, do you have a material that helps the students develop the writing in a way that the CASA test is going to be graded? Does that make sense? Like what are the things that whatever software is being used to grade it is looking for? And are there materials to help teachers teach that? So the question was, what curriculum do we recommend? Or do we have any materials that we suggest? And yes, that answer is coming in the third section. We have, we've been gathering suggestions from agencies in the pilot program, and we are open if there are other ones you have. That I'll kind of just jump ahead to answer your question because I hate to get left. When I ask a question, they tell me to wait. I hate to wait. Okay. Within CASAS, we have Quick Search Online. And that has the ability to look up keywords like conclusion or inference, which is two important things when you're writing if you're trying to go for a high level essay created. We also have one of our agencies like the, it's called Pre-GED Writing One and Two by New Readers Press. And we have another school that said, we're going to write our own. And I have a link coming up to share for you with that one. It was called Hook, Line, and Sinkr. I love that title, which is talking about how to write an introduction that hooks the audience. And I do have a copy of that PowerPoint on CASAS.org. We can get that for you if you're interested. It's posted in the, if you go to CASAS.org and you click on the national news and updates webinar, you'll find that PowerPoint in there. Is that a start? Sure. Sounds great. Okay. Super. Thank you for your question. So the essay test and the language test, I like to think of them as complementary, where the essay test is looking at the lyrical flow of the language, the creative part of it, and the essay is. And then the language test is more of the grammar and mechanics down at the lowest level. The language test is 20 multiple choice questions, 20 minutes in a sort of a sentence completion format, fill in the blank. That's also scored immediately. And there's a report. We're going to get into the reports in four slides. So I showed you what the essay test sort of looks like and what a sample question looks like. Let's take a look at a couple of the sample questions from the language test. The chef cooks, meals at the restaurant. Okay. Cooks. This is an example where a question is aligned to the verb category in the report. The language is looking at nouns, verbs, phrases, and punctuation. Okay. Let's look at another one. Okay. So the class that he signed up for begins in the spring. This is a question that's aligned more to the clauses, the category of the report. Again, going back to which students these questions would apply for, I think you might agree with me that they need to at least be probably a higher intermediate and up ESL student to be able to grasp these kinds of constructs. The pilot sites that use the essay and language test, as far as timing is concerned, that's kind of leading into my next slide. Let me hold that thought for a second. This is inside the E-test configuration screen. Any proctors or E-test coordinators out there, this should look familiar to you, right? So we added three different essay prompts, and we also added the language test. And that language test is, we have great vision, great high hopes for it to be expanded into multiple levels with multiple, you know, pre and post test forms. As of now, it's labeled a field test because we don't have the full battery that we normally would launch in a CASUS test suite. But this is what you do. You basically, just like any other E-test, you go in, you start the test, your students log into the testing stations, and they start to write. And as essay test runs under E-test, it creates that secure environment so students can't copy materials from a website or another source. And they can't exit while taking the test. Then when they're done, the personal score report is displayed. Then you start the language assessment and get that report. I like doing both of them at the same time. Even though we allocate 60 minutes for the essay test and 20 minutes for the language, we found looking at over 200 essays in the pilot and language, they don't take that long. Most students will finish up, you know, a decent two paragraph, three paragraph essay. They can crank that out in a half an hour. And when they go to take the language test, that's usually maybe only 15 minutes, just for planning sake. Okay. So here's the essay personal score report. It's got a little green bar, looks familiar. It's got the student's name and information up at the top, and it gives them a score. I remember waiting, the frustration of waiting for my essays to be scored by my teachers, and it would take like days. And it pops up immediately as soon as they're done with the test. Huge improvement. It's not just a pass or fail. It's anywhere between one and four as the score. An inadequate essay is given an asterisk along the same lines as the other CASAS tests. And at the bottom of each report is a specific areas for improvement, which is on this slide. Main idea, organization, word choice, grammar and mechanics. This divided up into those areas. So anybody who scores a two, let's say, would be given a set of suggestions on what it would take to get to the next level, a three. Let's dive into a little bit more on the bottom item here, the grammar and mechanics. Because this is the part that the language test really gets into. It breaks this up into a finer granularity. So grammar is the structure of language, where mechanics refers to the rules of written language, such as capitalization, spelling, that kind of thing. So when we run the language test, you'll see that that report reflects that. With nouns, I'm looking at the language content areas, nouns, verbs, phrases, and punctuation. And the way that I recommend teachers look at that is they go over to the far right and they look at the percentage correct. And they look at the areas, like the ones I've highlighted here, where the student needs the most work. There's also a CCR language level, which is helpful for you to pick out the right curriculum level for the student. So this is the end of my section, second section, which is to sort of summarize that the way you use these two reports, you have a student that's got a comprehensive roadmap on areas that you need to concentrate on to help them to improve their writing. So the next section, we're going to talk about how to teach writing. And again, keeping in mind the student populations, it could be A, B, E, N, S levels two and up, and ESL high intermediate and advanced. There's more than one way to teach writing. Again, old school, I used to use an outline. I'd get a piece of paper and start drawing things on a piece of paper and I would come up with an outline, and then I would switch to the keyboard or whatever and start creating it. I find the generation Z students, they go straight for the keyboard. So we have to consider how we're going to teach them. Are we going to teach them to look at the prompt and pick out keywords? Or are we going to teach them to grab phrases? I know writer's block is a real thing for a lot of people. So that's something that we need to be considered and how they're going to effectively use their time. I mean, the pressure of a time test sort of is created by just saying the fact that it's a timed test and they don't know what the topic is and that adds to some of the anxiety. However, I think that you'll find that the benefits outweigh some of the initial sort of discomfort. These were tools that I mentioned a moment ago about inference and conclusion, the introductory hook line and sinker PowerPoint from Andrew and Kay Hartley. I think she's in the audience here. She's going to be teaching at the Casa Summer Institute in just a couple months some more ideas for writing skills. So hats off to that group. And again, the new readers press pre-HSE writing one and two was the other tool that I mentioned. Can you share your slideshow so we can slide? Yes, absolutely. Did I already do that once? There it is. tinyurl.com slash casas hyphen essay. So to kind of bring it all together, we're talking about this cyclical nature of composition, which is writing and editing and rewriting. And hopefully this has given you a little bit of a sneak preview on how the CASAS assessments could be used in that environment. We'll be planning a few more in-depth presentations in April. So if you're interested in doing that, please send an email to me jfera at casas.org and I'll get you onto the list for that. We do have field testing opportunities for other tools, but these are the two that today I'm super excited to be sharing with you about. And now I'll kind of take a little break before I get into some weird techy stuff. If you have any questions so far? Cynthia, do you have any questions there in room 209? No, I have one question. Hi, Janice. Hi. I can't see you, but I can hear you. Question, Janice. This particular assessment is more for in-class information and not necessarily for placement purposes. Or if it is for placement during an intake, how would you use that in combination with life in writing or life in reading, life in work, or the Goal Series? Excellent question. So the question is, would you use this test for intake purposes? And if so, how would you go ahead and how would you use that for blending it in with the other levels? Let me just kind of throw that one out to either Barbara or Kay or Portia. If either of you three possibly have a suggestion on that one. I have a couple ideas, but let me just ask the California Program Specialist if they wanted to tackle that. I don't know. Ladies, no? Well, I know that one of the adult schools were considering using that in place of the oral assessment. And they felt that preparation step by taking a student that hasn't been through any of the preparation discussion about, be sure to put a space after the period or how to manipulate an online word processor, that that was asking a little too much for a lot of students. That they wanted to spend a little bit of time with them to prepare them before putting them into the essay test. So the agencies that I talked to had already put the students through an appraisal and had already level tested the students before the essay was administered. Wouldn't it be challenging to figure out if they're at a high enough level before administering it? I mean, it seems like you might be putting them up for failure if you gave it to them without discerning whether or not it's at an appropriate level. Yeah, I see. My thought is it's primarily for instructional information rather than replacement. And that's what I'm hearing from you, Kay. Yes, yes, definitely. If, for instance, you had an ABE class or a pre-GED class where they're preparing for the test, it'd be an excellent tool to determine whether or not they're beginning to approach readiness for essay writing for the test. But for standard ESL, I think it would be a real daunting experience if they haven't had been appropriately leveled before they took it. I would think that you would want, especially for your ESL learners, you would want to find out how literate they are in their native tongue and go from there because some students are, you know, have advanced education in their native tongue, but yet they're still struggling with English. But, and a lot of those students can write much better than they could speak. So it could help you that way as well. Thank you, Barbara. Great. Do you think, one more question, do you, to the group, do you think the language assessment, the story language one, that one would seem to be more helpful for placement purposes, right? The grammar one, correct? Yeah, I don't think so. The question is, would the language test, would it help if I showed that, that report again so that you know what that looks like in case you're not, this one? I would think this would be very informative for you as a teacher and for the student, especially if they are educated in their, in their native tongue, or your advanced ABE, AS, your advanced ASE student. Also a question in the chat from Carol Lazarus asking, can it be used as a post test for moving up a level? Answer to that is no. It's not considered a post test. Remember, a pre and a post test have to be of the same modality. And the CASAS SA test isn't one of the ones that is already approved by the NRS as having a pair. That's something that we're going to talk about, you know, going forward once we have more agencies that have administered it and we have more statistics, we're hoping that we can use it as a predictor for success for the GED test because of the writing component that's part of the RLA within GED, but we need a lot more data and more time to be able to come up with that kind of a reference. Okay, is there's a question here? I'm sorry? There's a question here in the room. Yes, go ahead. So one of my instructors who teaches high-set, we just started implementing computer-based testing because of the problem with high-set. But with one of the problems she's running into that the students are so used to and I think it came out in your example, seeing the underline of the error, how can we prepare or what can we do so that the students, like, not depend so, because that's been one of the challenges that the student is so used to seeing, oh, there's a grammar error, there's a spelling error. So in their mind, they're like, oh, it's on the computer, so it's going to show me the mistake, but how or what advice would anyone in the room or online suggest for us to try to relay the message to the student? Because we've tried, apparently, one been successful, but that they're not going to see that or is there a practice tool, kind of like what you showed, that's not going to show the errors of the student to kind of start training them that it's not going to show you your errors in the period of the film making sense, right? It's not going to underline the mistake. It's not going to un-collect itself for a grammar error. So as you recall, when I showed the E-test sampler, that that shows that as they're typing and they get that red underline that they need to right-click to change, to fix the spelling mistakes. If you had a Chromebook and you turned on the spellchecker, invariably you also get a grammar checker and that's a blue line underneath. Agencies have to kind of decide if they want to have that turned on or not. And we think it's a kind of an unfair advantage to give the student a grammar checker and then they find themselves on a high set environment, testing environment and they don't have that anymore. So one agency that has Chromebooks in their E-test lab decided to turn off both the spellchecker and the grammar checker because it's like buy one, get one free within the Chrome environment. It's not a problem on Windows but that's how Chrome implemented their spellchecker. Any ideas from the field? I mean, this is a panel discussion and I'm so glad that all of you are here and so many of you are teachers. Yeah, Anders come in about auto-correction to be turned off by the individuals. It's not actually auto-correction, it's merely flagging the error. It's not doing any auto-correct for them. And thank you, Luke, for saying more eloquently than I did about the correlation with the other modalities. Thank you. Kind of an interesting fact I picked up. So the agencies did a lot of their, and the pilot, they did a lot of their pre-tests on their students, the reading and then they gave them the essay tests around the October, November, December time frame. And now I'm circling back with them and saying, no, so they're in their writing classes now, they're in their boot camp, they're in their workshops. What are you gonna do for the retest? And it kind of brought up a really interesting bunch of questions. Is it valid and is it actually beneficial to give the student the same essay question after the writing assessment instruction is completed to see how they do with the same question in terms of their use of words, in terms of their flow, the grammar, the punctuation? Would that be the way that you and your agencies would use it to retest on the same of the three prompts? So if you started with the first one about, I think one is about employment, one's about kind of a health sort of a thing, and then one's about consumer skills. Would you, well, I'll just kind of give you the answer for the essay, the pilot site that I was working with is they said, we want to test on the same prompt. We want to see what they do with it. It's not like they're memorizing a bunch of answers. We want to see how they've matured in their writing style. And that way that they can kind of cycle through that and see if they've gone from a one to a two or a two to a three. Maybe they've made a mega jump and gone from a one to a four. We're still waiting for that data to become available to us on the retest. Dennis? Yes. I would think that the student would really like to see that, to see their own progress. I think that would be really inspirational for the student himself or herself. Good point. I like that. You're right. Because I know when I used to, I'd get back my essay into being redlined and the teacher would sit next to me and she'd go, okay, this is what you need to do differently. And she would send me back and I would have two more weeks or a week and I would have to redo it. And then I would come out with something that was so much better than the first one I did. And that's just such a joy of a teacher, really, to watching the student mature from where they were into their new abilities. Yeah, that is the writing process. Angela, to answer your question, when the student is in the essay test, yes, they are shown the spelling mistakes. No, they are not shown the grammar mistakes. The one caveat is the way that Chrome has, the Chromebooks have implemented it, which is you either get both of them or you get neither of them. Okay, a question from Danielle. After they take the essay test, can we as teachers view what they wrote? Am I being recorded? Oh my gosh, that's a giant, okay, I'm going to take off my COSIS hat for a second. That's a giant debate within COSIS. How much, you know, how much we're going to open up the entire database to let people pull out chunks of text? So currently, yes, you can see what they wrote. I don't know if they're going to lock it down or not in the future, but as of today, you can go into Topspro Enterprise, you can go into the tests, you can find the student in the test, and you can bring up their text. Okay, COSIS hat back on. That's just, that's how we've implemented it so far. And that was how we found the mistake with the punctuation, with the no spaces after the periods, that one of the agencies called me and said, this student Janice is so capable, we can't believe she got an asterisk. I went, what? So I have some magic that you guys don't have that I can actually do e-tests and copy and paste from other places outside of the secure browser. So I grabbed the student's essay out of TE, plopped it in there, and then scored it, got an asterisk, and then I put spaces after all of her commas and periods. And it was a beautiful essay. It was like, you know, four paragraphs long, nice choice of words. She got a three already, just for making that one kind of important correction. And I mean, if you think about it, she's got maybe 18 sentences total in three paragraphs, and every one of those ones ended up with a weird word with a dot in the middle of it. And that was enough to throw the auto-scorer into thinking that that wasn't a great essay. I have a question, please. Does it weight the grammar versus the content? Ah, there's lots of magic in them, our noodles. Yes, there's all kinds of weighting about which things are more important. Is the punctuation more important than the creative use of words? And that's all part of the AI behind the scenes. But I do have a few things to share with you if you want to go into the chat GVT discussion. GVT. Anybody from there with them? Go ahead. In my opinion, speaking and writing, there should be a huge emphasis on content. And we can kind of overlook the small little errors because are you understanding the individual writing and speaking and do spend time, not you, but us as a group, going backwards to picking apart all these little details I think is unfortunate. And your example highlights that. Fantastic. I mean, let's give her 80% on content and 10, 15, 20% on the other in my opinion. But I'm not an expert. I see your point 100%. And all I can say is that's something that I would take back to my development team and take back to a few others to discuss a little bit more that the value, the weighting on content versus little nitpicky things like punctuation. For now, all I can say is that the amount of hundreds of hours of training, let me kind of go back to that couple of those slides that have gone into training those three prompts that we've prepared so far would suggest that just a little bit of that advisory document that I shared earlier saying that, you know, these are a few tricks that you need to teach your students so they score highly on the test. This is, these are just a few things that they'll need to know because if you forgive them and then they get to the high set test or the GED test and it's not as forgiving, there's going to be a disappointment there. So unfortunately teaching them the way that an online scoring algorithm works, it's a bit, you know, it's a bit of a dinosaur at the beginning but it's, it will serve them in the long run and it's really not too hard to teach those little tricks. But I hear you loud and clear, content is super important. I mean, look at the stuff that, you know, the kids are texting each other and it's almost unintelligible but it's not unintelligible, right? It would fail an essay test but it would, it conveys meaning. Had a class in college. What is the meaning of meaning? You know, UCLA linguistics 601 and we sat around and talked about what is meaning? I get that. Any other questions in the chat that I've missed? Let's see. You want to see the last few slides I have for, because this is the technology and distance learning and technology being the big word there. I prepared a couple slides that I didn't share with the national audience just to kind of talk a little bit more about the technology behind the, let me skip ahead, the auto-scorer. I talked briefly about the algorithms and the way that we pre-trained the prompts. We used CASA's program specialists. We used experts. We've been using these prompts within our NEDP national external diploma program, which is popular in a lot of states besides California. We share with you the scoring rubric and just kind of give you the advice that these are the things that go into the auto-scorer. And we feel that the fact that it's, it, it'll, it doesn't have any teacher bias. Not that you guys are biased, but the bias of what's one teacher migrate as an A versus an A minus versus a B plus. This thing is a machine. And you give it the same essay 15 times, it will give you the same score. And there's a consistency that comes with that that some, some find is, is beneficial. Here's a question I got from one of the agencies. How lenient is the auto-scorer? How hard is it to get a four? So the good student, I sat down and I wrote a bunch of essays. I wrote a good Janus quality, you know, hard, good, I got a four on it. And then I backed it off a little bit and I got it to, I got a three. And then I made some mistakes and I took out a paragraph and I got it down to a two and I got it down to a one. And then I mess around with it a little bit to see what kinds of things it flags for an asterisk to score. Like too short, too many certain types of errors, you know, violent speech. So there's a, there's a bunch of things that are included in the rubric, which describe the things that would give it an asterisk. But I found that it's not easy to score a four. And if any of you have heard about chat GPT, which is a little online tool, it's free currently. Students are kind of using it a lot, especially in the community colleges. I hear about that a lot. So anyway, you feed it a prompt and it gives you some information that's snooped out of the internet in different places. So I took the CASAS essay prompts and I fed it to chat GPT. And sure enough, I gave it that, you know, healthy eating habits are important prompt. And it produced this little, you know, a couple paragraphs essay. And then I took it and I fed it to the auto score. And what do you think it got on it? Two. Wow. So it's looking for something good. And I like to think of it as a pretty tough 11th grade English teacher. Okay. Full caveat. If I had changed the prompt around and I had told it to do something like create a collegiate level essay on healthy eating habits, which include it probably would have put out something a little bit more, a little more collegiate level. But the fact that you can take the prompts and you can feed it to chat GPT and it only scores it to gives me a feeling that we are going into some level of detail that is exhaustive and analytical. Let me see. Oh, here's a couple of references which are included in the PowerPoint. There was a great presentation by Christy Reyes of Maricosta College part of the lunch with Larick talking about ways to teach writing in a collaborative way with cool online tools. I love that presentation. Also TESOL, they had a note starting on page 60 about advice for teaching writing to students. Those are, I thought, probably two helpful things. I'm watching the clock. I want to go ahead and suggest that if any of you want more information, you can always email me, Jay, Vera, F-E-R-A, just one are at casas.org. You can talk to your teachers and administrators and if you think this is something you're really interested in, reach out to tech support. That's Luke and his gang and they will be able to install the different sessions on your E-test sessions. Remember that was the slide we looked at here. They'll be able to install those for you and get you set up with all the copies of the documentation. There's oodles and oodles. There's the rubric. There's a frequently asked questions which is kind of a short thing that I would send up to your administrators if you're trying to explain to them the benefits of it and kind of the short of how it works. Then there's a document that's also posted that's we call it the TAD internally. I can't even test administration directions. And that's got screenshots, sample reports. It's got the verbiage that a teacher would say to the students as you're sitting down getting ready to take the test assessment. Lots of information in there. It's been reviewed twice. It's had two different whole revision cycles since we started the pilot and I'm pretty happy with the state that it's in right now as being good and comprehensive. And those are all available through the tech support or on casus.org. So I want to thank you for joining with me today for thinking about writing and thinking about ways that your students could possibly benefit from having that skill set in their arsenal as they go out into the world and go into the next things. If you have any comments or questions, reach out to me please, jferra at casus.org. And we have a couple more minutes before we're going to say goodbye. So if anyone wants to stick around, I'd be glad to chat with you and answer any questions. And that's pretty much it. I want to say thank you to Cynthia for room hosting there and room 209 and Luke Philly for being in the chat. Thank you all.