 This is Anthony from OTAN, welcoming you to this webinar, Considerations When Implementing Distance Learning. And if we are ready to go, then I'm going to turn it over to Jay. Go ahead, Jay. Okay, hi, how are you? So here's the overview, the overview, very much the same as the other sessions. Again, I think about half of you probably have done this session before, but obviously COVID-19 has had a huge effect on California, a huge effect on adult dead. There's an OctaMemo, we've been covering it a lot. It's memo 20-3, we'll talk about that. That's the big directive from the feds. They sent out a big directive at the end of March. They very recently, I think last Friday, sent a follow-up to that memo nationwide. So we'll talk a little bit about that. And then there's three big issues when we're talking about considerations for distance learning. Issue number one is how to deal with classes. So we've definitely not come up with anything scientific here, but we have come up with what we think are the two bottom line basic approaches to how to create classes. So we'll identify and define those two different ways. The other big issue that's come up is how to account for hours. There was a session that Penny did yesterday. Probably a lot of you attended it. She got in depth with a lot of the different models and how it applies in the classroom. What she was just using are the three models, the fed stipulate, you know, of how to do hours. So we'll look at a couple examples and talk about how to do hours. And then we'll talk about assessment. So there's kind of another three-headed monster within the three-headed monster there. There's pre- and post-testing, placement testing, and EL-7s. The short answer for all three, as we're still saying, and in-person proctor is required for all three. But we've talked quite a bit about we're moving fast and furious on remote options. We started a big field study yesterday. So we've got progress on all three fronts, pre-post, placement, and EL-7s, more about that later. Then there's a recording distance learning and enterprise. There's one that Shanna Sfera has done a couple of times. My guess is a lot of you have been on that one, too, but we'll talk briefly about some of the TE features. There's one that Shanna Sfera does that really gets deep in the weeds on how to use TE. We don't really have time to get deep in the TE weeds here, but we will at least show a couple of those features with the short answer being you can record exactly which hours occur in distance learning versus exactly which hours occur in the quote-unquote regular classroom. And then a little bit of a preview. This is the difference between the slides last time in this one is we have put together what we're simply calling distance learning part two. There's been lots and lots and lots and lots and lots and lots and lots and lots of questions that have come up about this. Lots of different areas in which the questions have arisen, but I think maybe the number one area is with the local assessment policy that is throughout this when there's numerous times when the short answer is, you know, you can do what you want, but you need to carefully define what you're doing in your own agency's local assessment policy that's naturally generated a lot of questions to the effect of, well, hey, that's great. We're glad we have the flexibility. We definitely have that policy, but what exactly do we need to do to record it correctly in said policy? So we'll look at a lot of the models I've been talking about a lot of the models that Penny really detailed a lot yesterday and try to come up with examples. We're not going to get every single one, but we've come up with six or eight different examples that characterize six or eight different ways of providing distance learning, some examples with all three models and how we might document that in the policy. That's what we'll be going on with part two. We'll probably be looking, this is tentative here, more, we've got a few eyes to dot and tease to cross before it's official, but potentially we'll be looking at delivering part two on Tuesday, May 5th, and then we'll be doing some stuff on Thursday, May 7th as well. Okay, moving on, obviously COVID-19 has had a huge effect on all agencies, schools have closed, colleges have closed and so on. We've talked a lot about that OTAN COVID-19 page. Here's the link on the slide. Lots of information from CDE, from OTAN, from CalPro, from COSSIS. We have our FAQs on the state and federal accountability. There's a TE help document, there's official letter and policy statement from CDE. So anything you wanted to look at and more is on that COVID-19 page. Next slide. Okay, the other big reference here is Octa Memo 20-3. So it was mid-March when we started transitioning to this COVID-19 quarantine mode, you might say. Once everything was headed in that direction, we were eagerly awaiting the feds to weigh in on what will be and what won't be. They weighed in with Octa Memo 20-3. To be clear, for those of you who haven't heard these a lot lately, Octa stands for Office of Career, Technical and Adult Education. That is the component of the US Department of Education that pertains to us. So they came up with Memo 20-3. It had all of the updated federal guidelines now in response to COVID-19, you know, basically said, you know, yeah, we know that we've changed a lot, but we're still collecting data. We're still requiring all states to report to the feds. We added extra emphasis that there are non-testing MSGs that can be collected. It brought up a check box that I'll talk a little bit about later. There's now a check box and TE that we'll use to record everybody that was affected by COVID-19. We're still looking at these procedures for how they might affect us in California. I think as far as how to record classes and hours, we don't really have any more investigation we need to do. We've just had to dust off and clarify a lot of existing policies. Where the review fits in though, of course, is related to remote testing. A lot of that Memo 20-3 is about remote testing. So CDE and CASIS are still working on more issues related to that, more on that later. I'll add, there isn't, sorry, I should add before I move to the next slide. There is an updated Memo that the feds sent out. Can you just back up? Sorry, I missed the one in bold, thank you. The updated Memo April 17th. So the feds did send out an updated Memo last Friday. It basically reiterates most of the things they stipulate in Memo 20-3. However, it gives states a little bit more flexibility to explain why they may not necessarily be doing remote testing. In late March, things had just started to kind of hit the fan, you might say. And the feds, I think we're really sort of figuring that everybody would need to scurry and do remote testing as soon as possible. Now it's starting to realize that that's easier said than done and is not gonna be an expeditious process. And it's also determining that some states are eager to get started on remote testing yesterday. Others are not as eager, so it's provided. I would characterize it as more cautious language, where it's still looking at ways to move forward on remote testing, but it's looking a lot more on states just explaining why it may not be able to do remote testing and more emphasis on identifying those students that weren't able to test. So anyway, I'll move on to the next slide, just probably more than enough on that. Okay, so basic requirements for DL students, the short answer is individuals in distance learning have the exact same requirements as everybody else. Those who are in distance learning should mark special programs distance learning. You can mark it at either the class or student level. And here within the last week, we've added a new checkbox now that didn't exist before, but again, back to that octane memo, what required all states to report a headcount of all students affected by COVID-19. So we added in TE a checkbox that we're calling force majeure, which is the term the feds have been using to characterize the students that were forced out because there's a quarantine and so on. So in any case, force, there you go, somebody wrote it, great. So anyway, we have a checkbox now in TE. If you want some screenshots, you can get it from me, from Janus, or from Tech Support. It's been in TE now for almost a week. So a lot of you, I'm sure, have seen it. So the checkbox is nothing special, nothing really mechanically notable about it. But I will say we are suggesting that you just leave it blank for now. We did wanna get out in front and get the programming done sooner rather than later. But we still have more to discuss with CDE related to exactly which circumstances require you to check that box. Some are suggesting, well, hey, COVID-19 affects everybody, so it should be everybody. Others are saying it should just be those who can't test or just those who got closed out of class or whatever. So lots of possible options. No answer is right or wrong, but we wanna come up with something uniform so you're all marking that the same way. Next slide. Okay, what if the students in blended learning, and this will be a lot of your students who are in blended learning, that is if you look at it across the year or across a class, there will be some of that time in the regular classroom and some of that time in distance learning. The NRS or Feds have always had, or not always, but for the last 10 or 12 years, have had the 50% rule. That is if more than 50% of the instruction is outside the regular classroom, marked distance learning. If less than 50% is outside the regular classroom, don't mark distance learning. So it's just simply more or less than 50%. If it's more than 50%, obviously, mark distance learning and special programs. You don't need to specify exactly how many hours for DL versus how many were not DL. All you really are required for federal reporting is to account for the 50% rule, but I will add there is now a feature in TE where you can get to that level of detail if you choose to do so. That's a lot of what Janice presented in that TE related session. We're also pointing out a field in the class instance record called instructional setting. Instructional setting is an old antiquated field that we used in the old cohort system five years ago. So it's not required. It hasn't been relevant for at least five years, but we are pointing that out as an optional field to check. A little bit of that is for you because you're trying to track classes that you might've switched over midstream. You might've had a regular class through March. Now we're in COVID-19 mode, so it's all DL. It's still the same class. It's way less than 50% distance learning. So you're refraining from marking special programs, but you know that a funny thing happened on the way to the forum there on the lines of March. You know it's not like any other class. So we're saying that's an informal way that you can show that that classroom is affected and an informal way to show that distance learning is occurring if perhaps you decide you don't want to earmark every single hour. So that's an optional way that you can mark it. There's, I'll add, there's a lot of information in that federal memo that requires all states to provide narrative on all the different ways in which we're affected by COVID-19. That's a way where we can potentially provide anecdotal information that certainly can't be anything official. But if we can say, yeah, we anecdotally have this large number of classes that were not marked officially for DL, but were marked unofficially as less than 50%, but where you did have to address some of these issues at some point in time, we're thinking having some numbers to that effect would probably be a good way to report that informally with some of the extra things we'll need to do. Okay, next slide please. If it's less than 50%, again, you need to do nothing. That's where I would really hone in on that classroom plus distance learning where you know it's less than 50%, so you're not marking distance learning. You don't really have enough information to get in the details and record that hour by hour. But again, you know a funny thing happened on the way to the forum. You know it's not like any other class. You know that some of it's distance learning. That bottom bullet is a good way to show that you made some of these changes without necessarily rising to the level of recording distance learning based on federal rules. Next slide please. Okay, so the first big topic is creating classes or addressing classes. Or actually, sorry, so we're looking at those two methods. In doing that though, we have two key considerations we know we need to follow. We need to make sure that we follow the 50% rule so we can do that by recording the hours granularly or we can just look at each student or each class and market. Bottom line though, for what's required is you must calculate that 50% rule for every single student. So that's the part that's non-negotiable. You really need to figure out the 50% rule. Normally it's a small issue, but obviously this year it's a big one. The second one is totally optional. Janice and others have shown some of these sessions where you can now show exactly which hours are distance learning versus exactly which hours are regular classroom. If it's best or most promising or easiest for you to do it that way, of course I'm gonna say really, really great idea, but I'll point out you're not required to stipulate every single hour if some of that is easy to track and some of that not so much, you're not required to do so, but it is available if you wish. Next slide please. Okay, so I'm just gonna go to this one in bold here. We've talked about more than 50% or less than 50%. Bottom line is calculating this at the class level depends on how you structured classes in response to COVID-19. Sorry, I took a long time to build up to that bullet, but we'll go to the next slide please. So again, here it's repeated again, but not in bold. Calculating this at the class level depends on how you've resumed classes and how you've structured that activity in response. So again, not scientific. I can't find any official terms anywhere for this. So we made them up. One, we're calling pick up where you left off. The second we're saying is start brand new classes specific to distance learning. Next slide. So pick up where you left off. So what we're suggesting with pick up where you're left off is you're leaving all the classes as scheduled. So maybe you started them at the beginning of the program year. Maybe you started them in January for the beginning of the winter spring semester, whatever. But the thought is you have lots of classes in progress. Yes, we probably had to suspend the classes. Yes, we definitely converted them all out of the regular classroom and into distance learning, but we're still keeping it basically the same. The teachers the same, the students are the same. We're still meeting on Tuesday, Thursday mornings or whatever. The only difference is we're meeting Tuesday, Thursday morning at 10 a.m. via Zoom rather than in the regular class or rather than in the regular classroom. So we're just keeping it all the same. We're just not meeting the same time but doing it a little differently. So in that case, we just keep the class record as is, but we do need to worry about that 50% rule next slide. Okay, so here's an example. So we have an ESL class that started in August and was scheduled to run through the end of May. We're just using a sample situation where we kept that regular classroom instruction going till March 13th. We closed for a week or so as we were in COVID-19 mode, but a week later on the 23rd, we reopened the class, same teacher, same students, but now we're doing it in distance learning. Here's an example where we would not mark distance learning at the class or student level because less than 50% of the instruction was delivered outside the regular classroom. Again, we went early August through early March in the regular classroom. We've just got the final month or so distance learning so we would not mark distance learning in this example. Next slide, please. Again, pick up where you left off, same approach, but in this example, the class started the first of March, again, scheduled to run through the end of May, the same interruption time. We kept the regular classroom through March 13th, reopened it via distance learning on the 23rd, same general set of circumstances, but here we would mark distance learning because again, here the majority of the instruction is via distance learning. So in this example, we would mark it because it was more than 50%. Next slide, please. So in either example, whether it's more or less than 50%, you can specify that in TE, just saying, that's the session that Janice got into where again, we're calculating 50% we're not required to mark this level of detail, but there is a way to do it if you really wanna make sure you capture exactly what's held via distance learning versus what's in the regular classroom next. Okay, next approach is start completely new classes specific to distance learning. So we're basically under the same conditions, of course, this suggests that up until the first of March, we were doing everything in the regular classroom since mid to late March or so, we've switched 100% to distance learning just like everybody else, but in this approach, we're suggesting that once COVID-19 hit, we had to do everything completely differently. We had to close down all the classes now that COVID-19's hit and we have distance learning, the teacher assignments are a little different, the student enrollment is a little bit different because of that, we're now offering it at different times. The new world doesn't really resemble the old world at all. So we're deciding to create completely new classes specific to distance learning. So in this case, we're closing out all the existing classes in the regular classroom where perhaps we're giving new class records, new IDs, most likely giving completely new titles to those classes. So we've got completely new classes now for COVID-19. So the simple answer there is we close out all those old classes, the old classes were in the regular classroom. So obviously we're not marking distance learning. The new classes all are completely distance learning. So for 100% of those new class records we're creating, we're marking them as distance learning, obviously. Next slide. So first example, again, here's an example where we started everything up in early August. We were scheduled to run that class through the end of May. The COVID-19 timeline, we're using the same as those other examples. We closed the existing class on March 13th. We reopened new classes via distance learning on March 23rd. In this example, we had the class started in August. It will be non-DL, because of course that was entirely not distance learning. And then we just create a second class that we're marking for distance learning effective March 23rd because that's when we started that brand new class in distance learning. Next slide, please. The second one, again, this one we started the first of March. Again, it was scheduled to run through the end of May. Again, the same interruption due to COVID-19. We suspended the regular class on the 13th. We resumed the distance learning class on the 23rd. So same basic approach and answer here. We close out the non-distance learning class, give it an end date of March 13th, create a brand new class, give it a start date of March 23rd. The new class of course is exclusively distance learning. So mark that as distance learning at the class and or student level. Next slide, please. And then again, either way, whether you're marking it, you have that option of marking all the distance learning activity in TE, perhaps this is a little less necessary for this approach, probably for the pickup where you left off, I would probably encourage you to consider this for this other option where one class is 100% not DL and the other one 100% is DL, you may not need to really record this as much given that situation. Next. Okay, so moving out of class creation and then the second big issue that submerged, that is recording instructional hours. So just like we did with the class issue, we'll start at the top with the feds in the federal implementation guidelines. They report three different models of measuring distance learning, clock time, teacher verification, learner master mastery. We'll start at the top with clock time. Clock time is the easiest and most straightforward. It's simply using the clock that is, basically it's suggesting we're using the same method or approach to record distance learning as we would in the regular classroom, that is in the regular classroom. If the students and teacher meet for one hour, the default answer is you would record one hour of instruction for all the students that met for that class, clock time, model and distance learning is doing the same approach. That is one hour of regular classroom is the same as one hour of distance learning classroom via Zoom, for example. Teacher verification requires the teacher to observe the students and assign a certain amount of hours based on how the teacher observes the student to complete. So there might be a set amount of instruction for which you're meeting with the students. The teacher is observing the students. Some students might be hard at work. So you might record the entire amount of time for those students. Other students may not be hard at work. So you may not record as much time for those students as the ones who are hard at work. Teacher verification is doing that and allotting different totals to each student based on the teacher's observation of different students and noting some students to complete all the hours of instruction and other students to perhaps not be completing all the hours of instruction that's teacher verification. Learner mastery on the other hand, again, not using clock time, but it's assigning a fixed amount of time based on observation ahead of time, recording it ahead of each lesson, and then recording that amount of time once the student completes each lesson. So learner mastery suggests the same basic scenario as teacher verification. That is some students are gonna be hard at work. Some students are gonna motor through it because they're working very hard. Other students either due to not being hard at work or perhaps just being interrupted by many, many things take much longer to complete the same amount of work. Learner mastery assigns the same amount of time to that amount of work for all students and basically records that set amount of time for each student upon completion with the idea that some students might complete that hour of work in an hour. Others on the other hand might complete it in a few hours based on interruptions. So you use learner mastery and define the amount of instruction for that work prior to the students and engaging in it to record the amount that you defined ahead of time and thereby record that same amount of time for each student equally. Next slide, please. Okay, of the three you can probably detect from my manner of presentation, I spent the most time describing learner mastery not so coincidentally, that's the one that's explicitly stated in our California statewide policy, that is the proxy learner mastery model to record distance learning. To be clear, and I think Penny covered this in a really good detail yesterday, you're allowed to use any of these three models in California, the state goes out of its way to allow all three, it basically allows all three, but it does require you the agency to formally choose which model or models you're using, and it does very much require you to explicitly state that in your local assessment policy number one and also describe in detail how you're employing each of those models in your local assessment policy number two. So again, you have the flexibility to use which model you want, you can use multiple models if you choose to do so, however, once you employ that flexibility, that does also mean you have the responsibility to describe in detail how you're employing each of those models if you're using more. But again, learner mastery is the one that we've used the most in California, and I'll say, it's not just a culture issue, I think I've kind of been presenting of it as a, yeah, yeah, that's what we've used the most, which is true, but beyond the fact that it's the one we use the most, it's also the one that CDE has stipulated in the statewide assessment policy as the preferred model, and I believe it's done so for at least 10 years. So I will note, it's not just a, oh yeah, by the way, that's what most people do, it's what most people do, because it is the one that's been specified in our statewide assessment policy for quite a long time. But again, either way, you have that flexibility, but whichever one you use, you need to record that in your local assessment policy. And again, you can vary your approach depending on the model you're using, and obviously depending on the distance learning format in which you're providing the instruction as well. Next slide, please. Okay, so we've got some examples here. We're looking at facilitated online instruction, facilitated as the, meaning synchronous. We have another example we're using for educational software. There's a synchronous approach to educational software and an asynchronous approach. We've got a couple examples there, and then we've looked at that educational software and boiled it down to three practical situations, three practical examples, determining those proxy hours, depending on the exact scenario, kind of suggests using a different model, we'll get into a few of these starting now. Next slide. My audio is breaking up. Sorry, I'll try to speak more clearly. Hopefully this is better. So anyway, the first one we're talking about is facilitated instruction received online. Sorry, I might have had my hand masked. Sorry about that. So how do we do facilitated synchronous instruction? We're using clock time. So this is just the example I gave you earlier, where we're suggesting that we're meeting with the same teacher, same students, same days to the week, same time of the day. All 20 students are meeting Tuesday morning at 10 a.m., but they're just meeting via Zoom instead of the regular classroom. That's a synchronous facilitated example. So in that example, we'd simply use the clock time model. We recorded an hour of instruction for all those students on Zoom, the exact same way we record that one hour of instruction for the students that attended the regular classroom, pretty much no fuss, no bus, no fuss here. Use clock time, again, an hour of online facilitated equals an hour of regular instruction. And then if that happens to be more than 50%, Mark Distance Learning and Special Program. What if it's a recorded session? So we're using the same example where we're just doing the teacher talk into the students via Zoom. And this example will say we had 20 students, 15 of them attended Tuesday at 10 a.m., but five of them missed class, so they had to make it up by sitting through the recording. So if you do the recording, that's fine. We suggest that you do stipulate that process if you want students to make it up through the recording. I'll add you, the agency, have the power to decide this for yourselves. Some of you would say, well, yeah, we've got to make sure we give students that option. We've got to give students that flexibility. Others, if you are suggesting that's a slippery slope and you really don't want to allow that at all, I'll just say it's up to you whether to allow it or disallow it. Just make sure you stipulate it clearly in your local assessment policy. If you do allow it, if you do want to provide that option for your students, obviously be sure to document it. For that scenario, we would say that's more of an asynchronous example, an asynchronous non-facilitated example. As such, that would be an example where you might need to use learner mastery instead of clock time. That is, you just have the students complete that hour, but instead of relying or trusting the student and giving the full hour, you might have a series of exercises or a quiz or something that you have the student complete to show that that student really and truly did watch that recorded session. You would thereby have the student complete whatever it is you needed. Sometimes that's assessment, sometimes that's exercises, sometimes that's an oral interview, whatever, but whatever it is you wish to do, you would stipulate it and define it very clearly in the local policy once the student completes it. You document that same instruction and again, just record it in detail in your local policy. What if you're using educational software? This is obviously the really common example. We're saying not always, but usually we're saying that's learner mastery. It would be a lot like what I just described for the recording. Again, some students might complete it an hour because they're hard at work. Others might get interrupted and take a lot longer. So what you would do is just figure out, hey, this is an hour of instruction. So we know this lesson should be an hour. So for all students you record an hour, you can't really monitor the students. So you might again give a quiz, you might give a task, you might give an oral interview or just completed exercises. Once the student completes that, you'd record that hour of instruction for the student. Again, we're using learner mastery with the idea that the students that were there at the session completed that hour via clock time. They were all there in class and engaged for the hour. So for the students that made it up, you would wanna make sure they're treated equally and recorded an hour of instruction just like those that completed the class. So with software, we've sort of broken this down into three different approaches based on the software. Some software applications are pre-programmed for hours of instruction. That is some, you know, have it set where the student completes an hour on the online, or on the educational software. That software automatically records one hour of instruction. If the software has that advanced feature programmed in, obviously our suggestion is use that feature as implemented by the educational software. You know, just use what they give you, record the amount of time the educational software records into enterprise for official reporting. Again, use that fancy feature if it's provided for you. The second one isn't fancy, but relies on the publisher. That is the publisher doesn't program the hours of instruction into the software. But does provide some sort of software publisher best practice recommendation or whatever. If so, simply use that recommendation that the software publisher provides you much better to use their professional recommendation, save yourself some time so you don't need to do it yourself. The third one is if they provide you nothing. That is you have the software but there's nothing in the software, nothing in this instruction manual, nothing anywhere that provides you that guidance. So that's where you need to meet is that management team, assign a set amount of instruction for each lesson, each module, each activity. You know, your mileage will vary on exactly how you want to define the increments, but you need to meet as a team define that instruction and provide a set amount of instruction for the students, you know, based on how long it takes each student to complete that activity. So example one, the software records the amount of hours or amount of instruction into the software. Again, simply provided that total that's recorded by the educational software. So we're just saying we've got a 12 module, you know, software, the student completes lesson one of 12. The software records one hour for completing module one because that's how long they've already determined module one takes. So the software has recorded that into the software already. So simply record what that software module or what that educational software has recorded for you and mark that in TE as the students instruction allow example two, here's where they don't program it, but they do provide that test publishers recommendation. Then again, you don't have anything automatically recorded for you, but again, use that recommendation whenever the student completes that activity, just record that amount of time. In that case, piggybacking a little more here, a couple of asked this, I would generally say it could be either way depending on the dynamic of the software, but usually this would entail learner mastery, where like the situation where they don't provide you anything, you're defining what activities they're completing, module one or lesson two or whatever it might happen to be, but instead of you, the professional team at your agency needing to do the extra work to figure out time increments for each module, you're simply doing it the easy way where you're just peeling off the amount that the publisher tells you and just recording that in your local assessment policy, obviously way less work for you, obviously the best way to do it, the assumption being that publisher is gonna know better than anybody how long each of their own exercises is gonna take each student. Next example, so here's where they, you're using the software, but they don't provide anything. Here's where you need to meet as a team and define it locally. So there's no policy for exactly who needs to be included on this team, but we are definitely saying make it a team, not just one teacher or one administrator, we wanna have a team. So we're not putting anyone in the position of unilaterally determining this. It needs to be an agreed upon amount by a team, not one person. So again, same example, observe students, come up with an average, make sure that it's an agreed upon average and then simply document what that amount of time is in your agency's local assessment policy. You don't have to write a book, you don't have to write anything scientific, but we are definitely suggesting you should give at least a little information about how you observed it, why you came up with the answer you came up with. Next slide please. Here's just an example where we're just listing off the modules, providing an exact amount of time for each module under sometimes that might be the same, an hour for all 12 modules, a lot of times that might be a different amount of time, depending on the specific module or specific exercises, the student is completing, if you have a different amount of time for each, simply list it all in your assessment policy, specify the hours for each module and then record those hours in TE. You know what, in Penny's session yesterday, it talks a bit about providing some kind of assessment or exercises. What I'll add here is, in general, that's gonna be for these software activities where you're just expecting the student to go through it. For videos, I'd say that's really what you're gonna need to do because you don't really have activities. I'll add in these software modules, sometimes you can just use what the software gives you. Again, a lot of those educational software that has hours also has built-in assessment and built-in activities, that is the software won't allow the student to advance to the next module until that student completes all of those checkup activities to move on. If the student does that, you can use the checkup activities that the software provides instead of making up your own, just simply define that you're forcing the student to complete those activities and those activities that the publisher provides you is what you're using to check the student and to prove the student to move on to the next level. Okay, so general guidelines, contact your publisher for guidance on hours. Sometimes they provide a lot of information. Sometimes they provide nothing. If they provide nothing, we're still saying, contact your publisher, there's a good chance they can help you out anyway. Again, we talked a little bit about making sure it's a team approach and an average, not just the decree of one person at your agency. If you want to do it on an individual, by individual basis, again, that would be using teacher verification. Penny, Janice, Jay, we've all gone out of our way to say that's fine, it's allowed. If you think that works better, 100% A-OK, but we're saying it does require more work for you. Number one, it's more work on the teacher because that's suggesting that the teacher has a way to dutifully monitor all students. Through distance learning, obviously that's difficult. So there's a way in which you're videotaping the student while they're doing the distance learning so the teacher can observe, or perhaps you've got really, really detailed assessment or exercises that you're having the student do. That again, the teacher has some very detailed ways of monitoring what the student is or is not doing, but its bottom line is if you're doing teacher verification, it assumes that that exists somewhere and that it also requires you to document what sort of scenarios might require you to document higher low hours totals based on what the teacher observes. It also puts, so it puts more on the teacher to be very, very vigilant. It requires more on the management team to document that detailed process in your policy. Again, you've got to document that hour of instruction case by case basis. That's why the state and why we kind of suggest that learner mastery might work a little more easily because you're not generally observing the students in that much detail when you're doing distance learning is here for you to define that ahead of time and just check up at the end whether the student did or did not complete the requirements and document it that way, knowing that you're unlikely to be able to have that high level of observation. Either way, document the process and the local assessment policy. Next slide, I think we're switching gears here. So back up a slide, I'm sorry, I thought we were, but I couldn't see, sorry about that. But in any case, the one last thing I'll say is this is where part two comes in. We referenced part two a couple of weeks from now, May 5th, May 7th. This is what we're getting into. We didn't just make it up or please ourselves by this. We're definitely putting together part two related to this document everything in the local assessment policy because that's definitely been where everybody's questions have come in. So we've got at least one new session and potentially two. Of course, I'm biting off way more than I can chew as usual. So that's a little bit TBD, but here's where we'll get into part two and try to get into super concrete examples of what it looks like at the agency level, what types of instruction might relate to each module, kind of what Penny's done already to some extent, but taking it that next step of how does it look based on classroom instruction? How does it look based on distance learning approach? And then based on that, how do you take that next step to accurately document that approach and that accounting for hours and putting it in your local assessment policy? That's what everybody's been asking. It doesn't seem like a lot, but there's been lots and lots and lots and lots and lots of new material that keeps coming up. So anyway, a little bit on testing. It's the last section here. The short answer to tie it all neatly in a bow is that again, back to that Octay memo, it clearly leaves it to the states to make decisions. The California state decision is still in progress. It clearly leaves it to test publishers like Cossus to get the show on the road with this. I'll implore you to believe that we definitely have. There's a huge field test for remote testing that we started just yesterday. We haven't, I'll just informally characterize it as phase one, phase two, that is we've got a process at Cossus that we've already started that basically does the pilot testing for remote testing. We've got a phase one, phase two, phase three, phase one started yesterday. We're keeping it really, really cautious and conservative. That is it will include only five or six agencies nationwide. It'll include just one on one test administration. Again, we're starting very cautious, keeping the low hanging crude approach. We'll do that for two weeks, complete phase one at the end of next week around May 4th, start phase two and open up to a larger number of agencies and open it up to a larger number of distance learning options where we'll try to do a couple agencies with group administration, not restricting to one on one. We're a little unsure how long that will take, but hopefully it will be short and sweet like phase one. And then as we get into phase three, we'll kind of get more into simulating the way we do it in real time, do a little piloting kind of matching the way we do it in real time for a certain amount of time and then unveiling it to the public. But we'll be, we're starting phase one, we'll be doing a webinar that hopefully will include CASIS and CDE there around that first week of May as we start phase two of this pilot test. We've had some discussions with CDE on the ups and downs of what we should and what we should not allow related to payment points. So when we roll out this approach for remote testing, everybody will be 100% clear as to whether this is something that will be kind of planned for keep, so to speak, where yes, you will need to do this and you should keep payment points in mind as you do this versus yes, we've got all these wonderful options for remote testing and remote ELCIVIX, but it's not gonna be for payment points. It's more like a fire drill here. We're closing the book on payment points. I'm not saying I know which way CDE is gonna decide, but we admit that no matter what we do, that webinar will have a decidedly different tone one way versus the other way, we really need to get that decision 100% rectified before we can realistically do that webinar and expect it to not be completely sabotaged by that one question obviously. Okay, next slide please. So that's what I just described. You can move to the next one. So for now we're saying for pre-post testing, for placement testing and for ELCIVIX, an in-person proctor must be present for pre and post testing. We won't really have that changing until we complete those couple phases of the pilot. I'll add though for ELCIVIX for a few select co-apps as well as our citizenship preparation, citizenship interview test. We should have some more tangible guidelines here in a couple of weeks with that webinar. We've figured out some things kind of in the weeds where that really requires CDE to weigh in before we can really disseminate any of this. Just to be all convoluted with a typical J-right example, we know that ELCIVIX is different than pre and post testing. We know that theoretically ELCIVIX is a little bit lower bar, lower hanging fruit to do remotely than pre and post testing. However, if we're gonna unveil that is a great latest and greatest solution for collecting payment points given everything with remote instruction and remote testing, we realize that as is it's a terrible way to solve the problem because any ELCIVIX co-app payment point requires a pre-post pair to earn the payment point. So obviously if we're coming something to do that's a lot easier than doing pre and post testing in a remote format, there's a hundred percent chance of failure if we allow it for ELCIVIX if we don't make any additional policy decisions. So everywhere we turn those kind of things we're there ugly head. So we're hoping to be in a lot step between CDE and COSSIS before proceeding. So some of that relates to policies CDE needs to make at the state level on whether we're doing any of this for payment points. And some of that is on us at COSSIS to get a little bit further ahead on remote testing and remote ELCIVIX. So okay, let's move next slide. I'm not seeing anybody kick it to a curb. So again, for now it's still present in a couple of weeks. We'll be doing a webinar for placement. Same thing, we'll be covering this as well. But again, I'll just reiterate for now you can do things like oral interviews and writing samples and all those other things and lieu of COSSIS locators, COSSIS appraisals next. Again, here's lots of resources if you're having trouble with that next. For ELCIVIX again, we won't have solutions for all COSSIS, we'll be way, way, way short of that. But we do plan on having a few. Again, we're looking at COSSIS, a combination of popularity and appropriateness for COVID-19 and appropriateness for distance learning. We've got a short list of ones we will be offering. Again, we've got some things in mind for payment points and not payment points. The exact door we open and unveil for you on that one really depends on whether we need to use it for payment points. We have determined that if it's for payment points, we probably need to have a slightly higher bar than if it's not for payment points, so more to follow there. Next slide. And then again, for SIPPREP, we will not do government in history, but we will definitely offer citizenship interview. Citizenship interview will definitely be a prominent part of this webinar. I'm referencing for early May, the SIPP is an oral interview, so we feel strongly that that will be included as part of our quote-unquote battery for options for remote testing. We're not there yet, but we're very, very close. I'll just reiterate for EL Civics Instruction, we're not talking about co-apps or assessments, we're just talking about instruction, that of course you can do via distance learning. You don't need to wait for any webinar to get started on that. You've been able to do all along next. I'm running a little over. Obviously, my porridge was a little too hot last time, it's a little too cold this time, I guess. Again, I'm not gonna get into details, I've referenced it already, but in TE, you have the ability to record that DL hours versus regular hours. Again, there's an FAQ document that basically covers that issue already. Okay, next slide, I think we're about at the end. And again, okay, I thought I changed this, I changed everything, but not this one, but I'll just say we'll have more TE sessions in the future. There's a couple help documents that give you the details, so obviously if you attend this on April 2nd, it's not gonna help you much. It's not gonna help you much. Yeah, I see all that, Jennifer, that's gonna be covered. I've gotta say it's a lot, I've gotta be honest, it's a lot less of a problem than you characterize it. Yes, I'm gonna push back because I see the way you're characterizing it, I've gotta say it's more of a problem than it actually is. I've gotta provide that pushback because I know that's 125% true, but we will cover that, again, a lot of what you're talking about really depends on whether it's for payment points. If it's not for payment points, my pushback quite frankly on you is gonna be quite hard, that no, you don't really need to do that if your agency requires it, fine, but at the global level, sorry, I'm gonna provide pushback and say no, not necessarily, but again, that depends on whether it's gonna involve payment points, yes or no. But we definitely know some of that will need to be covered. It might be an issue of what we're doing now versus what we're doing started, but yes, that would be covered in the webinar.