 This is Anthony from OTAN. Welcome to this afternoon's webinar, Considerations When Implementing Distance Learning Part 2-Local Assessment Policy. The presentation slides are actually available right now on the OTAN website. So if you go to the OTAN website, otan.us, on the right hand side, if you're on a desktop or laptop, you want to look it for that COVID-19 field support button on the right hand side, if you're on a mobile device, you might have to scroll down a little bit. The COVID-19 field support button will take you to the COVID-19 field support page, and the slides for the presentation today are listed at the very top of that table. So you can go ahead and download those slides right now. And Jay Wright, I'm going to turn it over to you for today's presentation. Go ahead, Jay. Okay, great. So looking, there we go. Okay, there's the sharing. So here's our cover slide. We've got too many to worry about straw poles and so on, but this is what we're calling part two because there's that part one that we were just simply calling distance learning considerations that we've done collaboratively with the OTAN CASIS team four or five times now. So we did that a bunch starting there in late March when everybody was in quarantine mode. Obviously, lots and lots and lots and lots and lots of questions have come out of that to the point where we agreed yet would be good to piece some of that together and make a part two of this to start covering a lot of the things that came up during part one. A lot of that was related to the local assessment policy. I won't say it was restricted to that, but that was definitely the number one source of questions related to part one. So for the last few weeks, we've been sort of taking note of that and putting together this part two. You might say another funny thing happened on the way to the forum with this just like all things COVID-19 since mid-March, you might say, where things keep changing. In this case, the same basic approach is holding fast. But as we were saying amongst us CASIS OTAN folks before this, this has turned into a little bit of a transition from nice to do to need to do, meaning that we really developed this over the last couple of weeks with the idea that you had lots of really good questions and particular questions about how to address your local assessment policy. So knowing this was coming and knowing it was definitely best practice, we were crafting this presentation to provide tips and more in particular, lots of specific examples of ways you can craft it in your policy. The reason why I say need to do is you've heard a lot of you, I'm sure, have been to those CASIS workshops on remote testing and so on. You've heard a lot of those updates on the federal memos and so on. So it's kind of gone more from nice to do to need to do. That is the feds now require states to submit an addendum to the statewide assessment policy if they wish to do remote testing to be clear. California is definitely opting with yes on that. So thereby as a state, we're now required to do an addendum to our statewide policy because we're selecting yes now you as local agencies that decide to do remote testing will need to do an addendum to your local assessment policy. So that's why I say we've transitioned this literally in the last two or three days from what I would call nice to do to need to do. So it now includes some of those specific issues related to how we're going to deal with this as a state. I'll just hope say hopefully that makes at least a little bit of sense. I know it doesn't make a lot of better sense but hopefully a little bit. So in any case the agenda that's what I mean by updates will get into a few updates that have occurred again. We're moving fast at the state level. We're moving fast at the federal level. It's only been a couple weeks since we've done this presentation but that's how it's taken for things to have changed in terms of rules and regs and all that wonderful stuff. Because of that I'll recap a little bit on part one sort of talk about the updates and how that's affected things here. Part of the recap is a big part of part one of course was looking at those different ways in which we can assign instructional hours to distance learning. So I'll recap that and how that will affect what we need to do with the local assessment policy because most people quite frankly even if you say you know all about the local assessment policy sorry I know too well that the overwhelming majority of folks know little or nothing about the local assessment policy. Lots of things we've unearthed about that that's just reinforced that opinion. So we're going to go ahead and provide a brief overview on exactly what those things are to make sure everybody knows what we're talking about when we talk about modifying the policy if you don't know what the statewide policy is you don't know what the local assessment policy is obviously expending a lot of hot air on how you need to modify those things is not going to be much help related to this there's an appendix c distance learning no that is not new appendix c has existed for at least 12 years since at least the 0809 program year I think it dates back to 06 if I'm not mistaken but we've had an a special appendix c in our statewide assessment policy specifically to address distance learning for that long if you don't didn't know that don't worry you're in a very big club just about everybody didn't know that but there is an appendix c that's been around for a while that's done collected a lot of dust over the years quite frankly we spent a lot of time unearthing it the last couple months it's worth pointing out that it does exist and it does cover all those things we've been talking about now that the feds have moved on with these new memos and all these issues related to distance learning there's a new addendum that hasn't existed yet but we'll need to begin to exist starting here in 2019 2020 we'll talk about that as well and then at the end we probably won't have time to go into all these gory details but when you download the powerpoint when you start working on this on your own the thinking is there's lots of examples we add in here I want to say there's like eight or nine examples at the end again not enough time to cover each one but it should be pretty self-explanatory when you download the powerpoint and review the slides there's different examples based on what learning model you're using a lot of those examples that Penny gave in her presentation or elaborated upon here for accountability purposes and specified in terms of how you would do it in terms of local assessment policy so I'll just pause hopefully that sounds more or less like what everybody was expecting to get themselves into here this afternoon next please okay so here's the first slide of part one because it's so darn important it's the first slide of part two COVID-19's effect on agencies of course that's comprehensive we've all been affected by COVID-19 everybody's closed we're all in quarantine mode we've talked about the resource page that OTAN has that's been our official COVID-19 hub since about Friday the 13th in March it's got all kinds of FAQ documents official proclamations from CDE that were all affected by this that we're all looking at the same page FAQ documents from CASAS FAQ documents from CalPro and OTAN all kinds of health documents all kinds of information on all things COVID-19 I think we know that about that already here's the link at the bottom of the slide we'll move on to the next so here's a little bit of an update we've talked extensively about memo 20-3 that is the big memo that came out at the end of March again we went into quarantine mode mid-march you know at the end of March the feds came out with that memo where they went out of the way to provide all the additional guidance that all states were looking for given that we were all in COVID-19 quarantine mode that basically uh you know basically proclaimed from on high that yes COVID-19 is a really really big deal but that doesn't mean we're going to stop collecting data we're still collecting data we're still reporting at the state level we're still reporting at the federal level just because COVID-19 is a really big deal doesn't mean we're changing that but it at the same time gives states a lot more increased flexibility on how to handle on how to handle testing how to handle outcomes on whether you know to you know on how to handle instruction you know we all did it a slightly different way that basically loosened things up so all states could kind of find their own way and not worry about whether that's in violation of federal policy so uh 20-4 came out a few weeks later it basically just followed up over as simple as 20-4 simply up the ante on 20-3 again we're still collecting data but we know we're doing it in a very modified way it increased the flexibility to the states to number one implement remote testing because it seems like eons ago but when we started this our answer was we can't do remote testing because the feds say so the federal guidelines were really clear that all pre-post testing required a face-to-face proctor so they've loosened that twice now they loosened it up so each state can decide on its own whether to do remote testing but it kind of up the ante on it you know can you hear me better now here i'll turn it i'll turn this i'll uh turn my head differently so any case it up the ante more flexibility to states it basically said if you're doing remote testing however that means you need to submit an addendum to your statewide assessment policy detailing how your approach is going to be at the same time that also means agency is doing remote testing and term need to look update its local assessment policy so to be clear each state gets to decide whether it is or is not doing remote testing i'm sure a lot of you were in those webinars yesterday where it was very clear that cde has selected the box labeled yes for california meaning yes california is going in on remote testing as you heard yesterday we're not requiring all all agencies all students to do remote testing but we are encouraging to the maximum extent possible agencies and students that are able to do so to the maximum extent possible do so so we've chosen that option i'll just add one last thing that if we're exempting students from testing we have to have a detailed timeline for how we're going about moving away from that if we chose to do it that way and we also have added that force measure check box and te to identify those students that either on the state level or the agency level we're exempting from testing and or may be impacted somehow some way by covid-19 all right so moving to remote testing again this is not the workshop where we're doing that we had two very large workshops yesterday that detailed that this is more for policy and accountability but we did have two big webinars yesterday on remote testing so again cde has decided that california has selected the box label yes we are implementing remote testing so to be clear cde is drafting a document we referenced it several times in yesterday's webinars it's not ready yet it's not posted but it will be posted later this week where there will be an official cde document that will basically serve as the addendum to the statewide policy on how we're handling distance learning more specifically how we're handling remote testing in california for the 1920 year we're doing that because the fed said so so in that cde document they're basically approving remote testing but it's saying if agencies choose this they must have procedures in place to ensure number one the student can be identified number two any approved test that's administered is properly secured and number three the remote proctor can properly administer the test that seems simple i'll just say that's the oversimplified way of basically accounting for what we spent an hour two times yesterday detailing and that remote testing workshop that's just the simplified account of our hour-long account that we provided yesterday where we got in the details on how you can make it secure how you can identify that student and so on this is just simply stating that we as the state are saying it's okay as long as you can follow these steps so we provided that webinar at CASAS to give you the details on how we think everybody should be able to do that next slide please okay here's the details i'm not going to read this word for word but again here's the more detailed guidelines that you'll see in that cde policy in this case i really just did a copy and paste so this is not anywhere near the whole document but for what it's worth yes this is a copy and paste from that official cde document on remote testing so right now it's just saying CASAS GD and high set are the only ones that you're allowed to do this way you need to to uh complete that remote testing authorization so there is an agreement again that's something that we at CASAS are still working on we will make it available no later than May 11th it's still in process now if it's available a little earlier we might post a little earlier but May 11th is the date we're promising so i'm not going to give a date any earlier than that but this will be a probably a short little one or two page document where each agency is basically throwing its hat in the ring indicating that it's doing remote testing indicating that you're following all those additional security guidelines indicating that you've got the technology at the agency and student level specifying which CASAS assessments you're doing all these things that you're you're signing your life away saying you're taking these precautions signing off on the copy submitting a copy to cde submitting a copy to CASAS to basically have you be good to go to be clear this is not a training you're completing this is a two or three page document you're completing basically signing off that you're following on all the guidelines you'll have a copy locally that you'll have handy if you're monitored for any reason at least for this year because it's such a big deal for this year you'll submit a copy to your cde consultant you'll also submit a copy to your CASAS specialist that's for this year only i'm not sure that that will be a requirement every year but for 1920 while we're still in quarantine mode you've got to submit not one but two separate copies to make sure we're all aware that you're doing this because we're doing the full court press on getting everybody up and running and the future my guess is you'll continue to need to do this form but you may not need to submit copies every year it'll probably settle into being something that you'll need to retain locally like the rest of your local assessment policy but if you're monitored you want to have it available and ready for show okay and then again you need to have the training records bottom line is there's not a special training that you're going to be required to complete for this but you do need to have anybody that does this be somebody who's already completed that proctor training and you need to have the documentation that shows that person already has completed that proctor training and then again you need to submit a written standard you need to incorporate a written standard procedure for how you're doing remote testing and add it as an addendum to your local assessment policy to be clear that item under five is not something that you need to submit to cost us or cde but it is something you need to complete before getting involved in remote testing it is something you should have on hand if you're ever monitored for any reason so i won't get into all this but this is a slide from yesterday my guess is most of you recognize it because you were in one of those trainings yesterday if not both we had phase one that started a couple weeks ago it ended last friday we started phase two this monday we started very cautious we only have like five or six agencies nationwide we limited it to just one on one student student to staff testing so again by one to one testing we mean we did it by doing it remotely it's just one staff test proctor and one student completing testing it's a one to one ratio that phase one was restricted to one to one ratio now that we're in phase two we opened it up to a larger number of agencies and as you heard yesterday we opened it up to where you can continue to do that one to one ratio but in phase two you also have that option to do the one to five ratio where that test proctor is monitoring five students at a time instead of just one again we're staying pretty cautious with this but again it's lots of issues have come up so we need to keep it very cautious you know keep it slow that's been our approach on this we know that we need to go that way so we're staying one to one and one to five i'll add we also added government and history for el sevix we started adding new kinds of technology we're going to stay with phase two eventually we'll hit phase three and we'll start looking at the additional options again we kind of uh fell off the grid with that because a lot of those things really depend on how things proceed with phase two next slide please so this one i don't need to talk about a lot but this is just a screenshot on how it looks again students student and staff are on camera on zoom student is able to see staff staff is able to see student that's a big part of the test security on how we feel like we can get away with doing it remotely we can do it remotely but only if we can view what all the students are doing in vc next slide here's a copy of that agreement again this is still a work in progress should be available the next few days again no later than may 11 but again all agencies that ought to do remote testing will need to complete this remote testing agreement to be clear this is the one that you've got to do and you've got to complete and you've got to submit to both cde and cossus before getting started you can see here it basically it's pretty basic it's not rocket science at all you're just giving the name of your agency you know who's giving consent who the person is signing their name signing their life away specifying which assessments you're using and which ones you're doing remotely specifying which remote techniques you're doing are you doing all one to one testing are you dabbling into one to five testing which specific technology platforms are you using obviously you need to have this thought out before you just dive in so by forcing you to do the agreement first and then jump into remote testing it ensures that you're giving thought to the remote testing uh procedure the technology the assessments all of those things you need to think through before you do remote testing by addressing these basic decisions first it guarantees that you did exactly that before doing it next slide please and okay so i think that's all i wanted to cover on remote testing again the details related to security and id and the whole pilot testing process was obviously covered in more detail yesterday so we're kind of moving on but sticking with the topic that was covered yesterday that's that force majeure so again by force majeure that's the term the feds are using to refer to those students that were impacted by COVID-19 in particular it's been talking more and more about students that are exempt from pre and post testing because they're unable to do so due to COVID-19 so you've heard about this a little bit because it's been in te for a while we programmed it in te back on april 15th so we've got the check box in a few different places in te and again it's been there for a few weeks but we've gone way out of our way it costs us and same for cde to say don't mark anything related to this yet the cde is still completing its policy on when to mark things for force majeure so some of this has been as advertised for a few weeks what we've been saying is there's been different interpretations on exactly what that means some would say well of course everybody's been affected by COVID-19 so if we're just saying who's affected by COVID-19 by definition that means everybody so we need to check everybody others are saying it more specifically needs to be just the people that got kicked out of class others are saying it's more specific to testing so again cde is making a very specific policy for california that will clarify exactly what needs to happen at the agency or student level in order for you to check that box so that will also be available as part of the statewide policy here in a few days so again that's what we need there's a checkbox in te already another bullet that i put here i did this with some trepidation by putting this bullet on this slide but there really was no better place to put it way tmi but i'm saying all this tmi to hopefully use that as a way to circumvent the confusion between the force majeure checkbox that already exists in te and the 125 000 separate checkbox i'll repeat again because i know even though i say this people are still going to get confused even though i save this in as a heavy-handed way as possible completely separate checkbox from force majeure force majeure exists in te the second checkbox i'm talking about totally does not exist in te so as everybody exceedingly clear i know i'm making a fool of myself here to avoid all these silly questions but i'm gonna get all the silly questions but i'm huffing and puffing to try to avoid the silly questions two separate checkboxes in te totally not the same thing does everybody follow that okay because you know everybody's going to ask this question anyway right and get the checkbox confused it's the same thing even though i'm making a fool of myself right now right so anyway two separate checkboxes force majeure in te for those who are exempt from testing uh you know because of COVID-19 or potentially exempt from classes or exempt from services for that same reason the second checkbox relates more specifically to remote testing and it will be the box that you check for those students then do complete remote testing so there's a checkbox that's there that you mark for those that are exempt a separate one you'll mark for specifically for those students that complete remote testing most likely the second one will be in the test record but it will be something that you check for those specific students that remote that did remote testing most of you have pointed out a million completely verifiable understandable good reasons for why there's no way on god's green earth you'll be able to remote test everybody but hey as long as it's understood you're just doing it for some and not all hey you'll get on board with this you're all for it so again that means that we're gonna need to specify which specific tests which specific students those were that did remote testing so that's a checkbox that does not exist in te yet so don't go looking for it because there's 125 percent chance you won't find it because again it ain't there but it will be coming very shortly where you'll mark those specific students that do remote testing so enough puffing and puffing in that issue go ahead jay sure i've got two things one to share is um this morning i got a phone call from a consortium manager and she said that she was she was looking at some of her data and she noticed that one of her agencies had very aggressively gone in there and marked force majeure for a whole bunch of their students who were um it's kind of impacted so the advice back to her was please advise your agency go into the student in program years records you can grab them do a bulk batch edit and go ahead and remove that flag right because no problem right right no harm no three assessment i'll just say no harm no foul it's not a capital offense if you mark it anyway we're just trying to have some standardization so yes you can uncheck it just like you checked it i'll just say no harm no foul good and then jill had a question i left it open in the question box i wanted you to kind of look at that who says she says um did jay say that we also have to submit the local assessment policy with addendum to cde and casas as well as the remote testing agreement no okay good question i saw that coming i thought i cleared it but maybe not so yes that agreement that's just you know where you're checking off your technology and checking off which casas assessments that one you do need to submit to both cde and casas the addendum to your local assessment policy you don't need to submit at all yes you should do it but no you don't need to submit it i'll just say the policy for that of course is exactly the same as it is for the local assessment policy in general that is of course you need to do it but no you don't need to submit it anywhere definition of distance learning this is straight from pennies presentation again any learning activity where students and teachers are separated by geography time or both for the majority of the period there's all the details i'm not going to read that but if you want it there's the page in the nrs implementation guidelines that give you that information next slide so here's more review from part one again and generally speaking uh dl learners have had the same requirements as everybody else for since 2009 we followed the 50 percent rule simply meaning more than 50 percent just you know uh in a distance environment mark distance learning less than 50 percent in a distance environment don't mark distance learning and then again there's the new feature that tracks uh you know activities affected by COVID-19 called force major just talked about that next slide okay more review from part one here's the three models i went over in a little bit penny went over in a lot of bit but those three models in the nrs guidelines clock time teacher verification learner mastery and review clock time is simply measuring hours based on the clock that is if they're spending 30 minutes in the regular classroom that means exactly the same thing as 30 minutes in distance learning if you're using that approach that's using the clock time model the other two teacher verification learner mastery that's more for asynchronous where you've got things like software uh you know educational software or videos or zoom recordings or websites or whatever you're not necessarily with a facilitator you're having the student do it on their own so teacher verification is marking hours by having the teacher be be uh directly engaged and directly observing each student and having the teacher determine on her own or his or own exactly how many hours of instruction each student deserves learner mastery is the one we've recommended in California since about 2008 2009 that's suggesting that we go ahead of time look at that software look at that video or website or whatever assign a specific number of hours to that application assign it ahead of time and then that way you can mark the same amount of instruction equally to all students based on the fact that they completed that particular module in penny's presentation she talked about testing gotta say to be clear in our california assessment policy having a formal test is not necessarily required but that is one way you can do it the more common way is in some of the software applications you're using usually they have a built-in test already done module for module so most agencies simply use that test that the software uh that the educational software provides for you by the student completing those exercises or that test in the educational software each unit or lesson by completing that that demonstrates mastery you can use that as long as you recorded as such and have students complete what's already built in and the software to serve is the way that they demonstrate that mastery next slide okay then this is more review facilitated synchronous we're saying use clock time generally speaking asynchronous again recorded sessions videos educational software historically we said use learner mastery and then this last bullet is kind of what we've done on the fly where most of you are using software we've come up with those three general approaches next slide please so again if they uh with the software this is all in review some software are higher and they're a little fancier they have the amount of time programmed into that back end if you're using software that programs the amount of hours into the software already well then the obvious thing is use what the software gives you then you don't need to do any of these models because the software does it for you take what the software publisher gives you if you've got software where it's not programmed but sometimes it won't be any fancy programming but they will give you recommended time frames then again give what the software publisher take what the software publisher gives you use the recommendation provided by that software publisher again by doing that you're on firm ground by accepting what the publisher says then you don't need to have any extra responsibility on the teacher or the management team to do those extra steps number three is where you're kind of stuck where you might need to take those extra steps if there's if you have nothing in terms of recommendations or programming from the software then that's where we're most likely to recommend learner mastery that's where you need to look at it as a team and kind of plan that out ahead of time and specify how many hours you're giving the student for each module next please okay here's the general guidelines the first one hey if you don't get anything from that publisher a lot of times really good idea to do it anyway usually they're going to have some information no matter what they have good to get whatever information you can from that publisher if you have to do it on your own again fewer and fewer having to do this but if you have to again don't have it be one person it doesn't have to be anything that excruciatingly detailed but you should at least have more than one person be on the team so it can at least be an agreement and not be the opinion of just one person that another one that's come up a lot with the questions trial and error is allowed I think I've used this cliche already today but again when you're doing this as an agency totally understood you're not going to get it perfect the first time so don't worry about getting it perfect something is better than nothing if you need to tweak it later tweak it later again trial and error is totally allowed if you decide learner mastery is not your bag and you'd rather put it on the teacher that's a hundred percent permissible but if so be sure to document it in your policy if so be sure to you know you know that means the teacher has you know a lot more on her or his plate where that does suggest that the teacher is being very vigilant student by student is able it is so bit vigilant that she or he is able to differentiate how much time student X put into this versus how much time student Y did on this and by observation is able to make that level of granularity and then again if so describe that in your policy next slide okay so getting into the policy to do the assessment policy here uh again over and over and over again already you've discerned this was true in part one the same tired answer for a lot of this document it in your agency's local assessment policy I put the link here to that attachment B in our admin manual attachment B for many years has been the local assessment policy I'll just say if you don't know what I'm talking about when I use that term here's the link to attachment B that's been the local assessment policy template every year since 2005 next slide please here's what the local here's so again we've got statewide policy and local assessment policy just a couple things about the cde statewide policy if you have no idea what the local assessment policy is sorry but highly likely you have no idea what the statewide policy is either so just to make sure everybody's on the same page the cde statewide assessment policy 20 or 30 page document existed every year since 2005 statewide policies on pre and post testing again recommendations for pops it placement testing 40 hours rule requiring you to do the local assessment and so on all those things you know and then the down valid issue that's always been there that nobody really paid much attention to is again that policy for assessing and reporting distance learning next slide for the local assessment again just can description here as well again part of the statewide policy for 15 years now is you know we've got the statewide policy all local we owe a two agencies are required to develop its own local assessment policy and updated at the start of each program year so again it's always been flexible on exactly what the dos and don'ts are as long as you address all those issues specifically addressed and that cde statewide policy and then again you've never had to submit this anywhere the policy has always been you need to do it every year but the way you reported is through your fpm visits those there's no formal annual reporting or deliverable you necessarily need to complete next slide so i won't get into this this is uh you know this is going to be for a future when we do where we really roll up our sleeves on local assessment policy but a big long laundry list of things a good local assessment policy shouldn't tail so that last slide is just telling you yeah as long as you address what cd because cde says so you're basically fine but over the years we've obviously said a lot more things than a good policy really includes but it's going to get into more detail on how you handle placement pre and post testing but if you're a good agency you're maintaining a calendar for the whole year that includes testing dates and strategies for makeup dates for those who are answered you get into details on test security specify which cost us tests you prefer get into how you're addressing regional priorities next slide get more into hours between tests 40 hours rule training and so on and then you're also addressing some you know again it's kind of gets more in the weeds the part that you probably think about less but we're starting to think about you know also how do we provide accommodations for disabilities and there it is again what are our procedures for learners and distance learning next slide please so here's the sections i won't read off on this but again here in a week or two we'll follow up and get more into local assessment policy and detail it for now these are the sections that we have in our local assessment policy template these are the sections you would address in any good local assessment policy i'll just add there's the part of this that nobody knows about appendix c for distance learning and then there's the even there's the part that you know about even less because it doesn't exist yet where there'll be that new appendix for program year 2019-20 on how you're going to do remote testing so next slide please i'm also going to use this as a little bit of a sanity check did everybody follow that transition from generic stuff to more specific stuff as it relates to COVID-19 and distance learning just making sure okay that's your story and you're sticking to it thank you okay so again other issues here again these are not necessarily required it kind of depends on what you're doing at your agency but again accommodations EL civics any security that might be special to your agency differentiation between e-test and pencil paper if you've got agreements with your we yellow one partner okay consortia whatever you know and then again here's that in red that additional issue of distance learning which is another thing that you might hypothetically need to add it was hypothetical for most of us up through end of February now obviously for all of us it's not so hypothetical next slide please okay so here's appendix C i probably should have shown a screenshot of this i'm not sure if i have one or not so sorry if i don't but if you download that statewide assessment policy and that link i provided a few slides ago you'll get that statewide assessment policy template it's been at the same location since 2005 yes we do update it every year but by and large year to year the similarities are much greater than the differences but as part of that policy again the part of this that nobody ever knew about including cd and casus you might say we've been doing a lot of dusting here the last couple months is since 2008 9 at least there's been a section of this statewide policy called appendix c distance learning that provides our statewide policy related to distance learning nobody realizes we had this but the bottom line is yes we did we already had very specific statewide policies on distance learning the fact is however that nobody realizes they existed yet so just to get into this a little bit because we're all worried about distance learning appendix c is the part of the statewide policy that addresses distance learning issues it really and truly does address these issues and then it has three or four sections not three one two three four bullets not three definition of distance learning testing curricula instructional hours next slide so definition of distance learning is what we've said already it talks about the 50 percent rule now in this year's policy you're going to want to describe how you've transitioned to distance learning again we're talking you know you really need to make it more like an appendix d for that but the definition of distance learners is the same next slide please and then for testing again this is kind of wavering this is where it's changed a little bit but it's talking about distance learning must uh complete the same requirements as everybody else you specify the process for how you do it now you know we're going to have that addendum for now we're probably going to leave this alone in appendix c but we will have that addendum again at the state level they're coming up with an addendum for this for 1920 at the local level you'll come up with an addendum where you're specifying specifically what you're doing are you doing pre-post are you doing el civics etc etc for 1920 uh i think what we're saying is don't worry about writing a book just write what you need to do at a bare minimum to move forward with remote testing if you fought further than this yes that's a really good idea yes no i don't know what i'm talking about but yes i really see this as being something we'll need to do every year from this point forward for all the ridiculously obvious reasons next slide curricula again this is where you're specifying by curricula are you using educational software specific videos specific websites specific platforms here's where you're specifying which technology you're using successfully in order to implement distance learning and then here's where we're talking about hours this is the point of the area where we've really been talking about it we've been spending most of the time talking about this i really think because this is where most of the questions are the basics on testing and curricula well you know especially when we didn't really have remote testing pretty straightforward really not you know generating that many questions but the part on hours has always been the trickiest part so that's what we've talked about the most because it's the most confusing but again here's the quote where again this is all but it's still basically the policy the cde promotes a rigorous outcome based learner mastery model i'll just say here's why i always say you could use any model but learner mastery is the one we've always recommended in california if nothing else because appendix c of the statewide policy says so and has said so since at least 2008 and so again with designated levels for program completion verified by a process that may include teacher or external certification i'll just say that's why i say the test or the percentage they get correct if you want to do that great but is it required so sorry i'm going out of my way to say no because it's so explicit in the policy that it may include that it definitely doesn't require that quite frankly it goes out of that out of their way to make that an option on the part of the agency next slide okay and then for hours again it's describing methods for delivering distance learning models used to record hours designated a group from the appropriating area that's that you know team you know discussion for learner mastery we've been describing any additional activities included so that's been a big one where hey maybe the idea is mostly we're going into the software and completing modules but we know that there's a little bit of instruction that's taking place outside of that we really want to record that we really want to give the student credit for that again that's fine you but you do need to identify what that is in your policy and then you can use multiple methods the policy has always said that's fine again that's fine if you combine them use them separately whatever you think is best all of those answers are yes yes yes yes yes yes yes yes yes as long as you do to flea document them in your local policy next slide please okay then again here in 1920 here's what we're here's a little bit of need to do here again you know for 1920 we'll have that addendum so to be clear here's where i'm giving you tmi i'm telling you policies that don't exist yeah we've had appendixie many many years appendixie is what we are what we're gonna do is we're gonna make an addendum appendixie that basically you know has a paragraph or so stating the obvious that so anyway for 1920 you know we're going to include that extra information and then there'll be an appendix d that specifically for 1920 and specifically for how you're undergoing remote testing if i were a betting man i would say it will probably become a permanent part of everybody's policy and everybody's statewide policy that will need to have that specific issue related to remote testing for the sake of argument i'm going to call that appendix d so again we've got appendix c that addresses all those basics of how we're delivering distance learning and then appendix d is the more it's more specific component that addresses remote testing in particular okay next slide please so and the addendum we're gonna here's the this is straight from the cde policy incorporate a written procedure on remote testing security how do you identify the student how are you pre pre screening the student to verify identity again lots of questions came up yesterday it'll be a little bit different at every agency what are your technology tools technology platforms and so on what publish your guidance is there for the platforms and then in particular this is i we kind of addressed this yesterday i think all this should be clear if you were in the trainings yesterday but also you'll need to address what happens if somebody undergoes technical difficulty there needs to be a plan where you're the student and the staff both have have a step they know they can do in the event of technical difficulty if you haven't looked at that that's the part you really need to look at because the state is very concerned about that they're definitely going to add that as a required step for any distance learning addendum related to the local assessment policy okay and then the remote addendum should address these issues again this is kind of the same thing security technology tools orientation again plan to respond to technical issues next slide sorry that was unnecessary so security and integrity process for registering computers testing and maintaining a secure environment you know ensuring the platforms you're using are secure pre screening steps for how you're verifying student identity what ideas you're checking how are you putting the camera on the student are you taking pictures are you just putting the video camera on it what are you putting in place to make sure you know that student is the correct student certification or trainings to administer all that is is put listing just like you would in the regular policy which staff have completed that proctor training which staff have you designated for doing remote testing and then orientation what's the process for re uh orienting your students on how to do remote test delivery you know obviously you can't just throw the student in there and expect him or her to do it you've got to have a way where you're working with the student a little bit so the student like you knows what to expect the student like you is comfortable in that situation you'll need to orientate each student a little bit to expect that to happen in any way shape or form next slide technology so again here you're just specifying which specific technology platforms you're using we're a little bit limited now but we're going to be expanded even with our limited list everybody's going to be a little bit different probably and then how to respond to technical issues i'll just repeat again i feel like all of these are ones we've talked about a lot but this last one i pause i'm less sure about this last one that we haven't talked a lot about this so i'll go out of my way to point that out to say if you haven't thought about what it is you're going to do in the event of technical difficulty the state is saying yeah you absolutely need to start thinking about it like right now before you implement testing yes you do need to think about that and outline what your steps are if you get any sort of technical problems what are you going to do in response to that what is the student going to do in response to that and make sure before any staff or students step set spot in the remote testing doorway that you already know what the procedure is for any sort of technical difficulty next slide and then general guidance again models how are you going to employ i'll just say this looks the same written a little bit differently this is the more nice to do that we you know included when we were really looking at this training being more of a nice to do related to all those issues we presented in part one i left it here because these are good steps but they're not the official state guidelines so probably a little less important but here's another series of things you should consider it has nothing to do with remote testing but it does relate to all those issues we talked about in part one related to class setup class creation proxy hours you know distance learning hours models you know and so on all the different things we talked about in part one still really good idea to the maximum extent possible to put that in your policy next slide i think we're here moving to the transition i think that's the last one next slide please yes good all right so it's 201 and we're here into the examples i'm if you can just slowly if you don't mind going slide by slide i want to show as it kind of a proof in the pudding that these do exist but here's a clock time model example just stop here for a minute so we've got example one example two so we've got a you know i think three different clock time model examples i won't read it word about at the end but again we provided very specific word for word examples that you can use and lift and copy and paste into your local assessment policy so we have not one not two but next slide please oh well i thought we had a third one for clock time oh well my bad so we've got two clock time model examples and one teacher verification examples i think if you go to the next slide we've got not one not two but three examples on the learner mastery example yeah there i was actually right so so yeah let's start with example one because i made a big fuss about it so again two clock time examples one teacher verification again this is a little bit reading from the slide so i've got to say if we didn't read from the slide i wouldn't really have seen it as that big a deal because really you're gonna have a lot more success reading and doing this on your own than just reading it off or whatever but again here's a clock time model we're just saying everybody's in the same ESL class they attend for an hour on Tuesday and Thursday this is one of those we're an hour the distance learning is exactly the same as an hour in the regular class classroom same bat time different bat channel i guess you might say but same teacher and same student same bat time anyway so that hour that they're spending on Tuesday morning same as the hour they spent on Tuesday morning when they went to your school they're just doing it on zoom instead of in person so everything's basically done the same so i'm still not going to read word for word in the example sorry not going to do that but here's your word for word example on how you can type you all can read so no i won't read it for you you can all read word for word in copy and paste that into your local assessment policy you know i've just left it left it blank here where you can populate it with google chat or zoom or canvas or whatever it is you're using as the software platform to gather all your students next slide so here's a second example okay let's back up again we've got time so let's look at example number two there we go so they're using software that records the correct number of hours for each student so totally different scenario but another example where we might recommend clock time so the most obvious is that one example one where you're just having students meet via zoom or whatever for the same hour as they would if they met in person but here's a second situation where you'd use clock time so it's by model so the first one is the synchronous model here's an asynchronous where you've got the fancy software that records hours that would be another one where you might use clock time because again the software records the amount of time the student has put in the software so this would be another one where we use clock time model where the software is automatically recording the time the student spends on the software recording it in that software program so again the easy way is to use clock time for this example where you're just taking what the software gives you and recording what the software gives you into directly into te next slide please here's teacher verification we're here for teacher verification we're suggesting 20 students are using software that does not record hours and does not include any sort of additional exercises at the end of each module to confirm understanding so this might be a scenario where you want to use that teacher verification again this is the one where through teacher observation you're using you're putting it all on the teacher to decide student x gets this amount of time student y gets a different amount of time student z gets yet another amount of time where the teacher is dutifully and diligently observing each student in a distance learning environment so that's kind of why we're kind of you know skeptical scotty on why we don't think this is really gonna work that well but again if you know the teacher is able to view all the students and observe all the students and come up with good answers for each student in terms of hours then go ahead knock yourself out use the teacher verification model and this would be the example for that we're based on teacher observation she or he is recording the appropriate number of hours for each student again i'm not going to read the example but here you go below that in italics learner mastery number one here again 20 students using software it does not record hours but it does include some of those exercises for understanding at the end of each module so again in this case we're saying uh you know once again you're gonna need to use learner mastery because it's not recording hours so yes you will need to meet as a team uh you know and identify the number of hours but no you won't necessarily need to create any kind of activity for them to do because the software is dutifully doing that for you so here in your assessment policy you are coming up where you did come up with the method for recording hours so you can apply the same method for each student equally but you allow the software publisher to provide those exercises at the end of the lesson and your policy is just pointing your agency pointing the teachers and the students to those module exercises and suggesting if the student successfully gets through the exercises that the publisher provided for you that's sufficient for the student demonstrating learner mastery and moving on to the next module and getting credit for those hours in that module the student just completed next slide throwing a monkey wrench here's another example using learner mastery example number two where 20 students are enrolled in an online esl class for an hour Tuesday Thursday via zoom again totally different example most students attend as schedule but a few students who are unable to attend access the recorded session so in this example we're using clock time for the majority of the students but a few students couldn't make it Tuesday at 10 a.m so they're viewing the recording for those that made it Tuesday at 10 a.m we're basically going to use clock time but we had a few students that didn't make it Tuesday at 10 a.m so for those students we're going to need to use learner mastery so we add in our policy the classes that meet two hours a week you know use you know the clock time for students that can't make it at the advertised time we give them the option of accessing the recording if they successfully do that we provide a few questions they need to answer at the end if they get those questions right we'll credit them with the x hours of instruction so again you simply specify what we want to do for those that make it at the right time we have different specifications for those that don't make it at the right time I'll just add in part one we went way out of our way to say it's up to you as an agency to decide whether to allow that some of you say yeah we really need to allow it or we'll never make it our students will never make it others have said have noted that can be kind of a slippery slope to allow that so if you've decided that's too much of a slippery slope go ahead knock yourself out and disallow it next slide please so here's one more learner mastery example this example they're an independent study classes and they're watching assigned youtube videos as part of the required instruction so in this case again we're detailing how they do that so here's one what we're saying yeah there's not really any kind of confirmation for understanding there's no way we can really use the youtube videos at all and know whether they watched them or not so this is one where we really need to do a little extra effort on the agency's part and we need to specify exactly which videos they're completing and then once they complete the videos we give them a quiz and they need to pass the quiz with a certain percentage because there's no other way we have any way of knowing whether they did or did not watch said videos next slide please and then finally a lot of you have brought up can we use mixed delivery methods the answer is yes but yes you do need to specify if you use mixed methods it does make it a little more complicated but just a little bit so here's an example using mixed methods where we're saying they're using software it doesn't record hours doesn't include exercises it's real basic bare bones you know so we say some participate in a live session where we're all working on the software is one big happy family but similar to that other example if you aren't able to make it at the right time so they do it more on her own or his own so here we're saying you know where we're saying for some students that make the live session we're going to just give them clock time hey we don't care if they did a lot of work or a little work they sat in and they were there for the whole hour while the teacher was training so they get an hour of instruction just like they would for regular but if they miss it then we're going to apply learner mastery because the teacher isn't there and we can't really verify what the student is or is not doing so if they don't make it at that time we're going to apply learner mastery so here's an example where for the same students in same class we're employing two different models and we're specifying which circumstances use one model versus the other set of circumstances that use the other model next slide please and then here's another mixed delivery method we're using the software for up to six hours a week they're logging in you know there's sometimes when we're going to use teacher verification so same sort of thing as the other example where some students are there online at the same time as the teacher so the teacher is able to observe and evaluate each student other times students might be online at a different time from the teacher so thereby of course the teacher is unable to make those evaluations so in this example we're saying up to the point that the teacher is able the teacher is going to rely on teacher verification to determine this but when the teacher is not there then we have this learner mastery process in place so we can still record hours for students that are unable to attend the online sessions at the exact same time as the teacher next slide please okay and then here's a here's kind of a not a specific model this is the last one but this is kind of putting it together kind of the the overriding learner mastery where if you want to do it for a whole software module which my understanding a lot of you are doing where you've got a cd with like 12 different modules and the students are going through all 12 throughout the semester whatever here's another way where you can do it where you can start with you know putting the name of the software up at the top what does it address list the modules give the titles and the projected amount of time for each one of those modules so here we've got module one through four with a targeted amount of time for each of those modules again just follow listing all 12 modules record the number of hours to each module maybe it's what the publisher tells you maybe it's what you determine through learner mastery whatever but put the hours there and then again when students complete each module you have that little plan that you already put in place so whenever the student finishes the corresponding module you simply record that amount of time for the student depending on which module they complete and that's it just want to add that the q and a once we get a chance to go through it and shake out any of the weirdness we will be posting it on otan.us alongside of the powerpoint okay jannis thank you so much for that reminder