 our first presenter Dr. Carolyn Zachary from the Adult Education Office at CDE. All right, thank you very much Melinda. Welcome to our webinar. I hope that you're ready because we are ready to start remote testing and I am happy to be here with the cost esteem. We'll be talking about who will be giving you all of the details. I'm giving you a real high level view right now and that is the fact that we are going to be instituting a remote testing policy if you are going to be doing this. You will need to amend your local assessment policy and that local assessment policy is going to need to identify what tests you're planning to administer in a remote setting. Included in that will be an assurance that your agency is going to follow all of the remote testing requirements and an assurance that all of your staff are trained to administer those assessments. This is following OCTAE guidelines which are part of our NRS reporting requirements. So this will include that you have some way of or process for identifying the student to ensure that your proctors are completely trained for proctoring the test remotely, that you're also ensuring test security and that you are providing live proctoring. A template and guidance will be available for agencies to use by the end of the week. You may notice that in TOPSPRO Enterprise known as TE that there is an optional box for students that says force majeure not able to test. This is what we will be using as we provide you the guidance so don't do it now but as we provide you the guidance for what to do if you are not able to do remote testing with students. The reason we're asking you to do this is because OCTAE is very interested in the data that will be coming from states with those those numbers of individuals they weren't able to test. We will be including this information in our narrative report to OCTAE. Okay I know these two words cause a lot of anxiety for everyone so I want you to all breathe and I want you to be aware and know that the adult education office knows that you are very concerned about your payment points and that you're concerned that COVID-19 is going to dramatically impact the work that you're able to do with your students and the results of all of that hard work known as payment points. We will be evaluating all of the data that comes out of this after the certified by your agency and that doesn't happen until August where you do your certification so we will not be evaluating this data until the fall. I want you to understand that the data from the 1920 year is used for your 2021-22 grant award year so it's not used for your grant award that's coming up here in July. We really want to evaluate the data very carefully and ensure that agencies are not dramatically impacted by COVID-19 and the inability to test students because we know that this is an equity issue across the state. What we really want you to focus on right now is on your students and using testing to help your students see their growth as well as your teachers so that your teachers can see the growth that the students are making in a remote setting so we encourage you to use it for those purposes, relax about it for payment points. We will be focusing on those in the fall when we have the data which is what we really need to have to be able to look at all of this. I would also encourage you those of you that have issues with connectivity because of the remoteness of where your agencies are located to look into the Rachel device. This could help with your connectivity and I've asked Penny to put information about this in the chat and it is from the World Possible Organization and this may help you in your remote setting and with that you're welcome to email me any questions that you have so with that I'm happy to hand this over to Pat Rickard, president of POSIS. Thank you very much Carol and thank you all for joining us today to learn more about remote testing for the NRS approved COSIS pre and post test. We're very excited to share with you what we've learned, what we're doing and what we plan to continue to do to support you and we really thank those of you that have willingly agreed to pilot test and have given us some really great feedback. I do want to many of you joined us last week on Thursday to learn more about remote testing for the co-apps and sit and I want to put this session today in context of the other two. I'm looking at it as three different levels of remote testing. The first level would be our performance-based co-apps and those are developed by the local agency and you have in place trained staff and the security on those, definitely we want security but it's something that is within your agency and not across the state or national. The second level would be our performance-based citizenship interview test. It is standardized for use across all agencies in California that are offering English language citizenship preparation, you know, civics citizenship preparation and we have trained SIT assessors throughout the state in your agency and we talked about simply providing some additional training for them in order to administer the SIT. The third level which we're going to be talking about today and Linda is going to be covering is our nationally approved NRS classes pre and post testing which is multiple choice. We will also cover the EL civics government and history test which is multiple choice. Each of these three levels the co-apps level one the performance-based SIT level two and the NRS pre and post multiple choice level three each have separate guidelines and each have separate go remote instructions. So I just wanted to clarify for those of you that joined us last Thursday that what Linda will be covering today will be the pre and post testing and Linda with that I will turn it over to you. Thank you Pat. Hello everyone. Glad to be able to present with you today and thank you for taking time to come and hear about this. Next slide please. Yes here's our agenda and as Pat mentioned we are focusing today on the multiple choice tests the NRS approved pre and post tests and government history for citizenship. We're going to give you an overview some background about remote testing and how COSIS is approaching it. We're going to tell you about our clinical tryout and the pilot phase that happened mostly in April and then get into what's happening starting today May 4th where we're moving into phase two still piloting but also rolling out for agencies that have been piloting and rolling out different approaches that I'm going to describe then also we have a phase three which is you'll see then we will tell you about next steps where to find all of the documents and supports that we're going to be sharing with you today and then at the end and actually I'll be stopping along the way and pausing to see if there are any burning questions that haven't been able to be answered in the chat box. As there are so many people on the call we may not be able to get to all of your questions in real time today but we will compile them at the end and post that on the website so we'll hope to answer everyone's questions at some point if not today then afterwards. We're going to get into an overview now that is going into a little bit more detail about what Carolyn introduced in terms of the Department of Ed the office of career technical and adult education octay. They put out a memo on April 17th that stated very clearly that states have flexibility at this time with this disruption caused by COVID-19. States may allow local programs to exempt students from peer and post testing those that are still enrolled doing distance education and also octay the federal government wants to know what the time frame is for that in your programs and who or the extent of those exemptions. They know you're not going to be able to test everyone under the circumstances and they would like to have an idea of what happened. As Carolyn mentioned there are certain guidelines that they put out that are to be followed when doing remote testing whether with CASAS or with another test publisher and these are in these very broad areas of making sure that the student identification is authenticated making sure that test security protocols are in place and making sure that proctors that are administering remote testing have been trained and know what to do. On April 9th octay met with all of the test publishers and CASAS of course was there and then about a week and a half later we just we CASAS met with octay presented them with a plan to roll out remote testing. They have given us permission to do that and we were happy to hear that they were quite impressed with our remote testing plan and with the speed at which we were able to pull that together. So that is something that has given us made us feel that we are supported by octay and we are looking forward to rolling this out with you. We did conduct a webinar on the 20th of April with state directors in 30 states and told them what our plans are. We have actually had pilots going on in the last few weeks in probably 15 to 20 states including California. So it is really being tested by programs like yours all over the country. We are hearing how it is going and we are making adjustments in very real time as we go. We have been updating the guidelines pretty much daily or even sometimes more than that. So to really make it as streamlined as possible. So what are some of the benefits of doing remote testing? One certainly is that you are able to continue providing services in terms of assessment. You are able to post test. Now of course we know that you are not going to be able to test all of your students. We really want to make it clear that that is not the expectation. But during this time we hope that you will take advantage of it to try out remote testing. Post test perhaps select certain students that are priority or that you would like to see how they do with remote testing. So you will also of course be able to see how they did in their coursework. If you have incoming students and we know you do have a lot that are recently unemployed therefore might have more time than they had before to study and taking advantage of that. We know you would love to get baseline scores for those students and so if you choose to do remote testing you would be able to get those using E-Tests remotely as well as use these assessment to guide where to place these students in distance learning. We really want to emphasize that this time also gives you the chance to develop your agency's capacity to learn how to test distance learners remotely. And when all of this is back to at least some semblance of normalcy that will add a lot of flexibility to your assessment programs so that what we envision is that all of our assessment programs will consist of both a face-to-face component as well as a remote testing component so that you will be able to test students that have difficulty coming in to test at your test centers. So it looks like remote testing is the wave of the future so that we can do distance learning as robustly as we've now learned how to do or are still learning how to do and we actually are very much in a learning mode but feel that we're improving day by day in terms of understanding what's possible, how to do it more efficiently, more cost effectively, and with additional types of technology. So we do know that we're learning together and look forward to that. Now challenges, I'm sure you all could fill the chat box with all the challenges that you've already encountered with distance learning, getting that going, one of them first and foremost I'm sure is the students access to technology, the equity concerns related to that, the digital divide that has come into such sharp focus these days. When you're going to consider doing remote testing you'll need to have certain devices for proctors and students and I'll be giving you details for that shortly. Smartphones are too small as you I'm sure can understand for the test items that we have in our NRS approved tests that they would not be able to see them well on a screen and that would mean that in turn they would not be able to perform well on the test. So unfortunately although most of our students do have smartphones that is not a possible device for the pre and post testing. It can be used if you attended our session on Thursday, it can be used for the co-app assessment deliveries, it can be used for the SIT but not for pre and post testing. In terms of internet bandwidth that can be a challenge as well with Wi-Fi being much in demand in a given household there could be kids trying to use it at the same time as their adults working from home trying to use it. So we know that our students may have a challenge getting access to the bandwidth that they need so that's something that will have to be planned well ahead and hopefully dealt with in a way that will work out. Also in the students test environment at home they may not be able to get a private place to test or it may be challenging. Many distractions with others in the household so here again there are many challenges that we are well aware of and I will say now and I'll keep repeating to really reassure you that all of these challenges are things that we have considered as we are writing our guidelines. So whether it's for SIT or co-app or these pre and post tests, the remote testing guidelines that we are going to be posting that you'll will require you actually to study very carefully before you embark on remote testing. They have very detailed instructions step by step, procedures, scripts, here's what you do, here's what it looks like, we're going to be I'll tell you more later but we're planning to do some follow-up sessions with videos of what settings in certain types of platforms so we really are aware that you are going to need that kind of very specific help and that is provided in these guidelines. In this webinar today we don't have time to go into all those details I'm going to get into some level of detail but please be assured that these guideline documents have very very specific instructions. So it is going to be somewhat of a challenge for you and your program staff to learn and implement these new processes but based on our pilot testing with a little help from CASAS and a little practice we're sure you're going to be able to do a great job. We are we know that your proctors have been trained to give E-test space to face but they have not been trained to do that in a remote manner so we are relying on you to ensure that they have read the guidelines carefully that they take advantage of the training that will be posted as recordings on the CASAS website and that they are ready to go when it's time to go. Test security is extremely important and you know we're not going to be able to make it airtight but we hope that we can close up as many loopholes as possible and we really are going to count on the goodwill of our students to some extent but we are putting in place as many precautions as we can without making it too cumbersome for you or the student. Equity concerns have already been mentioned to some extent with the technology but there may be others. So as we researched all of this we looked at some third-party vendors there are lots of them out there we met with many of them. What we found is they're expensive a little bit steep for our adult ed budgets 15 to 20 dollars an hour to have a live proctor in a third-party vendor so that's not what we went to first we may reach out to them and perhaps have be able to crack a deal that is affordable at some point but at this point in time we are not offering that kind of solution nor an artificial intelligence solution at this time and we looked at a variety of video platforms we talked to state directors and their staff we talked to a lot of folks who are familiar with distance learning to learn what we could from their experience with that we made weight multiple factors um cost which in terms of the technology to use for remote testing uh we think is fairly minimal however there is a cost in terms of staff time and that is because uh with remote testing and the need to be able to observe the student in real time it's necessary to limit the number of students that can be tested at one time and so it's between one and five we are allowing at this point if we find we can allow more great but we understand that that is going to be labor intensive and therefore costly for your programs depending on how many students you decide to include again though I want to emphasize that this is a time when you're trying this out and you are not obliged to do this remote testing but if you do want to try it it will be more labor intensive because of the high ratio of proctor to just one proctor to only a maximum of five students so we understand that there's really I'm sorry nothing we can do about it but that is the situation at this time we know that there may be technology constraints both for proctors and students which perhaps you've already been working to alleviate when you've done uh distance learning now by uh learning out devices so perhaps that's a way that you can work this out as well test security is something that we'll talk about it takes somewhat different forms with remote testing ease of implementation uh we've tried to make this as easy as possible though it does take a bit of doing in terms of understanding how to set up the video platform and etc all the different steps but it's possible and doable and we know that many of you are very eager to get started so we've tried to address that urgency by making this available on a very short timeline from compared to how we normally develop new types of assessment in terms of the delivery it will be live proctoring so that's in real time synchronous uh there's something called a remote testing agreement where we'll want to have one per agency and that will cover all of the different remote assessments that are possible there's an agreement for proctors that you will have them fill out and keep on file at your local program and then students uh this is not something we've had to do before but because they're at a distance and not uh under our watchful eye uh next to us we will ask them to verbally agree to certain uh test security promises in terms of delivery uh it is on e-test so we need you to use certified e-test proctors who've already been uh ideally who already have experience using e-test so they must be certified proctors there's no by the way additional certification for the e-test administrators just the proctors and these proctors will need additional remote test training as we just said um we had originally talked about videotaping but we have gone away from that uh after thinking it through a bit more and getting input about it we have decided not to require videotaping of these remote assessments it's just a little too cumbersome and hard to figure out how to store them lots of reasons so that'll make it'll easier for you uh and also could be a privacy issue so no videotaping required also just to be very clear the CASIS remote testing cannot be delivered with paper tests uh if you're doing remote testing it can only be with CASIS e-tests uh for the pre and post tests now uh in terms of co-apps or SIT you may um you may the conditions are different and you still need to be able to see the student live but um there can be some paper involved but we'll explain that that that is explained clearly in the previous webinar and um we can clarify that it's all laid out in the guidelines for co-apps and the SIT as well but as far as the pre and post tests no paper tests also we know that in some programs they have figured out ways to do face-to-face testing even during these times uh of the COVID-19 crisis that could be in corrections programs we know that some programs have gotten creative and have found ways to have students take the test in their car getting a box you know that's that they come and pick up but they and it's never never touched they never touch uh uh anyone on the staff's hands anyway it's there are ways I guess to do it where you can observe social distancing and still have the student be doing it in person so any situation like that where you're giving the test face-to-face essentially uh the basic normal rules still apply that a proctor must be physically present so if you are doing that that the the usual rules apply and um you need a proctor so which tests can you use for multiple for remote testing with multiple choice tests it's the same ones you've been using the goals series reading goals and math goals for ABE and adult secondary for ESL it's life and work reading and the life and work listening 980 and also though it's not it's not a pre-test but it's a post test really the government in history for citizenship test a certification test that is given to ESL citizenship students is also a multiple choice test and can be used with remote testing following the very same guidelines that we're going to tell you today for the pre and post tests now this slide shows the phases and the one we have just completed is phase one where we did clinical tryouts and piloting and we restricted that to one-on-one one proctor one test taker and pretty much used only zoom because that made it a little easier to see what was going on so that was I would say quite successful we got a lot of great input and we incorporated those suggestions into our guidelines starting today we are moving into phase two where we are going to be having if you're just starting up we're going to ask you to pilot that is you know not go full scale right away but start small pilot and then roll out whether zoom or another platform like go to meeting webx or other ones one-on-one is one option another option is proctoring with up to five test takers and that can be with the with various devices I'm going to explain the details soon and that will soon include Chromebooks it does not include Chromebooks at this moment in time that's something that we hope to be able to have available within a week or two so we're talking weeks and not months and it's very very shortly we hope to be able to make it possible with Chromebooks but starting now you may test up to five test takers with these other platforms like zoom and go to meeting for example and then we are adding on the ability to do remote testing with the listening 980 series as well as the government in history for citizenship test in the future we are still interacting and exploring with third party vendors if we can possibly work something out to make that work there are other options we're looking into that if they can make this a more efficient and less costly proposition we will certainly look into them and let you know as soon as we've worked anything out all right let's go on to the next slide then and we'll so now I'm going to tell you in some more detail about the three approaches that we have come up with so far and that we're making available the first one is one-on-one and in this approach the test takers have a variety of devices that they can use that's one of the advantages of this approach they can use a pc they can use an ipad or they can use a mac and in this scenario the proctor needs to register their computer and it has to be a windows 10 because the proctor is going to run e-tests on their computer and then remotely share their screen with the test taker through the web conferencing platform so that's why this can only be one-on-one because the proctor is loading one test onto the proctor's computer and then sharing that with one student the proctor and the student are connected and be able to see each other real time using a mutually agreed upon web conferencing platform with a webcam and they need an internet connection that is two mbps or higher okay let's go on to the next solution in this one it is possible to test up to five students however the I would say challenge here is that the test takers have to have windows 10 PCs so that may be a high bar we know that but we do want to make it available in case there's a chance that you can provide students with these or if some students do have a pc with windows 10 then they could and you could also take advantage of testing more students this is necessary because that's they have to be able to run e-tests on their computer so another aspect of this approach is that the test takers computer has to be registered and just want to let you know that for the purpose of remote testing we are allowing there to be only one person that is the proctor to register the student's computer normally we require two certified staff to do that but for remote testing we are waiving that requirement and so the proctor would be able to register the student's computer then using the web conferencing platform that you select and the webcam you the proctor would be monitoring these one to five students and again you need a and I believe this is not a very fast internet connection but again given that it may be in competition with others at home using the connection it might be challenging so part of again these detailed guidelines step-by-step instructions you're going to get there is a link and all the instructions for how to test the proctors and the students internet connection speed there are a number of websites that do that and that's something that you'll have to do on the day of testing you can do it before to double check but also I mean to see if it's even close but also on the day of testing and this is in the guidelines you'll be checking their internet speed to ensure that day that they're going to be able to have a fast enough strong enough internet connection to do the job the third solution is coming soon and we know it's eagerly anticipated and that is with Chromebooks because we know that many of our students either have a Chromebook or their students were given a Chromebook or you can give them a Chromebook because they're so affordable and so we are moving as quickly as we can to make the Chromebook solution available and in this solution the test taker would register the Chromebook on their Chromebook that they have at home you would help them set it up register it again just only you and not two people then like in solution two the proctor would monitor using a live web conferencing platform and you can do up to five students but the difference here is that with Chromebook you can only run up one application at a time so in this scenario with Chromebooks the test taker would have to use their smartphone with the video cam to and they would set it up on the side so that the proctor would be able to see their screen and to see the test environment while they're taking the test so that would be how we do a workaround with Chromebooks because you can't have the web conferencing platform on working on Chromebooks at the same time as e-tests by the way one thing that is again in these very detailed step-by-step instructions is that we asked proctor to do a sweep of the testing environment and this is something we explained to the student beforehand we encourage you to have a session before the testing day or before the test session to explain to the student and they so that they can be aware and anticipate all of these situations that would be new to them including this requirement that the proctor will need to scan the room they'll let the student will have to scan the room so the proctor can see this is to make sure there are no people there who could be giving answers to the student no inappropriate documents papers aids that might help the student answer questions so it's a test security measure we know it could make students slightly uncomfortable but we hope that you'll have a way to explain to the students that this is necessary for um test security that it's only for that purpose we're not looking at anything else in the room just looking for that and this would be done before and after the test session or if the if the proctor senses that there is something amiss the proctor can also ask the student to do that anytime if they're concerned and there's also instructions in the directions that uh if you if a proctor sees a reason to be concerned then the proctor can interrupt the test at any time and try see if there is a real concern in which case they would they would stop the test and then as i think you all know you're able to start the test up again if appropriate or end the test and end the session so this is a mock-up version of these guidelines just to let you know they do exist they're quite detailed about 20 pages with examples and scripts etc so there is something called an agency remote testing agreement this is a form that will ask every agency that intends to do remote testing to fill out one per agency it will cover all of the possible tests that you can use for remote for EL civics or pre and post that would be the cost of center as to prove pre post tests the co-app assessment deliveries the sit or the government in history for citizenship we will ask you to tell us how you're going to verify the student identity and in that we're going to ask you to use the procedures and the ways that you normally do that in your local program so nothing will change there but you may have to figure out some ways to do that remotely that are different than you normally do it perhaps and then another aspect of this age and very important aspect of this agreement is that you attest to having read these remote testing guidelines thoroughly and that you agree to abide by them so now we're going to go out of the PowerPoint and into the actual document so you can see the this agreement and I'll show you a little bit more detail that's in this test remote test center agreement so okay yes so that at first it says you've read the guidelines then that the second paragraph down there is that you know we would like you to stay informed because we're releasing the version 1.0 but it's very possible likely I would say that we will be updating that with additional refinements and improvements so we will let you know when we release new versions and they'll be on the costus website we're going to have one page dedicated to California remote testing on the costus website and that's where you'll find all these documents all these resources and the trainings including any updated versions of these guidelines again to reiterate we're not asking you to record and that we would like you not to record because we would not like these recordings to get into the wrong hands so we thought there was really possibly too much danger in recording because you would need to then ensure that those the privacy requirements would be maintained also for test security all right so then here we would have you list which test series you're going to use by checking off which ones you plan to use and you know we're aware that when you fill this out you may perhaps decide to use certain of these tests and not others so if that changes over time that's fine you can work that out with your program specialist and your cde regional consultant to see if you want to revise this but just please do the best you can at the time when you want to begin we know it may change then we want you to indicate which of these approaches it might be more than one that you're going to be using in your agency so are you going to have the proctor heavy tests on their computer their windows 10 and have students have more options for the devices that's number one one on one are you going to opt for a solution where you can test more students at a time but they need a windows 10 and if we can get that changed at some point in time we will let you know it would be nice we know but at the moment that is the requirement for this option number two and then option number three which if you're going to fill this out this week you would not be able to check this one because it's not quite available yet as soon as this Chromebooks option is available we will definitely let you know and then you'll be able to use it so just to reiterate Chromebooks is not currently an option as of today but in the next week or two we expect to be able to have tested out all of that that whole process and have good clear guidelines for you to use Chromebooks as well all right then just briefly to share with you about the co-app assessment deliveries and how this is also in this agreement so if you're going to venture into delivering the co-app assessments remotely we want you to list which ones we want to you to briefly describe how you're going to adapt them if you need to for remote testing and then on the right we've listed from low to high tech some options for the technology you may have think of others but and you can if so you can certainly add them to this document when you submit it but this will give an idea of what you're planning in terms of the assessment deliveries of your civic objectives and then for SIT if you're going to do remote testing again it's optional but if you decide to you can use the if you happen to have the printed COSIS paper SIT test booklets either you brought them home or your agency can get them to you that's perfect and then really nothing changes you would as you would normally it's a consumable test booklet so you as you're giving the test as you know you write the 012 rubric responses right in the test booklet and that's how you administer the test that's if you don't have access to one of those paper booklets we have created and this is new a fillable PDF version of the form 973 and 974 they're not quite yet available but they will be available by next Monday the 11th of May that's when all of these things will be on the COSIS website so please have a little more patience we're almost ready with to roll these out but when that time comes that those booklets will be available for you to use if you don't have the paper booklets you might decide to use both depending on what your SIT administrators have at hand and then for SIT remote testing there's just two options for the technology and platforms and that's smartphone with the video camera or computer with the webcam phone then we already talked about the identity requirements and then when you're going to start estimated is what we want to know okay back to the PowerPoint and so then the proctor remote testing agreement is something that will also be on the COSIS website it will basically have each proctor attest to the fact that they've read the guidelines that they will ensure that the student verbally promises to also abide by the guide these agreements not to cheat or basically to follow all the test security rules and this agreement if you have proctors that do more than one assessment they just have to do one of these forms and it's something that you keep on file locally with your local assessment policy now we're going to get into the remote testing version of the going remote checklist we've had this type of thing for E-TES onboarding called going live with E-TES but this is the going remote with E-TES process so the first thing and extremely important is that your agency has already implemented COSIS E-TES and if you haven't already but you do want to try this out you can during this time take advantage of the time to go through the certifications and get all your staff certified and get set up in E-TES that's something that our tech support and other staff is available to help you with even during these times so please feel free to seize the opportunity to do that now if you're not already on E-TES you'll need to complete that agreement we just talked about the agency level agreement you'll need to make sure that the proctor has the equipment necessary to implement the scenario the approach that you decide to use or approaches and that ideally ideally that they're experienced in E-TES so that they will have a little bit of an easier time doing this you'll have them complete that proctor remote testing agreement then you'll make sure they know how to access the training materials on the COSIS website there'll be the recorded trainings there'll be some youtube type how-tos there'll be FAQs there'll be there'll be the detailed guidelines step by step there'll be also a quick start like a short version of that that once they get good at the using the the detailed script then we'll have like we do for E-TES the steps for testing day kind of document where they can have a shorthand version of that that they can refer to easily now that the training is not required but it is very much recommended and so it's optional but we do highly recommend that you have your staff do this training then and as far as your test takers having the technology they need or if you need to help them by providing them with some of the technology in order to make this happen registering the test stations is something that the proctor needs to do with the one-on-one solution where they share their screen or that the proctor needs to do for the student in the second and third approaches where the student takes E-TES on their own computer then you're going to review and select the appropriate test sessions and you really want to be ready with those at the time of testing so you're not holding up the process as you go to find those very important step nine is to conduct a trial run or more than one trial run with colleagues not with students but to try it out and make sure that you're comfortable with the process you really have thought through what might happen and how you're going to guide the students when you actually work with students and finally once you're comfortable with that then number 10 is go remote so now we'd like to go out and just show you the slightly more detailed version of this just see that it does and we are going to be adding to this you can see on the first one we've added the link to the going live checklist for getting started with E-TES itself so we'll be adding links to this document so that you'll be able to easily find those documents on the CASAs website that would include that second one the agency agreement the proctor agreement and number three if you scroll down a bit you'll see that there's some there's a little bit more detail about the requirements for the proctor they need a webcam of course then the training again we'll have a link to that into the various possibilities for training step five is all of this all of the different variables and alternatives for test takers so there are a lot of different ways to go and you'll have to see which ones will work again I want to apologize I know it's frustrating that it's limited to only five students or depending on what you kind of technology you have just one on one and we know that you're not going to be testing your usual 25 students in a remote proctoring mode it's simply not possible but we do hope that you'll take advantage of this opportunity to learn how to do this and prioritize the students that you think would most benefit from doing it now or that for whatever reason I think you'll have to talk internally about which students you decide to do this with and which teachers which classes I'll register the testing stations I think we've talked quite a bit about and I think the other steps we've already covered again the training tools these guidelines documents are very detailed FAQs will be forthcoming and will be ongoing we will keep adding to them as we get additional questions from the field and in terms of the training webinars there will be an initial one that will be up on the website no later than next Monday and we will also be offering subsequent mini webinars on different topics those will be rolling out starting next week to follow up on things like you know how do you do security settings what does that look like or how do you do the sharing part or what should you look for when you're monitoring and proctoring so there's there are quite a few topics that we know you're going to want a little bit more guidance on so we're going to be creating short how-to videos and short mini webinars to help you get more comfortable in different modalities for how to do these remote tests and there will be these for not only for the pre and post testing but also for the co-app assessment deliveries that I'm sure we're going to have a lot of really creative ideas for how to adapt the assessment deliveries for the civic objectives for remote testing and then also we'll have a whole set of materials also for the just to kind of reiterate that in terms of the preparation this is so important that before you're actually giving the test we'd like you to meet with this test taker it could be the day before it could be the same day confirm with them that they have the technology they need make sure that they're aware of the test procedures so they're not surprised by anything answer any kinds of questions to make them comfortable with the process especially help them understand that there will be somebody watching them and that can be uncomfortable so but if they're aware of it then hopefully they'll at least be prepared for that situation and then on the testing day although you've done things like actually beforehand checked their ID and however you do that locally and done an environment scan the room just so they understand what that's like you still do need to redo that on the day of the test because of course that could be different also in terms of checking the speed of the internet all those things you have to do again on the day of the test unless you do that that preparation session just before the giving the test which is possible easier to schedule but maybe not quite as comfortable for the students so that's up to you to see how you're going to schedule that but we do you do need to be sure that on when you're actually delivering the test that you confirm everything that you had practiced and explained about before okay then this is a screenshot that shows the proctors computing using the one-on-one approach where the proctor has e-tests on the proctor's computer and then is sharing their screen with the student you can see the webcam where the proctor can see the student can see the proctor this is also something I wanted to mention that we have the e-test sampler which is on the costus website it's free I hope you're all aware of that but if not please check it out it's called it's the practice items in all the modalities for all the test series it's called the e-test sampler and it has the all of these tests in the e-test format and these are the practice items for reading in the e-test format so this is something else you could do in the practice session with the student is to familiarize them if it's the first time they're taking a test with e-tests show them this e-test sampler show them how it works how you navigate through it and because they're going to be taking the test on their computer they're going to be controlling the arrows and and progressing from one item to the nest next when the when the screen is shared by the proctor it's actually the student taking the test on their own computer even in this one-on-one mode where the proctor has their the test on their computer the student is taking it on their computer so or on their iPad so especially I want to say that if they're taking a math test it would be really helpful for the student to see if they haven't already taken a math test how it's done any tests including the online calculator and other types of things I do want to mention that one of the recent decisions that we made is that we are going to allow for math only we're going to allow the student to use scratch paper and to use a handheld calculator if they prefer they can use the online calculator but they could also use a handheld one we we're going to restrict this but we realized it's just too difficult to take a math test without scratch paper so the the requirement is that you'll have to make sure that that's a blank piece of paper when you start the test and and see what the student has there at hand and at the end you'll have to make sure that the student tears up that paper in little bits in front of your eyes to make sure that it doesn't go anywhere else after the test so these are some of the things that are this is a very good example I would say of the kinds of things that are listed in great detail in the guidelines I see a question here about what does it look like when you're delivering the test with more than one student and I don't think today we can go through that in detail but what I can say is that in the guidelines it does go through step to step how you work it out in zoom for example they have what are called breakout rooms in other platforms they could think they call them private rooms where you can have the students all together then you can take the students into the break it into one by one into a breakout room to check their ID and work at work it out in that way so we have that all detailed out in the guidelines and yes that's how it would work all right next slide please so one of the things that is coming soon is that we are going to be able to we need to be able to identify which tests are given remotely and which tests are given face to face and we will very soon find a way to embed that in the e-test software so that you'll be able to indicate that that will enable us to compare student performance on remote as compared to in-person testing we certainly hope that with good preparation of the student that they'll be relaxed and will be as well as relaxed as you can be taking a standardized multiple choice test but that they will be able to do their best and have comparable performance as they would on an in-person test I mean they'll be in the in the comfort of their home perhaps who knows maybe they'll do better but we do want to see and also we will be as we as we always do we have ways of reviewing test security issues in the back end where we can see for example if a test was taken very quickly that can be an indication of guessing there are other data type indicators of test security problems so we will continue to do those sorts of checks as we roll out and ongoing as we do remote testing as was mentioned in the beginning there is a field a new one in Topps for Enterprise called unable to test due to force measure this is where you will be able to indicate students that you were not able to test because we know that there will be many of those and maybe I'll ask Carolyn to pop in right here and just comment on this because I want her to be able to tell you the guidance for this so the guidance for using the unable to test due to force for sure will be coming out later this week with coming from both the CDE and COSS related to remote testing we do have to work on a policy for this and that's in development but we need to we'll need to get that out to you later this week thank you okay now we can move on to some next steps just to make it clear where again kind of I think we've talked about this throughout but wanted to pull it all together by showing you that by when and where you'll be able to get all of this where is on the COSS website the there's going to be a page that's devoted to California remote testing on the COSS website and by next Monday May 11th we intend to have the following materials there one is the remote testing guidelines there would be the document for pre and post testing and government in history for citizenship there would be other documents other remote testing guidelines documents for assessment delivery of civics objectives for co-ops and another remote testing guidelines for the SIT for the going remote checklist there will be one for multiple choice tests for co-ops and SIT as well so there'll be different ones the one we saw today is for the multiple choice tests those will be there the agency remote testing agreement will be there as well as the proctor agreement that you'll keep locally on file the FAQs will be there and we'll be addressing FAQs in all the different for all the different assessments and then the resources for training will also be on this website as far as today's webinar the recording the PowerPoint and the Q&A responses will be on the OTAN website and just a minute I'm going to turn it over to OTAN so you can see exactly where that will be which I think you are familiar with the COVID-19 page and also on the CASUS website at this link is where we will have the recording of this webinar if you have technical questions you can as always contact techsupport at CASUS.org if you are interested in pilot testing with remote testing we would very much welcome that and if you would like to do that for the pre and post testing or for the government in history or for co-apps or the SID please contact your program specialist and you can talk about it with them see if it's something you really want to do and if so we can go forward and help you out to get started well in that case I'll ask Pat if she would like to make some closing comments. Thanks Linda I just wanted to reaffirm that the at this time with the COVID-19 trying to do remote pre and post testing using the CASUS assessments is not required this is a time when you might take the opportunity to get staff trained to actually try out and practice within the agency and to to build capacity and so we really encourage you to look at the long at the at the long-term outcome which is building capacity of staff in order to on an ongoing basis be able to serve students both in class students and remotely with following up with co-apps with the SID and with the pre and post testing and we are all very much aware at this time that the students lack that many of our students lack the technology in order to do this and I think Carolyn may talk about and has talked about the CARES Act and the ability to apply for funding for the technology so we really encourage you to think about this as a longer term capacity building opportunity and we're more than willing to share with you what the pilot sites are finding out we're going to be sharing best practices going forward we're going to be putting recordings up for the training that that are available 724 that your staff can access and learn more about the remote testing so we really thank you very much for on this very interesting journey with us and we are going to be working with you now and into the future so that we can really make this work for all of our students so thank you very very much for your participation today