 OTAN, Outreach and Technical Assistance Network. So welcome. We have our panel today, myself, Margot Teske, from CASAS, Program Specialist. Used to be SoCal, but now I'm North in California as well now. But we actually cover the whole state, all of us. Crystal Corbus is here with Pleasanton Adult and Career Education. She's Assistant Director, has lots of good stuff to share with us. And she'll introduce herself a little bit later with more in-depth information. But she's a cool person. Gilbert Leos has tons of experience. He's at Pasadena City College in the non-credit division right now. EL Civic Specialist and a Technology Specialist as well. He's done a lot for their program. And today you'll hear more about him. I'd love to hear more about you and your agency. But before we get to that, just a second, the slide deck is posted in the chat. If you can't get it there, feel free to get it on our Google Drive. Let me see if it will copy here. So that you can get it. I'll copy it into the chat real quick so that you can all get it. Some people were having trouble. Not everybody's on their latest version of Zoom. So sometimes those files are hard to get if you're not on the latest version. OK, we'd love to hear from you in the chat at any time. We'd love to hear from you right now. If you could just let us know your name, your agency, your position. We'd kind of like to know who our audience is. So we know where to go fast, where to go slow, things like that. Don't be shy. I don't see anybody posting. Come on, Kathleen, we'd love to hear from you. Also, we have set aside after each of our speakers some time for question and answer. So feel free to join us then and you can unmute at that point and ask us questions or briefly share about your own experiences. OK. Thank you, Amy. Thank you, Kathleen. OK, well, today we're going to talk about costs, this remote testing, various procedures for remote testing. We're going to share about remote testing at the local level. Pleasant and adult and career education will be sharing how to start small and then build your expertise. And then part three is how they're doing it at PCC, which is more of a larger agency, transforming from challenges into successes. Our intended outcomes are to provide an overview, thinking of everything all at once. But we're here to provide help and to reach out to you with our experience and to let you know what works. OK. I want you to hear from the field on tried and true techniques. Then start thinking about how this can work for your school. You're able to express some concerns or ask questions and discuss really what works at your school. But of course, we have to keep that a little bit brief. We have 90 minutes. We were going to try to do this in 60, but it didn't seem to work. So we added from we added an extra half hour thanks to OTAN. And so we have that time. OK, I'm going to get started, unless anyone has a question so far. Please continue to let us know who you are and what your jobs are. We have a remote testing person, personnel in the office, customer manager for Burlington. OK, Danilo, thanks. And Britt Peterson, records and report clerk. So some of you maybe are proctors or you're dealing with orientation and registration. You want to know more about what we do in in E-test so and EL civics. So good. And Lori, thanks for coming. So again, our presentation is posted in the chat. Just go to the very top of the chat. And if you can't get there, remember, we have that Google Drive link, so I'll post that again. OK, there are four main approaches to CASIS remote testing. And we'll call them approach one, two, three and four. Remote control is the first one. That's one on one. Then the second and third one are one to one or one to multiple test takers. Their first one is using Windows 10 PCs. The third down the list is oh, I missed an R in there. Sorry. A one to one or multiple test takers using Chromebooks or iPads. And the fourth approach has to do with oral responses. And all a really good description of all the approaches is on a YouTube video. And I posted that link there. Each approach really has their own unique requirements and guidelines. So I would recommend starting with the one to one remote control, especially for ESL beginners, people that are not very tech savvy. And CASIS has some great examples that use Zoom and Zoom breakouts are really great for approaches two and three. So if you can group your students by devices, intergroup into Zoom sessions, that also helps. And remember that one proctor per five students you need to monitor and help your students. Each student goes to their own breakout when they're ready to take the test. So I'll explain that a little bit more as we go. And if you have questions, please post our panels. Very happy to answer questions as we go. So one thing to remember about remote testing agreements is that you have to have about remote testing is that you have to have an agreement. And the first type of agreement calls an agency remote testing agreement. And there's one specifically for California that includes CASIS E test, EL civics and citizenship testing. So all three in one agreement. And that should be done once every year. Of course, you can change it and modify it as you go through the year. Both your program specialist and your CDE consultant should be given copies of those. And in addition to the school or agency remote testing agreement, also proctors need to be aware, even if they're only EL civics proctors, that they should do a proctor remote testing agreement. OK, now. For CASIS E tests, they also need to have testing certification. It's not hard to do. It's online. All this is online, so it's pretty easy to do. Those are the links. So when you get a copy of the PowerPoint, you can just click on the link. OK, so for students, testing agreements are a little different because they're not through the CASIS website or anything. What really helps is that you get students prepared all as much as you can before they have to take the test. So it would it helps you to set up some time to work with them one on one. That could take 30 to 45 minutes. But one of our my local network schools, Hacienda La Puente Adult School, they wrote up their testing guidelines for students in the major languages that they have for students, English, Spanish, Chinese, Korean. They put that together and they help their students. Once they want to get registered, automatically getting into a Google form and going through the rules and guidelines in these languages so that they could understand what's expected of them for a pretest and how to do that pretest. So they use sign up genius for that. And you'll hear, I think, for Crystal later, how they use sign up genius. This is a Google forms form that's used by Hacienda Puente. This is kind of a dummy form. I'm going to copy it for you right now and put it in the chat. Again, you can use that to kind of guide you as what might be possible. If you have questions, you can always contact Maria. Tell us if you don't know her email and you don't get this PowerPoint, you can email me and ask me for that. OK. So preparation, you got to schedule your students and set up meetings. The first part is to get them ready for the test. And then the second part, Part B is actual test. So sending meeting invitations to each test taker by email or text, you know, your students best, what works for them. And if you can include the Zoom link or whatever link you use to connect with them and send them the speed test URL, that will help them know how fast their computer is already. Hoping that they use the same computer when they're doing the test. Not always true, but you got to verify that and expect that from the student by letting them know you want to include a link for E test sampler. So the students have some practice and they feel more secure taking the test, knowing ahead what's what's ahead. You know, adults, we like to know what's coming up. So meeting with the prior students prior to the test is that can take 15 minutes or more depending on student tech abilities and also what you've sent them ahead of time. So basically when you meet with them, you want to talk to them about the basic procedure for the test date. Ask them about what device they're using and emphasize that they need to use the same device. Maybe tomorrow when they're giving the test, when you're giving the test to them. Do the speed test together if if they haven't already done it. Make sure they have two megabytes per per second or faster. Usually faster is better. Check their desktop. You want to make sure that they don't have a lot of, you know, possible programs like Google where they can search for the answers. Open, want to make sure everything's closed and it's great time to confirm that they're still available for the test date and time and then give them the URL link for the test for the appointment for the test, not the actual test. OK, so approach one, one to one. So I know some of you have some experience with one to one. If you have comments about one to one, a remote control, please post those comments in the chat. We want to make this interactive as well. So this is where the proctor gives remote control access to the test taker so that the test taker actually enters the responses on the proctor's computer. And this works with the Windows 10 PC. The proctor's computer is registered for testing and is remotely shared through this examples with Zoom. So it can be easily shared that way. Oh, Kathleen's been doing it. So Kathleen has it worked pretty well for you guys. It takes a lot of time one on one. Yeah. Yeah. Is it OK to talk? Yes, sure. Yeah, we've been doing it. We've got one, two, three, four, five people and they only work part time. So they're only getting to a day. So we're we're trudging we're trudging along. We got the help from the teachers. We're getting the teachers the list. We had a we had a list of students. We get we're trying to get go for the post test right now. And then we're also working on the pre test for high school diploma by remote and it's hard enough to get the students to to do one time. So we've been doing the part A and part B together. In the same session. Because these are post testers, so they've already tested on, you know, in person on a test on a Chromebook. So that's that's what we're doing when we do the pre testers. I'm sure we might have to do the second day because they are they're new to the school and they haven't taken they haven't taken the test yet. So I have a high school diploma students done better at doing it faster, a little faster. No, I haven't checked the results yet, but that's something all right. But that's something I will check because that's what I've heard from the field that as you know, the higher levels, they just go faster. OK, not not on the test, of course, but getting ready for the test. So the whole procedure. Oh, yeah, I understand that part now. OK, oh, yes, it's so much easier. And that's why we have we have like one Vietnamese liaison. So she's calling and talking to the student, the lower level students in their own language until they get to begin test. Yes, you can do that. So that's good. But our school mostly has Vietnamese, so there's more. There's more. So we're letting the we're we're encouraging. We're having our non-Vietnamese speak proctors schedule and proctor the higher levels of English so that they would have more and more and higher levels of. Yeah, digital issues. Yeah, yeah, great. So if you're new to this whole process, it's great to watch this YouTube video. I'm not going to show it now because it just, you know, doesn't seem like you guys are that new, according to the comments I've seen from Tamara and Amy as Amy, I think. So great. If you need to look at it, the YouTube links there, there's guidelines there. This this video just takes like seven minutes and it really helps train your proctors as well as get you ready if you're doing it. So it's great training for proctors, I think. The more they start with one on one, the more comfortable they feel. And then they can move into like approach two or approach three and approach two. It's it's still you could do one on one. But it's great if you can move up to doing like five to 12 students. So that's what they're working on in the field now is enlarging it from five to 12. So not every agency can do that. It's kind of by special request at this time. So keep it to five. But once you're comfortable with five, you know, that's that's not two a day. It's it's more like 10 a day. So that's great. And how it works is that the proctor is is setting the students up in a private breakout room. And so they have five break private breakout rooms and they they work with the students in the main room. And then they go to private breakout rooms, check their ID, make sure their room looks great and they get them started on the test. And then they come back to the main room, grab another student like that. So really, it's very helpful if you have two people because one can be the moderator teacher person and then the other proctor person can take them into the into the breakout rooms. And so students have something to do, you know, in the main room while the proctors, you know, taking them into the breakout rooms. They can also practice the sample test items when they're in the main room as well. So as you can see, if you had two proctors in addition to the teacher or moderator, you could do twice as many students. So the proctor's job is basically we'll talk about it. But a proctor's job is to to monitor once they get them all set up, monitor how they're doing and circulate from breakout room to breakout room. OK. And I know Crystal and Gilbert will be talking about that a little bit more too. These are just the basic approaches so you know what to do for test takers on Chromebooks and iPads. They have to be in a special mode. One is called kiosk mode. The other's guided access mode. And again, you have to check, you know, they have to register the computers for E test. They have to have a certain speed for that. And again, you can do five test takers at a time. And same as above in approach to. So it's very similar. It's just different devices. Yeah, I mentioned registering the students computers. So that's done. And again, a special video is available on the cost website. Well, on YouTube, so you go directly to YouTube, if you'd like. Proctors work with a student to register his or her computer. If you're familiar with T E and E testing, there's a special code that you get and you give that code to the student and they can register their computer. It's pretty easy to do as long as our speed is as good. And students understand, right? Approach for is is basically for Windows 10 PC for the proctor. And the proctor's computer is registered for Windows 10 PC. The first computer is registered for E test. And then the proctor screen is shared with the test taker. But the student is not able to. They don't have the ability to do like the remote control. So they tell the proctor the answer. And then the proctor writes down the answer for them. The test taker can use, you know, a tablet for that. Chromebook Mac PC or iPad, as long as they have a webcam. So you need to have a video on them. So and it's limited to one one one student at a time. If there's no webcam, you know, you could use a smartphone camera as that. There is some remote. I call this approach 4.2 because it's used a lot with EL civics for, you know, oral tests and for the citizenship oral interview. And it's best to do using a video calling app. What's the most popular video app that your students use? I know what I've heard from the field, but oh, good. Kathleen's working on approach to that's good to hear. It'll help. What's app? Yeah, Vicki, that's it. What's app is a very, very popular, especially with Asians, I believe. But, you know, other people are getting into it too. It's a really easy app to share with and it's more private, I think, than Facebook. But, you know, maybe I'm not up on privacy so much. But it works well with a small group that I work with. And it's actually, we're working with cell phones in a pilot test right now for listening. The cost is listening tests. So they're also working on it with goals testing. So we'll see how that goes. You know, can the cell phone be big enough to read things? We're not sure yet. And to do math problems. I think for listening, it's pretty much a natural. And it's working out so far, but there's another month of testing to come. And we can cross our fingers because students have smelt smartphones. So that might be the easiest way. We'll see. Probably we'll be out of COVID before all this is decided. In the future, you know, students still want to do some online testing. So. All right. I'm at the. End of my part. Are there questions for me? Anything that you want to ask for me. And of course, you'll have more questions as we go along from the experts in the field. So. Any questions about. Cause now's the time for the continuous improvement plan. Are you guys focusing on remote testing? What payment points are the most important for you. To focus on. As part of your goals. Let's say. We're just trying to get them. We're just trying to get paired scores. So we've been working on it. And we, like I said, we have a team of maybe 12. And it's, we're, we're just plugging along, but you know what? We're doing it. So that's going to be our narrative. That's good. That's good. And as you. You'll get a lot out of crystals. I know, because you guys are starting small and you want to grow a little bit with that. So that's good. Okay. All right. Well, let's move on to crystal. Crystal, there you go. Well, thank you so much, Margaret. I appreciate it. And thank you for inviting me to be a part of this today. Hopefully everybody can hear me. All right. I wanted to give you a little perspective before we dive deeper into what we did at our school. Just so you can kind of compare and get an idea of who we are and what, what we're doing. I came on as the assistant director this past year in October. So it hasn't been a long time. And the first thing my director said to me was we need a remote testing program. And he is not a heavy adult ed background. He's more on the career tech ed and does a lot for K through 12. So it was all about me jumping in and doing this kind of work. And the great thing was I had some background coming into it and we'll talk a little bit more about that. But I will say I had five to six years, about six years of teaching ABE and ASC and to being the department chair at Castor Valley adult. And from that experience, I was very comfortable with testing, of course, in person. That was not an issue. We did it all the time in our classrooms. We were smaller school too. So we just tested as students came in during a certain time of the year, a couple of times a year. And so when I came to Pleasanton, I felt comfortable in the in person testing and the one on one. I done quite a few sessions of the one on one testing over the summer and at the end of last school, you're trying to make sure we got those gains. And so a little bit more about the school I'm currently at and what we were doing. So our school is one that had funds swept and was reopened back in 2016. We are part of Pleasanton Unified School District at K through 12. And we're in the East Bay, San Francisco Bay area. Our demographics. We are predominantly an Asian population in our area and mostly ESL students. The majority of our students are and you'll see that. I think on the next slide. Age predominantly 45 or over. And female is 84%. So we have a lot of women in our program. And very highly educated in their home countries. And the percentage is really high of diplomas and degrees from outside of the United States. So very, very incredible group of people, very smart. Very capable. An older group. So maybe a little bit needing of technology support. But anyway, that kind of goes through that slide. Thank you, Margaret. I appreciate it. So I wanted to give you an overview of just how many students we're talking about. So last year we had about 326 ESL students. And right now we are down. We're at about 192. ASC and ABE programs. We had about 12 last year. So not a lot of students in that program, but it is growing and definitely with my background, it's a priority that I'm looking at being able to support more students and moving some of our ESL students that needed into our ABE program. And we also have started the ABE program this year. So that's adding a couple more people into our program as well, really focusing on those students. So I see that growing more and more. So next. So some of our testing challenges, like I'm sure many other small schools, especially those that didn't carry on over the time that funds have been swept by districts. We have testing challenges because we don't have classrooms that belong to us. In our case, we do not have any daytime classrooms that belong to us. We, at nighttime use the local continuation school, village, high school as our location. When we're not in COVID, right? And we use a scheduled kind of shared district room for in-person testing. When I say shared, I just mean we have to schedule it. And then it's our room, but we don't have a dedicated place that we can can test people on our own. We have to make sure that it's available. And there's up to 10 students that we figured out can be socially distanced in this room. And we use a Chromebook cart and Chromebooks for our testing, at least in the in-person environment. We have an older student population, as I mentioned. And so we needed to pursue remote testing because there was some vulnerability in that population and people that were uncomfortable coming in. We have a lot of parents and grandparents in our community. And they are really looking for that literacy and to support children and grandchildren. And so it's very important, but they also needed some technology support. And that was kind of a challenge as well to do that remotely, but they needed that remote test environment. So that is definitely one of our challenges. Next. Okay. So just to give you an idea of how many students we've tested. So I was actually really thrilled when I saw these numbers. So when I talk about it, we've had 306 students tested. They might be duplicated because they pre and post-tested with us in person this year. And they were using mostly Chromebooks, but yeah, they would have been using all Chromebooks, actually, in our testing environments. And then when I switched to remote, which was when I came on board in October and we started really working at this, I will talk to you more about how that took place and how the proctors were trained. But basically we started with one or two at a time and then worked our way up. And we've gotten to 83 students tested. Again, some pre and post-tested. So they're duplicated in that number. So that's, you know, 25%. That's not bad. You know, that we've been able to get through. I will agree with Kathleen with the comment of you just have to do that. It is not an easy thing. And you just have to say, I'm going to try to get 10 people tested this week or whatever it is. And recently we've gotten up to 10 people at a time and two proctors on. Actually we had three just to see if we were going to capture more than 10. So we can make it work. But it is, it is a challenge. Every test is a challenge. Every test is a challenge. Next one. So how did we handle our students signups? And some of this is going to be more along the administrative side. I'm trying to touch on it, but honestly Margaret did such a great job of covering the different options. So I'm just going to kind of cover some of the things that might be helpful that we took away from the experience, but not try to go in depth into every piece of the, of the puzzle as far as testing. So each week we would send out emails to our students, not the entire student population, but those that we saw had either not pre-tested or they had 40 hours of instruction or more and needed a post-test at that point. And so we would just gather that or data person collected that information each week and send out a report. And then we would literally cut and paste into an email and pretty much the same email went out every single week. Why change it, right? It worked. And it connected them to a signup genius form. We actually would have one signup genius form for every week, so that it just dropped off and we got a root of all those ones that were already done. And it worked really nicely and it was clean and the students obviously could do that. It wasn't a problem for them to jump on and to figure out, oh, I want this date and that time. We wanted that to give them the flexibility. I want to say in here that you need to be patient and professional and be just ask them to sign up. I mean, I think we all are, but sometimes in this game of trying to get as many students in, we kind of try to push them and be demanding of you need to do this by this date. I don't like that. I'm not that person. I just had graced with them and understood that they were uncomfortable and that it wasn't this easy thing I was asking. And so if I didn't get them that week, I just kept asking every single week, we just kept putting it out there. And then enough of them started talking about it in class, that they felt like I could try that. I could do that myself. Eventually nearly all of our students either signed up for the remote testing or the in-person testing, which I should mention, the sign up genius didn't just have the remote testing spots. It had the in-person as well, so that it constantly was in their mind that, oh, I have a choice. I can go in now if I feel comfortable enough or I can go ahead and do the online. And I mentioned the using the sign up genius. It was just the easiest, most flexible way that we could think of to get students to sign up. And then we would use it to send a reminder later and we'll talk more about that on future slides. Yeah. I put a note in the chat that it's a great tool for practicing. Calendaring. And EL Civic. They have calendaring. So I just wanted to show you kind of what that looked like. We are smaller, so we don't have massive numbers of students that wanted to come in every week or sign up for these. So we were doing a couple of sessions a week. And at first, of course, two or three people, and we'll talk more in a minute. So I just wanted to show you kind of what that looked like. We are smaller, so we don't have massive numbers of students that wanted to come in every week or sign up for these. So I just wanted to show you a little bit more in a minute, but I wanted to kind of give you an idea of what it looked like. And it would just go out to the students each week as a link saying this week is this link. This week is this week. And we do two weeks in advance, usually of dates that they could choose from. And we tried to mix up afternoon, evening, morning, some in-person, some, especially when we first were starting this in like October, November, and then we had a little bit of a mixture of in-person and remote. And it added, we just wanted to add that flexibility for the student to have choice. And like I said, we used it for the in-person testing. I think it just gave them the ability to maybe switch to in-person, which we could handle 10 people in-person. So it kind of made it go, especially in those early months, when we weren't doing 10 at a time, it allowed us to be able to have a few more people in in-person at a time. Crystal, one of the, Kathleen, I think asked how do you schedule more than one student at a time, at the same time period? It allows up to 10 people on the sign-up genius. Is that what you're asking? I guess, I guess it would be, how do you know, how do you know when the, when the 10 is up, when it's full? So will it stop after? Yeah, we'll let them sign up after. We'll drop and say not available or something. Okay. So they have to sign up for us to send them the link of how to get on the zoom. To do the sign up. So there's a setting that you can do for that? Yes. It's part of the programming of available spots. See how I have a Windows 10 and then it says 10 in parentheses after it. That's how many spots are available. Okay. Thank you. Yeah. And it'll, it'll go down, you know, and say nine or eight. Or whatever as you go. But yeah. Thank you. Next. Okay. So after there was this whole thing of how do you then, you know, know, and I agree, Kathleen, you know, how do you know how many people? Well, it was a process. We had to take off the sign-up genius. We left them on there, of course, on the list, but we would take their name, put it on to another list because our proctors and we'll talk a little bit about our proctors in a moment. But because they were not teachers, they didn't have access and we didn't give them access full on to ASAP. So they didn't have all the student information. So we had to have some mechanism to give them the information they needed, like email address, phone number, their language, their ID number, that kind of thing so that they could have that information available. And then the Google Sheet. Let me see. They need to have access to student information. Yes. And after the session, we also had that same sheet be useful to our people in the front office. They would then create an email and send out the test scores to the student and the teacher on a test score report. And so I had somebody else who would access that and do that work to send it out and communicate with everybody after the fact. And this is just an example of what that Google Sheet looked like. We would include the Zoom link, who was in charge of the session. And then I was always the second person, which I know is a lot. But if you don't know that you're going to have more than five people, you need to have somebody who can put some extra support in there, but be flexible enough to walk away when you're not needed. And I didn't want to schedule somebody for two and a half hours of testing and not have them do it. I'm very sensitive to hourly employees since I did work in that environment for many years. So hourly teachers are close to my heart. So I don't like to schedule people unless I know that they're going to be able to get paid for that two and a half hours or that two hours that I've scheduled them for. So I just made it. So every session I would jump in for a half an hour to 45 minutes, help out. And then if I wasn't needed for the rest of the session, I walked away. We also use the same document to put the station registration code for each of the sessions. It got a little bit confusing because we were doing a test and post test because our pre's would gather all of our information. And so we had multiple tests that a student could be taking and we needed to make sure that we had the right session station number for that. So we would kind of use that off to the side. We would just put it up there. And I think that's good enough for this one. Okay. And then the day before the student actually came, it was a great reminder that one of our staff members would send out to the student. Here's your testing session, your zoom link. We are looking forward to seeing you there. Here are the translated instructions in the language that you've designated is your first language. And it included a screenshot document. And we also would make sure that the primary proctor was also sending out that same document again and establishing contact in case there was problems along the way. I should also mention that the email would have our cell phone numbers because you had to have that quick contact with the student to be able to make sure that it and they needed to be able to get a hold of you if they something happened in their session and they needed extra help. The primary test proctor, yeah, also emailed them that language that translated screenshot. And then after the session, the front office staff would send out the score report. I probably copied that one again. Sorry about that. So a little bit more on or a little bit on selection and preparation for our proctors. So our teachers, they were busy. They had a lot going on. And normally they were somewhat involved in our COSIS testing in the ESL world, which was the majority of the teachers were talking about. And so we really kind of identified early on that that remote testing function needed to be done by others, not the ESL teachers. They were at home teaching and they needed to be connecting with their students. And so what we did was we used our high set proctors to support our COSIS testing. And so in any case, they had started from the beginning because they'd been high set testing. They didn't do COSIS testing. So when I came on, I started getting them trained. I worked with them individually and as a group in remote, in our sessions, you know, remotely. And we, they were really good with scaffolding instructions, which I felt was really important that they be able to just focus on what's the next thing we need to work on and support to get that student up and running online. And so I did some introduction to the whole testing process first by having them do in-person testing just so that they could see how do I set up a session, work out the kinks of how do I know if they are, what session they're needing to do, how do I pick the computers, all that kind of stuff. So we worked on that first just to get them familiar. And Margaret, could you move me on? And I'll throw in there again that I was the new director or assistant director and I had done in-person testing. Actually, the school had only done in-person testing up until this point. They hadn't even done one-on-one testing. So nobody was familiar at the school with the testing remotely. And then many students, they still needed the pretest and we, but many didn't want to do the in-person testing. They were really uncomfortable with it. So it kind of really pushed us towards this remote testing. And so we conducted one-on-one. I had conducted one-on-one remote testing at my previous school. I had a little bit of comfort, but I'll tell you, I was not like running, you know, towards this. I was like, that seems really hard. I've done one-on-one remote and I think I'm a little bit more comfortable with it. So I did one-on-one testing. And then I had the idea of two, let alone five. I don't know how I'm going to get there. And so I then took on, I can't just throw people at this that don't, haven't done this before. And so I jumped in and did the one-on-one again, just to remind me how to do it. And then I started working my way up, started slow, and then did one-on-two, one-on-three, and we started getting better and better at it. And then I started to do one-on-one testing. And then I started to jump into 10 or jump into five with one proctor. It was try it and work at perfecting it. And by the way, so go to the next slide. At the same time, I brought those proctors in and let them watch so that they could see, oh my gosh, what am I getting myself into? And they were so adaptable. I was so blessed to have people that were really aware and they just started supporting pretty quickly. And again, it was that whole, they, a couple of my students worked in adults with disabilities programs. They understood the students, they understood scaffolding. I'm not trying to compare, I'm just trying to say, if you understand scaffolding and, and going piece by piece and describing and talking through it, but showing through it too. Using different techniques when it's not working. They really did a great job of, of doing that. And so I, I stress to them though, be patient both with yourself and with your students that are on there. And just, just acknowledge everything that they do that gets you one step further towards this, you know, give them, give them a clap. If you can't pat them on the back, at least give them a clap and say, great work so far. This is not a great job. And I think that helped. And I love your bravo. And so the second sessions, I started transitioning to the second proctor and we started getting a few more students in so that my first proctor, the primary was now becoming my proctor that I wanted to, to run these sessions, but still not running. We were still just dipping our toes. And I'll tell you something, you were just clapping because you got, it took you an hour and a half to get all four or five people up and running. And then other days you were like, wow, that was like 20 minutes. How did I do that? So I thought that was pretty great when we were getting that much quicker, but every once in a while, you'd go back, you'd take a step back because it would be a different group of people, different issues would come up with the technology. And then around the third session, they were able to run the session. And I was able to take a big step back and be able to just support. And then I just continued to support the first 30 to 45 minutes. Rarely does it go much longer than that. So, so recently I mentioned, we've started to allow up to 10 students. Actually, I should say that we were prepared for up to 15 weeks. And so I think that's a good point. And I think that's a good point. I think that's actually, I should say that we were prepared for up to 15 one Saturday because we were trying to intake a bunch of pre-apprenticeship or apprenticeship students, adults that were needing ESL support from one of our partners in the area. And so we had set it up for 15 students. We only made it to 10, but we did have two proctors, three proctors on at one point, but then went to two proctors and it worked. And then it was a little bit more more of a plan, but we didn't really have much to say about that. And I guess that's kind of kind of that second bullet. You know, we don't have, have that many, but But when we do, I stay on the whole time if we do and I always schedule myself so that I can always step out if I, if nobody's there, but stay if I needed. teacher, he's very big into graphics. And I hadn't thought that through enough. And he just grabbed it and ran. And I felt so fortunate to have him willing to just jump in and translate. And then we found our district has some supports for checking our translations. And we have a student, we have a IA that supports us with translation for Chinese in our classroom, one of our beginning classrooms, as she helped. But we've we really found that this was huge. This was a game changer for us when we developed these detailed documents with screenshots. It just stepped them through the process so much easier and in a comfortable space for them. So it's a lot easier when you do that. It's a lot easier when you do that. Great. Yes. Highly recommended if you're not highly recommended. Yeah. And so getting students ready for testing. We had started back in I think the fall before I got here, they had been passing out the Chromebooks that we had in our Chromebook carts. So our students that needed computers were getting computers and all of those were preloaded with PASSE test. So it was already in kiosk mode. We already had all of that figured out. So that was easy. The harder students, of course, were the ones that didn't have the Chromebooks and were happy with their Windows 10 or their iPad at home. So we had to get used to using really all three. But we don't we don't use the iPads very frequently. It's not a norm. Students borrowed a school Chromebook if they didn't already have one. We found a lot of our students would come borrow it and then return it. They had what they needed at home. But we'll explain there are some issues that crop up that sometimes you need the student to just try using the school Chromebook instead that's already preloaded. And we provided them that translated document that really helped get our students ready to go usually the day before. So the testing procedures we we use the approach two and three. And we did both the one to multiple with Windows 10 Chromebooks and even iPads but not very frequently. And I don't know what my time I'm gonna do a quick time check. Am I running too long? Go a little quicker. I can do it. So we created binders that's a good testing procedure I think is create the binders for for the proctors to have right in front of them. Because you can't like be looking in multiple windows on other devices you really need to have it sitting in front of you. I think it's too hard to try to be talking on on on the zoom and be looking through all these different documents and guidance. It's better just to have something in front of you. And we use the quick reference sheets at the beginning. But they kind of tell you to go go up and do this part if you haven't already. And it kind of, you know, it was a little bit difficult. And so we just took the key components and made a quick checklist for ourselves based on how we were running our sessions. And I would suggest that it kind of helps to have it in the order that you're going to do it just to make sure you hit all the things you need to do. You had mentioned early Margaret that, you know, I think it doesn't talk about moving them to the breakout room necessarily before you're checking IDs. But I that is a I'm going to do ID checks and what the room looks like when they're private. I don't want to do that, you know, so the breakout room part was pretty important to us. And so we wanted to have that step through so it flowed. And that's good. So you can go to the next slide. So lessons learned. The Windows 10 computer, one of our big lessons is, especially in a population that has a lot of translation tools that are often embedded at startup. So when they start up, restart their computer, it just comes on in the background. And oh, by the way, it's their husband's computer, or someone else in the family's computer, they don't even know what's starting up. And they aren't very well versed in technology necessarily. So now they're in a room by themselves. And the cost test program shutting down. So we found that was something that happened. And yes, we I totally agree. Going to the help desk is a very good idea. However, when you're in the middle of it, you're kind of left with, we have to figure something else out. And so often that something else out was, do you have an iPad? Can you bring an iPad over? And I was the resident iPad person, because really, we rarely did it. But I knew to pull up the instructions and run through the steps and get it into the, it's, I can't remember if it's Kia, no, it's not kiosk mode. It's the other mode, but it's the guided access mode. And so I was able to manipulate it. Yep, I was able to manipulate it. The first time was tricky. But after that, I was like, Okay, I know what I'm doing, I can do this thing. But but it's tough when you got Windows 10 Chromebook and now iPad, and you're trying to coordinate all of that. And oh, by the way, your other proctor is supposed to be going in and checking just to make sure nobody's wandered into the breakout room or in the background. And so I feel that that's an important job of our proctors to make sure that the testing environment is correct. So you're helping people, you're wearing, you're juggling all of this, right? You're trying to juggle so much. So anyway, just to kind of that was one of our ahas was sometimes we had to loan the school Chromebook out and push the student to come in and get it, because it wasn't going to work on their Windows 10 or they'd come back the next session with somebody else's or their different Windows 10 computer and they understood to have whoever shut down all those other systems beforehand. Sometimes too, we would have is your son home? Is your husband home? You know, is is your wife home? Whoever can they come in and help you with the setup? And that I mean, literally, I had an eight year old child that was doing so awesome, putting a iPad into guided access and spoke great English and his dad was very grateful. And then of course, he left the room and I watched him go, you know, kids have learned so much for the miss online learning too. Yeah, and it's very empowering to the children for sure. Yeah. Okay. And so I'm getting close to the end here on suggestions. So start slow. I mean, that's that would be my biggest thing is don't go into this with I need to get to this point by next week. You need to be very methodical about it, but you need and have procedures and have it documented and have a system. But let yourself have some grace in. Okay, what did we learn? Definitely have some of that discussion afterwards. It's worth the time to get the system flowing smoothly, to be able to make sure that the students feel good about the process because you want them to come back and post test with you. That's pretty important. And so just be patient with everybody that's involved and be prepared with the documentation that you need. You need to have a list in front of you of the students you need to have the station IDs written down if you're doing it like we're doing it where you're giving them control of their own computer and them watching them. I really didn't talk about that you've got to be really careful about the camera situation. We really didn't discuss that in this, but you need to know how are you going to monitor them and know what that device is. So if it's a Chromebook, I need to be able to monitor them on their own cell phone and watch what they're doing. If it's a Windows 10, does it have is it a desktop that can't be moved? Well, then I need a camera that I if it isn't movable, I need their iPhone so I can see around and check their room. So you've got to be a little aware of the limitations of the different devices. So you might like Margaret suggested, want to only do Chromebooks one day and only do Windows 10 another day because it does get confusing. Okay, what are you on and what what do you have that you're using as a camera? Okay, show it to me. And I didn't I know I let me see your sign up genius kind of takes care of that, right? The sign up genius tells me that but I have to make sure I put it on the ES on the spreadsheet that I have so that the proctor knows because he's not looking at the sign up genius. And I need to make sure that the proctor understands the implications in their training of what it means that they're supposed to be able to see. And I don't know that I hit it, but the other nice thing to remember is that phone that they're using as a camera, it could be turned towards the screen when they're stuck, not when they're testing, but when they're trying to set up use it as a tool to be able to see where are they because you get lost in your head. The other thing is if you have the ability to have them screen share once they get in the breakout room, let them screen share and you can see exactly what they're doing on their computer. You know exactly where they are and what the process is that they need to do next to get where they need to go. So Crystal, we're kind of out of your time, but you have a couple of good questions, right? So one of the one of the participants asked, what do you suggest for someone who does not have a remote testing team? She's like the only one trained to do remote testing. Yeah, you're going to be stuck at five people max, right? Unless you want to try to get on the pilot or whatever, if they still allow you to join with the 12, and as soon as they approve the 12, then you're going to try to get to that. But that's a tough one because I always felt uncomfortable as a one on five with how do I help that next person and still check the rooms to make sure the students don't have people in there with them? You know, kind of juggling and going back and forth between all these students and if you get stuck in a 15 minute getting them onboarded into the test, I would say you got to utilize that meeting with the student ahead of time idea. We don't do that, but you could meet with the student ahead of time and get them all lined up to go. So there's a question like you need a second proctor, right? Usually, because of the main, you got a main room and a breakout room, but actually students in the main room can be told to stay there while the proctor takes them into the breakout room and the proctor goes back and forth. Well, you could actually put everybody into the breakout room. Right, Gilbert? I mean, if you have something to say. Well, that's what I do. I mentioned in my slides that we actually just send everybody to the breakout room, but we give them instruction to say stay in your breakout room. We're going to come to you because that is the most secure way we can can engage with the student. Okay, I agree. And Beth is asking about your Chromebook lending procedure. Can you type that in the chat while we get Gilbert started? Sure. Yeah, we're a small school. Thanks. Thanks, teacher. Yeah, not a big procedure. Good. Okay, but a lot of us are small, so that's cool. All right, Gilbert. Well, thank you very much, Margaret. My name is Gilbert Laos. I work at Pasadena City College's non-credit division footy old campus. I am the EL Civics Specialist there. And just to kind of give you a little brief understanding of how we work within conjunction with Pasadena City College. We are separate from the college. We have our own campus and we serve primarily adult learners and we're also, we owe a title II funded. Next slide, please. Uh-huh. Before COVID, we were serving approximately 2,500 students annually. But once COVID came around, our numbers dropped dramatically. We had a reduction from 2,600 students to 2,400 students in the 1920 year. This year alone, we are servicing at least 686 students. Which is a quarter to a third of what we were doing prior to COVID. So COVID has definitely impacted us dramatically. And so we are doing our best to, to reach out to our students and to find ways to, to engage them and to encourage them to come back to school. Especially now as we're moving out of this pandemic. Next slide, please. Uh, out of the 686 students, 75% of them were new students, which is very surprising. And 25% came back from the previous year. So a lot of our new students, a lot of the students that we have are new. And we weren't expecting that. We were expecting most of our old students to come back. And so we were very, at least pleased to see a growth in that area. Of our 686, we have about 74% of the student population are female. And we also service about 26% male. Next slide, please. Our ethnicity breakdown, we primarily also service the Asian population. But we also service our Hispanic, white, and black, and African American population as well. But the primary, the big majority of it all, of it all, is our, our Asian population. We do have a few others that we service such as American Indians, Filipinos, and Native American, Native Hawaiians, and Pacific Islanders. Next, please. So this is what our testing facility looks like. When you come in before COVID, we didn't have the plexiglass. We now have added that so that when we do go back to in-person testing, our students will have the opportunity to sit socially distanced from each other so that they can actually engage in in the tests at the lab. But in the meantime, we are using these computers here to give remote access to our students so that they can actually perform both E-tests and EL civics tests. Next slide. Thank you. So our computers, we have 45 Dell Optima Plex desktop computers, and they each hold 8 gigs of RAM. And they use an iCore, an i5 Intel Core processor. We have Windows 10 on each one of the computers, and right now we are using the Xcelon HD cameras. We just recently purchased those, and they've been a benefit because they have built-in microphones with them. We also have our computers loaded with COSIS E-tests, of course the three major browsers, Google Chrome, also Firefox, and for us, because our computer is a little bit older, Internet Explorer. Of course Zoom is loaded, and other applications such as Microsoft Office and Adobe Acrobat are available. Each of these stations requires that the staff log in to them individually. So this isn't just where we can, we're not at that point, but working towards it, where we can load them up all at once, but at the moment we're required to log into each station individually. In terms of using Zoom and testing, the way we go about it is that we log all of our testing computers into Zoom at the same time using the web signing sheet. The first computer that we designate as the host computer is logged in, and inside the actual Zoom application you'll see it as host. And then the subsequent additional computers that we add in become co-hosts, and we can therefore have them all logged in at the same time using one account versus using multiple accounts. Next slide. With regards to the breakout rooms, students are placed into each breakout room individually when they're taking the test. We use the share screen feature to give the students access to the test on our lab computers, and they do remotely access our lab computers. So it's not that we're giving them a link and they can go on there, they actually get a chance to use our systems. And so we know that we have control over what they're working with and what they're seeing and everything else with that. Next slide, please. What we consider to be acceptable technology for testing. For us, we want students who have computers with built-in or external speakers, microphones, and webcams, very essential that they have these. We don't mind that they have desktops, laptops, or tablets, including iPads. One of the things we have kind of distance ourselves from, and that is Chromebooks because we haven't had much success with that and encounter a few issues with it. But we primarily accept back-offs in Windows 10 operating systems. Next slide. So the issues that we've encountered, specifically within Zoom, has been with ShareScreen. We would initially share a test and then we would share something else and in the process, Zoom would crash on us every single time on our end and sometimes on the end of the student. So in order to remedy that, what we found is that by unchecking the ShareScreen sharing option under use hardware acceleration for, in the advanced section of ShareScreen setting, we were able to utilize ShareScreen in such a way that we can now share multiple Windows whenever we needed to. Another issue we have, of course, is with the iPads. They would not allow us to view our students while they were taking the test remotely and utilizing our computers. So one of our colleagues came up with this solution where if you use this side-by-side mode inside of the ShareScreen setting, that it would broadcast the actual camera from the iPad while the student was taking the test. And because of that, we were able to keep them on camera and watch them as they perform the tests. Next slide. At our campus, when it comes to EL6 testing, we test our ESL courses levels one through five. These courses roughly match the costless and NRS functional levels, our lowest level one being beginning low and level five being advanced. These courses are split up into two sections, AMB. And so our courses will range from English 1A to ESL 5B. Our class sizes are typically around 20 to 30 students. Occasionally, we have more or less. But that's how our courses are set up in terms of ESL. Next slide. These classes are tested three times a year when it comes to EL6. We have our ESL classes on an eight week session schedule. And so for the fall semester, we'll have a first session of eight weeks. In the, we'll have a second eight week session. And then in the spring, we'll also have a first and second eight week session as well. And during this eight week session, what I found best is to test our students within the fifth and sixth week, giving them sufficient time to acquire all the knowledge that they're going to learn for EL6 at that time. Next slide, please. And so during the 2020 year, we've decided that we were only going to test levels four and higher, just because in the initial part of our COVID issue, back in 20, a lot of the students were struggling with technology, understanding technology, especially our ESL students, they were not able to understand well, what is this function or they couldn't understand even the simplest of instructions. And therefore, because of that, we made that decision to just go with the students that we know who have had more success over the years and understand instruction. Therefore, we chose to chose them. And for the first fall eight weeks, we had 61 students actually participated in our assessment. The following eight weeks we increased by 76 and we're hoping that this spring semester we'll have even more. Next slide. When I prepare the materials, and I'm pretty much the one that does it, I actually start with the cost as rubric. And so what I'll do is depending upon which co-op we have selected for the year and we select usually typically three. I will pull that cost as rubric out and begin reviewing it, researching all the materials, all the information needed to teach and create the tests so that the students are have that information to not only just acquire it and learn it, but they're ready to go. And so we start with the rubric first. Next. Once I've done all the research and I've actually gone through I've created the test. I know what's going to be on there. I have all the materials and all the slides and all the nice little images you have to go looking for. I actually put all that into the Google slides. I found that that has been very effective in terms of distribution more than anything else. Prior to COVID I was printing out papers and pamphlets and booklets and it was costly and time consuming with the slides. It's much simpler and easier and it seems to be helpful with the instructors when they're trying to teach that material in their classes. Next slide. So one of the things that I did with my manuals or my booklets of prior to COVID was make sure that there was practice exercises and I continued that even today by creating a Google doc that gives the students the opportunity to see what type of questions will be on the EL Civics exam and to get a chance to practice and not only just practice have the answers so that they know what I'm expecting to see when I actually grade their exams. And so I make those available via link as well. Next. Our instructors what I do in terms of all the materials that are created and everything that we have I make sure that the instructors get this prior to the week before class begins. We want to make sure that our instructors have everything that they need and understand what's going to be coming up for that year so they can incorporate it into their lesson plan. And so I make sure that the links are provided for the Google sites and docs as well as the link to the student testing agreement. One of the things that we would like to have and continue to have is is that our instructors engage them engage their students in understanding what is going to be expected of them when they come and take the test and not just from not just here from us but here from the instructor and only that if they have any questions they can also ask the instructor what this means what that means. I also provide them a PDF copy of the cost assessment rubric just to let the instructors know how the test will be graded so that they can prepare better prepare their students and understand that it's not just me who wants to make up things that they actually get what they need. Also I provide additional information and instructions such as what to emphasize in their lessons and how and when we're going to take the test. Next slide please. With the proctors I'm pretty much the only proctor I do have we do have our classes coordinator and she does help me whenever we begin testing and what I do with her is I walk her through the entire process I make sure that she has the opportunity to have all the documents with a digital paper so that she knows what it is that we're going to be working on what information do we need to acquire from these students we also have rosters for each class that we deal with to make sure that we can note off who's accepted who's agreed to the the agreements and who's there and so she has all the information and after we've gone through the process we'll then have a Q&A amongst each other to see if there's any questions she may have or or I may have some questions of her and so we'll go through that nice little dialogue. Next slide. Again the instruction materials that I create from Google Docs and little slides all that material those links are given to our instructor and the instructor is responsible for sending out that information via email to their students as well as posting on their main Canvas page and so we want to make sure that the students they can go at either place to get that information and begin preparing for the exam. Now as far as the content being taught we allow the instructors the freedom to teach it however they wish whether it's asynchronous or asynchronously so that it works and conforms with their lesson plan. Next slide. Now the way we set up the lab is that each computer is logged into Zoom on the day of the test and then assigned to its own proper breakout room. I also utilize the Firefox web browser and that's where I go to the Google Form where the test has been created and have it ready and prepared so that they can be given to the students. The spell check component of the browser is disabled. Students love to use this feature and with Firefox I'm guaranteed to disable that and make sure that they're not able to use it. Next slide please. Next slide. Okay thanks. Our webcams are turned on and we make sure that they stay on until the student begins the test. Once the test begins we turn them off so that the student is not distracted by anything else and they're focused on the exam. Microphones are initially muted but when the proctor engages with them we turn them on and they remain muted once the test begins. Next slide. We have our students when it comes time for test we have our students come during their scheduled hours to the Zoom room and once everybody's there at the schedule time we give them an additional five minutes for any late comers that may come out on you and then we begin to give out our instruction. Next slide. And when we give out that instruction to our students we review the functions of Zoom. Some of the most important functions that some of them do have trouble with even when we get to that stage and that is with chat share screen and breakout rooms. We've had students who when we try to engage them in chat they don't know where it's at they don't know where the button is. So we make sure that we have a slide that explains to where are these functions at function buttons at and how to access them. We also review the testing agreement and make sure that they understand what is expected of them and ask them if they have any questions. Next slide. We also let them know about the breakout rooms and about what is going to be expected of them once they reach there such as they will need a photo ID they will need to make sure that their test area is clear and they will also make sure we need to make sure that they are the only person in their room. We do ask and emphasize that they please stay in their breakout rooms because we've had students who have exited the breakout room and had difficulty returning back to it. Once that all that information has been disseminated we then assign each student to the personal breakout room and send them off. Next slide. So once the the proctor goes to the station what they do with each individual student in their breakout room is they ask to see a valid photo ID and then they ask to see the student's room and then they ask the student if they agree with the testing agreement. Afterwards once all that has been completed the proctor will then inform the student that they can leave once the test has been completed and they're ready to go. Next slide. Once the student is ready to go the proctor will share the screen giving the student access to the lab computer and they will be able to remotely access it and we can see as proctors what is going on when they're taking the test so whenever they're having an issue with writing we can see that if they're having an issue with the question we can see all that. And so we monitor that throughout the process but once the student has access there we now move on to the next student as quickly as possible and so that way we can then monitor all of them all at once. Next slide. Now during the test again we monitor the students we make sure they're following the guidelines make sure there's nobody in the room and we're there if they have any questions so if a student goes teacher teacher we're there to answer their questions in any shape any way shape or form. Next slide please. After the test we regain control of the computers by then the student has already left the room and we close out the web browser and afterwards we just exit Zoom and once that is done the EL Civics testing is complete. I believe that's and that's it. Any questions? I have a question about I understand doing the remote control on the one-on-one I understand about assigning students to different breakout rooms and monitoring them but I don't understand how all all the students can be remote control over so you have explained that to me again. Okay so what it is is that for every one student you have one lab computer designated for them that is assigned to one specific breakout room. Okay and when you have that one-to-one relationship that student you can then give access to that one computer to the student and that student has access so it's everything is one-to-one. So it's just like on a one just like having one-to-one only you're setting them up okay in breakout rooms instead of that. Okay so then how do they do you send you send them one Zoom address and you make one host and the other host. Okay so for what we do in terms of setting up the lab is that all of our computers are logged into one account via the web the web sign-in of Zoom. From there we send each computer to its own specific breakout room the student will then be assigned to that breakout room where the where that computer exists so that we can have that relate so that computer can be related to that that student and then we can show that screen with that student. Okay does that does that make sense? Okay so you're kind of creating a Zoom meeting with your with your your your lab computers you're creating a Zoom meeting and the computers are the individual breakout rooms like it would be a person. Yes. They Zoom meeting and now you're adding the component of having a student go into the go into the breakout room and now gets remote control over the computer. That's correct. Okay now Gilbert you're still having you know one proctor for five students or something like that. Yes that is correct. Okay. So but we do Matt would you you're using the one to one remote control system so it's that's correct. Okay. Lori had a great question. I don't know if you want to go to a Lori on it. So Gilbert this is Lori. Hi. I was just wondering about which skills the low level ESL students need to learn in order to be able to participate in your co-op testing. If I understood correctly you said the co-op testing was only for the higher levels. That's correct. It seems that some of them had trouble understanding how to utilize Zoom how to install Zoom and also some of them were not familiar with computers. They were more much more familiar with their phones and we were trying we're trying to keep away from the phones for EL civics testing at the moment just because the screen size is just not big enough for the test. But there are EL civics that are oral oral testing. Yes the EL civics co-op that we selected was not oral it was written. Oh. But I was also wondering had those students been pre and post tested on the COSIS test previously so they would have had experience with the computer in the COSIS testing? That I would assume that they were but I have no idea as to at least for this year they would have. Prior to that I'm not sure if they were tested before back in the fall of 19 or early spring 20 before the COVID hit. And so that was the reason why we made this decision was based upon that information at that time and not on the 2021 year. Okay. We're having success with beginning lit students working with in Zoom classes now but it took time. We're actually getting beginning lit students remotely tested one on one. Yeah. Wow. Great. Great. Yeah there's a lot of things Gilbert I'd be happy to chat with you about EL civics and what you could do to to get reaching down to the lower levels. There's a lot of teachers that are really good at that. So. Start at the higher levels for everything. But no free EL civics. You know co-apps. There's a lot of co-apps that are really good for lower levels. And you know like Alyssa in your area Kathleen and then. Alyssa. Yeah Alyssa does. Alyssa Takuyuchi. She teaches beginning lit but she has her beginning lit students go ahead and do the beginning low assessments. Yeah. Class. Yeah. Yeah. So it's possible. You really have to help them choose the right co-app basically. So. Oh okay. Other questions. Good question Laurie. I like that one. Other questions. Gilbert has a lot of different procedures that he uses that are a little complicated. So keep that in mind. I mean start slow. Like Crystal said and build up. Build up over time. It's the best way. I just I just want to chime in again. Crystal and just saying everything you were saying we tried. We did the short check off list. We had them looking over the shoulder. We have the thing is the only thing we haven't done is the sign up. We've been getting the list from our resource room. The list of students who need to be tested. Giving them to the schedules the proctors then they've been working with the teachers in their zoom meeting in their zoom classes trying to get students signed up for their teaching their working days in hours. So I understand it's whatever works for you. You know you've got to be patient with yourself and your group and just try your best. One at a time. 10 at a time. Whatever you can do. And we have to think like to the future too. I mean once covid's over we feel like we can get back into classes. What can we do for those online classes still? Because a lot of students still want to do some online classes. They like that format. It's a lot easier for them. So. Well I know I would love to keep ill civics remote regardless or even if they just come into the lab and do it on the computer. It just seems to be a little bit easier. And especially when you did so many the information is just much easier to send out the link and have everything prepared than it is to print it up. I forgot to put the presentation slides link here but it is in the chat. Hi I have a question. This is Dominique. Yes. For Gilbert. I saw that you were using Google form for your test. How about for an application form? Are you changing it to like a Google form using Google form or what? At this moment we haven't actually done anything with applications. However I would most likely be inclined to use the Google form just because an application is nothing more than a form and you already have it there and you can customize it to whatever needs that you need. And you can pull the data out in the CSV file imported into either into access or Excel depending on how you want to grade it. Okay thank you. And Kathleen posted that they have a fillable application form. That's online. Maybe Kathleen you can connect with Dominique and let her know. No I'm just saying we didn't use it but when we were getting ready to do something we had some we had we were looking at one of the employment ones and I saw that there were fillable ones but we needed I needed somebody's help to turn it into a PDF or whatever that so we did not we didn't actually use it but I'm saying that there are there are reels you could set up a you know when they say agency authentic it or something yeah I of course I understand like you know a form is a form and we want it to be you know authentic but is it possible to you know just do a Google form instead of you know a PDF a fillable PDF a yes could have a an image with the numbers by the areas you want them to fill in and put that on the Google form oh okay that makes sense yeah and Google forms have come a long way since I use them a lot so keep that in mind but check in with other agencies too a lot of people use application forms so thank you good question Domenica and Bethany mentions that we we want to encourage you to do the evaluation let us know how we did let OTAN know how we did and Laurie Howard's here she said yes you can use Google forms sorry I couldn't find my unmute can I just say quickly that you can use it you know the idea of authentic that you would use authentic questions if it if you're going to do it electronically you could just put the authentic questions into a Google form because in class when we teach them we do share you know the PDF but it's it's so much harder for low level students to you know to do like the fillable document when you can always adapt for the low level students okay remember though you are getting them ready for real life so yes there's a a push and a pull there well thank you all for coming you know if you have more questions please feel us we're happy to chat and talk via email and thank you for coming