 OTAN, Outreach and Technical Assistance Network. What we're going to talk about right now is what I call, you know, the brass tacks of the program. This is where the rubber meets the road because it has to do with your deliverables. It has to do with your continued funding. And my goal in this presentation is to make it, I mean, to make this process, you know, regardless of what you had thus far, as easy as possible. So that by next week, you know exactly what you're on hold for, and there is no confusion about that. So at the end of this presentation, you should be very clear about certain things. So what we're going to talk about, we're going to talk about data and accountability entails both academic performance data. And it also entails some things that are related to the fiscal part of our program. So every year, the beginning of the year, we come out to the beginning of the year later. It stipulates what you're supposed to do during that fiscal year. I'll share that with you. And then we also provide you with a calendar of deliverables. And then within our program, we have program, I mean, we have agencies that have grouped together and applied as coalitions. And so that's what 1.3 refers to. And then as an administrator, there are some things that you have to address in 1.4 that relate to data collection training and reporting. I mean, you have to know these things and make sure that your stuff does them and on the monitoring party to be your work. Yeah, these are the sort of things that relate to 1.4. And you should be clear when we're done. 1.5 is about your local assessment policy. Jay talked a lot about it. The only thing I'd add to Jay's conversation is that he forgot to mention that we added a new line as the CDE to the local assessment policy. What we want you to do is tell us when you reviewed it with your staff. And that should happen every year because that's a working document or living document kind of guides your work as it pertains to your assessment. And if your assessments are wrong, it's going to affect your future funding and so on. So you need to stay on top of that. And then we'll talk about the end of year data submission and how that all works. Item two in our objectives is data privacy and personally identifiable information. We're going to talk about social security numbers. When you talk about individual text identification number, we're going to talk about consent forms, both the paper based and the electronic version. And then finally on three, we just talk about data monitoring. Like I said, you know, what you do in the interim as an administrator in those intervals, you know, between your calendar of due dates, which I do usually quarterly. Okay. So let's let's begin. This is a web page, you know, I mean, this is a slide showing you the beginning of the year later on our website is is titled program and accountability requirements. I'm hoping in the future will change that and make it the beginning of the year later because I think that makes a straight line of where all this begins. And then as a housekeeping type of thing, if you see on the slide, any of these items underlined, it means that you can click there and go to the web page, and then you can visit this. So this is obviously a sample that I'm in the picture, and you can see where it begins it talks about how to manage your local administrative costs. Aturo, you know, discuss that in depth. And then it talks about monthly attendance requirement to the next paragraph and so on. All right. And if you want to look at it. I mean I'm not going to keep clicking these links but it's taking its time. This is what it looks like. And it just tells you all the changes we've made like that are pertinent to that particular fiscal year because this thing changes from year to year. Right, like I just talked about the assessment policy. There's a new guidelines for this fiscal year regarding the administrator and local assessment policy with your staff. We want that to happen. So this part has changed and I've made you aware of it. There's memorandums that have been discussed by Neil, you know, and Catherine Peacock. So it covers all the things, all the changes that you're on the hook for. And these things are legal requirements, and yeah, I mean just follow them. That's all I can say. I'm going back to my slide. Back at my slide, you know, I've shown you how the slides are going to go moving forward. There's illustrations of what the webpage looks like. And this one covers the beginning of the year later program and accountability requirements. And then we're going to move fast enough. And then we also provide you with a calendar of grant deliverable due dates. Like I said, I'm going to make this simple. When I first came to the CDE and was working with my 13 or 14, you know, I mean, 23 esteemed colleagues. This is what they told me I must do. They told me I'm gala. You need to get the calendar of deliverable due dates. That's what you as a consultant and the CDE are on the hook for. And you need to help agencies do this. And then if you want to go deeper and deeper into the we are titled to program. And learn more. Just like this venue or this conference, you know what it is providing, you can go deeper. But by next week. You should know that this is what I'm supposed to address. So there's some things that happened that are required by September the first by September the 15th. Regardless of, you know, I know I'm addressing a crowd of experienced and some people that are not experienced. These are the things that you do. You are due for. And these things keep you in good standing with the CDE. Like David said, you know, how do you identify programs that are performing well. This is one of the criteria. And these things are not coming on time. It makes, you know, I mean, it just peaks our ears, you know, and we'll see where it goes. Okay, so be very keep that, you know, posted on some board somewhere. And, you know, as you're running the program learning more about these intimate details about the program, keep that calendar there. And that calendar, however, you know, now I'm on the next line. We have agencies that group together and apply for the grant as a coalition. And so some of those deliverables have to be submitted by the individual agencies, and some of them have to be submitted by the coalition. And we provide, you know, this table that you see in front of you, you know, to demarcate, you know, between those lines. So the fiscal agent, you know, they can see their responsibility in the second column of that table. And the individual agency can see their responsibilities. You know, demarcated in that table. Next. Well, later on, since I'm still going over the, I'm still in the overview, we're going to go over data collection training and reporting. And this is stuff that is separate from the calendar. And some of the things that are pertinent there is that you need to have your agency complete two trainings. Okay, two training somebody during this next fiscal year from July 1 2023. Yeah, June 1. All the way to July. I mean to not to June. I'm sorry, July 1 to June 30. You should have somebody always present that has taken the accountability training and the implementation training. All the time, and you should be able to provide CDE, you know, especially when the people go on FPM records, if we call you, because the data is not coming in correctly. I mean, this is one of the things we look at, you know, to see if you have someone, you know, that's competent to handle TE and understands what we're talking about right now in terms of what is what are the deliverables. And the CDE through CASAS, you know, offers you all these manuals, technical assistance, and please, you know, take advantage of it. Okay, the next thing. Oh, sorry, I skipped. I'm sorry, I skipped the pay. The next topic I'm going to talk about is the California assessment policy. If you go to this link. You will find a document. It mentions all the tests that have been approved by the feds for us to test our students for the next fiscal year. As Jay mentioned earlier, some of you may be continuing with the old test but there's a great period, you know, regarding that. So after a while, those tests are going to expire. And then some of you have jumped onto the new test. And so this document is a guide, you know, for you as an administrator to decide, you know, what sort of test you want to do, what sort of protocol, and so on. And this is a pass through item I need to stress that. When I say it's a pass through item, what I'm trying to say is that the CDE cannot decide which tests we should use for this particular program. They have to be approved for reliability and validity through the NRS. And then they can be administered, you know, throughout the country. And California is working with CASAs, you know, to do that. And so look at this document very carefully and it will inform you, you know, on what is valid, you know, for this fiscal year. And then so we pushed that down to you. That's why I say it is a pass through type of deal. I ask you to make a local assessment policy to guide your agency and to guide your staff and everyone who's in your data team should be aware of what the local assessment policy, you know, the content. You need to provide a template. And you need to fill out the content, you know, things like how does your orientation work? Who does it? What are your testing intervals? When are you going to pre-test? When are you going to post-test? Those things need to be expressed in your local assessment policy. And you need to share it with your staff. That's why I said it's a working document earlier in the prison. You need to share it with your staff. And then if you do remote testing, there's documents that help you, you know, get authorized with that, with CASAS, because there are some rules, you know, about being a proctor. And all that that you need to educate your assessment people on how to do, you know, the virtual testing. Okay, on next slide. And then since a lot of you are new administrators, maybe you haven't participated in this exercise of submitting end-of-year data. Four months, usually four months before the end of the fiscal year. Before June 30 of next year, we're going to send you some documents on how to submit your end-of-year data. And as you can see, like Jay mentioned earlier, you know, end-of-year data is due before July 15. And if you submit your data before July 15, submit what you have. Okay, submit what you have. I mean, do your best to clean it up, but submit what you have, because sometimes we get questions from agencies that sort of postpone the whole thing because they're trying to be perfect. But CASAS can look at that data for you and tell you where the holes are much quicker and help you fix your data. So submit what you have on July 15. You have a grace period where you can correct whatever data you submitted to us. It's about 15 days, so take advantage of this. And within that end-of-year data submission notification, you'll find documents such as the submission instructions. And this provides you with a step-by-step guide of how to submit your data in TE. I mean, as you can see, all those areas that are enrolled within this diagram, I mean, it just tells you the clicks to follow, you know. And if you have trouble with this, we have a troubleshooting document. And it allows you to look at the data integrity detail, payment point summary audit, data payment point flags, and so on. So if your data or the payment points are not looking like you have seen your agency, your agency's performance in the past. These are some of the things, you know, that will, you know, these are the low-hanging fruit to find out where the problems could be. You want to maintain your payment points and make sure that they're the same level as they always have been because that's what's going to stabilize your funding. And so that's what this relates to. Okay. Next item, we're going to talk about data privacy, securing, you know, personally identifiable information. We have a management bulletin that describes, you know, how you are supposed to treat social security numbers. We are updating it right now to include the individual tax identification number. But what I can say ahead of that, even before we approve, is that the same guidelines that are working for the S social security number are exactly the same as those for the item. Basically, there's three requirements basically with this stuff. One, you have to tell the student that it's voluntary. Not that it only, yeah, so one you have to tell the students it's voluntary and it's not going to stop you. Or it's not a barrier for us, you know, for you participating in this program. That's one thing that as a CDE consultants will look at. Right. So it is, it is not a barrier. And then the fact that it's voluntary. And then you also tell them what it is being used for. Right, just to track their employment status. And basically that's it. And so after reading the bulletin and learning more about the data privacy guidelines as indicated in number one. You can begin to formulate your own. And we have a template. And it relates to item number two. In this slide, and where it begins voluntary authorization to share, you know, personally identifiable records form. So that's a template that contains all the three things that I mentioned, and you can create your own. And it can be online, you know, when you give the student and they can read it on their own just like we do the bank records and all those kind of things. And yeah, I'll finish that. I mean with that on that part. And then we also have translations in the 16 most popular languages, you know, California, I'm actually proud of this page. So I'm going to share. So everything Arabic, Armenian, Cantonese, Farsi, Japanese, all those kind of things. And if you guys can make more translations depending on your location would like to see more of them. Okay. I'm going to go back to the slide. And then we have an FAQ, you know, you can dive into that. And yeah, that's all I have to say on data privacy, you know, there are the three requirements. We provide you a template, you know, see item number two here we have translations to deal with, you know, some of them in the most common, you know, languages that you will encounter the 16th talk. And then we have some questions that have been asked by previous administrators. And this is management, you know, bulletin 1701. Just a sample. This is the voluntary, you know, authorization to share, identify a goal. This is a template. And it has a lot of words, yours does not need to have, you know, all these words, as long as somehow you find a way to put the three requirements that I mentioned, you're good. And then electronic or paper form. Here's the place where we keep, you know, the translation. And if you click that underlined area, as we mentioned, as I mentioned in housekeeping, it will get you to that side. And then here the FAQs. And these FAQs also relate to the item. So we are going to update our documents because the item is a new development and essentially what we're going to be doing really is just putting item next to SSI because the rules are the same. And then, in terms of action, I mean, next we're moving on to data monitoring reporting. And I'm going to be talking about a lot of action items here. So you as the administrator right now you should have dealt with the program entry and update record because you have some students that you've enrolled. I imagine that you have tested them to determine, you know, what level they are at and you've inputted that data. That is the best practice and that should be an action item for you. And then as I'm discussing this, I also wanted to let you know that for this program, you have to enter attendance monthly in that entry and update record because that's how we track exits. And we mentioned this attendance monthly as a requirement in our assurances. And so everyone signed up and said, yeah, we can do this. And we really need this to happen because we're trying to increase our employment and I mean improve our tracking of employment outcomes. And if we don't do this, it's very hard to be successful on the other end. And we provide, you know, a wizard like J described in the morning that allows you to send texts and emails, you know, to conduct surveys. I really recommend that you take advantage of that and you make sure that you do that, you know, as prescribed, you know, in our calendar. And deliverables, you know, main. Okay, so that's one that these are action items, things that you have to do, you can't get around them. If you don't do them, you're going to be getting a call from the CASAS program specialist or just CASAS technical support. And then if it doesn't happen, you're going to be getting an email from the CDE consultant. And if some of these things are not completed, it's very hard for, I mean, well, it will delay your payment, not that it's very hard, it will delay your payment. We need this done before we process the payment. And on the survey, like Jay said, it's not necessary that, I mean, it would be fantastic to get the responses. That's what we're shooting for. But when I say full responses, I'm saying the student entering, you know, all that information is required. All we're looking for here is response rate. Did they say something, you know, in return? So don't be afraid to send the emails in the text actually to improve your number sometimes. I'm again, I'm sorry to interrupt. This is Jim. We have some questions. Would you like to take those all at the end or what's your preference? We can go right now. Okay. Maybe when you move to a new section each time, just check in with me and see if there's any questions. But we'll go over the ones we have. Just a moment, please. Okay. Our first one if we have all of our students enrolled in ESL and HSE classes fill out the CC apply admissions application. Does that count as a social security release form? The application included the consent to release information statement. Yeah. Because I mean, I, okay, so I'm saying yeah, because I imagine that the community college right when they designed the CC apply electronically, they have a consent form. Right. Yes, that's part that's part of the application I asked that question so I work for a community college and the students already fill out a lot of forms so I don't want to duplicate. You know, adding an addition to the social security number release form if the CC apply already counts because they have that the release of information, the consent to release information. So most of the students mark that they're willing to share their information, but there's some that do not want to share and we pull that information from our student information system. So I just want to make sure because right now we're in the, we're going to be monitor so I'm submitting all the documents and I want to make sure that we comply. Yeah, so, yeah, so all you need to do if someone was conducting the FPM and monitoring your agency. Or even if the feds came to and asked you about your data security policies, you just show them the portal where the students enter. And this is the language that's there. And then that's it, you're done. Okay, thank you. Great, and we have just a couple more at actually Janice just provided a link from cast us about student data security if you want some additional information you can find that link in the q&a and then our final question. Do we need to keep the social security number release form for record keeping and if yes for how long. I mean it's that's a really. I think you should destroy it after five years. Because we already have that SSN in our database. And we also have to destroy it too. Because after we've done our match with the EDD. We're confusing what's going on in that space because the feds want us to do longitudinal studies and the longitudinal studies take more than five years. They are confused, we're confused, but I would say like if you look at if you listen to. I'm not sure I mentioned, like, in the fiscal part like how long you're supposed to keep certain documents and before you destroy them. I think it's true. Can you help me on the documents that was supposed, I mean when we're supposed to delete them. Yeah, I am here for the federal regulation is three years. But the CD recommends by because the CD had all the feds have authority to go back five years for the federal part. I, I'm sorry I didn't, I didn't hear the specifics but like for example for personal information transcapes and all those, those are permanent those you keep those as long as you forever, and now you can keep those electronically. And, but it will depend whether the documents are federal or just the, they say that they're under the education console. It will depend on the specific but I get in general federal is three years says the regulation, but we recommend five because the gap. Requires or I guess authorized federal agencies to go back five years and look for if there was any miss spend funds or something like that. Thank you, Arturo. So, I think we should go with five years right because we're going to be viewed by the feds, right. Yes, I'm going to come to CDE and ask about agency documents and you know that's what they do sometimes they take a sample of the agencies to see if they're doing a job properly. So, in order for us to answer those questions properly, we need to agree. I agree. I just, I just mentioned the extra stuff because sometimes we make questions about a specific you know like transcapes or records that personal records, so those are different than the, these ones for data accountability and those that are required for the federal for the afla grant, which I agree. Five years is the recommended time. Okay, and Arturo, I posted a link on our records retention bulletin in the chat. Just for everyone's reference. Perfect. Thank you. We have a management bulletin on it. Thank you. And, and we had one more question that came in. Do we need to complete the sip every year. So, yes. I mean, well, in the years that we are doing an RFA, the CDE recently, you know, decided that in those years we don't do the same because we don't know who the successful candidates are going to be. And so it's a continuous improvement plan. You know, it goes over one year. So in the years that we do the RFA your application. You may not have to do the sip and that's more of a Carolyn type question, but that's what that's how I'm thinking about it. So the next three years, you have to do a sip. Right. And then that fourth year, when we go to the RFA application, you may not. Yeah, it's very unlikely that you have to do a sip based on our logic right now. Does that help? I hope I answered the question. Yes, it looks like we okay we just had another one roll in for students that re enrolled in the current year. They need a new sign form, especially for students and rolling towards the end of the previous year. Help me understand what the sign form. They perhaps are Laura could could you clarify are you talking about like the social security release form or if you feel free to go go off mute. Hi there. Hi. I am referring to to that form. And what I'm talking about is for those students that enroll, let's say towards the around May. They finish the year in June. They come back and enroll in the program in July. So do we need to have to one form for each program year. No, no. You don't need to do that. Right. It's not like you can avoid the fall. You remember when I doing the community college example, like if the students took off, and then they come back and CCC apply as that default page where you have to read it. So what are the guidelines. Yeah. Because, yeah, well, yeah, you do have to do it again. Sorry. If you're doing paper based, you do have to do the form again. Every year. Yeah. If they take off and then, you know, Jay talked about. Well, Jay. And then Carolyn have been hopping to you guys about periods of participation, right. You've heard that car, right. Yes, right. But what I'm saying is this is students are the first year is 30th. So in May, fiscal year and 30th. So they're going to come back. As of you like first. We have to do it again. Okay. Yeah. And we had one other one, a person's asking, does that mean no sip for 2324, but they will be required for this program. You're right. I'm a Gala. Yeah, so they said no safe for 2324. Yeah, but that that just supplied to last year because we were doing the RFA but for this year, it is going to be required right that's my understanding. So the last fiscal year, it was not required. And then the current fiscal year. I'm a little bit confused by the question because they're saying 2324. Yes. And it's not required. Well, she, she's asking, does that mean is, is no sip required for 2324 in April. No, it will be due is my in 2024. Right. Yeah, in April. This coming April. Okay. We have a sip too. Okay. Okay. She wants to clarify her questions. So then go, go ahead and go off mute, please. Thank you. That's exactly right. I know that a sip is due on in April 2024. I meant, we don't have one to continue on because we finished hours for 2223 that we had written the previous year. So we don't have any, we don't have one that we're working on right now, is that correct. What we're working on now is creating a new sip that will be due on April 20 in 2024. Am I saying that right. Is that correct. Okay. And I don't think you have to formulate something like radical change. I mean, I don't know your situation, but if you are continuing agency. I mean, your stuff has been continuous. Even though we said we don't want you to submit something. Does that make sense. I think so we are a continuing agency and so we're going to be working on creating a new wish sip right our goals will reexamine our goals and go forward. Yeah, I think I got it. Yeah, we were trying to deal with agencies that are not continuing. Okay. Because I mean that's why this policy is there that's what I'm trying to say it's like, why have them submit when they're not going to continue. Got it. Thank you. Thank you. So it should inform your future. That's all I'm trying to contribute. Okay. And then I talked about the two trainings that are required. This is another action item as an administrator. It's just one California accountability training course. And then one country implementation course, you know, which is item number two, how to use tea basically that's what that is and Jay gave you a little bit of a taste of that this morning. Next is the DIR. So I gave you a calendar of deliverables. I gave you some action items, you know, I shared with you all this stuff. But as an administrator, you also need to think about the things that you do in the interim periods. I mean the simple things. Right. Because these simple things. The DIR, the data integrity report in the interim can tell you that you're heading into problems before the quarter is over. If you look at the DIR, and there's no pre test after three weeks, you know, when the students have been there. That's a problem for you as that because that's going to affect your performance because we are all educators, you know, in this conference. And we know that the most learning happens at the very beginning, like, you know, at the very beginning the learning graph is like this. It's really on when the student becomes more proficient. You know, it becomes harder and harder to extract, you know, performance out of them. And so if you miss that first part, it means that maybe you've hurt your agency in terms of getting payment points, because that's when the student could have grown the luck of the most. And you would have your agency would have been rewarded for your effort teaching that student. And that intervening, you know, with the ABE ESL and getting them to the next level. I mean, this is a very, very important point for all you administrators to understand. And if you delay in testing your students. Sometimes you can delay too much. That is too late, you know, to do a post test with that instances where sometimes you find a program hasn't done a pretest. In January in the spring. Sometimes even late spring. And, you know, we at CDE and Casa Sakana scratching our head because once it gets to that point. It's this very little that we can do and it's going to hurt your data and all the work that you put into that student is not going to show. So in the interim, that's why the administrator needs to look at the data integrity report. And so I'm going to share some things with you on the data integrity report. Okay. So if you go to the Casa site, yeah, this is the DIR. If you go to the Casa website, you click on their documents that pertain to the data collection. There were some PDFs there that have the data dictionary. Those the data dictionary is above on a table. You know, it's more in the overview of the section and then Casa says a table and they have all these documents that you can look at that are PDFs. And one of them is called the DIR and this is our latest one. So with the DIR, if you look items one through 12. Right. All these things related to gender, no ethnicity in your data. Missing birth date. You can do a pretest and post test with that student. And if you're missing any of those things, your work will not come. I'm trying to stress with this. Okay. And then just to show you what, I mean, this is almost like a template of the DIR. What your agency DIR will most likely look something like this. Okay, so I create, I mean, I played around with the Casa thing I called it, no administrator school, because I cannot share the agency where I got this form. I was just hiding the data, but you can see, you know, they have 128 missing. I mean, less than 12 hours of instruction. You as the administrator in the interim need to discuss that with your staff. Hey, what's going on, you know, with our orientation. Right. Zero or, you know, these are the types of things you need to address in the intro, so that you have something to submit, you know, that represents the way, you know, you want your agency to be working and compensated, you know, after the programming. Right. So this is a working document. Like I said, I want to keep this simple. We gave you the beginning of the year letter gives you a calendar of the things that you need to do throughout the year in your first year. Very simple. Right. You talk to your data coordinator. You create a program for that. But then in the spaces in between to save your energy as an administrator because you know all these things are happening. This is the report from CASAS, the DIR that you should be looking at. It'll, I mean, it'll make your life easier. And then later on, as you become more proficient in the program, you can go deep in the weeds and, you know, start going in depth or if you have questions about some of those things in the calendar, some of these things in the data integrity report. Okay. Jim, do you have questions? There are currently no questions. Okay. I'll keep moving there. And I'm at the end of my presentation. And I mean, initially I wanted to have questions at the end, but it looked like it was working better, you know, during the presentation. So in terms of technical assistance resources. I mean, Jay shared with you the site. And here are a few links, you know, that may help you. So anything to do with California accountability, if you have questions, use the email address in item number one, shipping and payment questions. That's where you get your orders for electronic and paper assessments, you know, addressed with item labeled number two. And then on a short term basis, if you and your data staff are confused, item number three on the slide tells you where to get the tech support. And sometimes CASAS will come to your site to resolve these things. In fact, most of the time, if you guys are struggling and it's just not happening, do not hesitate to contact CASAS and ask them to come to your site. Because our program specialists are located in different geographical regions, unlike us, the consultants were here in Sacramento. And then it's their software, you know, so they know how to navigate it better than us. And then if you cannot resolve your problem within, you know, that space, then you consult with your CDE regional consultant.