 Hi everybody. Welcome to today's Community College Consortium for Open Educational Resources webinar on OER funding and impact measurement. I'm Matthew Bloom. I'm English faculty at Scottsdale Community College and one of the tri-chairs of the Open Maricopa project, also on the Executive Council of CCC OER. And I will be monitoring this webinar today. Very excited to have a number of projects and approaches related to securing OER funding and developing mechanisms and applying mechanisms for tracking the impact that the projects can have to kind of justify getting the money in the first place. We do have presentations from the Midwest Higher Education Compact, the California Consortium for Equitable Change in Hispanic Serving Institutions in their OER project. And then we also have someone from the InSpark Learning Enterprise, which is also related to Arizona State University's Center for Education through Exploration. Before we get into all of that, however, I just wanted to mention they'll have the opportunity to introduce themselves as they come here. We have Jenny Parks from the Midwest from Higher Education Compact, Katie Zabak, who is from the OER Cost Savings and ROI Research, and Smith OER librarian from West Hills College, LaMoure. And hopefully, I don't think he's on the call yet, but hopefully he'll join us a little bit later is David Schernstein, who is the head of InSpark, and the work that they're doing with EPX at Arizona State University. And as I said, I will be the moderator today. But first, a little tiny bit, this is kind of the obligatory overview of CCC OER for those of you who may be new to the work that we do. There are a consortium of basically members of institutions, community colleges primarily across North America. You can see that we have 94 members in 34 states, and that continues to grow and we're excited about that. What we do is we provide support to all of our members. The mission is to expand awareness and access to high quality OER support faculty in development of resources and their ability to actually use resources. Also, fostering regional OER leadership, and all of that, of course, has the purpose of supporting students in the end and making them more successful. So one thing that we wanted to try to cover here a bit and I'm going to pass it over to Una Daly, who's actually the director of CCC OER to talk a little bit about just how all the different kinds of funding streams that are actually even available for OER. Thank you, Matthew. Can you hear me okay. Okay. I surprised Matthew with this basically last night saying, hey, you know, some of the funding sources. We won't have time to talk about today, but I think it's worth mentioning. And one in particular that we're not going to specifically address today is the federal emergency relief funds. Many of you have heard about those higher education was actually allocated almost $40 billion just in March of this year and there was monies available before as well for higher education and this was around the pandemic. And so Spark recently conducted kind of an informal survey about institutions, higher ed institutions that had used this money for OER initiatives. And so I can report thanks to the lovely people at Spark that community colleges from California, Oregon, Maryland, Pennsylvania, Iowa, Wisconsin, and others have reported receiving monies to fund their OER programs. And how you do this is that these are grants that you get from the federal government or your institution would. And they have to be aligned with the criteria and OER aligns quite well with three criteria from the grants. And that's student reengagement, reducing financial barriers, and distance learning conversion. So, I would suggest that you talk to the folks at your institution who received those emergency funds and find out if they've been allocated, and what's still available because OER can very closely be aligned with the criteria for the grants. And that's really all I had to say so thank you Matthew for letting me share that. Absolutely thank you for providing that overview for us. Without further ado, we'll just go ahead and introduce the first of our guest speakers today who Jenny Park she's vice president of the Midwest and higher education compact. Thank you Matthew and thank you for having me here today. I'm excited to be here and to share the work that we've been doing at Mac and in conjunction with the other regional higher ed compacts around the work of OER. Next slide please. One of the people who will be sharing that work with you is a researcher Katie Zabak, a consultant on OER cost savings and R and the return on investment research that Mac has commissioned under a Hewlett grant that we share with the other regional compacts. Next, and she will also be sharing some information from some others. Kelsey Smith and OER librarian at West Hills College. I think what we want to do is go ahead and let, let Katie share her slides. Here we go. Okay, thank you. Then I'm going to do for okay so here's the thing at Mac, we have joined with the other regional compacts under what we're calling the National Consortium for open educational resources and see OER. We have a Hewlett grant to engage in a number of activities to increase the capacity for offering OER in all of our states but also with an emphasis on increasing equity in higher education doing so. One of the ways that we at Mac thought we needed to tackle this was to look at the ways that are returns on investment and the ways that cost savings are calculated for OER, because honestly, when you look at all of the different ways that these things are calculated and reported. Sometimes the message gets lost and the folks who are consuming that information just don't always know how to compare apples to apples and what to make of it. So again, I'll show this is the work that we're doing in conjunction with the other compacts and the specifically in Mac we're doing this research on cost savings and return investment calculations. I'm going to turn it over to Katie. Thanks Jenny. And thanks everybody for being part of this conversation can give me a thumbs up. If you can hear me. Perfect. Okay, so I am so excited to be working with Mac and this great group work group that you see on your screen right now to start to develop some standards around cost savings and how we measure cost savings and return on investment, which I'm trying to rebrand as cost benefit analysis. So, I think that the reason that Mac got involved in this project and the reason that they pursued this project as part of encore is because one of the best ways for our state legislate that our state legislators have any able to advocate for OER funding at the state level is by showing the cost savings so we are present to students. We know in the field that there's a lot more to OER than cost savings and there's a lot of other benefits to OER than cost savings but having the cost savings conversation and the return on investment conversation really opens the door to talk about some of those other factors. I know that prior to becoming a consultant, I most recently came from the Colorado Department of Higher Education, where I had the great pleasure of working to advocate for our OER programs at the state level, and being able to talk about costs, but also look at other factors that are improved by OER was something really powerful in our legislature. So, very excited to be part of this work. So, one of the things that we wanted to do. When Jenny came to me and asked me if I would be willing to work on this, I said yes, and I said that there's a lot of research and a lot of really good work already being done and so what we need to do is bring the smartest people in the field together to come to some kind of agreement around what it looks like to measure cost savings of OER and to put that down on paper so that people can reference it. And so we put together a great group of individuals who are on this slide, a couple of them are also on this webinar. So thank you so much for those of you who have already contributed. And we've been meeting regularly and then taking the ideas that get generated. And the other good work that's already been done in the field through both research and practical application to bring something together so that we can release a final report and a final set of recommendations. So, what are we doing at the basic level what we are doing is creating a set of principles to improve the consistency and reliability in the field of measuring cost savings and the return on investment of OER. The final product will make it possible for someone to recreate and or replicate a final number. We're not sure if we're going to publish a final number as part of this work. And, but what we do know is we want to create some standards around what the right approach is. The group that we are targeting this is our decision makers. So those are people who are legislators, their institutional leaders, their department heads who are having to make decisions about resource allocation. We want this to be accessible to students, parents, faculty members, but really the this tool is designed to help change hearts and minds at the top and to help advocate for additional resources for open education resources. And why is this important. So it's important for a number of reasons. The first is that the people like I used to be when I worked at the state of Colorado, who are advocating for OER at higher levels they need a concise statement clearly articulates and communicates the value. Decision makers people who have to implement or decide whether or not to fund OER programs. They need some consensus based metrics to use or to customize for their own decision making purposes. Leaders need to understand what good work has already been out there and then be able to adopt it quickly. For those of you who work with high level leaders, you know that they don't have the luxury of sitting around and figuring out what the best answer to with the best answer is they need information quickly. Practitioners who have limited time also need shortcuts. And finally, we all need to ensure that OER is continuing to add value. And so this the point of this work is to create some standards so that we can continue to measure the work. So these are some of the key questions that our research is aiming to answer. What are the current models. So, sorry, my, my zoom screen is covering up my second. That happens to me all the time. There we go. What are the current models cost savings of return on investment that are in the research there's a lot of them one of the things that I'm pleasantly surprised about in working in this industry is how much work you all have done to research yourself. What are the cost savings practices for states and institutions that are already measuring cost savings and ROI, you don't want to reinvent any wheels, we know that there's good work already happening. What's the difference between cost savings and return on investment is a really pertinent question that we've had a lot of conversations around. How is cost savings and ROI different for the state, the student and the institution, because they are different lenses. So that's fine or I owe we are when we're measuring cost savings and so that's something we're tackling as well. What's the time horizon of measuring the impact of all we are. And how do we account for non monetary costs things like faculty time structural changes, the returns to students, additional access, the various different returns that the research shows, how do we think about in integrating those into cost savings. So what you can expect to see from this is some guidance around what it looks like to measure cost savings. I put some best practices in here but as a general rule people identify the number of courses. They identify the number of students enrolled in those courses that have OER and then they develop some kind of multiplier the amount of money they saved with OER. And that's really where a lot of work is happening is for us to understand what's the standard around that what that multiplier should be. The second side of this is how do we give people guidelines for a return on investment calculation or some kind of cost benefit analysis calculation. The institutional and state investment side of things institutions and states put money into things like course development faculty release time integrating that the textbooks, and then the losses to bookstores, and then from an institutional side, in terms of returns there's things like increased course enrollment increased credit accumulation improved retention this is not an exhaustive list, but those are the kinds of things we're looking at as part of our cost benefit analysis approach. And then finally you can expect a set of principles from this work. So these are the working principles right now they continue to be massage through feedback from the field, but they start with the idea that all students should have access to course materials for their classes, if there's course materials, means that those are probably needed for the class and so we need to assume that all students should have access to them. OER resources support learning in the same way as commercial or all rights reserved materials and research shows they are equally or possibly more effective. We know that good course development happens regardless of whether or not you're using a commercial or all rights. If you're using a commercial or all rights reserved material versus OER, you have to do course development, and you have to integrate whatever material you're using into that course, developing new OER resources is not always required for our OER efforts. And so we need to take into account that new development doesn't always have to happen. There are benefits beyond direct cost savings that should be acknowledged and we hope that any cost benefit or any analysis acknowledges those. Consistency is important, but to scale estimates are probably going to be a valuable tool. So we're not going to get it exactly right all the time, but an estimate is a really good proxy for decision making. You need different levels of specificity in cost savings and ROI estimates depending on which stakeholder you're working with. So when I'm talking to the legislator, I need much less precise calculations about what my cost savings are going to be with OER than I do if I meet the head who actually has to budget on those things. And so those are the elements of our work and we continue to work on packaging those to provide more clarity to the field so that going forward there is more clarity in how we look at cost savings. Thank you, Katie. This is Jenny again. I just, I, a number of you have asked will these slides be available they will, because I agree they're fabulous, and I just can't thank Katie enough for her leadership and insight in doing this work. Well thank you so much for sharing that and again, I think just like a lot of one of the things that I thought was especially interesting about that was really distinguishing between the cost savings alone and the actual return on investment I think that that's very meaningful and something that a lot of projects maybe overlook, perhaps so that's really helpful. I do want to go ahead and share my screen here so I can get the slides back up and we have Kelsey Smith from West Hills College LaMoure. She's going to be talking a little bit of the Ed Grant project that they've got going on. Thank you. Let me pull up my slides. All right, so my name is Kelsey Smith I am the OER librarian at West Hills College LaMoure. West Hills College is one of the community colleges in central California. So today I'll be talking about, we abbreviate this because our title of our project is very long. We abbreviate it CC ECHO. It stands for California Consortium for Equitable Change and HSI. HSI stands for Hispanic Serving Institutions. For those of you who are unfamiliar with that abbreviation, open educational resources. So I am the project director for this particular grant. Okay, so our funding source for this project is the US Department of Education. This is the open textbook pilot program. We were one of four awardees for the fiscal year 2020. And I have the link there. I don't know if somebody wants to put the link in the chat but if you're interested in seeing the abstracts for the other three awardees. There is a page for that. And if you are unfamiliar with the open textbook pilot program is it is what funded the library text project from UC Davis. That was back in 2018. So every year since then we've had another project or projects funded by the open textbook pilot program. Our CC ECHO project will run from this year until the end of 2023. And I actually wanted to ask Uno or anyone else that may be familiar, is there plans to continue this open textbook pilot program funding every year like they have been. From what I've heard Kelsey, yes, there definitely is plans. But it's still being negotiated what the amounts and so forth would be. There's no information today and I couldn't sell. That's good news. So, for those of you who may be interested in what this program is, if there is future funding, here's a description, I won't read the whole thing except for the bolded part here. So, basically this program is focused on demonstrating the greatest potential to achieve the highest level of savings for students through sustainable expanded use of the open textbooks in high enrollment courses or in programs that prepare individuals for in demand fields. So, the open textbook pilot is very much focused on either high enrolled high impact courses or CTE career technical education. So CC ECHO is where focused on those high enrolled high impact courses, but a lot of the other awardees are focused on career technical education. This comes from the their site specifically so it's just if you're interested in applying you need to have at least three institutions of higher education in a consortium so CC ECHO has four colleges we're all California community colleges. You need to have some sort of ed tech person on board or a curriculum design expert, and then also an advisory group. Also, the, at least three in your consortium should have demonstrated experience in OER basically so the four community colleges which are West Hills College of the canyons College of Marin and Alan Hancock College. We all have experience in the zero textbook cost degree grant that was previously done in California, as well as OER programs on our own campuses so we're a very, very experienced in OER. And so if you apply for this or any other funding it's very important to highlight that highlight your experience in the OER field. The application for this was very intense. And keep in mind this is a consortium thing so at least three colleges. It was, it's difficult to do when it's not just one college working on a grant application. It's 60 pages. It was a lot of teamwork and trying to schedule time that we can all talk and it was tough so the most important things to focus on on the application is really establishing the needs so why do you, why does the country or the state need your project in particular. What are you doing that's so special, pretty much for CC ECHO we really focused on how the pandemic and the transition to remote learning really exacerbated the inequalities between underrepresented underrepresented racial groups in higher ed. So there are a lot of particular Latinx students in HSI's. The goals of our project were to fill the OER gaps in those high enrolled courses so there are courses on our four campuses that have a lot of students enrolled in them but they still aren't using OER textbooks for some reason either it doesn't exist or the faculty aren't happy with what's up there. And our goal is to create an OER textbook for them. We also want to create our OER textbooks based around a diversity equity and inclusion framework, as well as have them peer reviewed. And then another big goal of ours is to share out all the resources we either use or create during the length of our grant. So one way we established the need for what we are going to be doing was doing a initial OER gap analysis. Well, before we did the application. So we looked at all of the high enrolled courses on our four campuses. So which ones were using OER which ones weren't. And then we kind of decided from there which ones we would work on, and we will be doing a gap analysis every year, because as we do our work. OER is being created all the time. So what's a gap this year may not be a gap next year. So we will keep doing gap analysis and make sure we are working on what is needed and not recreating what's right out there. And another important thing to highlight in your application for any sort of funding is how you would be measuring the impact of your project, how we know it's successful. So we're going to be using qualitative and quantitative data. What's the return on investment. Every pretty much everything Katie just talked about, you know, the cost savings. How are you going to be figuring that out is the return on investment different from cost savings. Are you going to be looking at retention, enrollment, things like that. So OER has a contract with our P group, they will be doing our data collection and analysis for us. So we're excited to have them on board. Here's some of the performance measures. We will be using in particular, but this is going to look different for every project. The number of students enrolled in courses with the CC go adopt created textbooks. So when we create an OER textbook, if a course adopts this, we'll be looking at the number of students in that course. The number of students actually complete the course DFW rate, which is DF letter grades, which is a failure rate and withdrawal and the course and these OER courses compared to a course that is using a commercial textbook. So if one of our campuses has a macro biology course that's using one of our OER created textbooks and a traditional textbook, we'll be looking at the DFW rate in both of those. The average grades in those courses that have adopted our textbooks, the number of faculty that are adopting within our consortium and outside of our consortium too, and then cost savings. Another thing to focus on on applications is be sure to introduce your team. So who is going to be involved in your project? What would their role be their responsibilities? What's their experience in open education? Their education, if that's relevant, their credentials, their experience, etc. And be sure to highlight who will exactly be responsible for what, who's your project director, do you have a grants team on board, etc. We also, since we have four colleges involved in this project, we also laid out exactly what college, each college will be doing and what their responsibilities and duties will be. And then of course you have to create a budget. So how much money will you need to achieve the goals that you outlined earlier? And then keep in mind, we figured this out a little bit late, but when you're working in a consortium with different colleges, keep in mind that every institution pays faculty a little bit differently. Some will do stipends, some refuse to do stipends, some will do release time, some do hourly pay, and at least in California that hourly rate is different at every college. And then also if you're going to be paying faculty outside of your institution, how does that work? Make sure you know how that works. Some colleges can't do that. Others are fine doing that. And then you may want to have just one college at the consortium kind of handling the budget, or in our case each of the four colleges has their own budget and handles their own money. And then sustainability and scalability. So when you are working on an application for any sort of funding, be sure to describe how your project will be sustainable after the funding ends and how can it be scaled up. So for CC ECHO, all of our professional development courses that we will be creating will be shared publicly. Right now there will be on Canvas Commons, hopefully will be in a different format for other LMSs for adaptation and reuse. People can adapt these and edit them to use at their individual institutions. These will be self paced or facilitated courses. And then all of the OER that we will be creating, whether it's a textbook or course shell ancillary, those will all be shared to our website, which I have failed to put on this slide. So we'll be putting those on our website and then hopefully other repositories like deeper text, cool for that, etc. And then all of the data that we will be the RP group will be gathering for us, that will be published, and then hopefully especially HSIs, other HSIs will be able to use this data to justify and bolster their own OER funding on their campuses. So when we, if we get some great student success data saying that the OER textbooks that we created with CC ECHO, the students are performing much better in these courses, especially the Latinx students. Other colleges can use this to kind of show their own institutions why their OER initiatives may need more funding and things like that. I think that was my, yeah, that was my last slide. So I don't know if we're doing questions now or later, but that's all I have. Well, thank you very much for, for all that that's you. I mean, as some of the comments in the chat have indicated this is a very, very valuable playbook for for others who may be looking forward to trying to secure some funding in a large scale project like that. So thank you very much for sharing that we're going to, if any questions come up in the chat or something we'll save those for the end. We're going to go ahead and move here forward with our next guest. I'm sure my screen one more time. Okay, so anyway, yeah, so next up we have David Schoenstein. He is, as it says here, the head of the Inspark Learning Enterprise at Arizona State University where he works with the Center for Education through Exploration and full disclosure of the project, the open skill grant project that he'll be presenting. I'm directly involved in that as well in my role in Maricopa. And so we're happy to have him talk a little bit about his experience, not just on that grant necessarily but on some other projects that he's also worked on. So go ahead and take it over. Thank you Matthew. Hi everybody. Thanks for having me. I wanted to just say that I subscribed to, you know, everything that Kelsey was was saying so lots of really good detailed points there. And maybe try and just add some color by talking about the three grants that I've been working on and some of the strategies that we have used for acquiring the funding and winning some of these grants. And then also as Matthew mentioned, it would be great. I can to dive in a little bit on this Department of Education grant. So same open textbook pilots project that Kelsey was talking about where so that project is called open skill so we can we can drill down a little bit on that one. Maybe talk a little bit about the evaluation that we're doing and how that has even shifted throughout the project. I think that might be interesting. Yeah, and measuring impact. So the three big grants that I have been involved in was a four and a half million dollar grant from the Gates Foundation. That was the next generation courseware challenge. So that was all about scaling high quality courseware. So lots of active learning. So it was in an effort to really try and tip the market there towards the to, you know, to courseware and this low cost courseware specifically for students who are low income minority first generation students so. So that was a lot about developing and scaling. We, we have this Department of Education grant which is open education resources. That is similar. Although it's focusing on active learning with. Oh we are. And so there. We were in partnership with three big community colleges. Maricopa community colleges Ivy Tech Community College and Miami Dade College. And then there actually is another grant that I can't even really talk about too much right now but there's one coming coming down the line soon, which is a similar kind of courseware development so my expertise is really in these big kind of courseware development and scaling kinds of grants. So, at a high level, one of the, the things that have helped us are these are establishing early on the collaborations. So targeting a kind of development and scaling grant. The strategy that has worked really well for us is to is to have your, your development partners be like big collaboration so high impact collaboration so with the open scale project, we were able to. How you know we had built up relationships with some, you know, what are probably three biggest community college systems in the country, and then they are really were written into the grant as core collaboration and not and and development partners so the key with that grant was really to not just talk the talk is to really have the faculty engaged on the ground in iterative cycles of development in in the project. So that I believe is core to our strategy. You know obviously Arizona State University is a big place big institution. We have the expertise in the infrastructure and the scaling of high quality courseware. But it is essential to have the faculty on the ground to test assumptions and not get locked into any one particular solution. We have good user research on on the ground and release early, get that feedback and adapt. So, the open skill project is a very good example of that. Because we had faculty that were contracted as Kelsey was saying you know you have either set up contracts or stipends different states will do these things differently, but because they were funded. And the faculty had time to dedicate to the project. Then we were able to release mock ups and then beta versions of our courseware and get lots of really good implementation feedback. And we're able to rapidly iterate and improve on on the product. So, I would say that is a great strategy because when you have these large collaborations. You, you then have the feedback you need but also then the, you know institutions and the faculty feel like that, you know if they can see that you have actually provided critical feedback and to the product. Then they are invested in the product and then they are, you know, have a better understanding of of it and have an interest in trying to see it succeed and adopt it. So, I would say that's one of the core things that we have learned along the way and how to how to do that well. So investing you know heavily in those in those faculty providing those the feedback the. I would also say that it is important in terms of impact to have a really good, you know, measurably set your, your outcomes, but also be open to it changing so in the open skill grant. We started with you know laser focus on the saving students money for textbooks, and that is a very important objective that we, you know we wrote into the grant. We, but we were able to kind of pivot in this project. And, and focus in on what we're calling essential workforce skills and that was something that the Department of Education wanted us to include but it ended up being kind of core and central to our value proposition. So we, we had a pretty big pivot, you know, even in the middle of the grant and and the outcomes that we put down, you know, we were able to broaden our our goals and outcomes for for this project so we were actually able to extend them into well you know what is the cost of failing a course, that is actually a very large cost so we start to incorporate measuring outcomes in terms of skills. You know, and what is their ability to, you know, improving pass rates because you know that is also a large cost in the grant so the, the evaluation and those metrics have have kind of changed throughout the project. So if you'd like I can, Matthew just done a little bit more about open skill. And then happy to field lots of questions. How does that sound. Yeah, sounds good. All right. So I think I may need to have you stop sharing and then I can share. Okay, great. All right. Just very briefly then open skill is we are tools that promote this active learning and implicitly teach essential skills. And so we we our strategy there was even though we were designing for textbook adoptions in the specific set of courses. We wanted to be able to reach massive scale by having discipline agnostic tools, in addition to just the textbook replacements. So we saw the need to improve the quality. And through the improvement of the quality we would lead to the expanded use of OER, which, you know, meets the requirements of the Department of Education saving students money, but also improving student outcomes. And, you know, as I said we had a pivot on to actually targeting what we're calling essential skills that became a core value proposition for us. The tools themselves there are for tools that where instructor can actually come in and build their own assignments in a tool and then deploy it. So, as a funding source, you know this this was a very good strategy because we we built this out as open source tools. They can be adapted, they are discipline agnostic so they can go in massive scale. And, and we're not sitting on any proprietary technology or relying on any proprietary for profit company to deliver our products. You know, if it when we do deploy these through you know an LMS and and bundle it with OER. We provide the option to use a platform but that is again an open source collaboration with Carnegie Mellon and and a platform that, you know, it doesn't lock us into any one particular provider. Because it is an open source platform that we could just spin up ourselves at ASU if we needed to or you could. So, that's the open skill project, the tools like I said can be bundled with textbooks. And we are targeting these courses. And in interest of time I can probably leave it there and just field questions. There is extra time happy to show it do a little demo but I think that is probably my time. Yeah, thank you David that's very very I mean that's a great overview of not just the open skill project and the kind of the way in which I thought it was I thought it was actually pretty interesting to hear about, you know, project sets out with a with a really specific goal of saving students money but then as you are kind of going through that feedback process and learning from faculty in the classroom that there are maybe some other ways that you know that the tools could be developed to make the more effective that that kind of kind of necessitated a change, you know, in the measurements and trying to figure out like kind of even what the purpose was and so the ability for that flexibility and for the for the grant to evolve I think is is helpful to see. So, and it's exciting. And I'm just being for myself here because I'm actually an old net too but it's exciting to be in like the final year or the third year of this grant and see the tools finally coming to fruition. But yeah, so I mean I'm just going to go ahead and open up there's one question and chat we can probably start with that and I'm just going to say to any of our speakers here. Jenny, Katie Kelsey, David, feel free to jump in, I guess but so the question from Frank is any tips on finding and recruiting other faculty to join in your OER projects. This is a really important question I think as David mentioned already you know getting the faculty buying on the ground is really important but any thoughts on that. I could add just really quickly that I think what I'm not an expert in OER so Matthew and others know much more actually about other than me but my experience has just been that different institutions are in different places you know in their journey on on OER so what was really interesting about this project was that you have different institutions at different places and then by by creating a collaboration across the institutions, the, you know, we can support each other in that in that journey so at one institution they may have a very very nascent OER committee, you know so start you know we start there others you know like Maricopa have a very well established OER project. So, in, in my opinion, you know if you if you can focus and get on these collaborations and even if they're at different stages. So if you're having creating that network where you facilitate the sharing of resources across institutions effectively cutting out any kind of publisher. That is a good strategy. I'll respond quickly to one other quick thing here. Do you want to hear something. Oh, I was just going to see if Kelsey or Jenny or Katie had anything else to say about faculty recruitment but if not we'll just go ahead and move on go ahead. Sure. Well, well, Kelsey wanted to add something I'll just quickly respond the we need to add the MIT license for our open skill tools so it's just a response to the licensing. All the other OER is like the you know is the will be adding a creative commons license. As far as recruiting faculty there, there are just a lot of ways to do that, and probably far too many for me to go into at this point right now. There are a lot of online guides to that and then just a lot of people I'm seeing who are part of this webinar today, who you could probably even say our experts at it. If you wanted to contact me offline and be willing to be very happy to connect you with all kinds of ideas and resources. So actually, I want to take that back this conversation a little bit into the cost realm. If that's okay. So, one of the, you know one of the things we've struggled with as we talk about how we measure and consistently communicate. One of the things of OER is is different. Everybody does their faculty differently. And so trying to get to a place where we could quantify every little dimension of how organizations communicate faculty in the development of OER is would be a big mountain to climb and probably not one that we could actually get up. And so we've really been, you know, recognizing that there's so much variation in the field part of what we are trying to do through our class conversations is think about where are the commonalities, and how do we create a measure around that. And then, like, this is a really good example of where there's features of individual institutions and organizations. And so how do we create space in say our cost benefit analysis process for each entity to take into account their own work in their own system that they're using to approach that work. And so, I think this this point really illustrates some of the attention that we are working through through that work group. Well, thank you. There is another question in the chat here that I think is probably very interesting for a lot of folks and it says, I applied for US Department of Education grant for OER but I lost a lot of points for not having community stakeholders provide detailed input into our plan. How do you recommend getting local businesses or civic organizations on board. That's a tough one. We had an, you know, the timeline to create this to write up a grant was really aggressive for this one I remember. So we had to build up an advisory board. So maybe just having a workforce advisory board is a good strategy the Department of Education I think asked for this, when we had to submit in the, in the timeline that we had, we, you know, it was really hard to put together and that advisory board changed a lot over time but I guess at least just having a few really good names of people that could support that was essential. It's, it's starting to lead to connections directly to companies like big big companies like Boeing for example so it's very exciting so it is actually tangibly having an impact on the project and not just you know some meeting you need to do every month. I don't know if that's helpful though. I want to talk a little bit about what we did in Colorado for various different grants around stakeholder engagement there is, there are a lot of entities out there who are really interested in higher education right now. Whether that's an industry advisory committee as part of your parkings work, or a statewide workforce council, or more on the higher education side. There's a lot of advocacy organizations out there who are advocating for reduce costs. We know, I know that there's some particular, particularly active in states like California. So, oftentimes we don't think about those organizations as partners there you know advisors and they come do that one thing, but because they know our systems are only our coordinator in Colorado was really good at just asking if they can weigh in on stuff that they wouldn't normally be asked to weigh in on so think about what, and maybe search within your sphere of to what external groups already exist that might easily be able to kind of come in and bring some outside perspective. I could add a comment to, you know how to work with faculty if you want. Matthew just just generally. Because I think there are some questions going back and forth about about that. One thing that I think is how do you actually have, you know, like support faculty to transition to more evidence based teaching practices. You know, and I think a big blind spot in a blind spot in a lot of the research is actually implementation so you can provide great tools and you know courseware and things like that, but how it gets used on the ground you know enabling a lot of academic freedom and pedagogical you know ownership of these products means that the implementation is key you know that they're very different use cases, and so I think if you're going to propose something. You know targeting specifically one component of the learning sciences that you know that we know works and then how you know like active learning or sense of belonging and inclusion or metacognition. You know transparency in the grading, you know, some of these things you can focus in on and then show how like you wouldn't, you would have a plan for bringing faculty along you know whether it be for training professional development. And then, and then understanding the implementation piece you know saying that you are going to do an evaluation of implementation to really figure out what works for certain student demographics that would be my suggestion. Well, so are there any other questions or any last words that anyone wants to provide there I know that the, like you said Jenny the suggestions for trying to involve faculty is is that's something that we could. I mean we, I think we have done entire webinars on that topic so it might be something also if the questions are still out there. And you could just check out some of the CCC OVR archive because our, our, we've got all of our webinars archives going back years now so there's some pretty valuable information there. I guess I'll just go ahead and go through the last couple of slides that I've got here thank you to all of our guest speakers again today. We had a nice question slide that we could have used that we didn't. So we have some additional webinars coming up today obviously was this one to more this this fall we've got the intersection of EDI and open on November 10, and then we have a student panel on the impact of OER on December 8. It's nice to hear from the student perspective on that kind of thing so I hope to see you all there and of course you are invited to join the community email if you'd like to stay connected. There's a lot of there's always great resources and discussions being shared on that email list serve so please feel free to join that you don't have to be a CCC OER member in order to do that. And, and also there are impact stories and blog posts and like I said we have the webinars archived as well so there's a wealth of information on the CCC OER website so come to check out thank you everybody thanks again to all of our panelists or guests today and we will see you next time.