 All right. Everyone, thank you for joining us. I see we're still getting folks kind of trickling in as we go. We've got a good number. You are in already. Thank you so much for joining us to talk about tracking key performance indicators or KPIs for OER. This is a webinar with the CCC OER group consortium as part of Open Education Global. And as we are a global organization, a national organization, it's always fun to see where you are coming from. So if you wouldn't mind taking a moment, throwing your name, saying hi, and where you're from in the chat, that would be great. So on the agenda today, we are just going to do some introductions, look at an overview of some things we're doing. We'll have a panel, we're going to have a presentation from each of our panelists and then we'll have a general panel discussion. Free to ask questions in the chat, but we'll try to move those questions to the end. So we'll do a Q&A and finish with some upcoming events. So my name is Nathan Smith. I'm an OER coordinator of faculty and residents. I teach philosophy at Houston Community College and these are our panelists. I'm going to go ahead and reserve introduction for each of them right before they speak, give a little longer introduction to what they do. It's really nice. We have a very broad range of different folks who are engaged with OER at different levels. And I think they'll be able to give us some really concrete guidance on sort of what we should be looking for in terms of tracking success metrics in OER and presenting those to our stakeholders. The CCC OER mission is to expand awareness and access to high quality OER, support faculty choice and development, foster regional OER leadership and improve student equity and success. That's what we're all about. And I hope that today's webinar helps serve each of these goals. As you can see, our membership spans across the United States, both continental and including Hawaii. And if you are a community college and would be interested in participating in this community of practice, please visit the website cccoer.org slash member and become a part of the organization. We do lots of different activities that connect you to other people doing excellent stuff in OER. Finally, I just want to let you know about a campaign that's going on, I think is really important and speaks to some of the core values of OER. This is the free the textbook campaign. We know that the publishers and bookstores are promoting a textbook saving service that they are using a lot of the same language that we use in OER in terms of saving students money providing first day access. And they're saying that this this program is really the way to go in the future it's sometimes called inclusive access their proprietary names for that. So what I want to say is, you know, let's not be, let's not rest with, you know, textbook savings or savings students money. Let's get to free let's get to radically open let's really make it let's really open up the textbook. That's what OER does it's really a unique thing. So, visit this website free the textbook or find some in material there that you can distribute on your local campus, maybe you're seeing these programs come in. And I think it's important to know what they really are to read the fine print, and then to sort of to understand what students are getting into. Okay, so we're going to kick off the discussion with Judith Sebesta. She is the executive director of the digital higher education consortium of Texas or Digitex, and they are a consortium that provides support to community colleges in textbook in Texas. Importantly, for our purposes she co authored a 2019 survey of the use of OER in the state of Texas, and what people around Texas are doing. Judith. Thank you Nathan, I really appreciate that introduction and I'd like to thank CCC OER for inviting me to share our work here at Digitex. Good afternoon everybody. So, at Digitex we engage in a variety of initiatives historically we've been focused on inter institutional core sharing across Texas, but over the past couple of years we have expanded our initiatives to include leading the Texas Quality Matters Consortium, and also to support open educational resource adoption, adaptation and implementation at public Texas community colleges, and of course beyond. One of the first initiatives in which we engaged under this OER work was to partners you can see on the slide with the Institute for the Study of Knowledge Management and Education, and the Texas Higher Education Coordinating Board our state higher education agency on a statewide survey on OER policy and practice. We conducted the survey in spring of 2019 and the report came out in the fall of 2019. And when I finished speaking I'll drop a link to that record into the chat for you so you can go directly to it and see more detail. This was a survey on OER programs policies and practices at both to and for your public and private nonprofit institutions in Texas. And we garnered responses from 100 institutions painting a picture of really a growing commitment to OER across Texas really the responses that we got were quite exciting for this baseline statewide survey, which is the first that's been done in the state of Texas related to OER policy and programs and practice. Next slide. So questions top question topics ranged from things like policies, drivers behind OER work faculty training and incentives budgets, partnerships, OER course marking barriers to adoption and data collection and impact. And for the purposes of this webinar, I will focus briefly on the latter two. So one of the key findings that's pertinent to our subject today from this survey was that data collection on the pedagogical and financial impacts of OER in Texas is nascent yet promising. Approximately 20% of the institutions that responded to the survey said that they are collecting data on the financial or teaching and learning impacts of OER. And these were the KPI, the key program indicators that they were offered to respond to quality of teaching, student academic performance, student persistence in courses or programs, student engagement in courses or materials, availability of high quality materials, cost of course development for the institution and cost of course materials for learners. Next slide. So I just wanted to focus on this slide primarily for today. The question that respond that these 20% of the total 100 institutions responded to here was where data are available, please rate whether OER has improved each of the following at your institution. And so you can see here that they responded whether it stayed the same increase decreased or didn't know. And so I think you can just let me just pause here for a bit you can see I mean only 17 respondents so this is really baseline data for us we're certainly hoping that in future iterations of the survey which I'll talk about before I finish that we will have more institutions that can respond to this one. And those that didn't respond by the way said that it was it was really for a variety of reasons that they had not yet collecting the data that what the information they have was primary anecdotal their programs were in their agency, but they did many of them did indicate that they were hoping to collect data in the future. The smaller the institution was the less likely it was that they had engaged in this kind of data collection. You know, obviously the good news is here from this slide that quite a few majority indicated that the student cost of course materials did decrease so that's that's good news for us. So in conclusion, I just wanted to say that to the next iteration of the survey we hope to start engage start conducting in this coming spring we're having our kickoff meeting next week with our same partners isch me in the THC be and we have to perhaps include for profit institutions as well and maybe ask questions about things like the impact of we are course marking equity question questions on equity and the use of our new state repository. Oh, we are texts. Thank you, Nathan. I'll turn it over to Richard. How standing. Thanks so much, Judith. So, our next speaker is Richard Sebastian. He's the director of open and digital learning at achieving the dream. Achieving the dream is a large nonprofit supports national network of community colleges with some very important goals for student success he importantly Richard has led the achieving the dreams. OER degree initiative. And he's currently monitor managing a follow up grant project and investigate innovative uses of OER in the college classrooms so thanks so much for joining us Richard. Hi, everyone. I'm great to be here. So, if you can move to the next slide, Nathan be great. So I, you know my, my perspective on this is is pretty global at least in that national in regards to the kind of KPIs that that's that, you know, can be a value to many different kind of institutions in the US community colleges in this case. And the OER degree initiative was really kind of a very large initiative. Just to give you some statistics of kind of the final kind of data that came in of the really two and a half three year initiative involved 38 community colleges across the US. And there were 2,000 instructors were involved in the courses and course sections were part of this initiative. And there were close to 6700 course sections created of OER courses, involving about 160,000 students I'm rounding up or down on these. So you can see it's a pretty, pretty sizable initiative and it really, it's, you know, we're funded by the Hewlett Foundation, among others, the Gates Foundation, Cindy M. And it really gave us the opportunity to look really broadly at, I guess what I would frame as institutional kind of scaled OER implementation through this kind of OER degree model. And so we produced out of this two interim reports and one final report and I'll share the link for those after I'm done here. But let me talk a little bit about kind of the scope of the project and some of the some of the data that we collected. We focused on for the research aspect of the research study of this project three different areas so implementation of these OER degree pathway projects at these 38 colleges. What worked, how did it scale, what were some of the barriers so I'm really kind of on the ground, data on what it really takes to launch the kind of a scaled institutional OER program. So we're also looking at academic impacts and this is a little different than many of the, you know, the OER research has been out this because it was looking at multiple OER courses so really looking at the progress of students who take more than one up to four, or even beyond OER courses to have any kind of impact academically as they were able to take advantage of more than one OER course. So really looked at it kind of more about not really longitudinal but you know multiple semesters of data. And I think I think also a kind of a unique piece of this, of this research and the report that emerged from it were the economic impacts, really kind of multiple aspects of really you know what does it really cost to launch an OER degree. And, you know, what are the real costs and what are the real savings to students. So, you know, when I know when I was in Virginia running kind of OER programs there, I use $100. You know, I didn't have any other days I use $100 for savings right so if you count up the number of students in the course, and you know who use an OER in that course and then you know, you just rounded that $100 to how much they saved and really it's more complex than that. So we really, we worked with an RPK group, really dig deep on to the savings part as well as the cost part. So how much is it, you know, if you pay a faculty member a stipend of $1,500 for example to develop an OER course. That's not the measurement really because they use, they spend much more time so we had, you know, time logs to kind of record that they use kind of administrative, you know, a fluvia and people to support that. So really kind of getting down to really detailed costs for these course emails and they were hot. Of course, you know, even for just kind of adopting kind of an OER course even in a group, you know, it was pretty steep. So we collected a lot of people were able to collect data from site visits, interviews, focus groups, classroom observations in order to kind of produce these kind of robust reports. Next slide. And, you know, I encourage you to look at the final report that we put out and other two reports as well but as you can see we were able to get a nail down what were the real cost savings to come up with a model for that for students, what were the real pathway costs over time, right, over the course of time and, you know, what are they, you know, really cost for colleges to launch and the effects on credit accumulation and there's really good news all around in this report. You know, the scaled OER, save students money, institutions break even or even make money and students take more courses. That's the too long didn't read version, but I encourage you to read that. So just a few key takeaways since I'm running out of time. Next slide. You know, really it's important to think about who you're, who you want to report to at the end of this of your project or, you know, in midway to say who's going to continue funding it or who do you need to persuade or who's this data going to, you're going to need to, you know, achieve whatever goal you have for that project so think about the KPIs in that regard, and maybe some less tangible or obvious ones. There's some kind of interesting ways of looking at some of this data so tuition recovery right so you you're attrition drop 6% you have more students finishing a course what's that. What's the benefit of that of holding on to that to tuition dollars and how can you repurpose that, for example, and just one of many. And, you know, if you can try to kind of be consistent if you have multiple projects or if you have a multi year project of doing some kind of long longer term. Look at, you know, completion of courses, maybe even of degrees because I think that's going to be that's data that's going to be really important to know. That's that's what I got. And I'll go ahead and stop there. Thanks, Nathan. Thank you so much Richard and I'm sure people will really appreciate to see the links to those reports in the comments. Next up we have Michael daily who's the director of operations at SUNY OER services. SUNY OER or services is a big state University of New York's OER project. So in his role, Michael works with campuses and faculty to drive adoption, and he's also worked on collecting data he's worked with some external partners in OER and RPK group, and develop sustainability plans. So I'm here to join SUNY OER services he spent 10 years as the instructional public services librarian at Fulton Montgomery Community College. So I'm going to turn it over now to Michael daily thanks. Thanks Nathan and thanks everybody at CCC OER and everybody joining us today certainly happy for the invitation to share and learn together. Next slide. So I'm going to talk a little bit about the challenges and maybe some of the opportunities that we have found in the State University of New York, in New York, in particular working with our community colleges little background. SUNY is a large comprehensive system the community colleges are about half a half the number of our campuses so there's 30 community colleges, and they make up a little under half of our overall enrollments. So what you're seeing on the screen is a glimpse into the back end course section and student section or population section reporting system, known as Cirrus and SUNY so all campuses automatically report to this system every semester. And as we are courses sections are tagged correctly locally that information flows into Cirrus so by design theoretically there's a good system in place, however good data in good data out. So if your section isn't tagged or gets mis tagged. We have potential problems. As you can see perhaps also there are any number of combinations of interventions that might compose a student course experience those intervention might be federal or state economic programs so student can be receiving Pell Grant tuition assistant program which is a local state of New York assistance program they could be on WIC, the federally funded women infant children food program could be first generation college student. And their course experience could also have certain attributes that can be involved in a course as a leadership component to them as part of an honors college, but is also using open educational resources. And so one of the questions we often asked to unpack by administrators both at the system level. And when we work with campus provost is which matter, and there's any number of elements of showing a screenshot which matter and which matter more. I do want to make note in Cirrus there's there's some critical interventions that either aren't tracked for a very good reason, or tracked and not attached to a course reports those those might include elements such as mental health our students, making themselves aware of mental health services, food security, every SUNY campus community college or otherwise by state law as a food bank or food bank system. Students that utilize that service you know aren't tracked appropriately, and how do those interventions impact a student's overall course experience. And here I'm certainly nodding to I think the Hope Center's real college movement and some some important awareness that they're giving to some of those elements. There's other things that happen locally on campuses tutoring services room locations where room is where room is located on campus can students get there efficiently effectively you're at the start of the day or in between dropping their children off at a daycare center. And so all these things kind of compound, or potentially make really getting true impact of OER data, somewhat challenging. All that to said I think understanding that there's a lot impacting the lives of our students and we are is certainly one positive intervention so these challenges should not shy away from trying to dig into are just a creating the data. Next slide. We've heard a lot about RPK group SUNY was part of the achieving the dream. Oh, we are degree initiative that Richard just talked about we had five SUNY campuses participate in that and through that we got to know RPK group and some of the people that work at RPK group and reengaged their services in the fall of 18 to work closely with our system offices and campuses to construct a framework or an approach to sustaining oh we are over the long term. We have some focus groups in 2018 and we've moved on to sustainability cohort so these are groups of 15, 12 to 15 campuses going through a series of 10 in depth workshops over 18 months, where they really internalize and unpack that framework. And in recite and fundamental ways in which their local campus culture resources and infrastructure might inform or aid their OER program. Above all, I think our refrain when we work at campuses in talking about KPIs, you know, to dine on self culture to dine on self program campus institution you know be true understand what works for you. One key program indicator or success indicator at one of our community colleges may not be an important one 30 miles down the road and making sure that's that's a clear and apparent it is important as all of us begin to think about KPIs for own efforts. In many ways the broad and narrow elements of the framework are selected by campuses as their KPIs and some aren't selected. And these choices very often extend to other efforts underway, and in the works. Next slide. So I think a lesson that we've learned over the last three and a half and even longer because some of our campuses have been involved with that we are for going on 10 or 12 years now is that we are doesn't have to be or shouldn't be the thing. We very often be the thing that touches a lot of other things on your campus. And so an emerging successful key performance indicator for a number of our community colleges is the number of other student success initiatives in which OER plays an integral part. So Rockland Community College. The success OER course being a part of their realignment with guided pathway schools. And so their entire guided pathways program shows OER as their college success entry course, such as community colleges, uses an open math homework system as one element of their placement rubric for math courses. So it's a multiple measure tool. Monroe Community College, I think Michelle is on the call today with us. I just started using utilizing OER and co rack math courses or math courses where students aren't yet getting core college credit. And multiple, multiple community colleges and SUNY are extending their OER efforts to K-12 school districts that are part of concurrent enrollment programs which is a large percentage of enrollment for many of our community colleges. Certainly willing to share more information but I just want to give a glimpse into one key program indicator, which isn't necessarily guided by student success but on a programmatic level which is touching other things. That's great. Thank you so much, Michael. So our last speaker today, and again we're going to be able to take a bunch of questions and do a little bit of panel discussion afterwards but this is Mike Mills who is the Vice President of the Office of e-learning innovation and teaching excellence at Montgomery College. He oversees MC Open, which is the OER and free textbook program at the college. They were also a recipient of the Achievement of Dream OER degree grant and I should mention that Mike Mills is a current executive committee member, council member of CCC OER. So thank you for joining us, right, and the floor is yours. Thanks, Nathan. I appreciate the opportunity to share a little bit about what we do at Montgomery College and the KPIs we look at and it's a joy to see so many friends joining us today. So if you'll, yeah, that's fine. Montgomery College is a two-year school in Montgomery County, Maryland. Three campuses were rather large and we've been doing some OER work prior to 2017. Maybe we started 2013-2014 but we had absolutely no way of capturing the data that we thought we knew what was going on but we were having to individually ask faculty members how were you using OER, the number of students who were benefiting until we were able to be part of the ATD OER grant that helped jumpstart our OER efforts because we had to tag our courses. And so beginning in 2017, we had a firm understanding of the impact we were having. And you can see through this slide that the enrollments have increased every year and if we use Richard's generally accepted number of $100 and that's what we use at Montgomery College simply because it's easy to use. We've estimated about $6 million in savings since we were able to start calculating the numbers of students enrolled and if you'll go to the next slide please. And along with the enrollments, the growth has been evident in the number of sections that we've offered. So this fall semester we're just north of 500 sections. And it keeps growing and it keeps growing because the students are demanding that it grows. So, you know, one of the key performance indicators that we track is this growth. And it's because of the student demand. We have students talking to faculty members about wanting to use OER. We have students who are questioning why they have to pay $300 for a textbook and their friend who's taken the same course, different section, doesn't have to pay anything because the faculty member is using OER. So we don't mandate that we are at Montgomery College. And quite frankly we haven't had to because the students are helping to do our bidding for us and that's been a really big benefit for us. If you'll go to the next slide. The savings are great and students tend to focus on the amount of money that they save. It's still a key factor for them. They want to know whether they have to pay $300 for a textbook. But when we started working with faculty and really selling the benefits of OER to faculty and administrators, we had to drill down to the success. And when we started collecting the data, our goal was to not do any harm to students. We could save them hundreds of dollars in textbook costs and provide the same quality of education than it was a win. And so we've been collecting data, student success data since 2017. And without fail, the success is comparable or better in every demographic. So we break down by gender and by ethnicity and you can see just a couple of the past semesters here. The spring 2020 is an anomaly in so many ways because of COVID. You know, I don't spend a whole lot of time analyzing the spring 2020 data. But prior to 2020 it's been very comparable and I'll be happy to share more about that. Next slide, please. So Richard talked a little bit about return on investment and we too were part of the RPK study. And when you looked at what our students have been doing as a result of our OER work. If they're involved taking three or more OER courses, they took almost six more credits than students taking no OER courses and that does a couple of things. One is it speeds up their time to completion and ultimately benefits the economic stability of Montgomery County because they're able to take that money and reinvest it in either education or other aspects of county life. And then from the college standpoint, our ROI was just over a half a million dollars. Once our senior administrators saw that it was an easy sell to continue the program of OER. So we, you know, we have continued our work. I'll be happy to talk more about it. We've moved on to a lot of open pedagogy aspects of OER at this point, primarily focusing on social and global justice. So with that, Nathan, I'll turn it back over to you. Thank you so much. You know, this is great start to the discussion and I really thank you for sharing your expertise with us. So I'm definitely watching the chat and would be happy to throw questions you all have to the panel. I have a few questions that just to get us kicked off. One of the things that that came up in a couple of the conversation and couple of the presentations is this issue of course tagging and collecting data, you know, good data and good data out. I wonder if you could talk a little bit more about that. And Judith, I know you, you raised this as an issue in our earlier conversations. So I'll just, I'll kick this question to you if you wouldn't mind. What's what's going on with the course tagging and what, what should we be thinking about there? Well, I appreciate that, Nathan. It is the subject I'm very passionate about. And we did have one question in our survey about that I can come back to that in just a second briefly but I think, you know, what Michael daily talked about in terms of their work in SUNY. It's a perfect example of the advantages of engaging in course tagging or as we call it in Texas course marking, because of the kind of data that you can collect whenever you're marking the courses that utilize. And I think I probably don't, you know, I'm preaching to the converted if I talk about the advantages to students of that kind of discover ability. But I, Texas back in 2017, our governor, Greg Abbott did sign into law H a Senate bill 810, which requires that Texas higher education institutions share searchable information with students on we are. This specifically open educational resources and that's defined largely using the five Rs that we all know about. But our survey revealed that of the 100 institutions that didn't respond 61% have course markers in place. Not all of those though are using the term we are they're using a variety of terms from free to affordable and a couple of others. Texas does not mandate compliance so while while they mandate that you mark your courses, there is no one tracking compliance for Texas institutions and I, you know, I think what our survey is revealing that not everyone is complying that's for a variety of reasons and we shouldn't necessarily blame the institutions for that because I think those of us who have engaged in that know it can be can be quite complicated. But I think what I want to say here for the purposes of this is that we, we need to start working on collecting robust data on course marking from whether or not institutions are doing it are complying but also what is its efficacy both for students for the colleges for faculty members, what are the challenges what are the barriers. I think that there's a lot of robust data to be to be collected and as I mentioned we're hoping that we might be able to build a little bit more of that into this next survey. Now that our colleges have had some time to work on implementing that on their campuses. Nathan, can I jump in and piggyback when something Judith mentioned about the terminology used in the tagging, because when we first started talking about tagging courses at Montgomery College we first used OER and then what we realized was and this this was again 2016 2017 was that students may not know when they go to search for classes what we are meant. And so we, we changed our terminology to Z course for, for, you know, zero textbook costs. But it, it, it was interesting to see how students respond it now you know today OER is more prevalent the term is more prevalent and they may respond to that a little better than they would have, you know, five years ago. That's a great point. Anybody else have something to add on this. There was a question just to stay with Mike Mills for a second from in the chat about how it is that you collect data on enrollment and for your, your MC open classes who do you work with on campus to do that specifically. We, we had a report written by our IT staff that my assistant can, can run, and because the courses are tagged they're coded. It's easy for her to count up the number of, of courses that we're offering. And then from the student success standpoint we take those courses and create a report in Blackboard analytics and crosswalk that is the gender and ethnicity breakdown. That's excellent. And then, Michael daily do you talked a little bit about the, the central system you use but can you describe sort of which departments are involved in that process and what tools you use. Yeah, but you're gonna have to buy me a beer nation because it's gonna take a while. So it really, it really varies locally so we, and it depends on the size of the campus and kind of the local structures in place sometimes it's an IR office as Mike Mills just mentioned sometimes it's the registrar. Some of our larger campuses, academic deans of schools have that responsibility to make sure it's tagged before it goes to the registrar before it goes to IR. So through our sustainability work with RPK group we're really trying to broaden the understanding that reach is absolutely necessary when working with an OER program that it can't just be one person kind of you know championing OER that you're going to have to have a robust awareness across campus of what needs to happen to put in accurate measures to have this to have this happen. And by and large, I would say, you know, 99% of our campuses want to do this well. They're really eager to do it well it just at, we're at a point where scale is absolutely becoming a problem with doing this accurately and soony. You know we have 100,000 hundreds of thousands of students every year that are coded as we are in serious 10 to 1000s of sections. You know, and we know that we're missing sections and students and we also know that the reports are getting our inaccurate reports. To me, I struggle with why it can't happen if we can accurately code online courses and modality. Can we just look at OER or affordability or cost truckers as a modality and kind of ingrained that into cultures to make these kind of tagging processes happen more regularly. Thanks a lot. I wanted to turn the next question specifically because you mentioned, we've mentioned a couple times this RPK group and studies they've done and return on investment. And, and maybe Richard you could start us off here. Can you describe like a little bit in a little more detail like exactly what is being collected like what are the, what are the inputs in that that you have that we should be thinking about when we're looking at a return on investment study, and and and maybe walk us through a little bit of what that looks like on a campus. So it's pretty, it's pretty complex but there's a couple areas to it so the so think about student savings, right. What RPK group did were collect some external data on it to really develop a kind of levels of use that students purchase levels for students so you know students don't go out and buy brand new books right they don't all buy from the bookstore some rent, some don't get it also they use some, some external data to develop a kind of scenario of percentages of you know how students buying patterns and use that to and the textbook textbook costs to actually come up with a more precise number so on that slide. So if you had a range, like if you use the kind of traditional kind of calculations, I think there is savings of something like $112 for for that but if you look at the kind of the, the actual buying patterns of students that the state savings per textbook or per, you know per student is probably more realistically around 6668 dollars somewhere and you know $80 in that range. So that's how they kind of worked with the kind of student textbook purchases for for institutional costs around we are they were, you know, they had these pretty detailed time logs. So if you were, you know, part of this your faculty member developing a course, even collaboratively, you kept kind of detailed time logs of when you're working on we are and where and kind of what resources that you are using. And then and then they calculated, you know, from faculty salaries and from administrative budgets like what that cost actually is going to be. So they came up with kind of initial snapshot of, let's call them, you know, develop developmental costs for for OER. And as I said, you know, it was pretty steep. It was actually kind of disparaging when we first saw the numbers because it was like well this is, you know, this is super expensive for the institutions but but then you know, over the course of the study, obviously, they also took in the expanding course sections. It was a really super successful expansion of courses with the multiple sections and then kind of growing into additional courses. You see that cost, you know, if you continue that cost, you know, per over a couple of semesters, it drops significantly down to it went from a couple hundred dollars to like $21 per course, right. And then they then they added in like well in these these increased course enrollments and what does that so how does that take away from the, the, the overall cost from, you know, from the, from the development of this. So, there's a couple of the elements for for kind of coming up with these calculations but it was, you know, is really kind of some detailed costs and revenue data from from each of the colleges and they put that together to come up with their, you know, over the course over kind of a time timeline. That's great. So, so the, so the lessons I'm hearing is that over time the cost per course gets cheaper. Yeah. And then, in addition, as more students take our courses enrollment increases and I suspect retention increases or at least the number of courses that students taking these courses. The number of courses that they take in a semester actually increases, which, which is increasing revenue into the institution. Yeah so so looking at the, let's call them non we are students and we are students, you know, traditional we are students took additional credits, different kind of statistically different institutions but overall, it showed a kind of increase overall increase in course credits earned. And so that's that's, there's a there's a dollar amount to that right out of college for having a student take additional credits. So, yeah. Outstanding and Mike Mills, is this something you track at your institution the enrollment intensity or the the sort of number of credit hours that a student taking OER takes on a semester. Well, we, we, I'll tell you it's not easy to track and and, you know, our PK did a wonderful job of doing that it's, it's a lot of work. We're looking at that but it's it's not not easy data to to collect and you know I think it's something that we should all probably want to take a closer look at. I will tell you that in looking at data from a couple of years ago and I shared student success data. Two, two years ago we had the highest graduating class of Hispanic students that we've ever had at Montgomery College. And I don't think it's coincidence that when you look at their OER success the that Latin X population does really well with that we are and so it's a decrease in time to degree completion, because students are able to pump money back into their education. But to answer your question, Nathan, we're not collecting it as well and following as well as we should. That's good. Yeah, I mean I think this is the this is a challenge I think it's a great metric, but it is hard to track. So I was asking in the chat what enrollment intensity mean, and I just through it's refers to the number of credits the student takes in a semester and what our PK group found and then another study that did that looked at sorry, Tidewater Community College found also that that enrollment intensity increases when a student takes an OER. And, and that suggests that that maybe there's a return there. One thing I wanted to turn to maybe like daily I went and Judith if you could touch on this. I really like Mike Mills data on disaggregating by ethnicity. And, you know, one of the things that we, we know with OER we want with OER is to address the equity issues in education. And so and I really liked the way Michael daily noted like all of these different ways of equity like all these different parameters. But Judith, is, is anybody in Texas doing stuff that is trying to disaggregate the data on that level to look at are we having an equity impact. That's a good question, Nathan. I'm certainly there are off the top of my head. I really am not aware specifically. I can say that I'll brag on her a bit my digit Texas associate director Ursula Pike is in the current cohort of the spark open leadership fellowship, and her capstone project that she is going to be working on very soon is on issues of equity and I know participating in CCC OER's executive committee equity group Luna you probably need to put that in the chat exactly what that's called. But, oh, gay Lynn says that it also community college they just aggregate their data. We've seen some impact that supports more equitable outcomes so I assume that that's going on across the state but off the top of my head. I don't have examples for you Nathan. I will say that in terms of the study that we did the survey that we did what we did see in on a number of the questions was that there was a desire to, to collect the kind of data or be able to respond to the question but if they were a small college and that that probably was often a small community college they didn't have the resources to do what we were asking or to engage in that kind of work at their campus so I think that's that's an equity issue in a way, I would say. Excellent point yeah that's a really interesting thing. Well, did you want to, how do you disaggregate the data on some of those thornier issues you were mentioning. Yeah, we were actually, we talked about key performance indicators that's that's one of the charges from our, our there's a student provost to the provost that kind of oversees all the provost and student he's very interested in understanding the impact that the use of we are or affordable materials can have a students of need and define me broadly, whether that's Pell eligible students actually receiving Pell students of, you know, various difficulties, even down to gender. So that's something we're working hard on it kind of goes to the, can we get the best data, most often. We, we lead with the assumption that yes it's going to have a positive impact on those students and we can make learning resources more available at a lower cost, and that that's going to be a benefit. We just haven't been able to kind of disaggregate the data to a level that were comfortable yet, but certainly within the realm of possibility. And another thing we look out to, and one thing I should mention we don't get any faculty data in SUNY, which confounds the problem. I think it would be really interesting if we could get faculty to opt into participatory study where we could understand the diversity of faculty teaching with open educational resources. We know in SUNY that we lack a diverse faculty that, you know, matches that of our student population that's something we're working hard to address. But if, you know, if it's not a one to one if you don't have diverse faculty teaching diverse students using open educational resources, you have a much larger problem than tagging OER courses. Hey, Nathan, I think it's important to to see kind of OER use, especially now we know more about kind of his impact on students as a tool for closing equity gaps right and and that's a, you know, that was something that we, we recommended at the beginning of the OER Degression Initiative, but it was just too challenging really for colleges at the time. But really, can you, can you, you know, hold, you know, some seats or make sure that the students who really need these, you know, who need this break on textbook costs can get a seat in those courses because what we know is that it's word of mouth. There's just a couple of sections, you know, the, you know, the students who need it don't necessarily get into those courses. So, so being more intentional about that and, and really thinking about how you can actually more directly make sure that those students benefit is is another, you know, I think challenge for colleges to help close those equity gaps. You know, Nathan, I'll say one more thing in Texas, we've been very fortunate that our legislature of the past couple of sessions are our sessions happen every two years in the biennium that they've really been interested in supporting OER and providing resources to to help promote OER practice. Of course, what that what that is going to mean is that I think Texas institutions are going to get me to get better in the way that I think Austin Community College already is perhaps partly because of their work in the Achieving the Adream Initiative. They're going to have to get better at collecting the kind of data that the legislature is going to want to see regarding the impact of OER. So I suspect Texas probably is not the only one in that respect but that's I think that's what's going to need to happen if we're hoping that the legislature will continue to support OER at the level or maybe an increased level that they have. That's a great point. I did want to, I want it reminds me of something I wanted to hit on as well but I want to go to a question that Galen asked in the chat. And, and that has to do with, you know, when we look at student success, you know, we have the issue of students, who which students are we talking about which is what Richard brought up you know we want to try to target the students that are most at need and that can be challenging. But then we also have this issue of like well which faculty are self selecting to participate in OER and you know are some of the success metrics just a virtue of like sort of really engage faculty who care about students doing good teaching you know in the OER program. This can be really difficult. I know, and you can do it in a, in a kind of confined study where you look at before and after if of where you look at, you know, student, student success prior to adopting OER and then student success after adopting OER and the same faculty you can control for faculty effects. But what can we do at an institutional level to do this or is this just going to be something we have to live with. Nathan at Montgomery College. You know, we certainly have not mandated that anybody use OER, but a number of disciplines have decided on their own that all faculty teaching a specific course would use OER. So we, we, we try to account for those differences that way. And it's been successful. For example, we have in our economics department, all of the faculty chose to use OER for econ 101. And then one faculty member, it just wasn't working for that one person so that faculty member opted out. You know, if you can get disciplines to adopt OER, you can negate some of those concerns, I think. And also how many, how many contingent and or adjunct faculty are being left out of the ability to participate in this movement because it's not valued for them in terms of their, you know, compensation in terms of their time. They would value it, but if an institution doesn't for them that that concerns me as well, because we very well know it, particularly at community colleges, how many adjunct faculty teach students. So I just I really, really am concerned about the number of faculty that might want to participate in this that can't. Great, great point. Another equity issue for sure. So we are kind of coming to the end of time I wanted to point out just kind of lift up a comment thread that was happened in the chat where Ellen, Ellen range asked a question about those small colleges that have limited resources. And then Michael Dailey replied with I think really helpful, you know, just start small with the program you have and then build on the results you get there. Maybe just one final word from folks we've got about two or three minutes if maybe each person could just say one final thought kind of like, what do you think is the biggest impact thing that you have shown your administrators or stakeholder or funders that that has kind of kind of sold the OER program at your institution. Sorry, I'm gonna have to call in some way aren't I. Okay Judith, why don't you go first. Which one do you see in Texas institutions that you think like is is probably the most salient. Well, I think that the grant programs are statewide grant programs we now have to, I think are going a long way towards assisting faculty and colleges in implementing OER. And what yeah what do you think the legislators want to know about those programs what do you think is that their key concern. Is affordability. Okay, great. Thanks, Michael Dailey. Yeah I would say, you know, we're also legislatively funded in New York State SUNY and CUNY. The first question for legislators is affordability how much money did students say on the ground, I think the biggest win has been working with faculty to let them understand that OER is another choice in the portfolio than what they already had, and it's a choice that if you know chosen correctly provides them an awful lot of flexibility and more academic freedom than maybe they've ever had before. And that's been a real win for us. Awesome so echoing what Jessica Egan says in the chat remix. Mike Mills what do you what do you think what's the big ticket item for external stakeholders, funders, it's the student savings and the equity part internally of academic success that we're not harming students by by using OER. Outstanding and Richard. Yeah I think I think the we are degree initiative gives you gives people give people at colleges a range of different pieces of evidence, depending that could that could be useful for a lot of different stakeholders it shows that yes you can. Students will save money through OER programs that institutions can invest in a scaled OER program over multiple years and see a return on investment, similar to maybe not somewhere exactly Montgomery but you know either break even or or, you know, actually make money and that students take more courses. Assuming that they may even complete more quickly and so you'll see kind of increased completion rates that wasn't part of our study but so I think it's a win win win. And that hopefully the report will be be helpful and making that argument for for folks at community colleges you want to support their OER programs. Outstanding thanks y'all so much I think we've fielded a bunch of questions. So, as always, these webinars are free to the public and they are available recordings are available on the ccc OER website ccc or or slash webinar. We've had four webinars this this semester so you can see what is going on there and we are building our schedule for the spring in the next week and so look for the spring schedule to be coming out soon. Thanks Liz just shared the link in the chat. Today in the loop get involved join our very active email lists serve just loads of helpful information gets shared there. It gets archived and put on the web as well so if you have questions about you know do you have an OER in this or what do you guys do what do you all do over here. You know yet that's the sort of thing that this community really is beneficial at and definitely check out our equity diversity and inclusion. We have a series on the website. I think you know we believe equity is a critical issue in education, and we believe we are can address it but I think we need to be intentional about how we, how we do that so please check out the blog posts and the student impact stories. And, well, we will be back in February 2020 you've got the list of contacts here, our presidents, Lisa young and Sue tassion, as well as our, our, our stalwart administrators unidally and Liz yada. Hey Nathan, I don't think anybody wants to relive 2020. So you mean 2021 right. Excellent. This is like, we're going to be done. Please say it's not like Groundhog year. Oh, my. February 2021. Excellent. Thank you so much. Okay.