 There's plenty of time for questions. Well, good afternoon from the East Coast, and welcome everybody to Course Marketing for Research on Impact. It's our summer conversation brought to you by the Research and Impact Committee for CCC OER. We're glad to have you here to talk about course marking for open and affordable course content and how it provides institutions with access to a wealth of data that can be used for reporting to funders, policymakers, and for other research purposes on campus and off. Our conversation will focus on initiating and the challenges of sustaining course marking at your institution for research on impact. Our conversation will start with a brief panel discussion and conclude with a conversation and questions from you today. And I'm sure everybody will have a lot of questions because I know that this is a very popular topic. Our panelists today are Bob Awkward, Debbie Baker, Lisa Young, and Sonny Pie. And I'd like to ask each of them to go ahead and introduce themselves before we begin. Bob, let's start with you. Thank you, Michael. As he said, my name is Bob Awkward. I'm with the Messages Department of Higher Education. I'm an Assistant Commissioner for Academic Effectiveness. My portfolio includes both leading our statewide efforts on OER, leading our statewide efforts on institutional assessment, and being one of the principals leading our equity agenda work, all within the public sphere of our higher education system in Massachusetts. Debbie? Hello. Debbie Baker, I am the OER coordinator for the Maricopa Community Colleges. I'm also an instructional designer in our Maricopa Center for Learning and Innovation. Sonny? Thanks. I'm Sonny Pie. I am a co-lead for OER work at the University of Hawaii Community Colleges. We work closely with our three four-year institutions. So a lot of the things that we do is at the 10 institution level. And this is University of Hawaii. It's our state public university system. I'm a digital initiatives librarian at Kapilani Community College. Lisa? Hi, everybody. I'm Lisa Young. I serve as the faculty administrator for open education and innovation at the Maricopa Community Colleges. I get to work with Debbie. And I also serve as the vice president of OER Global. And I'll turn it back to you, Michael. Thank you so much. And I'm Michael Magna from Delaware County Community College, where I'm the information literacy program and library services coordinator, as well as a professor of library services. And I also serve as the VP for research and impact here at CCC OER. And again, this is a summer conversation. So as our panelists are presenting about their institutions and their systems, if you have any questions, feel free to go ahead and post them into the chat. I'll be collecting those questions. And after the formal presentations, we'll have a chance to hear from our panelists. So if you have any questions at any time, feel free to put them in the chat. We'll make sure that we address those with the panelists. So Bob, if you'd like to go ahead and start our conversation today. I will. Thank you. First, let me thank everybody for coming. I know it's summertime. So it's hard, hard dirt to get people to come for professional development at this time of year, competing with, well, nice weather in some places, heat domes in some places, but in all kinds of, all manner of weather in other places. So we're greatly appreciative that you're here today. I'm here from Commonwealth of Massachusetts. And as I've already alluded, I'm really reflecting on the public aspect of our system. Obviously higher education is both public and private in Massachusetts, but we have an interesting state in that we only can really have real authority over private institutions created since 1946. I mean, we're in the state of Harvard and MIT for goodness' sake. So Harvard was created before the country was created. So how are we going to tell them what to do? We don't. So I'm representing, as you can see, our UMass system, our four campuses, our undergraduate-serving institutions, the nine state comprehensive universities and our 15 community colleges. And I'm pleased that Dr. Noe Ortega is a relatively new commissioner. He's only been here about eight years. He came from the Commonwealth of Pennsylvania, and he's an active OER supporter. Next slide, please. As such, I'm basically at the SHEO from Massachusetts. So I'm really talking, we have, as you can see, 28 undergraduate-serving institutions. And what I'm going to talk about is from the perspective of those 28 institutions working together, but all independently working together. We kind of call it loosely coupled system. And our work on this really started, as you can see, in June 2021, when the Board of Higher Education approved us to move forward with course marking as a, for all of our public institutions, as you can see, had only been implemented in 11 community colleges. And when UMass campus, at that point, they each had independently done so. But we basically did work with those institutions, learned from those institutions, and then talked to other institutions, places like Washington State and New Hampshire, looked at the famous course marking. I want to say the terminal tech book on course marking and use all of that to help us develop our own implementation guide to begin our effort. So our work in this space is relatively new, right? That's only a couple years that we've been working away at trying to get our other institutions on board with this because we recognize, and the Board of Higher Education recognize that doing course marking obviously enables our students to know what courses are using OER. So they'll be more likely to choose those courses, recognizing that creates its own dynamic, but it's okay because the point is we want students to use these books, free books if they can, because why not if they're available to be used? Next slide please. In Massachusetts, and this may vary in other institutions setups that you're going to hear from, we basically have done two categories of marking. We started off when we were just going to do OER, but then as we kept talking about, we realized, well, OER are not the only thing that there's no textbook to purchase and no cost for students, right? The students, if the faculty member uses all library resources, those have no costs to the students, they're free, or if the faculty member chooses to just make copies of things and use it and do it that way, again, there's no textbook to purchase and it's no cost for students. So we understand that open does not, OER and open are not the same. OER we understand that the five R's is coined by Dave Wiley, but to keep it simple and simple is important, but of course, marking purposes, we just check no cost and it can include those things that you see or low cost, which is $50 or less. Again, that may be different in other places. Some folks, the purists don't like low cost but we had institutions that were doing it 50 or less. We had folks who were doing it 40 or less and we did a compromise and said 50 or less because, for example, myself, I teach principles of macroeconomics, I went from a wonderful 225, probably $250 now, textbook, Cengage textbook to Lumen learning, that's $35, which puts me into the low cost teaching and learning materials and I made the case as did others. You want me as a faculty member to make that choice and give me some credit for making this move. So I should be able to get under the tent with other folks. In fact, the only reason I've basically used the Lumen learning is because they already have all the interactives and since I have a full-time job at the department of higher ed, I don't have time to create interactive so I need to have it come with all the materials so I can just teach with it. So we do both, no cost and low cost. Next slide, please. In order to support this now, you might think, oh, that's easy, everybody's gonna go gung-ho. Yay, let's do this. Well, if only it were so. There are people who get it. There are folks who don't wanna do it and then the folks in the middle who you're basically trying to move, right? Trying to use the folks who get it to pull their colleagues, recognizing the folks who don't like it probably will never like it just because that's the nature of the way it is. So we've had to do a lot of things you can see both in 2022 and again in 23 to try to help ensure people understood what we're doing, why we're doing this, what tools and resources we're providing for them, answering their questions. We created sessions with people with the same SIS student information systems could meet with each other and learn from each other on how they had implemented their campuses because we realized that was one of the challenges. I mean, not only do you have the faculty members that say, you know, if we do OER, all the students will take OER courses and we won't have courses. And I say, no, I mean, yeah, the OER courses will probably fill first, but if you're, since most of that's in the gen eds, they're multiple sections. And so the students who don't sign up as fans will sign up for your courses. Of course, you can always enact an OER course in your class and one of your courses that would help as well. But part of the issue was just understanding how to work their registration systems and work the logistics to be able to do these. So those sessions are very good. Again, this year in 2023, we did a series of seminars for people who were beginning, for people who had started but seemed to be a little bit stuck and the people who basically already had it implemented. So we customized these seminars for each of these groups. Again, trying to help people get traction and get them moving because it's continuous effort required here. Next slide, please. How have we fared so far? We're still kind of where we started off from. So I'm saddened to say that, but we're gonna continue with this work. We're not going away and they know we're not going away. Some of these issues are real. One of the issues beyond the faculty members individually at our state universities, we run into issues with governance, that we have state universities who say, we've got to run this through our governance process, yet they're not too quick to run it through their governance process so we can get this thing moving. So that stickiness is causing us some issues to get it moving. Although they say within the next year or two, they're going to get moving. And I've got a few ideas about how to get that group moving. The mass system is also in that middle group as well. And they're also committing that within this coming year, this coming academic year, they will get moving. So if I get to do this a year from now, I'm hoping that middle group will have moved into the top group and will be dragging the other folks in behind them. So that's kind of where we are. Next slide. And this is where I get to kind of say, because this was about research. So what was the point in doing all this stuff? Well, one big one of course is for students to know what books are, what's teaching, learning resources are open and low cost. Two, quite frankly, they encourage other faculty to join in implementing this in their among their courses. But the third, and this is where the research aspect is, is to collect data on what we call our key performance indicators and how we're doing with OER across our public institutions. So we adopted the COOP formula, COOP, which is Cost, Outcomes, Usage, and Perceptions, is the P, that's open ed.org, openeducationgroup.org. That's their work. COOP formula. And we selected a few. When you go in and look at this site, there's a bunch of things under each of these. Big dark headings. We selected a few that we want to work pursue. Last year, FY22 was our first year collecting data on all the ones that you see in the black. The two ones that are in red, what we call our equity measures, we're doing this fiscal year for the first time. Last slide, please. And here's what we learned. Out of the 28 undergraduate serving institutions, 17 of them reported 61%, which is good. That's good. We hope we'll get all this year. Next slide, please. Look what we found out. Of those 17 institutions, they had saved their students $7.6 million in doing OER and low cost, because we tracked both. That's a return on investment. As you can see, for every dollar that we spent, we were able to return $123 in savings to students. And then if you look at it in a full time, an FTE basis is $164 per student. So there's a caution in that. Of course, some of these savings is all not just an FY22. Some of this work was already ongoing. So that's a challenge. Once you do it, you keep getting a year after year after year, you get those benefits. But when you do data, you capture one time. At just one point in time, that's what we captured. We are delighted. So we move on with course marking in Massachusetts. That's our story to our next presenter. Thank you, Bob. So Maricopa Community College District is comprised of 10 separately accredited community colleges across the Phoenix metropolitan area in Arizona. We serve up just over 93,000 students. Again, that was last year. And in the 2021 to 2022, we conferred just over 22,000 different awards. These are transfer certificates, degrees, or occupational awards. Next slide. So how it started at Maricopa. And I'll jump there, Debbie. And it's so nice to see so many friends in the audience. Hi, everybody. So how it started. About 10 years ago, we came up with this idea called Maricopa Millions. And our goal was to save our students $5 million in five years without how are we ever going to do this. We had huge debates as to what do we call this? For these cost savings, do we call it OER? Do we call it no cost? Do we call it low cost? We had never thought of ZTC. And we had a debate that we were going to call it no cost, low cost, because at the time, we were really interested in cost savings. And the Washington state, Bob mentioned them earlier, they were doing $30 as low cost. And we surveyed our students. And our students came up with $40. So we were like, great, we'll go with $40. That's what our students feel low cost is. So we had a manual process. Paul Golish and I used to independently watch football together. And Bob, we had no football stories today in your talk, which I was really bummed about. But I also love football. So does Paul. And we would sit there, and we would go through the schedule and look at our courses to determine which looking at the textbook costs to determine what courses meant our no cost, low cost ceiling there. And then we would do the calculation. So because we have so many courses, as Debbie just mentioned, we couldn't go through every single section of every single course. So we just looked at the top 50 high enrollment classes. We had a very conservative estimate of 20 students per class. And we went with $100 per textbook. And we calculated our savings. But then we were able to, we were the first institution in the nation that put the search in, the course marking in our schedule. And so we had done focus groups. And our students were like, we are mining the schedule to find low cost materials. And we thought we'd rather have you spending your time studying or working or caring for your family. So we did it for them. And that made everything so much easier to calculate and then to dive into deeper data, which we're going to show you in a minute. Of course, we marketed this to students, did student awareness. We calculated those costs. And our process was very student-centered. We really didn't think about the data that we needed. We thought about the data that our students needed, which we're really proud of. But that leads us to where we are now 10 years later. So Deb, what are we doing now? So here we are. And we really want to know, looking back at the data, how much of those courses were using OER? Where was it? Simply zero textbook cost. And what percentage of these courses is falling in more of the low textbook cost range? So we have gone through over this past year to put in place a new course, new codes for the course marking. So coming up, I believe in the spring, is that right, Lisa? Spring 24, getting my years straight. Then the course marking will then reflect or have the faculty will have the opportunity to tag their courses as zero textbook cost or ZTC. Low textbook cost, LTC less than $40 or less, and OER. So some of these courses may be able to be tagged more than once. For example, something using OER could also be zero textbook cost. And so we would like to then be able to use this data for reporting, for providing options for students, for targeted faculty development, and also so that we have a sense of how faculty are leveraging these options to provide more affordable resources for the students. We know going into this, that this is going to require a lot of professional development for all of the audiences involved. So we've started conversations with faculty that are already using OER or low cost options. We've started conversations with scheduling. We've started conversations with the IT folks that have to put these codes in place. We know that we have a lot of professional development that has to happen in order to have these courses tagged correctly once these new tags roll out. We also see this as an opportunity. Go ahead and go to the next slide to further tell our story. So I forgot to, oh, thank you. The, we have a website was on the previous slide that has a ton of detail about our course marking. And so thank you, Liz, for dropping that in the chat. So using the data to then further tell the story that you want or need to tell at your institution. So here's a snapshot of some of the data that based on the current system of simply marking it as low cost, no cost, that one of the colleges was able to provide to then share out and say this is where these low cost, no cost resources are being used in that college across those departments. But as we move forward, we're thinking about what other stories we'll be able to tell with this data. So not just what courses are they in and what disciplines or departments are they in, but what programs are these courses living in? What meta majors or majors or Maricopa, we call them fields of interest are these courses living in? How does this helping the students in those spaces to be more successful and looking at a drilling down even further in, within those spaces to look at demographics around those courses. So what percentage of, for example, first generation students are in courses in these departments or majors or fields of interest that are also leveraging OER. So Lisa, do you wanna talk a little bit about student success on the next slide? Good, we can jump to the next slide, but before we talk about student success, one of the things I mentioned was cost savings and that our goal was $5 million in five years, which we did in two and a half years. We were absolutely thrilled. And after 10 years, we're over $30 million. And so that's some of the data that, $30 million that we've saved our students through no cost, low cost textbooks. But there's also a lot of other data that now that we're going to be able to disaggregate this to ZTC, LTC and OER that we can dive into. So ZTC is zero textbook cost, LTC is low textbook cost, $40 or less and OER is actually OER, openly licensed materials. So some of the things that we want to do, I was talking to Leslie Forehand over at Long Beach City College and she shared some of this amazing Tableau dashboard that they've built there. And just to see what we aspire to and what others are doing. So this is, there's VTC offerings, it's by programs, so you can see there's Associates of Sciences, Associates of Arts and you can see the total students declared and the percent of courses with ZTC zero textbook cost. So they're using this to identify how close they are to a ZTC degree. And if we can go on to that next slide, they are able to dive deeper. This is more of that information and I thought I had in the slide but we can go to the next slide we can dive deeper and drill down and say, oh, here is this Associates of Arts and Philosophy for Transfer Social Sciences and you can actually see which courses are not ZTC so then they can go and talk to the faculty in those departments and say, hey, would you consider offering a ZTC course? So this is really helping them go from ZTC courses to ZTC degrees and they built these amazing dashboards using the data from course market. So that's really exciting. If we go on to the next page, this is some work that we've done and there was a question regarding like, how is this impacting students? And so one of the things that we've seen just through our low cost, no cost courses is that community colleges, as you may know, have a six year, you know, that we look at completion over six years and what we're seeing is that students who are taking multiple low cost, no cost courses are staying in the system longer and not dropping out either their transferring, they're completing their goals or they're staying in the system longer and so we're very excited about that. And then if we just jump into our next couple of slides, College of the Canyons, I got permission from James Glopp-Agroslaw to share this information as looking at student success rates. So this is fall success rates. The next slide is retention rates. And so they're able to look at that based on everything down by race and then if you look at the following slide, we're looking at fill rates. And so we're seeing that courses that they're seeing that courses that our OER are getting filled faster as well. So there's a lot of ways that we can leverage this data. There was also a question in the chat about are you taking more credits and Tidewater College has seen that that's in fact the case as well. And so with that, I'm gonna turn it over to Sunny. Take us away. Hi, everybody again. Let's see. So just to repeat, I am a co-lead of the textbook cost zero program at the UH Community Colleges and I work with Wade or Shiro at Leeward Community College on some of this planning work. But we have 10 separate accredited institutions, three or four year. We started this whole thing as a group of volunteers for it started with a few librarians who thought this is a great idea. You know, we heard Keva Green speak and we got to go to an open ed conference. We rapidly invited instructional designers, instructors and even administrators started joining our group. We've been meeting every month since about 2015. The, in 2015, the UH Community Colleges Vice President's Office gave us some innovation funding to start a two college pilot then grow the program to the seven community colleges. Our 10 institutions use banner. So in 2015, the UH Manoa Continuing Education Program that's a four year. Initiated using a marking in the comments field, textbook cost, dollar, zero. This was outreach colleges like an adult education. So it's summer session. So our librarian there started that innovation and Leeward picked that up right afterwards and Kapilani picked that up right afterwards. So basically we're using the comments field in the banner registration system to inform faculty and students what courses were textbook cost zero or ZTC. But of course we were using a phrase so that makes it more complicated because you have to make sure that all the secretaries type it in precisely especially if you wanna do counts. So I was downloading our course offering data for Kapilani Community College and doing a search for dollar zero. It gets complicated if people have looked for, have done searches for strings. But the advantage of using the comments field was that each college had control over what they put into the comments field. So as a startup, we did not have to make the argument to the system to make this a system wide operation. So colleges kind of signed on one by one and by 2020, all our 10 institutions were on board and were marking. Then the definition of TXT zero, what is textbook cost zero? Basically, Leeward and Kapilani, we had to work it out as in our pilot program. And I just wanted to tell everybody that all of these links are working. And actually, if you download the slides, there's a lot of information in the notes. So don't download it as PDF, download it as a PowerPoint, so you can get to the notes. There's a lot of detail there that all of us have put in. So basically it was dollar zero, no costs at all. It's leans toward all the teaching materials are no costs, exams, no costs, materials that they have to read. If you have no cost teaching materials, but you have a lab fee, that's okay because that's equipment and it is instruments. And we have culinary and health services. So let's see, administrator started noticing our impact because we were going after cost dollar savings. We tried several times to get a banner attribute, first with a college institutional research person, then higher up the community college system, administrative level, no success. So you can see right away that this is very grassroots, we're trying to grow tall. Then in 2019, a 10 campus system level administrator, Hey Okimoto, she was associate vice president for student affairs and director of academic technology. So please notice she's doing the student affairs side, student services side, and she's also doing academic technology side. So she conveyed a meeting with us and she said, you guys are doing a good job, how can we help? So we asked about a banner attribute. So she knew the right people and before we knew it, we had a TXT zero attribute in banner, which went into effect spring 2018. When I say attribute, it means you just click one data element in the banner database. You don't have to type out precisely the phrase that we're asking everybody to use in the comment. It's also much easier to pull, to aggregate data and also just sort and select. So then that same administrator asked in 2021, our system, our 10 campus institutional research and planning office to generate regular TXT zero reports by which we can track statistics by campus and across the entire system. So this was very helpful because pulling all that data for each of our groups of OER campus advocates at each campus was a real headache. We have teaching faculty, we have full-time librarians, everybody is very busy. So now we have one report that comes out after each semester and also at the end of the summer session. So we have a sense of what's going on across the system. Essential stakeholders in all this process were our volunteer OER campus advocates, instructional faculty, department chairs and secretaries who have to click all those things to get it into Banner. Our campus registration specialists, they're the ones who answer the questions once a secretary doesn't know how to make this work. And faculty senates for some campuses. We got resistance from faculty senates for some campuses. So we had to get their support in some instances. And of course the academic affairs administrators at the campuses. At the upper level for us, it was the community college vice president's office, the information technology services and banner office for the 10 campuses, our student affairs office, the academic technologies office, 10 campus level and our institutional resource office at the system level. Next slide please. Next slide please, okay. So this is a very, okay. This is a very confusing, this is our output. This is our prized report that comes out now. We just got one for summer 2023. So when we approached about this report, we were asked, well, what do you want on this? So we basically constructed a data report that reflected our data collection process that we were doing campus by campus anyway. And our goal was to generate a report that could lend us insights into the diffusion of TXT zero practice and cost savings to students. So we wanna, you know, we're looking to see whether or not there's growth across the campuses to compare a campus with us, you know, with relatively low enrollment, fewer students than with a campus with higher enrollment. We look at percentages, such as the percentage of instructors who teach at least one TXT zero class. So that's helpful and people can kind of gauge how the knowledge is spreading, the adoption is spreading across the campus. So this data helps us calculate rates of class, TXT zero adoption using the course reference number, rates of adoption by subject matter slash departments and rates of adoption by instructors teaching at least one TXT zero class. You can also derive on a different sheet how many instructors are teaching repeatedly. And of course you can calculate cost savings to students using the $100, you know, generally accepted metric. So this data and trend data over multiple semesters is distributed to all campuses once the report is sent to us and the OER leads at each campus are encouraged to report them to their administrators, their own campus administrators. Wade and I as CC leads, we compile this information, we build trend data graphs and we send that to our vice president's office who continues to provide us with funding for this work. Next slide please. So each campus has a different workflow. So this is just to make things easier. This is my college's workflow. Again, all of this is clickable if you wanna get more information. I appeal directly to department and program chairs and their secretaries to get their faculty to remember to mark their classes TXT zero. Our curriculum specialist in the office of the vice chancellor for academic affairs fields any questions that secretaries or chairs might have about how to mark those classes. For Leeward and Kapilani, the courses are self certified. It's honor system for the faculty. However, we check on any questionable markets if they're reported or if we discover them. At Hawaii community college, shout out to Leanne who's in the crowd here. They have far fewer courses. So the OER lead requires her faculty to send her a document verifying that the materials are TXT zero and then she herself inputs that data into banner. So you see that little volcano in the bottom left of the PowerPoint slide. If you click on that, you'll get to see her criteria for faculty. As classes are entered into banner, they show up as TXT zero on the course availability list. The my UH portal store class registration system and the UH OER website. Then I appeal to all academic advisors to work with their students in finding TXT zero classes. You can see part of the flow here too is we have to inform our students. So it shows up for our college. This, that bottom in the middle column at the very bottom it says find your classes. A video specifically built by one of our library graduate students with the help of Leanne at Hawaii CC. We put together a video from the student's point of view because students have, they have screens that they see that I don't see. I don't have access to how to register classes for banner. So our library student worked through all the different ways that a student can search for TXT zero classes and what does that look like when they go to register for a class? So right now a student can say, I wanna take English 100. And if they click the right search parameters they can get English 100 off of 10 campuses. That's kind of radical. For faculties teaching at my college they know that a student who's looking for botany 101 can see whether botany 101 is being taught textbook cost zero at Hawaii CC. And if it's online, so that if you think about it, that's radical. Did we get pushback a little bit? Yeah, I got some pushback from faculty on my campus. But what we're talking about is we're trying to tell students where our budget is super important. They're trying to take care of their family. They're working part time. This is a really important advantage to them and we wanna make that available to them. Okay, so I recommend that you folks watch the video because it's very cute. And it also has a Hawaii perspective, which is really nice. So next slide, please. So in spring 2023, roughly 35% of classes at the community colleges were TXT zero. 8% of the classes at the universities were TXT zero. Using the $100 per text thing, the community colleges saved students over $1.7 million and the universities saved their students about $815,000. And since 2015, the community colleges have saved students over $17 million. One of the things that we really try to push and encourage tell everybody that this is a college-wide effort. So we have the staff involved, the faculty involved and the students involved in this process. Thank you. Okay, on to the next. Thank you so much, Sunny. And so we have some really great questions, but just to kind of get our conversation started with our panelists, can you highlight one or two challenges you had in starting course marking at your institutions? Sunny, I'll start with you. Nope, you're muted. For starting course marking, it was getting support from the right office. I mean, we kept going up the chain and so that was a little difficult. And then finally we hit the right person and it was great. Her response was, I can help you with that. So that was really wonderful. And also another challenge was getting support from your campus course marking staff to establish a new practice. So this is people on our own campus. And then we ran into faculty Senate problems with other campuses. The faculty were resistant. That's for me. Debbie, Lisa, Bob, do you wanna jump in? Go ahead, Bob. I would just say that from Maricopa, I can echo what Sunny just shared in regard to engaging the faculty Senate and having those conversations. We didn't include them. We had a great inclusive steering team, but we did not include faculty Senate. And that when suddenly our finding class marker came out, there was a lot of concern that's being echoed right now in the meeting chat of, well, my class might not fill or students, or those classes are cheap or what have you. There was a lot of conversation and there was a lot of, we really needed to talk about the quality of open educational resources and no cost low cost resources. The benefits, we had a lot of information from students and data and focus groups about the challenges they were having financially. So we were able to share that with our faculty and we got our faculty Senate engaged in our steering team, which really I think helped. And then also, we listened to them and they said, this is very prominent. Like you can't even search for whether a class is online here, why do you have this here? Can't you have it with other areas which we were happy to do. And so, it's an opportunity for win-win in terms of making sure you have the stakeholders there and engaging with them. Yeah, I would say in terms of challenges, the faculty Senate governance issues that I mentioned before, quite frankly, I have a lot of CAOs who are talking to talk but aren't necessarily walking the talk around support of course marking. And of course for me, it's easy to say because to me, I see it as low hanging fruit and totally in their wheelhouse because it both is about improving, expanding the options that faculty members have to teach from, which is teaching, which is in the CAOs wheelhouse and lowering cost improvement student learning, which is students, which is also in the CAOs wheelhouse. So to me, it's hard for me to understand why they aren't doing it leading at their campuses in a way they talk about this. Now, someone as I understand around political capital, they have to trade off what battles they wanna battle with their faculty. And there's no doubt someone saying, you know, this course mark is fine, but I got other battles and I'm not gonna use my political capital on this. In fact, it's one of the things I'm gonna do is get together with a group of the CAOs from the state universities and have them tell me how they can help move this needle on this issue because I can't do this on their campus. I can't just mandate it. Like Washington state, I don't have a legal mandate. So I have to do it through conjoling, information, education, influence. So, and then quite frankly, this will be interesting one to see how others reacted is that there's a, one of the issues with doing course marking is getting faculty to provide textbook information in general, nevermind just on OER textbooks, but just to get them to provide timely textbook information anyway in general, how long is that higher education act required that? I don't even, long enough that I can't remember how far back we've been required to do that and yet they still are resistant. So those have been some of the challenges. And so to kind of continue along this thread as we talk about faculty, we have a great question in here in our chat regarding, when you're publishing these reports or communicating this data internally to faculty, do faculty feel pressure to then switch and adopt OER, open text, affordable course content? And have you maybe done any research with faculty to see what their perception is? Now, Amy offered slides from a presentation that she and a co-author made at OpenEd 2022, but I just from the panel's perspective are faculty feeling pressure to adopt so that their course will enroll because we hear that that's some of their concern and have you done any research along these lines? I don't know, who would that be? Well, my quick answer is I hope they do. And I'm sure they do because we do hear the buzz about, well, you do, if you do course marking, well, you're not gonna, you know, my course is only gonna populate and then I won't be able to teach, yadda, yadda, yadda. I don't specifically, and that's a great research question, I just have to add that to what we're collecting if the system can handle it to see if in fact, how doing this is impacting other folks or other people feeling moved by it. I know that doing education of faculty is increasing their interest and willingness to adopt OER and thereby hopefully pull other colleague faculty along with them, but just doing, I'm not sure at least in Massachusetts is course marking is having that same full effect, I don't know, I'll be curious to see what other folks are learning. So if I could jump in here, I don't know how faculty in Maricopa, I don't know if they feel pressured to adopt OER but I can say that it's a good thing to start the conversation with them. But we talk so much about student success and removing barriers to student success and this is an easy barrier to remove and it's a good way to start that conversation and we do a lot to support faculty that want to move in this direction and we talk oftentimes, sometimes faculty are concerned about, well, I don't wanna write a whole book or I don't wanna do this, this feels too big to me. So we encourage them to maybe start with what they can do. So you don't have to transition the whole course all at once, maybe start small and help work with the librarians on the campuses at the different colleges to start identifying the open resources or the free resources that can help their students. From a course markings perspective, we really wanna move to a place where we can differentiate that data to say how much is really OER and how much is library resources or how much is low cost. Like where is this push for affordability really happening across the colleges? But it's okay to not have to go all in and jump into the deep end all at once but this is a great place to start removing barriers to success. Yeah, that's well put Debbie and we do have librarians across our tank campus system that are even just for starters are assisting faculty with starting small or finding commercial copyrighted materials that the libraries, we're already spending money on these resources. We're spending a lot of money on resources. So it's a great way for faculty to take advantage of our resources so that they can teach with those materials. Again, we are TX20, we don't require OER. And Lisa, that's a great comment about it allows faculty to connect with other faculty who can mentor them through this process. So that's excellent. Now, I'm gonna pivot a little bit and get off our highlighted questions on the screen because I think this brings us to a question that appeared in the chat related to how do we separate out low cost, no cost, textbook zero, ZTC, library license content? How do we separate all that out when we're pulling reports and communicating that data? Do you separate that out? And I'll speak from a library perspective. I know that library license content is getting a lot more traction with faculty. And are you pulling out that specific set of data because I'm sure that would help with library funding arguments on campus. So I don't know who would like to start there from our panel. Okay, Elisa, you wanna jump in? I'll jump in. And so this is the situation that we're in that we didn't have it disaggregated and it was really challenging for us. And so we did a lot of work in creating those definitions. We have 10 separately accredited colleges but our librarians are very engaged in the work that we're doing. And on some of our campuses, they are really looking at partnering with us as we look at ZTC degrees and how they can help leverage their budget and e-books. So an example of that is I teach a course that I have a great textbook and I'm like one of the OER people, me and Deb. And it's a great textbook and yes, I could take the time to write something but I love the textbook. And it was under $40, so it was low cost but I wasn't really feeling the low cost being my role. And so I went to the library and there was an e-book and we had one. And so I said, what would it cost to have an unlimited license? And it was $12 annually to have an unlimited license of the textbook. And so we were able to leverage that. But I think that, so that's just an example of ways that when you're partnering with them, you can really help to get that data to them and have them help you. But I think that the work that we did in creating, breaking down our definitions, do the work that we're doing in professional development of our faculty and schedule builders, the awareness campaign that we're about to launch for our students is all aligned with that. And if you look at the website that we shared, we have all of these scenarios that walk the faculty through like, is my course OER, is this DTC, is it LTC? How do I figure it out? And I think all of those pieces are a part of that. But we're so excited to have this data available to us come spring 24 so that we can hopefully do some of the things like Long Beach or College of the Canyons are doing and some of the work that Sunny and Bob have shared as well. So like we just can't wait or I can't wait. And so this question came up a bit and there was a great conversation going on in the chat. And so, Lisa, you're talking about, and I want to hear from everybody, you're talking about having to provide definitions for faculty so faculty can understand what course material falls under which category now. Here's my question to you. And I know Maricopa did some research to ask students what they consider low cost, which I got to say is great because I know that's often the debate, the debate is where does low cost start? Is it Bob, as you were saying, $50? Is it $40 at Maricopa? Is it $30? But from a student perspective, and this was a great conversation, wouldn't just free be a good course marking for them? Do you think there's value? Cause this was the conversation that was happening in the chat. Is there value to having these multiple definitions out there to educating students? Now it's great that you're having an awareness campaign. So I'm going to start with Maricopa. How do you get the word out to students? How are you educating students or students receptive to that kind of conversation? So I'm going to start Devin and then jump in. Like Sunny, we have student generated videos. We do a lot of awareness campaigns. Why we didn't just go with free? That's a great question. Free isn't free. And so it's free to the students but it's not necessarily free to everyone else. It's a zero textbook cost to the students. And then when you talk about free degrees, that's like a whole nother piece there. But I think that like we had a lot of debates about OER and using that specific course marking. And I dream of the day that our students actually search our schedule for OER for being able to leverage OER-enabled pedagogies or open pedagogies and being able to have them searching for those classes. So I'm just going to jump, throw that out and be quiet. I'll just add to that. I think Lisa is absolutely correct. Like that is kind of our hope for the future is that students will look for courses that are more open and that that will give us more and more opportunities to introduce faculty to open, not just open pedagogy but how to teach with an open educational resource. How to leverage the fact that these resources could potentially, depending on the licensing, be revised, be remixed, be contextualized to be more inclusive to localize the information or to provide information for the students in the specific situation that they're in. Excellent. And so I'm just looking at the time and I just want to offer the panelists last chance for remarks and we could just start and order Bob final thoughts on course marking as it relates to researching impact for students. Course marking, it's just so important both as I said, both encouraging more student demand and thereby increasing more faculty participation and for being able to collect data to see what we're doing and how we're doing. And just, you know, I use the course marking just makes it so much easier to do the kinds of data analysis and be able to disaggregate the data to see what's going on and what's going on underneath of those numbers. So of course, marking just makes it so much easier to do that than, and that's part of what's slowing down our ability to do some of the assessment we want because these people aren't just gonna do it manually. They're just not gonna do it. So that's why we continue to keep pushing on it. The other thing that we did, it was very helpful by the way is we did an implementation guide. We wrote it based on everybody's practices and talking to other states. We developed a guide that has definitions in a very specific so people know so the campuses are using those consistently across. So we're making sure when we're collecting the data we're putting apples to apples together when we're collecting the information. Excellent. That's our first. Sorry about that. Debbie, final thoughts? Excuse me, I'm looking forward to having, to being able to disaggregate all the data coming, moving forward. Oh, excellent. How about you, Sunny? Oh, well, we're not there yet, but we'd like to work with our institutional research office to look at student success factors like GPA and drop fall withdrawal rates for Pell, part-time Hawaiian students, Filipino students. We have slightly different categories and time to completion would be very cool. We do have one project where they're gonna start looking at student learning outcomes. So I'd like to see what they're doing and see if we can apply that also. Excellent. And Lisa, final words? I'm just really excited to look at how the course mapping works. Like if we can do something like Long Beach has done that I can't wait to see it. And then I also look forward to getting into the qualitative data again and doing focus groups again with our students and our faculty and some of the things that have been shared by everybody. I'm really excited to try. So glad that you mentioned qualitative data because qualitative data is so rich. Of course, it's time consuming, but I think we have some great ideas about really researching faculty, right? And their impression about course marking. But I wanna thank our panelists today. What a great conversation related to course marking, seeing what's happening at your institutions and your systems and how we can apply them at our institutions, our systems, our local. I'd like to thank everybody for attending today and please feel free to take our short survey and let us know your thoughts about today's webinar. And I know Liz put the link in the chat so you can go ahead and connect there and just know that a recording of this webinar will be posted online so you can recap and hear some of the excellent stories and the different resources that are available. So from the Research and Impact Committee for CCCOER, I wanna again thank the panelists for such a great conversation. I have pages of notes here and thank you everybody who attended. And we look forward to seeing you in future CCCOER webinars and events. So thank you so much.