 Good morning, everyone, and good afternoon to those of you on the Central and East Coast Times. This is Una Daley from the community college consortium for open educational resources at the open coursework consortium. And welcome to our April webinar. This is actually the fourth webinar. This is the fifth webinar in our series this spring. We had three last month. And this is a very special webinar on OER impact research, both from the faculty and the student point of view. And this morning or this afternoon, depending on where you are, we're going to hear from Rob Farrow at Open University and Quill West at Tacoma Community College. Now very quickly, for those of you who haven't used the Blackboard Collaborate system, on your hand side of the screen, you should see a list of participants, including yourself. And at the bottom of that is a chat window where you can type in comments as we go along. And we welcome your comments and questions. We will be holding most of the questions until the end of the session. But please do feel free to type those into the chat window as we go along and we'll answer as we can. And then we'll try and come back at the end and get those. So I'd like to ask those of you who have access to the toolbar, which is in the middle of your screen there. If you pick up a little star and drag it over, you can, one of those icons, you can show us where you're located on the globe. Let's see. So we've got some folks from up in Washington State. If you can use that toolbar, type in the chat window and let us know what college you're with or organization. We'd love to hear who joins us at these events because they're open and free to all educators and students and interested folks. Great. Thank you for sharing your locations. Looks like, oh, my goodness. Well, thanks, Rob. And of course, Rob Farrow, one of our speakers from the Open University is over there in Britain. And I did see a little star over there which disappeared. Thanks for that. But mostly it looks like we're North America today. All right. So after some brief introductions and a little overview, we're going to get right into hearing from Quill on her liberated project at Tacoma College. Then we're going to hear from Rob about the community college research that he's done here in the United States with various community colleges, over 20 community colleges that he has worked with. And before we continue, I just wanted to mention April is National Community College Month. So another reason to celebrate this great research at community colleges by both Quill and Rob. All right. If you haven't had a chance to introduce yourself in the chat window, please go ahead and do that. And at this point, I'm going to go ahead and introduce our speakers and let them tell you a little bit about their day jobs. And I'm going to start with Quill West, who's the OER project director at Tacoma Community College. She's held that position for the last couple of years. Prior to that, Quill was a library and faculty working on the open course library in Washington State. So Quill has had a long leadership role within the OER community. We're very pleased to have her this morning. Quill. Hi. Thanks, Luna. So I'm Quill West from Tacoma Community College. And I actually have a new title now, and we're testing this today. It's Instructional Designer and OER Project Lead. And that is a strategic move on the part of our college to take a two-year project and make it a part of our ongoing institutional planning and integrate it more with the work that we do in instructional design work and consulting with faculty. And I'm really excited about it. So our project has been going for about two years, and we have saved students $643,000 to date on textbooks. Wow. That's really impressive. And two years ago, you set out with a goal of $250,000, and you've more than doubled that. Great work. All right. Next, I'd like to introduce Dr. Rob Farrow. He is the senior researcher at the OER Research Hub at the Open University in England. But Rob's actually been spending a lot of time in the United States over the last year and a half. It's been three or four trips, right, Rob, that you've made. And last October, Rob was with us for almost a month. So say hello, Rob, and tell us a little bit about your day job there in England. Hi, thanks, Amy. Yes, I'm Rob Farrow, and I work at the Institute of Educational Technology at the Open University. I'm not really an educational technologist by background. My degrees are actually in philosophy. And I made the switch into educational technology about five years ago, and since then, and since then, I've worked at the Open University. I've worked on mobile learning projects, accessibility, course evaluations, and some other stuff. So for the last three years or so, I've been working on OER research previously with the Open Learning Network and now with OER Research Hub. And so the work that I've been doing, we have a team of researchers, and I have the community college focus. So partly it's survey-based research, partly it's observations in classrooms, interviews, focus groups, very much mixed methods. And as I'll kind of explain at least one of the sessions, there are kind of good reasons why we're taking that approach. But it does mean I get to go to the West Coast of America for a month, as you say. So, you know, it's a very job that someone's going to do. All right, thank you, Rob. All right. For those of you who might be new to the Community College Consortium for OER, our mission is promoting the adoption of OER to enhance teaching and learning. And our mission statements, our supporting statements mirror very much Community College mission statements. We're all about expanding access to education. And we do this through supporting professional development for faculty and students who are interested in finding out about OER and how to adopt that in the classroom. And of course, this webinar today is part of that. And our focus is the Community College, although we do work with the four-year colleges and universities because many of our students do go on to attend those schools. And just to let you know, we are over 240. I think we're probably at about 250 colleges now in 17 states and provinces. And if you aren't a member, we'd love to have you join us. And if you have neighboring colleges nearby and you want to talk to them, send me an email. At the end of the webinar, I'd be happy to connect you with folks that are doing OER within your area. Now, before I introduce our speakers, I just wanted to give you a little bit of background on why this webinar today and why this is so important. So Rob Farrow actually contacted us. Well, Rob's team can't contact us a couple of years ago to do OER research work at the community colleges. And we were very excited to work with them because we are often asked for evidence that OER is making education more effective, that it's improving student learning, it's enhancing teaching, and we all have anecdotal evidence to support that. But in terms of actual research, that of course is more difficult to set up. And so we've been very happy to work with the OER hub to do this. And as you know, OER research evidence really informs curriculum development. It can inform academic centers in terms of policies. It can inform student advocacy groups around promoting the adoption of OER in their courses and when they talk with faculty. It's also very important for writing grant proposals and also for the reports at the other end as you're supporting the work that you've done. College board of trustees are very interested in evidence of OER effectiveness. And this helps the whole institution to craft open policies better. And also, as we know here in California, but quite a number of other states, our state legislators are quite interested in the effectiveness of OER too. And in some cases, our writing support of legislation. So this evidence of OER effectiveness really has a very wide purpose. All right. At this point, I'm going to turn it over to Quill to talk about the liberated project and with her specific focus on how students are impacted by OER. Okay. So as I mentioned earlier, the liberated project is a part of the TCC OER project, which has been going for about two years. And I think the first thing to know about this project that's really important is that it was funded by our student government. Not entirely, but about 50% of the funds came from our student government. And that was a process where our administrators who were interested in open resources asked the students to fund this project and the students thought it was important. And so I think because it was born out of a kind of administrator faculty student cooperation, it's really important to include the student voice in our assessment of success. So we have a project now. We're calling it We Are Liberated. And it's about student voice. We Are Liberated is all about collecting the student and faculty voice, but mostly the students. If you'd like to see some of the product of this project, you're welcome to visit that website that you see on your screen. And I'll put it in the chat window when I'm finished speaking. I often get asked where the title of the liberated fits within the overall TCC OER project. And so I kind of wanted to give a sense that the OER project includes four parts. And a lot of that is the outreach and development and assessment and working with faculty. But the liberated is specifically the work that I do with students. And it's the work that it supports the student voice in the project. And I think that's one of the most important things because I think no one has a better perspective on student experience than students. And they are often not told what power they have as are the reason why our institutions exist. They often feel like their voices don't matter. And they matter the most from my perspective when we're talking about student textbook costs because they're the ones who are bearing the cost. Beyond that, they're also the ones who are in the classroom having to work with these resources on a regular basis. And so it's important to collect their voice. So where the liberated project came from? It started with me here at TCC two years ago making a claim. And it's a really well-intentioned claim. It's a very nice claim. I still believe it's true. But I am obviously a believer. And so the TCC community asked me to prove myself, prove it. Prove that you really are. That this is really true that OER offers greater access to education. That's part of the point of the project. When it started, it was a pilot. It was a chance to get 10 courses and see how it affects student learning, teacher engagement, and a host of other things. So when I'm making broad claims like this, I have to be able to back them up. And we try to back those up in multiple ways. We track how much money we're saving the students based on how many students are enrolled in classes and how much the textbook would cost if they were paying for it. We track their satisfaction data. So we have an official survey that we use and we send it to the students and they kind of participate in the Likert scale. And then we look at achievement data. So we're looking at things like how many students completed this course with this year better compared to students who maybe a year ago or two years ago before the OER. So we do a lot of that kind of assessment, but it's really, and it's authentic and it's important assessment, but it wasn't getting the voice. I wasn't hearing from the students. So when we say a student goes to Tacoma Community College, how are they really feeling about the resources they use in their classes? So I set out to find out. One of the first experiences we had with that is we were having an OER leader come to our campus to talk to faculty and I thought we'd be really smart to have the students weigh in on that conversation. So we had a student panel and they said some incredible things about how they choose to learn. And they said some great things about how OER for them was really supplementary material and they told us that they really associate e-learning and distance learning with open resources and I wondered if that was going to continue throughout the project. So we did that panel and that was like, I don't know, April of the first year. I think I started April first and we had that visit like April 30th. So we put together a really quick student panel. And then we started surveying the classes and as part of that, because we were surveying the classes, the teachers started to teach their talk explicitly about their choice of learning resources and why they were using them. And all of a sudden, the students were actually having this discussion about the resources. And they started to have opinions and wanting to share more about it. So they started writing journal articles. They were journal pieces and just journaling and giving them to me. And so I have this stack of student journals, journal writings on why OER matters to them. And I can use it to assess the project but I couldn't share it with anybody because it's the student's intellectual property. So we started going to their classes and teaching them about intellectual property and asking them intellectual property and open licensing and asking them to put open licenses on their things, their writings. And that went really well. So then students started to pick open resources and creative comments licensing and textbook choices as advocacy issues in the classes where they were writing papers or doing presentations where they had to pick an issue. And we had some really innovative things. We had a group of students once say that if a faculty member makes you buy a textbook and doesn't let you and you don't use it effectively in the class that used as a faculty member should have to answer to another board of students, which that's a really authentic student voice talking about their experience. So it doesn't mean leading them to any conclusion. Their teacher did not tell them what to think. This is an authentic student voice saying, we still see this as an issue in our education and we want our institution to do something about it. And we as students want power in that conversation. And it was a really wonderful moment for me to hear students take some of the responsibility for what happens in the classroom, but also ask for accountability on the part of the people who are teaching their classes. Because that's why we exist. That's what we do as an institution. That's what education is about. So out of that conversation and out of the students saying they wanted a bigger voice and they wanted to share more of their information, we started a qualitative survey where we asked the students to talk about how OER has affected. If they're in an OER class, how has it affected your learning? What has it done for you as a student beyond the price, beyond the dollar savings? And we started asking for creative commons licenses on that. And one of the things that I want to really mention here is that Rob came, Rob Farrow, who's going to talk to you about this visit, came to TCC in October of this year and we put him in front of a group of students. We put him in front of a public speaking class and they had a wonderful conversation about what OER means to them. And they shared really cool things about how they like studying on their cell phones, which is something that is an instructor part of me says, oh, that's how are they ever getting the reading? But this is their experience of studying and it was really neat to have them have that conversation with an outsider. So we're trying to do more of that. We're trying to bring in people who have no affiliation with TCC to talk to our students because we're hearing better from them what they're experiencing with open resources. So I refer to that as a happy accident. And that happy accident really was asking somebody who's a leader in OER to come and talk to students directly instead of just talking to faculty because so often we have those OER leaders come and they talk to our faculty or they talk to our administrators and they do a great job of promoting OER from that perspective. But the student perspective is kind of missing in that so we're going to try to do more of that as we go forward. Born out of that was We Are Liberated. So We Are Liberated is our current campaign. We do things like the picture you're looking at on the screen right now is a banner that we had printed for We Are Liberated or that I printed. And we go post it in places where students are and ask them to stand in front of it and tell us what they think about open resources. And oftentimes we get a quick thing about costs. Other times we get real things about their learning. So on the top right you have a student talking about on the top right that you're looking at talking about how she liked reading her OER material and how it connected her more with her instructor. So we collect those voices and we put them in on our blog when they're willing to share them. And then the other thing that we do is a video which I think we've created an open, a video called We Are Liberated where the students stand in front of a video camera and talk about what OER means to them and how it affects their learning. And I think Luna is going to take over. Can you show that to us right now? This is only a 60 second clip of the overall video and I'm going to put our blog up in the chat window and you can go see it. Okay, thank you, Quill. I'm going to try this. So I'm going to take everybody's screen to this video and you need to click on the play button in order for it to play on your screen but you will be able to hear it through my microphone. And as Quill mentioned, it's only 60 seconds so if you can't hear it now you can always use the link that Quill is going to put in the chat window and you can listen to the entire video which is three minutes or four minutes I think later after the webinar. So here we go. I want to click on that play button at the bottom. Great, about OER, I think that it really allows the teacher to have a little bit more of a passion about what they choose and that comes out in their lectures. I feel like when there is an OER system in place their lectures tend to follow it a lot more and they'll put together a lot of resources to cover the whole course. It's not just a text book. Instead of just a text book it means you can interact a lot more. You can watch videos and what more communication is going on. So this whole thing is really cool because that actually don't have to be a boarding class. I know it sounds crazy but it keeps the whole thing really interesting as opposed to just going through a book and highlighting. We get to watch videos. We get to do things like that instead of just keeping it. So we can really have great in-class discussions because there's more articles being posted and more things to talk about. I think it's pretty good deal. All right. Did everyone get a chance to watch that or hear it? Okay. I see two people did. Wonderful. Anyway, it was good. And I think it was great to hear from those students their kind of honest feedback on how it's working for them. So do if you get a chance, watch the full one after the webinar. All right. Quill, do you have any finishing remarks? No, I think that's everything I really wanted to share and I'll answer questions when we're done with the whole session. All right. So the students got the last word on that one which is a good thing. All right. Now, I'd like to take this opportunity to introduce Rob Farrow who has been the lead researcher on the community college collaboration with the OER Hub and we've been very excited to work with Rob and he's got some great information to share with you based on both faculty surveys that he did with our colleges and also site visits. Speaking with faculty leaders and students. Rob, are you ready? I am indeed. Thank you. Yes. Thanks for that, Una. So yeah, as Una says, I'm doing the research for community colleges as part of OER Research Hub. A little bit about the project and about the Open University to get started with. If you're not already aware of the Open University, we're based in the UK. We've been going since the 1960s with the biggest distance learning provider in the UK. I think in Europe too. I'm not sure about further afield, but we're definitely one of the kind of leaders in this area. Excuse me. So we're also one of the largest educators of disabled students in the UK. I think we have something like 17% of our students have a declared disability from cognitive or motor disabilities through sensory impediments. So we do quite a lot of work around accessibility as well. So in the OER Research Hub project, we've been funded for two years, started late in 2012. We're going to be going for about another year, I think. And we're not just looking at community colleges, although that is one of my focus areas, but we're looking at globally what is the impact of open educational resources. So we're working not just in colleges, but in universities. And we're also interested in informal learning and how informal learning might support formal study, so in terms of recruiting students or retaining students and that kind of thing. And these are all issues that affect the open universities as much as community colleges. So there's an interest there for us in that. So our own research is structured by a set of hypotheses. Now, this is not the most user-friendly presentation. So the key point here is that there are lots of claims that people like to make about OER and the kind of impact that OER has. So, for example, OER saves students money. OER increases access to education. OER helps teachers to become better at what they do and so on. So here what you have are 11 hypotheses that kind of reflect the claims that people make about OER. And these hypotheses are really the kind of scaffolding for our whole project. So this is what we use to kind of pull everything together. These are all on the website if you want to look at them in more detail. I'll speak about them as we go on. So we have a collaborative model for our project. So part of what we do is work through organizations like CCCOER, but also people like Sailor, P2PU, CIVULA in South Africa, Creative Commons. We've got a very good network that we're trying to kind of build up. And it's not just about us getting data from people. It's about sharing expertise within that community and encouraging people to be open about what they do to share the data and for everyone to benefit from that. And I'll speak a bit more about some of the ways in which openness informs the way that our project works as we go on. So just talking briefly about the CCCOER collaboration. It is already described a bit of what's happened. I would just like to say at this point for the record, a great big thank you to Una for all the work that she's done in setting up research visits, setting up surveys, and all these kind of things which really made this collaboration work. Whereas it might not have done. It's quite hard sitting in the UK to try and get all this stuff out of people in community colleges in the US. So thanks, Una. So around about last February, we started developing the survey. For us in the UK, that was quite a big task because the survey bank had to be applicable to all sectors and all stakeholders. That means we've got a pretty good collection of questions. So around about last May, the surveys started to go live. We closed them around January this year but there's still the possibility of doing some more because we've now got this extension into next February. So if anyone is interested in doing some surveys, we already have the questions, everything's ready to go. It will pretty much take no time at all. The main issue is IRB potentially. So if we can get around IRBs, then there's certainly potential to do more survey collections. And so over the last sort of year or so, I kind of did the traveling around the US, visited some community colleges, spoke to a whole bunch of people about what was going on there and tried to work out how do we actually find this evidence or what does the evidence really look like for these things. And so we're at the stage now where the survey data has been processed. I'm going to mainly present that today. And then we also have all the recordings, all the data from all these conversations, the visit that Quill mentioned. We were going actually talking to students just hearing what they have to say about it. So it's scheduled for the next month to actually go through that and do a proper analysis on it. But I can give you some general kind of things about what I've found. And lastly I'll say that we're going to be putting all this onto a map, which I'll show you shortly. So here are the colleges that participated in the research over the last 18 months or so. I won't read them all out but you'll probably see some familiar names on there. Maybe some of these are even your institution. And so some of these have had quite a lot of input. Some of them have been more of a light touch, just do a survey, just talk to a couple of people when it happens. But we're keen to build this network of colleges that are participating in that because part of the way that we're working is to try and ensure that when people do share data we add value to that and we share it back to everyone else. So just brief word on the sample that we had for the survey. So this is kind of composite reporting of a number of different surveys put together. So this is the overall collection we've got for college educators, community college educators in the U.S. So we had 136 responses, at least 7 youth experience for 84% and 96% had a postgraduate degree. So they're not kind of green, they're not kind of just qualified or anything like that. They're experienced teachers and they know what they're talking about. Most of them are teaching full time and some of them are involved in online instruction as well. So we did have quite a kind of mix and a mix from places around the country as well. So you see the places that gave data for this survey, DeAnne's or Foothill, Houston, northern Virginia, Rome State and South Florida. And as I say, we put together the analysis of the qualitative data from the various visits that we've done as well. So on to the findings. So first of all, a look at the kind of behaviors that college educators have around where we are. So we asked people just to say, what have they done various things? Have they adapted OER? Have they created OER? Have they added OERs to a repository and so on? So you can see about half of the sample that they had adapted OER. Just about a quarter said that they created OER. One of the interesting things here was if you compare the second column in descending order and the fourth, of those people who create OER, only around half or less than half are actually publishing them and are creating a common license. So that was quite an interesting finding straight away. You may not be that surprised to find that because we know that it's much easier to create them than to take the extra time to license them and so on. But it does seem to suggest that a lot of this stuff is happening without open licensing being a part of it. Looking at the types of OERs that people use, so if you look here at the top, video is by far the most popular OER people reported using. And then images. So multimedia is the most popular. Going down, down the list, if you look down to the very bottom, datasets, so open data where you could kind of take a data set and use it with students to do stats analysis, something like that. Not many people doing that. Similarly, not many people using the whole course to teach. So the impression is that people are just kind of taking bits, taking things or maybe they see a video on YouTube and they think this would be just my students so they share that. It's not so much people taking an entire course that's already been done and mapped to a curriculum and taking that and making that the basis of their own course. Most people don't seem to be doing that. So it's kind of slightly magpiesh maybe, taking bits from here and there. Very much in the spirit of OER and remixing and so on. But you might be surprised there. I mean just over a third are using open textbooks with these educators where we hear a lot if the open textbooks have a big thing that's having the impact in colleges. So there's a snapshot of kind of the types of OER that people are using. Now if you look at this slide, this shows you the repositories that people reported using. On the far right there, YouTube is by far and away the most popular OER repository. If you're willing to consider it to be one. Then Ted talks, then iTunes you, and then you then you start getting into kind of Khan Academy, Merlo, if you like the more well-known core OER providers. OER is their mission. And quite low numbers really for a lot of them. So obviously we didn't ask about local repositories. So people might have an institutional repository in their own college and they use that. So we were only asking about the external ones. But still, it wasn't really what I expected to see. So moving on, we asked people what factors were relevant to how they choose OER. So the most popular answer here was the resource being relevant to particular needs or interests. I guess you could call that perceived relevance. The second most important factor was whether or not they trusted the institutional person that had created the OER. So that does seem to be important. The idea that an institutional or a personal connection can be a proxy for the quality of a resource. You make that surprising. After all, that's kind of how it works with textbooks and so on. So that's not a uniquely open thing. I only just over half said that having an open license was a significant factor for them. So again, we see that kind of slight discrepancy between the theory and the practice of using OER. So we have some other factors mentioned in the body there. You go down to the bottom where I've highlighted the last two. So the resource featuring a catchy title or attractive image, 9%. So this seems to me to be kind of an interesting kind of outcome because one of the things that some people will say is another proxy for quality is the kind of presentation of something. So if something looks nice. And as we'll see, there are people with worries about the quality of the OER, but those worries don't seem to translate into the presentation. So people don't seem to mind it if it's kind of a bit rough around the edges. And the bottom response there, being required to use OER to complete a project or task, only 7% said that. I guess you could read that two ways. You could say, no one's really being required to use OER so it hasn't kind of really got into the mainstream of what people do. But on the other hand, 7%, that's not insignificant because how many institutions are actually asking people to use this stuff. So I'll tell you an interesting finding. I'll be interested if anyone's got any thoughts on that. So moving on. Now we're looking at the impact on teachers and students. So we asked, obviously these are all questions that are asked of educators. We didn't ask students in these surveys. This is the teacher's perception of how it's impacting on them and on their students. So this chart shows, we ask people to say whether they agreed with these statements. So OER use, does it make, do you now, as a result of your OER use, make more use of culturally diverse resources and so on. So what you're looking at with the blue is agreement and the red is disagreement. Now the first thing that you should take away from this graph is that these are all positive pedagogical indicators. So these are all things that people associate with better teaching. We have a hypothesis which is to do with critical reflection. So if we're interested in does OER use make people reflect more about what they do? Do they think more about their teaching? Do they spend more time preparing and so on? But it's hard to look at that directly. So these are our proxies for that. But you can see there's much more blue than red. So there's much more agreement with these statements than disagreement. And they're all positive. So people think it's having a positive effect on their teaching. So making more use of culturally diverse resources was the one that had the strongest agreement. But the one that had the most agreement was the fourth one down, up-to-date knowledge of subject area and the second one down, collaborating more with colleagues. So North Seattle College ICT skills are information and communication technology skills. The more collaboration with colleagues that is again something that people like to claim about this stuff and we see it happening. We see people collaborating on textbooks and sharing stuff. And this supports that view. Now there was quite a lot of, if you like, the least agreement was around ICT skills. So there were quite a few people who didn't agree that their ICT skills were improving as a result. And similarly, looking at the sort of second-last row there, I reflect more on the way that I teach. So you can see that there was a significant number of people who didn't really agree with that either. So there were some people who think it's having more impact and some people think it's having less impact. But overall, that seems to be the perception that it's having a positive impact on teaching. So moving on to the next slide and looking at students. So this is the perception of a teacher of what's happening with the student. And so we asked them, again, do you agree or not? And again, these are positive. So when people agree, they're agreeing with the kind of positive thing. So you can see here, the strongest one in terms of the strongest agreement was that OER is increasing people's sense of experimentation with their own learning. The one that had the most agreement rather than the strongest agreement is sort of the least or wider interest, sorry, interest in a wider range of subjects. And the third one down there, developing independence and self-reliance. These are all really good things. These are things that are hard to get people to adapt to and to change their way of learning and become more responsible for their own learning. So again, these are quite positive findings. And you can see here, we're talking about 60% agreement with these things. So again, these are a positive, increased enthusiasm for study, more satisfaction with the learning experience, building confidence, and so on. The one that you could argue had the least agreement was increasing enthusiasm for future study, which is about five up from the bottom there. So you can see the red there, but it's still less than 10% of people that disagree. So again, positive, probably slightly more positive than the impact on teachers, arguably. But again, it's good stuff in the sense that it's all going in the right direction. So moving on to the financial savings. So we asked people whether or not their institution is saving money through OER. So you can see here, 44% agree, yes, it has. And only 19% said no. Understandably, perhaps, there's quite a lot of don't-knowes. Not everyone has access to that information. It's not always maybe that clear how OER might be saving institution money. It might depend on the various deals that are in place, especially with bookshops and that kind of thing. But overall, it's a positive finding. Most people agree. Looking at the comments that people provided, partly, there's this idea that students are coming back and they're staying in school. It's not a direct institutional benefit, sorry, not a direct financial benefit for the institution. But indirectly, it's a great benefit because if you've got people taking the classes, then that's how you make your money. You've got this comment on the bottom right. Some instructors are only using OER which provides savings for students. So some people are getting at that level of the conversation where they're trying to work out before they make their decision. Am I going to do this or am I going to do that? Which one's saving money for students? If you look at the bottom left, this is one of the negative comments, which you may recognize as a sort of slightly old-fashioned view of where we are or maybe someone who's not really in the loop about the resources that are out there because, as we know, there are very up-to-date modern resources that are peer-reviewed and aligned to curriculum. So I mentioned that mainly just to not to agree with those sentiments, but just to tell you that there are people out there who do think that way. And I guess there are people who need convincing. So anyway, institutions saving money about 44% said yes. Moving on to the students have saved money. So you can see here that 62%, almost two-thirds, said yes. There's looking at the comments. Bottom right, someone says that they recognize saving their students about $100 each. And that's a figure that I've had come up in various places that I visited. About $100 per student per class. If you want to kind of rule of thumb, roughly how much would it save? $100 per student. Interested to know if anyone's got kind of if they agree with that figure or they think it's too high or too low. But this is quite positive. Two-thirds roughly say that students have saved money. Is there the potential to save money? But yes, they have actually saved money. So we also asked about student retention. So this has been back to this issue of where we are can actually encourage people to stay in college. So less agreement with this. So about 50% not sure. But still, 25% agreed. 12% strongly agreed to put them together. That's over a third agree that OER use is promoting retention. And I'll say something about the reasons on the next slide. But about half of it is not having an effect. So one way you could look at this is to say, well, okay, there's OER. And there's the practices that surround OER. And maybe it's not just enough to sort of stick a CC license on something, but also to think more about kind of how the openness of what you're doing can influence and change the way that you're educating. But again, there's still that kind of 10, 12% of people who disagreed. And some of them disagreed strongly. But yeah, I think still a kind of positive result, certainly not a negative result. So we asked about the factors that would be important in terms of retaining students. And you can see here that the top answer was the reduced cost of study materials over there on the right. Now, one thing you might have noticed about this graph is that the number of people who think OER can be a promoter of student retention is greater than the number of people who think it is a promoter of student retention. So going back to the last slide, you've got 12% disagreement. Sorry, you've got about 33% agreement. But 60% of people say that it can be a factor. So there's probably more people who think that it can be an important promoter of student retention than people who think that it is. So what I mean by that is to say that the people who are not convinced yet maybe are still open to the possibility. So here are some of the comments that we got about this. So some people will say, if you look at the bottom right comment, retention is nothing to do with this. It's just to do with time, insufficient preparation. It's not about the books. It's not about the resources. There's obviously the idea that we can't... Yeah, I agree with you. It is complicated. The factors are also not the same in different places as well. So it's hard to draw a very firm picture around this. But at the same time, I think there's a slight thing here which is about technology and about access technology. So the people who said that OER isn't the solution were pointing mainly to people's inability to use the technology in the first place or people's inability to access technology in the first place. And these things being more important for student retention. So maybe OER isn't the thing that makes the difference there, but if these other factors are more important, OER certainly fits in quite nicely there as part of a coordinated approach. So moving on to open licensing. So we asked how important is open licensing? 34% said very important. 20% said crucial. If you look at those two figures together, that's higher than the number of people who actually do it. So if I said at the very start that we had about 11% of people practice open licensing. You've got here 54% of people but it was at least very important. So that seems significant. Two interpretations. One could be that people don't feel that comfortable doing the licensing or they don't quite feel confident about how it works or that kind of thing. Another explanation could be that they just don't have the time to do it basically. And it's just an extra bridge too far to license it on top of actually producing it and using it. So I mean, an interesting finding there about, you know, how important is open licensing to all of this? Is it the thing that makes it all happen? So just to pull some of this stuff together, most of the respondents we asked have used OER. Only around a quarter actually create OER. So most reported positive effects on teaching practice. Mainly around peer collaboration and increased knowledge but also around increased independence and self-directed learning. Small proportion feel that OER use these directly to improved teaching practice. Positive effects also identified for learners as a say around self-reliance and experimentation and increased interest in the subject. So about two thirds of people feel that students will save money, less agreement around whether institutions were saving money. About third, believe that OER is improving, well maybe I should say improving student retention rather than attrition. About half believe it's not having an effect. Only around half of OER creators use open licensing. So you have adapters or about half a sample creators about a quarter of the sample and about 12% of the sample using open licensing. And there are some people within that sample who are very sort of, if you like, OER power users. So they're doing all the things they're supposed to do. They're licensing, they're applying to repositories, they're reflecting more on their practice and they're kind of very sort of pro OER. But then at the other end of the scale there's the kind of maybe anti-OER is too strong that people who sort of want to be convinced maybe there's a better way of doing it. One thing briefly just to mention coming out of this work is about the importance of a sort of a timely IRB process. So if you do want to do any research like this so you want to use our own survey questions and that kind of thing, get on top of your IRB process as soon as you can. So just sort of what's happening next with this stuff so we have some time for questions. So we've got some more work to do on this survey data but we've also got to cross-reference it with our other surveys. So for instance we have a sailor survey with about 3,000 responses. That's all informal learners, some of whom are college students as well, some of them are using sailor materials to get through their college courses. So once you kind of put that together with this what we've got we're going to get a nice online fill. It's an institutional review board, ethics panel basically. Sometimes it's called a student research panel and these kind of things but it's basically where you get your ethics and approval from. We have some stuff to do around trying to pull together the qualitative data that we've got and we're also really interested in institutional metrics and where we can find them. That is the goal class. That's what's really hard to get where a college can say we've got these students using OER and these students not using OER and here's the difference between them. That's what's very hard to get. So anything like that that you might have would be great. All the data is being disseminated openly so we're currently working out ways to publish all the raw data online and all of our research instruments like surveys and interview questions and that's all free for anyone who wants to have it. It's all openly licensed. We would ask that you share the data back with us but that's not up to you really. Just to finish on this map that we're producing, if you want to go there, it's OERmap.org. You can go there now, it's live. I've got some slides here to show you what's going on there. So when we find evidence of OER impact, we're recording it on this map and we're doing it in such a way where if you look in the bottom left quadrant of this slide, this flow diagram, you have the hypotheses running down the left, the sectors that we look at on the right and in the middle you've got positive and negative evidence. So you've got a picture there of what's happening overall. You can see most of our evidence, positive evidence about financial savings is coming from the college sector in green to purple at the top there. You can see if you look at the bottom left, C, access, the evidence there is split. So we've got some negative, some positive. Most of the negative evidence is coming from the informal sector. So that's to do with sort of MOOC claims about improving access, turning out to not really be that justified. So we've got a way of kind of trying to capture the whole kind of world picture in one, which is the idea. But you can go further into the data on the map as well. There is the next slide. You can see the United States there. And the flow map, the sort of diagram in the bottom left, that updates automatically when you select a country. So you can see how the picture is different in different countries. But you can also go through to sort of fuller map, which is a bit more like a Google map. So here you see OER projects that we've got moving on to the next one. Evidence for community colleges in the USA that's been entered into the system so far. Orange is positive and silver is negative evidence. And you can see here, if you look at the top of the slide, you've got drop-down boxes where you can filter all this stuff. So if you just want to see, right, show me financial savings for colleges and positive evidence, you can filter out that stuff. So you can see straight away where is the impact happening, where has the evidence been found. When you click on any of these nodes, by the way, you get taken through to a WordPress blog style report or summary. Another map that we've got, this is the policy map. So what you're looking at here are policies that have been entered into the system where R indicates a regional or local. I is for international, N is for national, and I forgot the other one, but there's four of them. And the idea there is to record all these policies, and we're trying to sort of, if you like, work out, is there a kind of correlation between where the policies are in place and where they find the evidence and which comes first and this kind of thing. So it's about trying to draw that picture together and also provide something that's of use to people to go in if they're talking to a policymaker or a dean or a government official or anything like that and you want to show people this is what's happening, which should be impact. The idea is this is a tool for you to use as much as it is for us. So that's the policy map. I encourage you to go and have a look around the site. This shows you the summary at the moment of what we've actually got into the system. You can see we're more focused on positive evidence and we've got quite a bit of it, but we're in the process of seeding this data. We'll be doing it all over the whole summer. So we're expecting to have quite a few hundreds of pieces of data in there as well as our own surveys for people to explore themselves. So quite a lot there. If you want to go and check it out, are you on map.org. To just conclude, in terms of what we're doing, the real point of all this is to build up the evidence base for everybody. So it's for everyone to use. And the idea is that if people share with us, we can kind of pull everything together into one place and share that back to the community and the idea is that the whole is greater than some of its parts. So the more people share with us, the more this evidence base grows, the more easy it becomes to make a case that you can take to your dean and say, we should be moving in this direction because look what's happening there. Or even if you just want to get in touch with them yourselves and do it that way, that's fine. So the last thing to say, I suppose, is that we want to put you on the map and your college, find out what's happening in your situation. So I'm going to go through the bio-means, drop me a line, get in touch with the project. And yeah, just let us know what's going on. Let's have a conversation and we'll see if there's any way that we can help on what you're doing. Thanks. All right. Thank you, Rob. Lots of comments and questions in the chat window. I'm going to take us real quick through just a couple of slides. By the way, the Open Coursework Consortium, our parent organization, is having a conference, the end of April, in New Zealand, Slovenia, which is just a beautiful country. The registration is still open. A few of us from the community colleges are going and we'd love to see more. I mentioned that next month in May, Wednesday, May 14th, we're going to have a webinar specifically on open licensing and also trademarks and patents. And we're going to have Kathleen O'Molo from Open Michigan who is quite expert in this area and she's going to share with us. So come and invite your faculty to that one. And at this point, I know we're very close on the hour, but we're going to go ahead and open up for questions. And I want to thank everyone for coming and thank my speakers for their wonderful information today. And while we're waiting for some questions to come in, there has been quite a lot of interest around surveys and Rob's email is there. If you want to contact him, we also at the consortium, you can contact me at my Una Daily email. I share surveys from our other members who surveys that they've used for both students and faculty. And so I'm happy to share those with you. Quill also has some wonderful surveys that I usually share with her permission. So do contact us for that. And I do understand we had a comment from Anita about, I think it was Anita who said that many of her faculty are interested in this new open education movement, but they don't really understand open licensing and OER. And how can her surveys address that without confusing people? And so something we'll probably try and talk about in May, but we're also happy to work with you, Anita, on your survey questions. One question we had from SPSCC e-learning was faculty collaboration. And Rob, did you want to speak to that as an outcome? Were faculty collaborating more because of OER? Did the surveys show that? Yeah, well, I would say we got some supporting evidence for that. We have a couple of places that we're thinking about following up and doing sort of bigger sort of case studies on. But there's certainly, we asked, if I can open up the slides, because there was one of the slides this came up on. So we asked people when we were asking about impact on teaching. And there was certainly, 22% said that they collaborated more with colleagues. And something like a further 50% agreed. So yeah, 22% strongly agreed and a further 50% agreed. So there's some evidence there to suggest that that's happening. But we don't know exactly what kind of collaboration that is. So whether we're collaboratively producing text or working out curriculum and these kind of things, that's an open question. But there's certainly some supporting evidence there for that claim. Thank you, Rob. And I will say that the Community College Consortium has also done some, did some previous research back in the 2010, 2011 timeframe that supported increased faculty collaboration. But I don't think it's, so there has been supporting evidence for that. Was there a question? Thank you. Rob has shared the full set of survey questions there. And I know that we had at least one question here who, one question I believe this was from Ashley and she said, Rob, is there going to be a report on these findings that could be shared with administrators because her administrators won't have the time to, say, watch the recording of this presentation? Yes, there will be reports. Quite what format they'll be in. I don't know yet to be honest with you. You're kind of entering, Rob? Yes. So we'll be entering our kind of final dissemination phase in the fall this year. So between now and sort of August, we're going to be, so obviously we've got conference presentations coming up at OCWC and OER 14. So once these are out of the way, we're getting a bit of time just to kind of go back and put everything together properly. So by September, that's when we're going to be kind of pushing stuff out there. I think it's quite possible there'll be some sort of maybe, sort of, you know, brief glossy report on this stuff, but I don't know yet. At very least there'll be a sort of e-book, a short e-book or something like that. But it'll be widely available. These slides will be online as well on the website if you want to see them. Or you can just be in touch with me and I can give you a few slides that will give you the kind of basics. You know, if that's something in particular that you want to sort of share with people. But everything will be out there sometime this year. Great. Thank you, Rob. And I think you have these slides and there's a fuller deck of slides that are available. Either email me or Rob for that information on this presentation, which may work in the interim. Let's see. We are just about running out of time because I don't want to hold our captioner much beyond the hour. So, Quill, do you have any concluding remarks that you would like to make before we turn off the recorder? No other than contact me for questions and I'm always happy to answer them via email. Yeah, that's what I'll stop with. Okay. Thank you, Quill. And of course, thank you, Rob, as well for your presentations. I just wanted to mention that Phil Dandetti is attending our session today. He's a communications professor at Clover Park Technical College and several of his students are with him. And they've been posing some interesting questions in here as well. And we'll just take one of the students' questions. Layton is eager to know how people who use OER are able to reach out to provide access to others who are technologically challenged. So, good question. Good question, Layton, about students helping others who are technologically challenged. And I'm going to stop the recording right now on that one, but Quill, do you want to take a shot at that one? I was talking.