 My name's Tiffany Halt and I'm going to just kind of give you an idea of how we in Utah sort of started this and how I spent two years of my life hoping that I wasn't going to go to jail before I met Kathleen Allen and figured out on a state level how to make open educational resources really work for us. So we started with a research project about two years ago in Utah looking at science texts using open resource. We weren't 100% convinced that the textbooks that teachers had in their high school classes were sufficient. We were hearing a lot that they were really big, that they used them on the shelf and that maybe once or twice they would reference them but they weren't 100% user friendly or student friendly and if you've seen them, they're also very large and very heavy and fill up the backpack and they're also very expensive. So we put together a research project. We started with about 15 teachers. We ended up with about nine teachers where we went and created a science textbook using open educational resources using a website called CK12 which is a foundation that has been focusing on science and math, vetted resources for high school and middle school students. So we put this book together and we were working with four different districts. We had teachers in four different districts and we were just not sure how the idea of open educational resources was going to play with school boards, how it was going to play with teachers or how it was going to play with students and especially how it was going to play with parents. I don't know how you are in your states but in our states, if parents aren't happy with the textbook, they will let us help you board. It ends up on the front page of the newspaper because it's very, very public what happens. So we brought these teachers together and started to put together these textbooks and discovered that we had two different models that happened. One was a girl model and one was a boy model. I hate to be gender specific about it but that's really how it worked. We had our female teachers edited the heck out of these books and came up with about a 250 page textbook and the gentleman in our group just took everything that was available, and put about a 1200 page book and said that they would edit as they went through the year. So we had these two models but we put them out there and decided to see what it looked like. Long story short, there's another presentation later today about the project. In detail, we found many things to be very, very positive. We found that teachers felt very empowered that they used every page of the textbook that students liked very much having access to open educational resource textbooks because they were available online. They could get to them from their homes. We also, through some grant funding, provided some print copies of the books that students were able to actually take home and write in. And we found that we were receiving some very good feedback from students and parents. We had a couple of issues that we needed to be aware of and I'm just going to talk about a couple of things that we found. As we started putting these together and using them in schools and the first one was that everybody is so busy we really needed to get better at quality control. We had one textbook that we printed with the teacher's name who was the teacher of the class spelled incorrectly. That was the first year. The second year we spelled it on the front of the textbook. We spelled her name correctly but we misspelled word chemistry. So just a little thing like that. We discovered that there really are issues with our own credibility. We knew that content was good but we had pages where stuff would run off the end of the page or where maybe the formatting wasn't perfect. So that using these resources we need to remember that if they are reflecting as educators that they need to be very, very good. We know that the printing when we were able to print wasn't in color but we were able of course to link electronically. So a lot of those resources that came up were fine. Oh it went away. This is, sorry, this is Brian. He's one of our teachers and he has his book in Utah. This is our UEN Utah Education Network page and they all have it. If you just click that. This Press and Master Biology book. There it goes. It's having some maintenance right now. Another one of those small issues is a lot of electronic stuff but we know that. But we also discovered that as we worked with teachers they would often be concerned with parent perception of how these books were being used. And so one day I was sitting in a football game at this high school where this teacher teaches and I don't know if all of your football fields are set up like this but the home side is always the side where the sun is not in your eyes. And so since we only really just care to cheer when my stepson kicks the ball which happens like once again. We were sitting on home side and on the visitor side and so I was sitting next to some parents and so of course because I'm nosy I just said, oh so do you have children in the school? Well who do they have for science? Oh they have Mr. Blake. And I said really how is Mr. Blake as a teacher? Oh he's fabulous. My kids have a book that they can write. Actually our youngest son has a book that they can write. It's just like he's in college. It's so much better. How come my other kids didn't have this? And I said well I actually know the answer to that question. It was very interesting to hear the positive reception despite the fact that words were spelled wrong or things weren't perfect. The idea that parents were so open to this idea that we were providing additional options and ways for electronically and for students to access immediately accessible and scientifically valid reliable resources. Our fear through this whole process was of course that we hadn't gone through all of the right channels. So that's how we started to work with Alan and Kathleen with this project so that we could figure out the right way as a state to begin to adopt open educational resources and be able to use them without a fear of going to jail for doing anything comfortable. I'm Alan Griffin. I work with the instructional materials recommendation process. Used to be called textbook adoptions. They've changed the name to Instruction Materials Recommendation. I also work with a group called NANSTO which is the National Association of State Textbook Administrators. I've been kind of crafting with them the evolution of all these new materials, particularly with digital materials. And my concern, sometimes I feel like I'm the vote acre holding everything back, I guess my concern is to make sure that we have really valid high quality content materials in our schools. And I've known Kathleen for about three weeks and known Kathleen for considerably longer. But as we went through the process of trying to decide what to do with digital materials and now open education materials, it's brought, you know, we can think about a whole bunch of new things. It's typically when we adopt a textbook, you know, we have the publisher send us samples and we go through the samples and we have a list of core objectives that we try to match it up with. And, you know, we consider things like the quality of the binding in the print on the page and all kinds of things like that. Well, now we have open education resources being used by schools. And, of course, my initial question was, have they been through my process? My process is the best. You know, they've been through my evaluation process and we kind of throw the doors open here and say teachers can get resources from anywhere they want or should they be validated. And so we're trying to wrestle with the reasonable process for validation. And so when Tiffany came into the office in connection with Kathy, we talked about open educational resources. Typically a publisher is a champion of their book and they will come and market it. They will come and push it out to me and, you know, tell me that it needs to be reviewed and we review it and all those kinds of things. Open educational resources is the champion. You know, there isn't a salesman who comes to sell it. There isn't somebody who pushes it. Well, does that mean that I should go out and solicit it? I'm a little apprehensive to do that because I deal with all the publishers as well. So one of the initial questions was, who's going to be the contact for the open educational resources to see that they get into the schools? Otherwise, you know, we've got the publishers in the free market system. They're not anxious to make a buck and so they will go out and actively market it. Open educational resources are not that way. Another difficulty is how do you validate the quality of the curriculum? You know, who is the validation? I mean, should we pull in a bunch of university professors and get them to validate it? Should we just rely on the publisher because they have an economic interest in it? They will validate it with open educational resources. Nobody has an economic interest so, you know, how do we go there? Or should we just let Google do that? You know, we just say, well, who's using it the most? And we'll validate it that way. Well, we wrestled with all these issues and we came up with five areas and I've got handouts for some of you if you'd like. The five general areas that NASA and myself are looking at to evaluate digital materials and I think probably they're going to apply just as well to open educational resources. Things like the content, the equity and accessibility. You know, I've heard that talked about a number of times today. But in public education, we want a resource that everybody can get to. Everybody can get to and there's no restriction necessarily on the type of device you're using. How is it assessed? What do the teacher helps? You know, if it requires technical expertise, who do you go to? Well, once again, we have the open education community. Does it need technical expertise or not? And then design and support. You know, we kind of rate them on a little scale there. At this point in time, we went ahead and because one district we knew was using some of these open educational resources, we allowed them to be the champion and said to them, send us the resource if you find the resources that you like that are open educational resources. Send it to us and we will evaluate them the same way we evaluate all the other textbooks. In other words, we're putting them in competition with the Pearson apprentice hall, McGraw Hill and all those others. And saying to them, you need to meet the same standard. Which is probably only fair, you know. But understanding that, they're a unique kind of instrument because open educational resources are designed to be adaptable. So the teachers can change them and use them as they will. So our evaluators need to understand that this is a different animal than what is sent to us by a publisher as well. So we went through the process. We got a teacher committee to evaluate them. We put them on our database. It's a starting point. I don't know how well they're going to do. I mean, the reviews are not sterling in comparison with other kinds of resources, but the fact that they're very cost effective, you know, that they're essentially free, makes a big difference to teachers. So that's kind of where I went with it. Now I'm going to turn over to Cathy and talk to you about her perspective on the issue. Okay, so the question is what's a book? Well, they took a digital book that was created from California Open Project and then they created a subdivision of it and then they actually printed it. So we recognize that's a book and it's just content. And what they added to the book, they added some of their own review questions or vocabulary terms. So they took the content that came from a great resource and customized it and made it themselves. And because it was put into paper, it was a book. So we understand what a book is. But what is a book in this digital age? How many of you have initiatives in your state where they're doing one-to-one where they're having every kid in the classroom have a digital device? And you have that going on in your states? A few of you do? Okay. So in the context of what's coming with education, one's a book. One of the jobs that I have is I'm principal of the Utah Lake County High School, which is the state virtual high school that's been around for 17 years in Utah. And we pay teachers to develop curriculum for all of their classes. And then because first of all, there wasn't anything to license back when we started. And second of all, we have the lowest funding for people in the nation in half a year, so we can't afford to pay ongoing licensing fees. So we had our teachers develop the curriculum. Is that a book? So we don't have a regular classroom teacher put their syllabus through Alan's process. Should we have to have an online teacher put it through the process? So let me show you real quickly what it is we've got. This link right here takes you to page, this is a back-end page. Back in 2009, we released our entire curriculum with the Creative Commons license 3. And this is the back-end page that will take you to all of the content that we have. And this isn't something that is not viewable pretty page yet. But as you scroll down through here, you can see all the different classes. And we teach our classes by quarter. And right here is financial literacy. So if you click on that link, it pulls a database view of the content for that first quarter class of all the notes that we have in the class. If you go to this view, which is what I'm going to take you to, that takes you to all the content for the first quarter. And then as you scroll down through it, that's all the content that a student would have. So our question is, is this a book? And should it have to go through the textbook adoption process? Well, we've also been thinking a lot about these mobile devices and how kids are coming to school with these devices, whether we're supplying them or not. And do we need to put this in a format that's more accessible for them? So back in 2009, we had been, how many of you are Blackboard lovers? Not for users. Not for them. Not for users. I said lovers. No, no, no, no. We got the other content out. Yeah? Okay. So we were in Blackboard. And like I said, we've been around for a while. And we bought Blackboard back when it was 1.0 kind of a thing. And at that time, it was like $8,000. And then it went to $108,000 a year. And it was like, and like I said, we're poor. So we made the strategic decision that we wanted to move away from Blackboard and we moved in to move. What had happened for us in developing our content? It was locked inside the learning management system. And it was nested hell. Because we have one class, the teacher had designed it, that it seriously took seven clicks before you could get to the first thing in the class because it was so nested. And so we made the decision that when we moved from Blackboard to Moodle, we were going to separate the content from the learning management system. So if you go to the front page of our class, one of our classes, let's see if I've got this right here. Hopefully it hasn't locked me out. This is what our inside our Moodle system looks like. And this right here, all of our classes are divided into these four sections. And we have, this is just information about the class. And then each one of these is a link into a unit that's out on our content server. When we separated the content from the learning management system, it made it so that anybody could go see that without having to log into our class. And then of course, the kids turn in their assignments and stuff down here. And links to their proctor tests and stuff are inside the learning management system. But the content is outside of it. So now that we have the content outside of it, what's the next step? And for us, the idea was, well, we need to create it in an EPUB or other kinds of accessible ways for kids to access it using their mobile devices. So we've been playing around with creating EPUB documents. So there's a place online called EPUBBUD that it's just a website that you can take and cut and paste and put stuff into it. And then at the end on this website, it will kick out an EPUB document. So we have, this is a place where we post some of the content, the media device resources that our teachers create. We've also added a link to the EPUB versions of that page. And so as our content has evolved over time, then we'll kick out new EPUB documents. We needed a process that was quick. So we've tried pages and then this EPUBUD. And the EPUBUD, even though it's not as pretty as the pages, EPUB kick out, it works pretty well. So our thought is once we get this process down pretty well, we're going to submit our EPUB documents to see what happens. So we'll see if it gets good comments or not. Not so good comments. So, great. So we would like to have some conversation. What do you think? Questions? Comments? Back in the back and then up here. I have a question for Alan. You said that you're going through the vetting process of these textbooks that they didn't get sterling recommendations, the older textbooks. Could you elaborate more? Well, we pull in a committee of teachers and district specialists to evaluate the textbooks. Okay. And so they go through them and number one, they compare them with the core curriculum to make sure all core objectives are covered. And then they evaluate them on the basis of things like, does it engage the student well and most kinds of things. Most of the books that we have seen at this point are content and they're made there and they're designed to be adaptable for teacher use and student use. But what's happened, I mean, our evaluators are used to seeing a student edition, a teacher edition. Glossy and in color. Yeah, CDs, DVDs, all kinds of ancillary materials, bins full of stuff. I mean, publishers send us an extreme amount of stuff. And so they're used to seeing all this glossy stuff that goes into... and I'm not saying the name is bad. I mean, it's wonderful stuff. But they're used to seeing all these teacher helps. And now what we're seeing is content and the content we feel pretty good about is valid. I mean, they looked at the content and said it was pretty good. But didn't feel like, you know, a teacher could pick it right up and go right into a course with it. It's not involved to that stage yet. The teacher needs to have some expertise in going through and focusing on the content and then adding his own whatever else that needs to go with it. I think it was mainly because there just weren't enough materials that went with it that it just didn't look like the other kind of content. And part of that was how they were developed. When they were developed, we asked the districts to give us the very best teachers because that was one of the vetting processes for us. Those of us were working on it. We weren't science teachers. And so we said we want the best science teachers. And so when you get that, you get the teachers who already have their lectures ready and they really just created a book that was a support to the student, not to the teacher. Another thing that they did is that they put their cognitive assessments that they developed in as part of that or quizzes and lectures or quizzes for their lectures that they had in place. But again, it was very, very much driven to support what they already knew that they were doing in the classroom, not to support a new teacher. If you were a new teacher coming in to teach biology for the first time and I handed you Brian's 250-page book, you might struggle with that just a little bit because he has another set of files on his computer that govern his content and what is taught in class and how his labs and activities work. So what we might do is take the old one that CK12 does and submit that to his process and then, because that's much richer and then see where we go from that and then know that that would be the teacher edition of the book and then the student edition would be Mr. Blake's where he pulls out and adds to. There was no problem. I'm actually something separate to both of you. As you were speaking, a thought I had in realizing as you acknowledged that it's a different animal as we look at this new environment where the old criteria doesn't fit well if you've got a restaurant and you have a connoisseur, a judge, but you can go next door and there's a free vegetable garden. Your criteria for the restaurant and the culinary experience, you stand in the vegetable garden and you ask, scratch this. We've got to deal with something brand new here. And just one thought based on not obviously got no insight to your environment so this comment might be misdirected but it might be useful. As I found out from Wisconsin, in our environment where we've got a bureaucratic, and I don't mean that in a negative sense, a bureaucratic institutional way of evaluating materials, that often you find that there's a bit of a dissonance at times between the evaluators and the teachers themselves of how they think about it. And that to me is traditional and I think a new way would be to validate both and not let the bureaucratic one be the final vote because that's also traditional but make sure that that harmony is created because sometimes you have teachers that say, yes, yes, yes, it's not there. When you pay people they do that for you but we do it ourselves and we really engage and kind of like I get into the vegetable garden I chop the zucchini myself it's not a problem. Instead of saying how was the zucchini presented? And where you can bridge that variation and then a quick question for you when you split the RMS and your content do you have regular people adapt the content there or do you have some people that are it's a trick that our teachers created it's the content our teachers did and so there's some feedback so the teachers find it easy to work with the content environment it works out okay they're standing up because we're standing up for you at lunch so if you'd like to continue the conversation we'd be happy to be here for a little bit thank you very much for your time