 Well, we're the last session of the day, so that means we're standing between you and food or drinks or something. So we may not go the whole time if we don't need to go the whole time. So this is an open textbook pilot study. We want to report on data from last year, so this is 11-12. So we've been the 10-11 school year. This is a partnership between the State Office of Education and Tiffany Hall will probably wander in here at some point, as well as a John Hilton and myself at BYU. And if you were there for the keynotes this morning, you already heard Superintendent Shumway make some reference to this. So I just want to describe the pilot, describe what we found and where we are currently in this process. So Utah's approach to textbooks is that we do not have a mandatory list that's maintained by the state that every school has to adopt from. We have a database of recommended instructional materials that textbooks get vetted and reviewed for and then are approved to be used as a primary resource in the text room or as a supplementary or a limited resource in the text room in the classroom. It's been a long day. And so districts are actually free to do essentially anything they want to do. They can choose a book off of the recommend out of REMS, the recommend instructional materials list, or a district can say we want to do something else and then the district is just responsible for documenting the process they went through to decide to make that choice as opposed to choosing something off of the recommended instructional materials list. Now this pilot we've been doing in the context of science and science textbooks in high schools and so one of the things that we did was just went through and looked at every book in every science category at grades 9 to 12 and just found what is the average cost of all the books in the REMS database. And so you can see that through some happy circumstance the average cost of the science textbook is exactly $75. In what's in REMS currently today. Now this is particularly in the context of K-12 budgets. This is a relatively high cost and that high cost has some impacts. The first is that when a high school buys a textbook they typically buy that book and then they hold on to it for a number of years. We've heard numbers in the 5 to 7 range as we've talked to people here in Utah. When you take a book and you hold on to it for 7 years and you hand it down from generation to generation to generation to generation on the 7 times of students, you know you end up with books that are out of date. Obviously by the end of that process books are in very poor physical condition typically if you're the 5th or 6th or 7th student to receive one of these. But a bigger problem pedagogically for these books is because they have to live a 7 year life is that as a matter of policy you tell the students you may not to face this book in any way under those circumstances highlighting in it or writing in it or taking notes or drawing pictures or anything like that because other students have to come along after you and use this book. And so one of the impacts of high cost that we don't typically think about in higher ed I mean it's hard for me to remember what was happening earlier this morning let alone what was happening in high school to know that I wasn't allowed to write in any of my textbooks and now trying to imagine studying without being able to highlight or not take or do any of those kinds of things is a challenging thing to imagine. And also because the cost of these books are so high it's not uncommon to see this approach of someone just buying a classroom set and putting 35 or 40 books at the back of the room and then saying okay now we're going to use the textbook everybody go grab a book off the shelf and now we'll read something for a minute and I'll put them all back in a situation where a student can't take a book home even to read in the evening or over the weekend and we want to feature prominently Tiffany who's walked into the room and I actually just an opportunity to embarrass you. So the high cost of these textbooks have a variety of impacts. I suppose I don't have to say too much to this group about open textbooks but they are free and legal to download, to revise, to adapt, to redistribute the four R's we call these reuse, revise, remix, redistribute. They're traditionally distributed in digital formats so they're available to be used on an iPad or a Netbook or something like that but even though they're distributed digitally, originally you can't take of course and run these through a ground-on-demand process so that kids who want a paper book or a teacher who wants a paper book not only gets access to a paper book but gets access to a paper book through a process of going out and finding who will print this book for me the cheapest and being able to kind of shop around and get a good price on the printed version of their open textbook. So specifically about the Utah Open Textbook pilot program this was originally funded and ongoing funding has come from the William and Flora Hewlett Foundation and also there's time that's been contributed from BYU in support of the initiative. We started this process by going to CK-12 and didn't either make it into the room? How many of you know CK-12? If you don't know CK-12 and you work in the public school space at all you should go check this out because they provide a wide range of books in math, science, pre-engineering and technology. On math now as far down as grade 6 up all the way through high school so it's a complete set in the case of science of course of earth science, biology, chemistry and physics. The teachers that were participating in the pilot were earth systems, chemistry and biology teachers. We didn't have a physics teacher at the original pilot. So we identified these textbooks and we had a very interesting conversation with Lisa's really interesting for me because it didn't go the way I thought it would go. Tiffany and I sat down with curriculum coordinators from some of the districts in what's called the BYU Public School Partnership. It's five districts that we have an ongoing partnership with the BYU works together with and I thought we could tell them hey there are these books that you could use that you could save a lot of money kind of to Jim's point earlier this morning. They said well yeah I think we could save money but you're missing the point that if these books were really this inexpensive to print that opens up a whole new range of things that we could do with these books that we haven't been able to do in the past. So after some buy-in from that district curriculum coordinator level we got teachers together again using funding from the Hula Foundation got them together through some professional development to teach them what OER are but yes it's okay to edit and adapt and tear parts out and make changes. And also some technical training on the CQA 12 system they knew how to use their editor to actually make the changes that they wanted to make to the book. So we ran this process over the summer and revised these books and the majority of the revisions were really just pulling out things that either they don't cover aren't part of Utah standards or something that the teacher already had a great thing that they did in the classroom to teach that that they didn't need that to be in the book and pulled out a bunch of this material that was essentially irrelevant for their purposes. So phase one this is 10-11 we had 7 teachers and 1200 students in that group and two things we learned a lot about trying to affordably print open text books and this is all covered in quite a bit of detail in a paper that's forthcoming that we've got written up on this that covers the cost figures in some detail. But we let the teachers basically be free to try a number of things so some teachers printed the books on loose leaf paper that was three hole punched to go into a three-year binder and some sent wanted to do print on demand through a service like Lulu which generate a real honest to goodness paperback book and things like that. And what we found was that there are a wide range of ways to print open text books that are much more expensive than just buying an additional book. We learned this lesson firsthand but until you get out and try you know you're never going to know but we did learn some things about ways to actually make this process work very well which we'll talk about. And we do have some efficacy data because one of the things that we included in our initial proposal with the Hewlett Foundation was that anybody will believe you can go find some free stuff on the internet and download it and print it off for five bucks. Sure. But you get what you pay for your student outcomes, your learning outcomes are just going to tank. So we got a track and figure out what the impact on student learning is. So we'll talk about that in a minute. And I did want to say that we had a very successful phase one in terms of teacher attitudes and student outcomes and for phase two we've gone essentially district wide in the Nebo district here in Utah where there were 20 teachers and 2700 students and we took all the lessons that we learned about print costs in the first year about what not to do and what to do and we're able to purchase text books at the following costs this year for those 2700 students. So these prices don't include shipping but when you add the shipping cost here, the average cost for one of these books that we use is $5.35 for a primary text for a science class, a year long biology or chemistry or systems class. So I think we have I think it's safe to say that we've absolutely cracked the nut on the price part of how to use open text books. And this does not include people that already have net books or iPads in their classrooms that just used the digital version that covered 200 kids and didn't incur any cost at all. Those people aren't included in these figures. If we included there's extra students and divided by what we paid to print, obviously the cost of using these open text books would be lower. And we don't have anybody in physics currently. So one way to think about this in terms of price comparison is to think if you buy a $75 or an $80 book and you keep it for 7 years, what are you spending per year on that book? Or if you only keep it for 5 years, what are you spending per year for that book? And if you use the open book, what are you spending per year for that book? So I think these comparisons are pretty interesting and pretty compelling to say that even on a 7 year cycle you're saving over 50% and of course you're saving a lot more if you're just on a 5 year cycle. If you look at this between the lows and the highs, we're saving somewhere between $5.40 and $10.30 per book per year. Because in this model we're going to print a new book every year. The next group of kids comes in and I'm going to print another book and hand it to you and say that book is yours, you can keep it, you can highlight in it, it belongs to you, knock yourself out. And with over 150,000 students in grades 9 through 12 doing science in the state of Utah if you take this range and do the multiplication and we're somewhere once this goes statewide we're talking about saving somewhere between 800,000 and 1.5 million a year. So the financial impact on the state will be very real. The impact is already very real. We've shown this loud and clear. This is great. So this is a good finding. So financial benefits. To talk about pedagogical benefits for a second, well this is a bit of what I was saying, isn't it? Each student gets a brand new book every year, they put their name on it, they keep it forever. They are allowed to highlight, they are allowed to annotate, they can take notes directly in the book, they can take it home over the weekend to study and read. Some of you might have heard this earlier, but Tiffany just has a fun anecdote to share. Tiffany do you mind telling this little story about football games? My apologies. So I was sitting at a football game and my stepson is the kicker and it's not as if we have to be very involved and the way the football stadiums are set up is you always put the visitors facing the sun. So we decided we'd sit on the home side so we wouldn't have to face it to the side. So he's only the kicker one, so it doesn't matter. In fact you might not even kick that game, but I got to talk to parents and so I was sitting next to a parent who had several children who had gone through and had one son, his younger son, who was a biologist and I said, oh you know I just hear that the new kid is at the cutting edge. Cory are you hearing me say this? Nebo district is just on the cutting edge. They do some really amazing things in science. Who does your son have this year in science? Well he has Mr. Blake. Oh I say he has biology. Yes he does. And the dad looks at me and says, and you know what, he's got a book this year and he gets to bring it home and I want to know how come my other kids didn't have that. And I said, well strangely enough I didn't answer that question. I was reading you that way or anything. But I just kind of, you know I talked to him and I said, what do you see your son doing differently in this biology class that your other two or three sons, several of them, was kind of big kids running around the football field, but they didn't do. And he said, my child brings the book home and he writes it. It's just like when I was in college, he highlights stuff and only what's important is in the book and he brings it home and it doesn't weigh 500 pounds so he can just leave it in his backpack. We never have to go back to the school to get the homework book. You know his parent perception is always one of our concerns whenever we try something like this. Well they look at this softbound book that isn't printed in color, you know, isn't perfect and glossy and shiny like the ones from the big publishers on our big front parlor book that costs $80 or something. And well they think that their child is getting a lesser education and the opposite wasn't actually a fact where he really viewed that as my child has a resource that my other kids didn't have and I see that it's making a difference in his interest in biology, in his ability to do his homework, and how engaged he is with the text. So I thought that was really important to have that conversation. I mean do you go to a football game expecting the person next to you to say, no my kid can write and isn't good. That's not the conversation you're expecting to have in football. Thank you for bringing that up because that's a really fun story. So on the professional development side I want to say that there are a handful of things that are really super important, several of which we did not do in the first year just because we were running as fast as we possibly could trying to keep things working and running. We did get the teachers together and we did talk to them about what openness means. We gave them the technical training on how to use the platform but we never, so we did deal with helping them understand that they can make use of the four-hour permissions that they have in the books. But we didn't go out of our way to tell them about any of the new pedagogical possibilities that they really ought to be thinking about. Different things that they could do, different things that they ought to do or in one of Tiffany's areas of expertise in literacy. Do you want to say a little bit about the science literacy kind of possibilities here? Well, we just, we ran out of time. We just didn't have time to do everything we wanted to, but we really wanted to talk with some of, you know, with them and help them understand how to use these books to help teach science literacy. To help teach, you know, since they can touch the paper how do you read charts and graphs? How do you really begin to understand a science text? How do you transfer that into your labs and activities? How do you transfer information from your lives and activities back to your study of science textbook? We didn't really have a chance to do that in the first year. The second year, with the much wider scope of people, we have two things going for us. We have several science teachers who are in the second year of the pilot who have specialized in science literacy. So they are already ahead of us in that regard. And the second thing is that this is the year that the state of Utah has been a student. So we're hoping to begin to gear up our common core state standards in science and science literacy. And so we're hoping to be able to find some connections between that and be able to have these types of books that teachers might like. So in terms, when we think about the learning impact, which are the next couple of slides I'm going to show before I wrap up, we really didn't have an opportunity to do any professional development at all. We essentially said you had this book before, take this other book and just do the same thing, just use this book instead. Yes, we know it's paperback, yes we know it's black and white, yes we know it's not glossy paper. All these kind of surface features that you think are the things that go into making a good textbook that it's 1400 pages long, hardback, whatever. So we really didn't have a chance to do any of that training. We just kind of swapped out books and said do whatever you do. So I want to look at the learning impact. These are graphs for these seven teachers. This is the pilot year. These are changes in the percentage of students that demonstrated proficiency on the state standardized test at the end of the year. So 2011, this is the group of kids that took the exam that had used the open textbooks. And then we compared them to kids in the same teacher's classrooms for two or three years before, oh I'm sorry, one or two years before depending on how many years of data we'd get access to. So we kind of control for teacher factors, kind of control for the types of kids that are in the school. Same types of kids, exact same teacher teaching the same class, all we've done is change the textbook. And so what you see is you see well this guy was 100 and he did all the way to 99 the year before and he was back to 100 with the open textbook. But essentially you see the one person staying the same, a couple of downward trends, a couple of upward trends. When you throw this all into a pot and just do some descriptive statistics on it, what do we find? Well, you can talk about central tendencies in these distributions a couple different ways. If we talk about the mean change across these seven teachers, there are 3% more students that were proficient in 2011 than the average of the years before. If we talk about the median of the distribution, it's 1% lower. So essentially basically no change across the group. Three up, one down. Basically they've maintained as a group, kind of maintained where they were in terms of learning outcomes. So to bundle all this up in a summary I think what we did show in the first year of the pilot, which was admittedly smaller than it might be, but still 1200 kids is not bad. Really significant cost savings. The ability to maintain learning outcome levels and that's without any PD at all. And then we've started to understand what the new pedagogical opportunities are and we want to share that now with teachers going forward and see if we can actually move the learning outcomes up at the same time that we're holding the costs down. And I do also want to put a plug in to say that if you have an interest in common core math, we're in our second year of the science piece now, but we've also been having really interesting conversations with people at the state office just in the last what seven days. Yeah, I mean something. Anyway, our plan now in Utah is to take what we've been learning in science and start to take it over into math and to common core as well. And I think if you heard the superintendent's enthusiasm this morning as he talked about some of this, we just have great support leadership wise at the state office from the superintendent down to the woman that owns for example math and science curriculum to Tiffany to everybody on that side. And of course, we're really committed at BYU. So I think they're really great things to come. And I think I've saved us a little time so we can do some questions or possibly even end early. Now, Corey, what question could you possibly have? You're in the district. I'm the instructional technology specialist right now. So it's kind of our responsibility to monitor the teachers and handle the technologies. And I think going into this project is when we started off, the intent was never to save money. The intent was to do twofold. The one was to reach channel money and two, in fact you'll have to so it's going to save money. And two, we want to build capabilities that of course doesn't go by in things and adapt things and things like that. And because of the success of the program in that first year and continuing to be the secondary of the district is saying this is the way to go across the war. Obviously there are some teachers that are never going to get there. But the district administration is on war. It worked out that last point in years of pedagogical possibilities. And it's important. I have to make a plug for Corey. You know, we've got CK-12 giving us science, math, pre-engineering. Corey needs social studies. I need social studies. If you know anything about collections of high school, age, appropriate social studies, OER, please, please, please come either tackle me or Corey when this presentation is over. Because they really are. They're ready to just go as fast as we can find appropriate content. The first year we kind of worked with kids being able to highlight and come back in their iPads and we didn't have their same notes and it didn't work. We'd kick them out. Standing in the hallway trying to get a connection. Yeah, and so it was a district commitment and it was having a champion like Corey who was like, what? You want to quadruple my workload? Bring it on. I mean, you know, it was really the district that was ready. And that's one of the things that is you're going to expand to look at this. You really have to say, are there going to be people at the district level for the superintendent on down? And especially the people who are going to actually help make sure that the technology is teacher and student driven and not driven from some kind of policy piece that doesn't connect to what happens in the classroom. Because that's really critical to letting anything like this happen. This is a very commendable project you're doing. It's very impressive. I'm curious about a feedback loop. Since you're so hands-on, you really can make for the students. It's not like some company out there that just ships you the books and they made their money and hopefully you're happy. Do you have a feedback loop where you maybe get focus groups of the students because they write in that they can really connect with the material? Do you kind of see what work for them and what would they recommend? Yeah, so we didn't end the first year but thankfully this year we've got a master's student who's taking on that piece of the research as a master's thesis project. So we are gearing up to do that but I don't have anything to talk about in the regard other than that we finally found the graduate student who's willing to kind of own that and do it. So it's exciting. Next year hopefully we'll have interesting things to say about that. Four minutes away. So we can quit early or you can raise your hand. No, I'm just kidding. Go ahead. Just a quick question on what the SMA PD cost was to develop the curriculum for the end. So I think that okay, so I'm laughing because part of our informal analysis of how this played out is that there's a boy model and there's a girl model of how this got done. Keep in mind that on the PD side you're just talking about the technical training to do the adaptation and things, not the pedagogical stuff. So the short answer to your question is that the average was about 60 hours is what each team put in to go from, and this is an important part that I didn't address I think now in retrospect, to take these books from 1200-1400 technical kind of science books like you're used to and get them down to 250-225 page really customized tailored books where we're not printing anything that we don't need. And it was really striking that across all the teams it was almost exactly 60 hours for chemistry, for biology, for air systems. What we found that really made us chuckle a little bit was that the approach if I can say boys and girls, the approach that the girls in the group took was they spent, we met, we just talked about this, I think we met on the first time on July 1st or sometime that's the first week of July and all the girl teams, but particularly chemistry had everything completely done, completely revised back to a sweet printed a book, Lulu, print on demand and they used it in school that year. The first time they heard of OER was the first week of July and they were using books that year. The boys took the approach of well let's just print the whole book and as we go through the book this year I'll just mark out the stuff that I know I don't want next time. And so they spent almost no time up front and then at the end of the year went back through all their notes and said we don't need that, don't need that, went into the digital, went into the system, pulled it all out ended up taking about 60 hours again by the time it was all said and done. It was just interesting to watch the different gender groups that we had. But so depending on what you pay your teachers for PD I think we're using $30 an hour I think is what we were paying that Hewlett Foundation was supporting so you say 60 hours all in including the days that we had them there for two days and whatnot. So you're a little under 2000 in on the adaptation side. And when you buy 1200 books each year for several years by the time you advertise that across all your books it's pennies on the book. You know it's not like this entirely new process it was never paid for before. Some of that you've got to do one way or another whether you choose a proprietary book or an open one. And many of them add a common form to assessments that their departments had already developed so when you measure in that graph where you were showing the test scores you were showing percent proficiency that's the y-axis. It would be interesting to see actually how the distribution of scores changed because one of the things that you might add to that is that actually kids at the lower end of the distribution might benefit more from having access to textbooks and access to home that they come home. Whereas kids at the upper end of the distribution have probably wealthier kids who have more access to resources anyway. So in fact you might see that even if you're not getting more kids to proficiency in one year that you're still having kids that have a better science understanding. And what did purple teacher here do? Purple teacher was kind of flat for two years and then jumped 23 percentage points and adopted an open book. What the heck went on in that classroom? You know I mean I'd like to understand that. The teacher has part of an impact on the test and so I think that the jumps and the drops are the teacher's ability to use these new materials and as they become better to use these materials we're just going to see that come back up. Which again comes back to PD. We gave them no PD at all right. We handed them a new book a week before school started and said you can just use this book instead of the other one. And just the fact that they didn't all just fall off. It should have been invaders drop. Right because it's this J-curve of innovation where you have to get used to it first and then you don't see their turns till second or third year which is not. You can see a little bit of that at individuals but across the group literally no change. No matter how you look at that it's no significant difference. Anyway now we're one minute old.