 I hope we set the stage for Barbara, Vick and Katie. The audience is engaged, I'm thrilled. There is good energy in the room and we're now looking forward to your presentation and learn about your strategic view on the topic as well as how you would set the priorities. Thank you. Thank you. Well, hello, how are you? Is everybody excited, ready to go? Yes, so, pardon? Mike. Yes. Speak into the mic, Kathy says. So, hello, it's me again. I'm Barbara Cham, the education program director at the Hewlett Foundation and it's just really wonderful to see so many familiar faces and actually we're still a little excited when we see a lot of unfamiliar faces, I gotta say, because the OER community I think was small enough, for long enough that you could literally recognize everybody else in the room and now that is beginning to change a lot and that is an exciting moment. So, I'd just like to add my voice of welcome and thank you for, now many of you have traveled for many, many hours to get here. So, thank you for that. I'd also like to thank our colleagues and hosts at the Berkman Center. They've done a huge amount of work to put together a really terrific program for you so please join me in at least thanking them for the first time. So, when you think about it, in many ways the Berkman Center and their work, their sort of seminal work is, this is actually one of probably a number of intellectual homes for the whole OER movement. So, it's very fitting in some ways that we're having this conference here. But before we head into an all OER all the time extravaganza and in part to launch that, we just wanted to share with you a broad overview of the Hewlett Foundation's education program including an update in some of the areas that you may not be as familiar with. To do this, I'm just going to kick things off with three sentences about our broad aims in education and a short update on our deeper learning strategy which we introduced to many of you two years ago. Kathy Nicholson will then follow with a kind of the classic state of OER in 2011 and I think there's some very interesting and surprising statistics that she's going to present to you and trends that you'll find very interesting and exciting. And then Vic is going to sum up with his own version, I think of the heat map, you just saw the trends and the opportunities. He'll connect our OER and our deeper learning work together. Then he'll end with four asks. I'm just going to get ready for them because there are some specific things that we think you can do to really move this movement along over the next year. We're also actually joined by two other Hewlett Foundation colleagues. I don't know if they're here but Chris Shear, are you around Chris? He annals our policy work in the deeper learning arena and Mark Chun who is right there who just actually joined us last week and works on new models and our research agenda. We have a lot to tell you so we won't have a lot of time for Q&A actually right now although we'll try to do a little bit of that but we're really very eager to get your feedback so please find any of us and give us your reactions to what you've heard at any point during the conference. Moving on to the Hewlett Education Program. We support three key areas at the foundation. The first really involves advancing educational equity in our beloved state of California through policy advocacy and innovative research with a particular focus on state policy. I'm not going to spend a lot of time on this because it's maybe of less broad interest but please grab us again if you're interested in this piece of our work. The second includes our commitment to open educational resources. Of course that's going to be the bulk of our presentation today so I won't say anything more about that. Our third component supports an initiative to promote what we're calling deeper learning in K-12 classrooms throughout the United States. So let me start with an update on deeper learning. At the broadest level our goal is to provide students as you see the skills and knowledge they will need to be successful in 21st century work and civic life. When we asked ourselves, we asked you, we asked experts and we interrogated the research base about what that actually entails. It is these kind of core competencies that actually emerged. The ability to think critically and solve complex problems, to communicate effectively in both written and oral form, to work collaboratively and to learn how to learn all applied to the mastery of rigorous academic content, math, science, literature. Of course these skills, we didn't invent them. They've been around for a long time and our best schools I think have always delivered these competencies but it would be our contention that in the 21st century knowledge economy they will actually be required of anybody who aspires to a living wage job. Our public education system is the most important tool we have to deliver them, particularly to children in communities of poverty who have few other means of acquiring them. That's why we're in this area. So here's one data point from that research. It was a survey of Fortune 500 companies. They were asked to rank the most important attributes of their new employees in priority order and you can see that these key skills were at the top of their list. These skills are not only important actually for a career, they're also important for college. In addition to basic math and literacy, colleges are demanding these critical reasoning and communication skills. For example, a faculty study conducted by the Academic Senate of the California Public Education System identified the disconnect between the deeper learning tasks that will be asked of students once they go to college and their students' preparation for those tasks. They indicated that only 33% of incoming students are ready for those analytic writing tasks. So these sort of studies give you some sense, I think of the lack of focus on these higher order competencies in K-12 classrooms. We've also been looking for sort of a broader gauge measure to help us determine or assess where our schools stand. The best proxy we've come up with is the PISA. I don't know if you've heard of that. The Program for International Student Assessment, it's run by OECD, which is an international economic body. PISA measures communication and critical thinking in three subjects, math, reading, and science. Against that measure, the results for the United States are not positive. Hard to see? The very bottom? We clock in at 31st, in 31st place. So I think what that means, if you sort of put some of these things together, is that those Fortune 500 companies that we were just talking about now have literally dozens of other countries from which to source their talent. And for the United States anyway, that we're not only at risk of losing our lower skilled jobs, but also those involving more advanced skills. So from here I'm going to pivot from the problem that we're trying to solve to the solution we're pursuing. In our view, economic success and civic participation will depend upon the acquisition of deeper learning skills. The school systems are a major vehicle for delivering them, but they're not doing so, at least not in large numbers. So what do we do about that? How do we get schools to change? Kind of our answer comes in the form of four core areas of grant making. The first has to do with resetting the goals themselves. The learning goals and the requirements for schools serve policy work to actually shift to these higher aspirations for students and to hold themselves accountable for achieving them. The second testing for deeper learning is the measurement system. It's the way in which schools actually measure their progress towards meeting the goals that they set for themselves. More about this in a moment. The third, strengthening teaching capacity has to do with the tools, materials, professional development programs that support deeper learning changes in the classroom. We think that the OER community can play a hugely important role in many of these areas, but particularly here. You'll hear more about that from Vic in a moment. The last learning, evaluating, and demonstrating what works refers to a laboratory of innovation, innovative model schools where we can test out new approaches that can be replicated nationwide. So High Tech High, for example, is one of the model schools we work with. Larry Rosenstock, if he's not here right now, runs that school, and he'd be a great person to talk to about that. So let me now switch to just one of these areas just to give you a sense of what we're trying to do more specifically in what our grant-making strategy is. So that is on testing. I just want to be very concrete about the shift we're trying to enact. To do that, I'll draw from a couple of examples from U.S. AP History, the U.S. AP History Exam. So are you ready? Here's a question for you. This is from the 2006 U.S. AP History Exam. I'll give you a chance to read it. You have an advantage, because Massachusetts is on this list. But any ideas? What the answer to this is? No. Well, it turns out the answer is Massachusetts. B. But that's neither here nor there, right? It doesn't tell you a lot about whether a student knows anything about history. They could have guessed. They could have memorized this. It does not inform you. Well, here's actually a question from their new U.S. AP History Exam. And they haven't done this yet, but they're redesigning their exam. And this is a new question. So I won't maybe just sort of breeze through that. I think it's clear, at least it was certainly clear to us, that this is a better question. It asks students to understand these historical periods, to be able to explain why they're important, to choose dates not even on here. And I think you can imagine that a student responding to this question is doing a much better job of demonstrating their critical reasoning and writing skills. So that's kind of what we're trying to do. At a broader level, the question then is, how do we get a lot more questions like this one to permeate state exams instead of the kinds that we have right now? And I think the good news here is actually that the testing world is in a state of huge transition right now in the United States. And in 2010, the federal government awarded $350 million to collections of states to redo their tests, and we're hoping to measure these higher-order skills that we just talked about. It's a huge opportunity. Actually, at Hewlett, we've been very involved with the work of these assessment consortia. But with the opportunity comes challenge, and one of the biggest ones is that of cost. So if you kind of look at this question again, see a couple things. It says write. It says write again. That involves writing. Writing usually involves human graders. Human graders add both costs and delay. And in these very tight fiscal times for states, anything that adds costs can be something of a showstopper. So what can we do about that problem? Our answer to that has actually been, I think, a really interesting project that Vic has been leading around creating a competition, which is actually being run through the same folks who ran the X Prize to demonstrate the degree to which technology can be relied upon to aid in essay grading. In other words, can a machine come up with roughly the same grade as a human? To answer this, we're actually running two competitions. The first actually pulled together all of the major testing organizations that have scoring engines into kind of a bake-off. So they're actually testing their algorithms against a common data set. The second, which I think has been really fascinating, it's a competition on the open web right now where we're sort of letting anybody try their hand at the same challenge with the same data set. And so, in fact, you could do it. There's about two weeks left or a week left, I think, in the competition. There's $100,000 there. And what's been really fascinating to us, particularly, is that some of the front runners in the second competition are all data experts, but they're not education experts. So, in fact, leading this cheerful fellow is Martin O'Leary. He's a glaciologist from the University of Cambridge, and he's actually been at the top of the stack. Most of the folks actually in this competition have similar backgrounds or actuaries. They predict the weather. They're predicting your scores on essays, too. So that's been great. We're actually going to announce the results of this in a couple of weeks, so stay tuned for how that went. And that's just to give you a sense of, again, how we're approaching some of these issues. So that concludes my brief update, and I'm going to turn this over to Kathy. Great. Thanks, Barbara. For those of you who I haven't met yet, my name's Kathy Nicholson, and I work with the education program with our Open Educational Resources portion of the portfolio. So today I'm going to provide an overview of our OER infrastructure portfolio and also provide some highlights of the past year since the last time that we met. So to recap, our fundamental beliefs are that knowledge is a public good and that openness and principle increases access and increases quality through multiple contributions. So as you can see here, our ultimate goal is to equalize access to knowledge for teachers and students around the globe through OER. And with such an ambitious goal, how do we get there? Well, we hope to make open as much of an expectation and an aspiration in mainstream education as green is in the environment movement. We define mainstream as reaching millions of teachers and hundreds of millions of students, and that requires building and supporting an ecosystem that can support mainstream adoption, and it also requires research and supportive evidence and constant innovation. And we see these two goals as mutually reinforcing. So to support integration into the mainstream, we need to demonstrate how OER can help increase educational capacity. And we have four pillars here to support that part. The first is high quality OER supply. And this is a support of OER producers who are sustainably providing high quality resources for the core academic subjects in K-12 and higher education. And our key issue here is to continuing to develop the resources that we have, improving it, and aligning it with standards where that's possible. The second piece is supportive OER policies. These are policies that remove restrictions on OER funding and implementation and provide incentives to support OER. And as OER continues to gain steam, we need to be able to share more examples of sample policies and continue our strong advocacy and support of OER. The third piece are implementable OER standards. And these are standards that guide OER development and increase discoverability, interoperability, and accessibility. And here this is all in support of continuing to help educators and learners find and use OER. And then the fourth piece of this is under field building. And those are conferences like these that are about OER and support OER. There are awareness raising efforts. There are collaborations within the field. And here we see a key issue is expanding the breadth and the depth of OER adoption, including our own community but beyond our own community. How do we create more outreach and support for OER? Capacity must also be supported by evidence and innovation. So here we have influential research. We talked a little bit about that before in the heat map piece. And this is research that we think would spur demand for and guide the production of OER. And the need is to shift the question to not just what is OER, but what impact is OER having on teaching and learning and gathering more evidence to support that. And finally we know that change is constant. So we must continue to grow and change as a field. This opportunistic innovation portion of the portfolio supports innovations that demonstrate strong potential to transform teaching and learning and reach the scale that we're aiming for. So I think you'll see that over the course of the next two and a half days we've structured much of the agenda around these pillars. And one of the goals of this meeting is to encourage more integration among these pillars. And we hope that you'll have many opportunities to meet colleagues old and new and continue to further these conversations about how to integrate these pillars further and continue to move the field forward. We've talked a lot in the past couple of years about crossing the chasm. And what we mean by that is moving OER from the niche into the mainstream. And we know that that's certainly not a fast or an easy change. But over the past year we've been excited to see more and more examples of OER making that leap and of large educational institutions moving in that direction. First, I'll note there was an increase in academic recognition for students who take free online classes. Higher education institutions like MIT are exploring expanding the university's free online courses and allowing would-be students to earn official certificates from a program called MITX. And Stanford University is taking the democratization of educational resources to another level with their massive open online courses. The artificial intelligence course alone that was open last fall registered over 100,000 learners from around the world. And while these two efforts are not openly licensed we think there's a lot of potential for OER as institutions recognize the demand for high quality, free and open classes and the need for academic recognition for taking these classes. And perhaps one of the most progressive in this area is OER University which is creating a global network of post-secondary institutions who will accredit learning from OER courses towards credible credentials. And OERU to date has succeeded in securing over 15 anchor partners in support of this goal and counting. And moving from the institutional to the state level, the New York State Education Department is supporting the statewide implementation of common core learning standards through the New York Region's Research Fund. And the state recently issued two RFPs for the creation of common core aligned curriculum modules, tools, resources and professional development and also for a video library that would demonstrate effective teacher and principal practices. And all of this will be openly licensed and support over 200,000 teachers in New York. So these examples are just a few of a growing number that demonstrate the evolution of OER from small pilot projects toward mainstream implementation at the hundreds of thousands level. And we still have a ways to go before we reach the millions that we're trying to aim for, but we see this as a very positive indication of growth. So beyond institutions, we also see OER starting to increase or to cross the chasm at the state, federal and global levels as well. In the United States, legislation supporting the use of K-12 OER was passed in Utah and in Washington. And California is exploring requiring open licenses as a priority in a community college open textbook funding RFP. At the federal level, open licenses were listed as a priority in 11 funding programs. And during Open Education Week, which happened not that long ago, in support of the launch of the Why Open Education Matters competition, Department of Education Secretary Arne Duncan voiced his support for OER, stating, quote, that OER can not only accelerate and enrich learning, they can also substantially reduce costs for schools, families, and students, which is a pretty strong endorsement. And around the world, we have several examples of OER starting to make that leap. In the city of Sao Paulo, Brazil, legislation requiring all educational resources bought by the city was passed, and it was requiring all of those resources to be openly licensed. In Poland, recently announced the equivalent of $14 million to support digital and free textbooks for grades 4 through 6, all licensed under Creative Commons Attribution License or equivalent. Breaking news today, I'm not sure if many of you have been on the email list, but the World Bank just announced an open access policy and is launching an open knowledge repository all under Creative Commons license. So that's pretty exciting news that broke today. I'll say also governments in New Zealand, the Netherlands, and Australia also made commitments to release their data sets under open licenses. And then later this year, UNESCO and the Commonwealth of Learning will host the 2012 World OER Congress which will not only celebrate the 10th anniversary of the coining of the term OER but also encourage governments around the world to support the new Paris Declaration on OER. And Sir John Daniel and Zana Baroglu have a short talk on this on Wednesday and I'll speak more about that. But we should note that increased attention to OER also made OER a target at multiple levels. Openness was threatened with proposed legislation in the US including the Stop Online Piracy Act or SOPA and proposals to strip OER from the Department of Labor funding. But thanks to the public education efforts of the OER community and many others these proposals were taken off the table. But I think we can be assured that as OER garners more attention that these attacks won't be the last. In the commercial sector we had a few pleasant surprises. Blackboard which is one of the largest LMS learning management system vendors in the world because of high demand from professors and their students added a share button to their software which allows faculty to share their course materials and select a Creative Commons license for materials that they choose to make freely available. We're all pretty familiar with the largest or if not the largest online video platform in the world with over 4 billion views per day. YouTube is already pretty OER friendly because they enable language translation on any materials for any content that's closed captioned already. So that is nice. But this past year they also added an option to select a Creative Commons license for any content that was uploaded. So that was great. These developments make it much easier for educators and learners to choose and to openly and find openly licensed materials. We'll also note that in the small business sector the University of Pennsylvania hosted a business plan competition out of which seven were OER startups. So we're seeing support for OER at the large and small business levels. Now let's look at OER by the numbers. For the past three years we've run a small pilot project called OER Analytics. And about 20 OER sites have volunteered to put a specific line of Google Analytics code on their sites so that we could create a dashboard roll up of all of these sites and take a look at these analytics so we can learn more about how they're growing and how users are engaging with these sites. And we use it as a loose proxy to understand the growth of the OER movement and put some numbers behind it at least in terms of access and the user base. So one of the metrics that we look at is web visits year on year. And it has decreased for some and wildly decreased for others. I didn't put up all 19 or 20 lines up there. But we would expect to see that in a fairly new field that is starting to mature. But overall we've seen about a 10% increase in the number of visits compared to last year which is consistent with internet growth overall. But what's particularly exciting about these numbers up here is the fact that for every single site in the pilot study the number of mobile visits vastly increase. So this is a real opportunity. Vic is going to talk a little bit more about that in his section. That was pretty exciting. And while we're on the topic of numbers let's touch on a little bit more about the data that we've gathered. And again I'm just going to highlight a couple of examples. I just mentioned the OER analytics project which helps us understand more about the number of visits, what countries users are accessing OER sites from, what language, how long they're engaged with a particular site for example. And one insight came from an analysis of the hippocampus site which showed that its materials were consistently being used during the school day. And why does this matter? Because this indicates a possible shift of OER being used in the classroom as part of everyday school instruction and not just as part of a project or part of an after school program. And this bears further investigation certainly but you can see how this is use of data that helps us lead down a road of further inquiry about how OER is impacting learning and teaching practices. Another example is a Utah open textbooks which is a small study this past year that created a customized version of the CK 12 flex books that were printed and delivered to schools for about $5 or less per book. And the study compared test scores for students who used open textbooks versus students who used traditional textbooks and found that there was no difference in student learning leading to the conclusion that OER could provide significant cost savings. But there will be a follow on study for this later on this year. So you can see that we're starting to gather data about OER impact and efficacy but I'll note that this is the most nascent area within our OER field and we certainly need to gather more information about and data and evidence about how OER is impacting teaching and learning. My last slide here is about a stronger OER community. I spoke just about a few of the many highlights since we last met and unfortunately it's impossible to cover all of the excellent work and the accolades that were won over the past year. But we hope that you get a feeling for what a tremendous year the past year really was for OER. And all of this is possible because of you the OER community. It's things like the persistent collaboration with university chancellors, teacher trainers and staff to integrate OER into the fabric of their institutional policies and classroom practices. It's the Perg's cross-country van tour of 40 campuses in 30 states to increase awareness about open textbooks. It's the work of communities like the OER Advocacy Coalition, the UNESCO OER community, the Open Courseware Consortium and informal networks of educators and educators around the world. And it's all of you who attend conferences, organize conferences and get OER on the agenda of new conferences like South by Southwest ADU. And it's everyone who provides peer reviews, conducts research, writes reports and publications, leads workshops, builds curriculum and organizes events to increase OER awareness. And all of these efforts have led to national international OER coverage in numerous education journals, the New York Times, USA Today, the International Herald Tribune and the Huffington Post. A blog post about the recently released NMC Horizon report noted that the concept of open is no longer just a trend, but a real value entwined with ideals of transparency. So if we don't say it enough during our meetings, our calls, our emails with you, we just wanted to say thank you for a job well done over the past year. And we still have a way to go, which Vic will cover next. But we really wanted to take this moment to express our appreciation for all of your efforts and to keep this field moving forward to continue to equalize access to knowledge for teachers and learners around the world. Thank you everybody. My name is Vic Vuchick. For those of you I don't know. I'm a program officer at the Hewlett Foundation and I co-manage the OER portfolio with Kathy. And I also manage our scaling strategy for our deeper learning portfolio, which includes kind of the how do we implement high quality assessments and how do we support teaching and learning at scale. So I'm going to touch a bit on opportunities and risks that we see in the next year. And then I'll jump into a little bit of how we're really blending our two portfolios of deeper learning and OER to get the mutually and symbiotically beneficial improving learning but at a large scale. So starting with opportunities. One of the first trends that we expect to see is continued growth at the policy level. I think there's a lot of momentum going here. We've had a number of big wins this year and policy tends to be one of those things where after you get a couple of wins it's easier for other places to follow. And then also what's helping catalyze that I think is the new Common Core standards. Over the next two to three years states are essentially going to be working pretty hard on figuring out how to support the implementation of these new standards, which includes revamping curriculum, professional development, and other supports. And in this they will be reviewing curriculum adoption policies and processes and looking for opportunities to help innovate and dramatically improve. And OER I'm sure will fit into many of those opportunities. On a global scale as Kathy mentioned we've had a number of big wins this year again and we expect that to continue. UNESCO this summer will have a big event we just heard today from the World Bank and again this is an area where once you get the first couple of wins it tends to be kind of a snowball effect. Grand opportunities you'll be happy to hear we expect this to continue. Some of the big ones, the Trade Adjustment Act has a couple more waves of $500 million of pop coming through. So that's a big one. New York State will be continuing to issue some RFPs around implementing the Common Core. We're excited this is actually an area where one of our deeper learning grantees, Expeditionary Learning recently won a $1.7 million grant to develop curriculum for elementary school Common Core English Language Arts. And all of this will be openly licensed. Next is Open Textbooks. This has been a pretty hot topic a lot in the press getting a lot of momentum. I think one of the things I just mentioned briefly here this is both at the K-12 and the higher ed level. People talk about textbooks and we understand that there's a funny comment actually by David Coleman one of the authors of the Common Core Standards at our board meeting and he said there's one thing that the US is still leading the world in and that's textbook size and so but I think what's important about textbooks and thinking through Open Textbooks it's an interface into the system it's often how policymakers think and can reference and understand OER if you talk about content modules and playlists and all these things it's difficult to grasp it's difficult to put a price tag on and things like that whereas if you talk about textbooks there's a budget line item for that they understand what it is they have all used them and so it's a way to engage them and I would almost consider it a Trojan Horse in a way where people use Open Textbooks they can go to online courses and use simulations and start doing adaptive learning but it's just a frame that the system really understands and tends to react to very well Credentially in MOOCs Kathy mentioned a number of these efforts what I just wanted to highlight here is I'll be bold and I actually think this is kind of a catalytic moment especially built on a lot of the work that you all have done over the last 10 years both across OERU MITX and Stanford in addition to the work over the years from Athabasca Open University UK, Peer to Peer University among many others this has really captured the imagination of people and I think it's changed the conversation I've noticed this as I'm talking to people in institutions where traditionally four-year higher ed institutions actually operate in a world of exclusivity and scarcity it's often seen that higher education is a scarce resource it's expensive, it's difficult to access and it's even in the system where there are incentives for these four-year higher ed institutions to accept fewer people based on acceptance rates which often get rewarded in ranking criteria in various things this is tragic in many ways but I think with these efforts and the conversations that I'm seeing this suddenly changed people's conception and the institution's conception where it's not just how can we deliver education here but it's a race to how many learners can we reach hundreds of thousands of learners at a time is possible and there's still a lot of work to be done a lot of design work, we have to understand assessment and many other things but I think this year is actually an inflection point that has changed the imagination and we've seen venture capital running into this starting new entities we've seen universities major all over the world start to think about this and the conversation shifted not from why should we do this and how should we get engaged next Kathy showed some pretty dramatic data here on mobile and tablets I think this actually poses an opportunity and a risk for the field I'll dive into this a little bit this is a former area of expertise I used to work in the mobile industry but there's been a lot of hype for ten years that mobile is coming and it's changing the world it's taken a little while I think for it to really get rich enough that you can start to understand how this can impact learning at large scale but it's obvious that this is an area of huge growth but it ends up the content publishing ecosystem here is different than in the generic open web the way that a lot of content is reached on mobiles is through apps and sourcing from specific iTunes app store, Android app store Windows mobile app store and things like that and if you think about it smartphones aren't designed to scan hundreds of courses and read those short descriptions and this and that and do a lot of things that traditionally OER requires that you look at a lot of things and you kind of pick and choose and mix and mash but you look through a lot of data and information at a time so I think it's really important for the field to start thinking more strategically I often hate leading with technology as a design principle it's often the wrong thing to do that people fall into time and time again including myself but this is one area where I think it changes the systems dynamics and it's clear that if we don't engage and get better at this that you could be left behind so some of the things I would recommend in this is the benefit here is that these app stores and these app this app format is wonderful for rapid iterative design and testing out demand and testing if things work so you can rapidly put together various apps put them up and within a few months you know if they're going to get users or not you get that feedback very quickly rather than just putting up an open website and then monitoring over a year or two how well you do in the search rankings so it's actually something that you can really leverage to test out demand test out use and if you think about how people use mobiles it's often especially smartphones very targeted they have a specific question for a very specific resource or something like that and it's much less of a browsing context what I would recommend for people to do one thing is go to the app stores look at what are the top 100 education apps what are they, why are people using them what's motivating them and what services are those providing obviously people find that compelling and then think about how OER can help in those and work in those areas as well another thing is to ask students if you're teaching in a class what are the devices last night to help them with anything and what do they use them for basically start to gather information and really understand the space better and intentionally start to design for this because if we grab on to it it's a huge opportunity and if we miss it unfortunately it'll be tough to catch back up and then last and opportunities we are launching a project and it is to help curate and improve the open educational resources page on Wikipedia this is a reference that people point to time and time again and it came to our attention that while it has some stuff on it it could be much better than it is so I think this is something we encourage people to engage in this is something the community should help curate and feed into this project on the risk side as Kathy mentioned as we get policy successes there will be attacks and I think this is somewhere where we need to continue to provide advocacy efforts to help inform and support the public in terms of understanding and appreciating the value of OER we need to continue to drive demand you know we often talk about from supply side push to demand side pull the good news is we have some evidence of us succeeding in this blackboard putting a share button on due to the demand of professors is significant that's the number the biggest market shareholder and we're influencing their product design that's not easy to do similarly YouTube huge platform and due to demand is integrating with creative comments this is something we have to continue to drive and as you as we succeed in that I think we'll see our jobs come much from kind of pushing the boulder up the hill to running ahead of it as it goes down the hill making sure it gets to where we want it to go then there's a risk that a supply needs to improve as our opportunities get bigger five hundred million dollars we're not used to fulfilling that kind of capacity as a field in many ways in terms of curriculum development similarly in New York it's tens of millions of dollars for curriculum and if we aren't able to fulfill the supply there's a risk that some of these opportunities could not be renewed and could turn to proprietary or more traditional measures I mentioned before some of the risks of mobile and I think globally we've got some wins we've got some momentum but this is somewhere else where we just need to be diligent continue to participate in pushing to broaden global funding for OER to make sure it can scale and have the impact all around the world so now I'm going to touch briefly on how we're blending our OER and deeper learning strategy I'll take two examples the first one is in our testing strategy so Barbara highlighted we're working with some consortia we're doing this assessment prize there's another area where we made a contract to a small for profit start up actually it's called show evidence and what it is it's essentially a portfolio assessment platform teachers schools and districts can go online create rubrics for what they want to assess and then the students can upload any media whether it's written text audio video anything that they want the teacher can then go and tag any of those resources anywhere on the resources for evidence of what's in the rubric so two minutes and thirty two seconds into the video here's an example of this or in this part of the paragraph this is evidence of this and then it has the whole data and analytics engine behind it to gather all that information and I will say the student can also tag themselves and say this is what I'm demonstrating here this is what I'm demonstrating here and that gets collated as well and then it can do in a rate of probability across multiple graders etc etc for deeper learning this is critical for a number of our proof point schools and ends up we are also finding a project for four of our proof point schools to actually develop rubrics aligned to the common core standards that will support assessing project based learning for the common core standards now the open part of this was part of this effort was number one that the rubrics in that project would be openly licensed for anyone to use or reuse but part of the deal with show evidence which they were enthusiastic about was that they would set up the default on their platform that all rubrics will initially be creative commons attribution only license and so people can always change it and do what they want but the default is actually open and then they're creating an open community around the rubrics and trying to curate that and grow that so people share create divert derivatives etc etc so this is a foray into kind of rich assessment and open to help us reach impact at scale second is on the teaching capacity side we engage with Buck Institute Alfred Solis is here from there they are one of the leading providers of professional development for project based learning and deeper learning overall last year they trained over 5000 teachers face to face and they're growing at 30 to 50% a year where they say demand is astronomical for this so one thing we helped them was give them a grant to help them improve their technology strategy and help them scale and to start to experiment with methods that would allow them to grow as fast as possible while maintaining quality to give them credit before we even approached them they were publishing some of their stuff under creative commons license but now they will integrate some of that as they build even more as they build their online presence to support online courses for professional development and project based learning so in summary to wrap up as for us we have first is to continue building awareness about OER this is a cross advocacy RFPs supporting supporting policy efforts and op-eds etc this is critical and as I mentioned even though we're getting a lot of wins and momentum this actually gets more important because this is where you have to run ahead of the boulder as it starts really rolling and make sure this stuff gets done well and that requires everyone's participation and engagement second is to encourage your government to support the Paris OER declaration at UNESCO this summer this is important this is another inflection point and everyone here can play a role in helping that happen third continuing to organize and improve the research I think we saw earlier how the research was the lighter part of the heat map we need to make sure that this research is asking answering key questions that it supports policy based on data and impact and that helps practitioners improve so that they're impacting learning directly and finally help us integrate deeper learning into OER outcomes so we can truly get the high quality of learning that we all want to happen at a large scale in a very participatory participatory and engaged community so with that thank you everybody and I think do we have about five minutes for questions so let's open it up this for Barbara when you were comparing the researchers and the independent data scientists and some of the big testing providers and who's succeeding how are they doing relative to each other just out of curiosity actually we can't quite tell you that yet for one thing the data scientists are still at it but day tuned in a couple weeks we'll be able to reveal I think you'll be very very excited about it right now because we sort of know what the numbers are it's actually there's a public leader board and you can actually tell where the public data scientists are and we kind of know what the other number is but we can't tell you that yet but we will soon but I'll just add what we can say is that the vendors did very well and actually most of the vendors really did approximate or even do better than human scoring in terms of reliability so we can say that as a group without announcing specifics um are they going to make the their algorithms for doing those analyses open so other people can use them so that was a that was a interesting and difficult decision that we made given that this was the first competition and we wanted to maximize momentum we did not require that they open up the IP on the individual competition we will go in and partner with them and who knows if they're things that we're interested we're in a position where we can provide funding to help incentivize them to open it up I will say that there is one open source solution that was in both competitions from Carnegie Mellon in fact um so what's interesting is that there is at least one open source solution here that could be used in many different contexts question Susie how much are you looking to on the point of open source how much are you looking to um look to the future of funding more open source projects I'm looking at open infrastructure as well as open initiatives and open content so you're talking from development technology perspective as opposed to content yeah the innovations yeah so I think we're pretty selective on those um where I would say we're more in the content range and that's where we are focusing a bit more but uh there are certain occasions where we may support an open source tool or something if we feel that it's a clear barrier and could potentially kind of have a trajectory shift and also has a demand based built in so I think you know we are open but it's less than what we did before hey Vic uh advice from Dan could you talk a little more about what um your interests are in the mobile and tablet space and what sorts of uh projects you might be looking at or content access or evaluation or all the above it's a good question and I think it's something that we're in the sort of formation stages um I think uh part of it and what interests me most is assessing demand more and doing more rapid iterative kind of experiments to see what people will downloading and engage with um and I think mobile really provides that the other area actually to just touch on um is mobile devices especially smartphones you know obviously no one's going to write book chapters on a smartphone um but what they are good at is first of all they have wonderful cameras video cameras as well and they're great for sending tweets and things like that and so I think the field is still pretty naysan overall outside of OER in terms of really using mobile for learning powerfully I think there are possibilities though for potentially capturing student work pretty dynamically posting that um and facilitating interaction given that this is obviously where learners are pretty comfortable these days and uh the whole notion that we should have them put away cell phones when that's really what they're doing is a practical one in the long term so Carolina thank you we did a lot of research in the use of mobile in Brazil and other countries such as India and the Arabic speaking countries which are the focus of the foundation now in the global team and they are really not even editing Wikipedia right which we would think they are not even editing Wikipedia even if they have smartphones and many times they have smartphones right in developing countries so there is a lot of data now and we did a partnership with the internet steering committee in Brazil to do some research on that so there is a lot of research public research on that that you guys can see on how people are using and what are the apps people are developing to help people edit like something as simple as Wikipedia and that there are a lot of barriers so I think we are still really far away from big edits so there is just input there because there are ways to go I think in this interactivity go for it and just a quick comment I was at a are any of you familiar with the nonprofit technology conference it's a couple of folks are okay so it's an annual conference consists of like nonprofits and NGOs from around the world and I'll talk about their use of technology and I was at this conference last week and it was a lot of conversation about mobile and I think that one of the points that we are making here is that if you are thinking about mobile as a platform after the design phase then it's too late what we are encouraging is to think about mobile as a possible platform while you are designing your curriculum as you are revising your curriculum or the modules that you are working on as a possible platform and I'll just throw one geeky thing out there try mobile fi it's a very quick thing that you can take any website and run it through mobile fi they have a free version of it and you can see what your site looks like on a mobile platform it's very easy to use David we are working with the deeper learning work one of the places it seems like commercial content development is going is to diagnostic and adaptive doesn't seem to be a lot of diagnostic and adaptive work going on in the OER space partly that's a platform question it's also partly an assessment question cause diagnostic and adaptive is powered by assessments and the overwhelming majority of OER collections don't publish any assessment at all they don't openly license assessment it's just a gaping hole in the field around open assessment how do assessment and how do diagnostic and adaptive play into kind of future plans didn't hear a lot about that today but it seems a pretty big risk for us as a field go ahead go for it got something I think one thing to think about too is that this is a good opportunity for us as a field when we have the cluster meetings when we're talking about these issues when we're meeting with other people who have identified as builders or learners or facilitators to think through those very specific questions and say what are the interventions that we could do as a field or what are the suggestions that we have and how can we bring those forward because we'll have some thoughts on it but we are certainly not the holders of the answers of the field and we're highly dependent upon the expertise of the field to come up with those ideas and answers so just one one plug for the clusters and I think I'll echo that I think it's a risk and actually I limited the risk to five just because I didn't want to be a downer but data data integration and assessment was one of the ones that I trimmed off and I think it is critical it's something that we need to work on open assessment I think part of the strategy is first just the field of assessment in general is very black boxy there's five groups that have a thousand PhDs and nobody knows really what they do and so that's why part of the competition is actually we're going to do a series of competitions where we're going to move next we're going to do short response written response which is actually harder to do than long form then we're going to do mathematical manipulation and I could see in a year we could do crowd sourcing portfolio assessment and move to some things that are pretty innovative plus we will be pushing the open side I think once we get some critical mass participation but we see this as a field building effort that we would like to do to help grow the field of assessment make it more of a community and hopefully open it up but that said there's also just technical details of thinking what an appropriate strategy is in terms of embedding data in OER having good assessments that can start to track whether they work or not and feeding back that data we are funding some work actually that reminds me with the fat sign simulations and Dan Schwartz from Stanford to actually track the click streams use that data and see if we can use them as embedded assessments kind of as pre and post and for things like critical thinking problem solving and even learning to learn the metacognition side of it so we're dabbling but that's something where we definitely are looking for help and advice thinking through how we're doing time wise I think are we alright sorry let's wrap up please feel free to approach any of us we're looking for continuous feedback and improvement over the next three days so I'm in somewhat uncomfortable