 My name is Anish Chopra, and I have the pleasure of establishing the foundation for what will be a very exciting panel. Let me first introduce the folks who are coming up on stage so that you all know who's with us today. Judith Murray, raise your hand. Judith Murray is the campus executive officer at Altus Education and Ivy Bridge College. Dean Flores, there you are, raise your hand. Thank you so much. Chief executive officer of 20 million Mines Foundation. I presume that'll be you, Richard Young, right there in the middle, Richard Young. Senior education architect at Microsoft. And rounding us out, is that you, Tammy? Tammy Windcup, the chief operating officer at Everfi. Thank you so very much for being here. I have the pleasure of helping us walk through a couple of ideas, some very provocative comments, by the way, from what you did this morning, thank you. But before we get started on the panel, I thought I might just take the liberty of a few moments up front, maybe to set the stage on the perspective that I carry to this conversation, and that might be the basis on which we would have a nice discussion. I served as Governor Cain's Secretary of Technology from 2006 to 2009, and then served as President Obama's Chief Technology Officer up until last year. And I wanna take you to Danville, Virginia as a starting point. There was a plant, Corning, owned and operated the plant, that used to, among other things, make cathode ray tubes. The plant announced in that time period, when I was in the governor's office, that they would shutter their operations, and they would lose the final 200 employees that were working out in Danville. And it was, metaphorically, the challenge of modernizing and new technology and new product, how many of you own a cathode ray tube television anymore? Any of you buy any of them? Probably not. Yet, that very same time period, we received a review, the Governor commissioned of scientists and engineers, who took a look at the Virginia physics textbooks, because we wanted to get a sense for whether or not we were up to standard. Are we presenting the right information to kids so that they can compete in the 21st century economy? And what does it say in the Virginia physics textbook? It says that the main component of a television is the cathode ray tube. So you have students in Danville, proudly coming home to say to their parents that you are at the center of the television revolution at a time when their mom and dad are out of work. Clearly, we needed to change the content and to upgrade or modernize what we were teaching folks so they would be up to speed. But the textbook adoption cycle was like molasses. I think it would have been 2011, maybe 2012, when we would have gone through the formal reviews, the evaluation, the production of new textbooks, the acquisition cycles. And that meant several years where students in Virginia would lack access to the information so that they could proceed in their career path with hopefully information to prepare them for the future. I came to open education resources not as a technology chest-thumping champion, but out of necessity to bring new ideas and new content to market. That very summer, when we learned of this gap between what kids were told in class, what the textbook shared, and what was reality, we decided to crowdsource the solution. I asked Governor Cain if he'd be kind enough to put out a no-dollar solicitation since we have no money, and he did. We thought of this challenge in the summer. The governor puts the rollout in September and says, we have no money, but if you have interest in helping us to upgrade our physics information, please volunteer, we'll thank you, we won't pay you. But dozens of people around the country, university professors, students, even a couple high school students said, we wanna be a part of this movement. And in collaboration with the CK12 Foundation, Virginia launched in less than six months from the time the governor's announcement to the selection of the folks who would help us through the four rounds of peer review to make sure it met quality, the Virginia Physics Flexbook, all of which is freely available online for remix and reuse. And we did so out of necessity because we wanted to make sure that the kids in Virginia had access to the information that mattered on modeling and simulation, on biomedical imaging, and so forth. This wasn't a fight between intellectual property rights and the commercialization and the open source movement. This wasn't about price. This is about accuracy and ensuring our folks could compete. Because of this effort, Virginia went on to pass three pieces of legislation to standardize and treat fairly open education content so that the monies we allocate to schools for acquisition of materials could include open education resources in addition to licensed material and could essentially, I call it, chunking up the content so you could buy a collection of learning objects as opposed to the full enchilada. I share this with you because we're going into this conversation now, I think, asking three questions and we're gonna be posing a number of these to the panelists today. But from my vantage point, we're at a time where we're looking at a tremendous amount of change in the economy. While we still have a high unemployment rate, we have millions of jobs that are open and unfilled. So question number one, what are we teaching? Is the content or the materials we're teaching preparing our workers for the jobs that are available? Second question, how are we teaching it? We have the traditional classroom model, but I love the trough of disillusionment as it's upcoming on the MOOCs, but whatever the model may be, new models are emerging that are coming up, may be successful, may fail, may need modern, may need tweaking. But how are we teaching? And then obviously in any innovation, it's not just about the trial or the prototype, it's about scale. And how are we gonna scale what works? These are the questions that have occupied my time in the public sector. A quick litany of opportunity. We've now started to see action in what we teach the Department of Labor and the Department of Education are investing $2 billion over the last several years and continuing to put new curriculum, new content that are aligning the needs of workers and the jobs that are available today through a community college, the TAA grants. With how we teach, Secretary Duncan and I launched something called the Open Education Data Initiative to open up assessment data and to build essentially tools that would allow new models of learning to plug and play, whether it be badges or other movements so that we can start to engage in new forms of teaching to demonstrate knowledge. And last but certainly not least to scale what works, we've begun through the learning registry, establishing a technical foundation so that we have, why I like to call it the teacher internet, so that what works is quickly and easily discoverable and shareable across the entire teaching community at little to no marginal cost. Today's conversation will focus on this next wave of digital education. I look to be a little bit provocative but to surface three big categories. One, a sense of what's happening. What is the changing landscape we're operating in? We're gonna ask our panelists to give us a lay of the land. Second, I wanna understand barriers. What is it that's limiting us or inhibiting or stalling any progress if in fact there is a sense that we haven't yet fully leveraged the tools? And last but certainly not least, the flip of this, where and how might we scale what works? And what's the right role for policy? What's the right role for technology and innovation, acquisition and so forth? You've heard enough of me and from me. What I'd now like to do is turn to my panelists and begin that very starting conversation. What's different? Why don't you start Judith and give us a landscape? Where are we and what's changed in the digital education world as you see it? I think there's been tremendous change over the last number of years. If you think back to more than a decade ago when MIT rolled out the first open educational resources which was really about pushing content out into the world for others to see and to use too. You know, the movement that's happened in the last few years which is all about reuse of those resources both by other institutions and by students in general or learners that are out there in the community who wanna access those. I think some of the most exciting changes that have happened with respect to open educational resources in the last couple of years is the movement to open textbooks. UNESCO actually more than a decade ago coined the term open educational resources and last year I was fortunate enough to be at the UNESCO meeting in Paris where there was a new declaration signed by countries committing to the fact that any resources that are created using public funds will be made available for free to the public and so that was a great move forward in the open educational movement. The second most exciting change in open educational resources and I think the biggest change that or the change that can have the most dramatic effect on what we're doing in higher education was talked about with the panel yesterday around the ability to actually give academic credit and to get credentials that are formed from open educational resources or learning that has happened through open educational resources. That's really the holy grail that needs to be cracked and when we crack that then we can really make changes. Thank you for that. I agree with everything she just pointed out and we're also seeing a lot of innovation taking place where online courses from major universities whether private or public are being made available not only to other college students but also to K-12 and even their parents we're seeing a lot of parents actually re-enter the education field simply by working with their kids and their kids are introducing them to YouTube and other social media systems where they can actually tap directly into classes that are being taught online. I'll use an example in my own situation was my mother. My mother's 70 years old and she is now becoming very interested in computer science. She never understood what I did before. When I first got into this field she used to tell people that when I explained to her that I was a systems analyst she thought that I worked for an actuarial company and so when she finally started to understand what I actually did and now being able to take these courses online it really speaks to the power of the internet getting back to our opening speaker's comments that the internet didn't really change anything it just became another vehicle for content. What we are starting to see right now is that people that we never could reach before from an educational perspective are now starting to leverage that but what is still king is content but content in the sense of can the individual user tailor it to fit their particular learning styles? And I think that is gonna be the next wave of innovation. How do we actually tap into the individual so that they can now really gauge how they're learning something and modify it in such a way that they can get greater value out of it? Well, since we're going down the row let me say I think what has really changed in this dialogue is not innovation I think the opening speaker has pointed out from the television all the way through the internet we've had these spurts of innovation I think what's changed in this space is the way we talk about it and quite frankly I think success in this space and the movement in this space has come because of mainstream press I mean we're no longer talking to each other in a Neco Chamber we Twitter each other all day long we used to do this many two or three years ago I think now you see David Brooks you see the New York Times you see mainstream editorial boards talking about the credit you see mainstream editorial boards talking about online education you see if you will a discussion in our higher education institutions that used to be quite frankly insider discussions now has been mentioned by Dr. Green the pounding on the door of the president by the trustee who happened to pick up the Los Angeles Times and the New York Times or the Washington Post and read about the MOOC madness that's occurring if you will in this nation and asking the question why aren't we doing this I think what's changed and has brought an inertia to this particular debate is this mainstream discussion I think if we continue to have had it if you will among ourselves at conferences on Twitter in our echo chambers I think that would have remained if you will closed I think today it's very much an open discussion where you see governors pointing in all sorts of directions whether it's Florida, whether it's California whether it's Texas everyone is trying to figure out how to fix a gap of growing cost and student dissatisfaction and I think we are now at this point in time I think I missed Mike Saylor's comment yesterday but I did read it on Twitter in our little echo chamber where he said one or two of us could actually change the world and I think what he was really saying is that if we get the word out in the mainstream to folks who are the policy makers have the ability to do that then I think we see lasting large change and I think what's really changed is this discussion and a much larger discussion Timmy? Well first of all, Dr. Green thank you I'm feeling very confident here in my red dress Very good You know I'd echo Dean's point I mean I think one of the biggest changes that I'm seeing is this real impatience for education, innovation and from folks in the mainstream but also from folks that weren't involved in the education conversation even as recent as five years ago and so I think that that that is not just an education conversation that's a private sector conversation that's a policy conversation where I think that echo chamber was much smaller we talked about this five or six years ago we remember kind of being the same hundred people who were interested in this topic or a thousand people across the country and I think what we're seeing is this broadening of conversation around it I think that's really important I think what comes with that though is a very noisy environment and a very noisy industry and I think that right now we're in this phase where we are kind of seeing who's going to play what role what role will policy actually play and actually opening up the industry for innovation what role will this very franchise specifically in a K-12 model you know where will they embrace that innovation so I think that that's the biggest piece I think the other thing that's happening is because of all the players whether they're content providers or now media companies or everybody who's in the education game I think we're seeing a good turn where the student as the center we're changing the conversation where the student as the consumer of education is actually playing a bigger role than they ever have before and I think that's good for the industry in general I'm going to dive a little bit deeper to cross up for anyone who wants to go there I've understood that you've said the technological advances are here that there's a growing demand from across the ecosystem so the question I want to understand is just go one layer deeper what are the promising or emerging business models that are at least capturing your imagination or giving you a spark that says this might make three and anybody want to share a bit about what they're seeing specifically not on the advances technologically I think you've all said that's been there what's the business model innovation that's got your interest or attention please Let me just jump on Dr. Green's comment about MOOCs for example and if you think about the business innovation what's capturing our attention out this year particularly I think MOOCs and this discussion about credit and how they will actually offer access to students for courses they may not be able to get is the debate in California for example and the governor and the pro tem of the senate are all interested in MOOCs providing a way of access they're not talking about a business model about MOOCs providing cost savings what they're really saying is that if a student can't find a course and that that university isn't offering an online course why wouldn't they be able to go to a third party provider and receive credit for a course that a university would approve now the big question and the fortress built there is the credit obviously it's the university's making the big decision of whether or not business wise they're going to be able to figure out the deal that makes sense for that credit I think that's the debate but I think the real debate big term in terms of where the MOOCs are going and a lot of them are in Silicon Valley whether it's Sebastian Thruns or Daphne Kollers or others I think the real issue simply is where will the MOOCs go I mean will MOOCs continue to be interest-based meaning free available for those who get what they need and drop out rather quickly thus the very low passage rates where will MOOCs move into this new credit type of function where MOOCs see universities with needs they can't fulfill then move in with some cost structure some business models that allow them to fulfill that and I think what the university should be really worried about is that maybe a third category of MOOCs so not an academic MOOC not necessarily an interest-based MOOC but I would call it a career path MOOC where MOOCs really start to look at businesses and say you know that university really isn't giving you the employee that you need and so what we're really going to do is try to work with you to find the faculty to find the 12 courses to offer it online for free in some cases but of course they're getting a pay-to-feed by the corporation but I think ultimately I think what the university should be threatened about is the fact that MOOCs themselves can move into a career path that many employers are very interested in and I think all of these MOOCs have to make some decisions are they going to be professional career-based MOOCs are they going to be academic MOOCs are they going to be in some senses continue to be the interest-based MOOC I think that's what's really going to change the debate here and if I could just leave you on the last aspect of the MOOC let me just give you a statistic and going back to Ken's television the example in the 70s the census told us that in 1971% of all cab drivers in the nation had a BA degree had a bachelor's degree the recent census now tells us 28% cab drivers have a bachelor's degree I don't know what that tells anyone out there but I think it tells- there's a lot more technology in the car you see, I understand well I think Uber's getting rid of that so, you know, but I think what's ultimately happening here is that we are seeing a broadening gap between the cost of education and with the value of the degree and I think what you're going to find is that a lot of MOOCs are going to look at that gap and ask the question should we go to employers directly which I think directly affects the university's role because what does it mean then to have a college education and a degree? So let's pull on the string you just raised an idea that to be fair the $2 billion labor department grants that's a lot of money by the way that's not chump change was to design I guess using your language the career path I don't know if they would use the word MOOC that probably isn't where they were directly going but the premise being what are the skills in the economy what's being taught and what might be collectively built so that that particular need is filled in the most efficient way possible are we seeing early evidence that any of that construct is working anyone want to comment on whether that career path MOOC as I guess you've described it is happening Dean anyone else want to react to that? Well I wanted to react to Dean's comment and that was when he was speaking about that I kind of had this vision of you know MOOC meets LinkedIn right where you kind of if you're going to go down that path of professional development and how do you reach out to people who in that category it's quite an interesting thing one of the interesting dilemmas though is when we look at business models for how to do this is that really we talk about free education and Michael Saylor talked about this yesterday and he's created a foundation to help facilitate that but really nothing's really free at the end of the day somebody is funding and supporting all of this and even with the MOOCs I mean they're able to offer them free now because they've gotten they've raised a lot of money but just like open educational resources that were funded and supported what happens when that money runs out if you can't figure out a business model that makes that sustainable over time and we see that with the open educational resources that were created a decade ago through the Hewlett Foundation dollars is where is the money you know gonna be to up to keep those up to date and to revise those as they need to be revised so it's very interesting I mean when you look at career paths and how all of this fits together is that I still go back to Michael Spence's signaling theory which I think he brought into play in 1969 which is a long time ago back to your comments earlier that you know a credential still matter they signal to employers and all of that university credentials still send a signal and until we can break that chain the notion of being able to do other kinds of education that sends a signal of a different kind whether it be professional development or badges or something like that that have credibility with employers I think we're still gonna struggle now before I get to you Tammy I'm gonna just one more thread of this and then I'm gonna switch to barriers you made a presumption I wanna make sure I pull it a little bit further cost who's gonna fund it that presumes the cost is roughly the same Richard I'm not putting in the spot but Microsoft has transitioned to the cloud so the cost of delivering email on premise versus delivering email as a service online at least for my memory in the government it was like 75% cheaper so when Judith says who's gonna quote come up with the cost if coming up with the cost is 75% cheaper because it's productized versus the traditional service model does that make that conversation easier and more specifically who's building emerging business models if you're looking at them that speak to that yes it's still cost but it costs dramatically less whether it's open or it's commercialized you wanna react to that? Well one of the challenges so that's a very good point but one of the challenges you have we're trying to take a commercial business model for an email system as you just described is that there's commonality with that particular type of approach whereas if you try to bring in the many different factions that exist within any industry especially education being as broad as it is there's a huge challenge one and it's kinda odd that we're having this conversation in the Washington DC area with all the intelligence agencies we have here they still have very much a silent mentality well guess what education is the same way you rarely have the departments that are trying to build learning management systems talking to the departments that built the student information systems and so on and so you have all these different technology systems that don't talk to each other directly you have to build some middleware piece to actually connect all of them and guess what students don't like that if you look at most of the solutions that are in place today that are directed at the student whether it's K-12 or higher ed they're not designed with the student in mind they're designed with the institution in mind that creates a lot of havoc and whenever you have that type of chaos you have extended expenses and so trying to look at a business model where you know we've taken something that was on-premises and moved it into the cloud you say hey we're getting huge cost savings when we did that we were dealing with a finite set of resources that we knew people wanted countering people wanted X that was very simple to do and we controlled everything Richard just to push back though haven't we done that with the 40 plus states that have adopted Common Core? They've adopted Common Core but how well have they implemented Common Core? No, but that's the issue The theory is I guess this is a question about what you all are seeing about the future if there are standardized definitions of math would that make it easier to have the equivalent of that I'm just asking what you're seeing, Tammy Let me talk a little bit about the model here and the provocative in terms of what I see is out there right now and let me talk a little bit at the K-12 level so I think there's two matrices on this the first is what you're seeing in K-12 is a lot of freemium and what you also are seeing as a part of that freemium is what I would call a lot of engineering and not necessarily education so what you're seeing is a threshold of where you're actually being able to engineer products at a cheaper and cheaper dollar value to actually come up with a product and those are being put into the K-12 market and we're all cheering for this because I think as a portfolio we need some of those to become big organizations and big companies in order to get to your last point which is the scale piece but what happens when that happens when you have a freemium model and you have a lot of engineering going on is that the dollars that it takes is to address the issue in K-12 that Dr. Green talked about which is the infrastructure around it the rhetoric around education technology is up here right now so we've brought in the conversation to mainstream but the implementation of it is a very different thing the implementation of getting it into not the early adopters in those T-curve but the middle of the curve where we can actually really innovate at scale is where we have to focus and frankly building that infrastructure takes dough we're lucky I see Matt Greenfield from Rethink Education in here who's one of our investors and I think that we are an education technology company we're a SaaS based company but people ask all the time with our business model you guys have this entire infrastructure that you're having to build you've got 20 numerous people over 50 people across 24 states are you the technology company and what I answer all the time is if you were really going to have business models in the K-12 space that technology companies that survive you have to be willing to build that infrastructure to help them in that adoption not just the early adopters not just the innovators but to bring the rest along and that takes time that takes great engineering and education products but it also takes an infrastructure and Rethink we're going to have to recognize that if we want to empower the ed tech industry to do that to compete with the publishers to compete with those other people we're going to have to support some models and let them win and I think the policy so to your point about Common Core the policy level has to enable that to happen we have to have efficacy around it we've got to be able to demonstrate but we've got to allow some of those models to play out and I think that's what's happening right now so let me just I'm going to get to you the first question chunk was around the changing landscape and if I summarize what I've heard from you all that there's been clearly a lot of technological innovation there's been more mainstream and bottom up demand and maybe 360 degree demand but we haven't we're still any number on it but say first inning, second inning on the business model test that we're betting into so we're seeing change technologically but not necessarily fully yet on the business model opportunity which then leads me to the second theme of questions we wanted to get into which is the barriers so Dean if you wanted to pick up from that thread given what Tammy just said you know if this is an area where you require a lot more infrastructure up front someone's got to put a lot more pennies on the table if you will to get started the chicken and egg problem is there is the market conducive to buy those once those investments in infrastructure are made or is this give us a sense of the barriers that you're seeing what's the system doing more generally either holding us back or enabling give us a feedback on that sure and let me connect your question about the department of labor's money because let's let's talk about that in the barrier category I think you said are we in first base or second base I think the problem first or second inning of a 90s take your metaphor but I think the problem is we keep using the same picture and by that I mean what we find is that we have good intention government money for example two billion dollars to build open educational resources to be taught to students who need skill sets and yet you know we look at the program and ask the question how much education resources are actually being created under that it's a very vanguard wish it's a very it's a very high order to think we'd have two billion dollars to create resources that Alana and Sailor and others could grab and utilize over and over professors and others and I think if you think about that for a moment the barrier to that is the fact that we give money and Dr. Green had it perfectly right on the head we give money and people take those and put them back into their legacy systems nothing changes they grab money and they say okay I've got twenty million dollars West Hills Community College in California one of the grantees I'm supposed to train students but I'm going to put that back into our legacy structure which means students who come to class park are sitting at seat time from hours between 9 a.m. to 3 p.m. and supposedly and I think it's going to work but I'm just going to give you an example of where money meets legacy this is our system we'll offer more courses to these students and everything will be fine well the reality is at West Hills less than 2% of those students are residential they all travel to the campus they all live somewhere between 50 to 60 miles away it's a very rural institution they're all over the age of 23 many of them work so the question would have been would we have created an online solution with these dollars in those same skill sets would that have been a different experience well pause Debbie Downer has anyone seen the opposite has anyone seen the opposite for every West Hills is there an XYZ that's embraced this to build the new infrastructure that was hopefully where I was going with this is anyone seen this give me anything come on Tammy I think it's those places in between we hear of these examples in big districts again for a K-12 example that are doing it but I actually see these middle sized districts that are really innovative that frankly have gotten the short end of the stick for a long time and are now using technology are you back in K-12 or are you at community college I think in both I mean I think we operate in both K-12 and higher ed and I think that it's almost the places that kind of have recognized that they're behind that have the opportunity to jump ahead we work with a variety of schools down in the Mississippi Delta and I think that we were able to pair demand-driven dollars to move down there and what we're seeing is in some ways this ability to kind of ignore their legacy systems more because they're frankly ready to innovate in a way that they that they're being forced to because they're behind so there are pockets of it but I think there are barriers I hear you that there are pockets but I want to get Judith tell me we have resources we've got technological advancements we've got demand A B and C are all active ingredients for success have we not put the systems together to do this I mean how how is it conceivable that the application from West Hills I don't know them from Adam to me West Hills kind of mess this thing up they did a great they did a great job I'm pushing back on you they funded a legacy I just want to understand what what Judith what is the industry in this room or participating in the future architecture are they just sort of passively sitting by watching these funds flow or are they actively participating in collaboration with schools to put proposals forward to fund the future what I want to understand Dean and maybe Judith you could comment this way is the is the language written from the grant to the point where they it was forced down funding legacy or did we just did the industry in this case the ed tech industry failed to embrace this and paint the better picture to fund the future the productization you described just give me a flavor Judith is this Debbie Downer sad or is there a whoa this is a great opportunity we're doing it I think it you know it shouldn't be Debbie Downer sad I mean we're never been in a better time in higher education or probably K to 12 or a more exciting time to be involved in this industry or this sector to look at changes but what we have to do is recognize that we're almost like throwing spaghetti at the wall and we're trying to see as we try new things and we you know we support these new innovations and we try them and we fund them that some of them are going to work and some of them are not going to work and we need to figure out you know how you collect the information from those on what works and what doesn't work and how you use that the learnings from that in terms of going forward so I don't think it's I think everything that people are doing is well-intentioned I think that everyone is trying to make a change I think that's a wonderful thing in this sector to see so many people engaged in thinking of doing things new ways and actually trying new approaches and I don't think any of that can be seen as a negative kind of investment so we've identified a huge barrier being the challenge of legacy and perhaps a limitation of our measurement capacity to figure out what works for scale. Are there other barriers that we want to get on the table before we get to the question of scale? I think there is a barrier in that the way the funds are allocated so if you look at K-12 as an example the money that came from the Department of Education around longitudinal data grants as well as raised to the top it was focused more on accountability so you're basically collecting data for the sake of doing analysis. I think if that same amount of money had actually been allocated and what you do is you set up a two-tier system the first tier is about getting states to demonstrate something innovative you pick winners there you have them build it and then you pick what is best and now you build grants around that so you're actually doing something that pushes back into the actual education and learning process more so than the accountability process I think that's been one of the biggest barriers we're seeing around the money that is flowing into education right now from the federal government in that we're providing a lot of money to education and it's needed but I think the money is not being allocated properly it's the consequences of bad intentions basically I mean we're going out there doing something that we think is right but unfortunately because we're putting so much attention on accountability we're losing people we're losing students because we're really not teaching them necessarily the skills they need getting back to an earlier point you raised we're really trying to figure out where they are and then also somewhat teaching to the test I think if we're taking that same amount of money I mean we're talking in the area of twenty plus billion dollars overall that we've been spending here over the last seven years or so if we're taking that money been a little smarter about how we used it I think we could have gotten much better results Dean or Tim any reaction final comments on barriers should we get to the question scale so let's all right well let's just let me let me summarize this piece to say lots of resources lots of good intentions lots of opportunity but the systems themselves are not yet embracing what's necessary to demonstrate the change or at least even to test a number of hypotheses if that may be a bit too negative I'm a more of an optimistic guy I think there's a lot more there but let's just pause there so what is it going to take to identify if we did this all over again and the good news is there's still resources not yet deployed and I'm just thinking about federal resources there's still non-profit and others but the question is what is it that we are doing what is happening right now that gives us a sense that we're creating the mechanisms for testing new ideas measuring what works and scaling give us any sense for a storyline or a threat as to what's giving you hope hey here's a pocket it's being well researched it's what's going on and based on the evidence we think there's a path to scale anybody wanted to give tell us a story as to how this does scale given what we've just talked about well I'll take the first shot and I think social media being introduced both in K-12 and higher ed as a learning tool I think is showing a lot of promise because what we're doing there we're seeing that students are actually helping one another learn and you learn better from your peer group we know that as kids you learn better from your siblings than you do from your own parents and we're seeing that in the education process as well and then also providing the teacher or professor insight as to what is going on in the classroom through the social media tools is a fantastic job I mean I worked with some folks down at the University of Central Florida where what they noticed was they were losing a lot of the connectivity between the professor and the students because a lot of the students were using social media tools that the school didn't have access to to actually collaborate with one another so we worked with them on a project where we moved a lot of that interconnectivity into the cloud that touch all of the different social media tools the students were using and now the professor could actually get in so basically we were creating virtualized classrooms within a particular segment of the calculus one section and now professors could actually monitor what was going on between the different clusters in the classroom provide input and the students themselves were actually working with one another it became much more of a collegial environment and the students got a lot of value out of it so there's an example of where technology actually was not a barrier it was actually an inducement for the students to collaborate with one another given their busy schedules given the disperse way that a lot of our college students are actually interfacing with one another and that no longer are we in a traditional classroom methodology where everyone is sitting in the room at the same time people have different work schedules people have all sorts of other activities outside of the classroom so that is one example of where technology is actually playing a very positive role in the situation and it's not costing the school a lot of money I'm pushing on this question of scale because that feels like in the first trunch of questions an example of an innovative way of collaboration sounds like it's cute but I'm not so sure it scales I want to get a flavor for what are the mechanisms for scale does policy have to play a role what are you seeing out there as we project out over the next several years do you want to go first? I think one of the really exciting areas now in higher education is outcomes based or competency based education first time we're talking about the financial model competency based education and the support now that is available especially through the dear colleague letter that came out recently for institutions to actually embrace competency based education and I think that that will have a very interesting effect on higher ed we're seeing in our organization a lot of institutions that are interested in going to competency based education but they don't really know how to get there and we've built a what I call the next generation learning management system I call it a virtual learning environment because it's really a different paradigm of a virtual learning environment that actually embraces competency based education and helps do adaptive and personalized education and so I think that the combination of the technology to enable that and the policy to support competency based education will help us create models for higher education that can scale that's what I was hoping for any reaction to you Dean and Tammy on that conversation? Let me just add one thing I mean I think there are two elements that are helping in terms of being able to do a lot of efficacy work around education technology the first is time and the second is the ability to have institutions whether higher ed institutions or K-12 districts actually divides implementation of this at scale because I think one of the things that we're finding in terms of doing efficacy in our higher ed learning platforms and K-12 and doing that type of control studies and things like that is that we've had to get the implementation right first and it's had to have been big enough that we can go back in a year three and a year four and a year ten and actually do that type of competency and assessment and so I think that's an attitude change right which is that people are willing to dabble in doing small pilots and small you know in a portfolio effect or I'll do a couple of pilots around all of this with just a few a few students and in order to really prove that this is working at scale you have to have some folks that are willing to take a risk I can give you two very brief examples so in Houston ISD which is a big public district down in Houston, Texas that we've been working with our financial literacy learning platform we are finally at that scale with them where we have enough students going through that we can actually look at you know providing different type of assessment models and really get some data back and I think that that has been a testament to their willingness to actually implement at scale and I think that's going to play a huge role in the policy side too to take those labor dollars or those DOE dollars and say how do we do this at scale and not just pilot around the country when I get to you we're going to have about five minutes or maybe ten max on questions from all of you so maybe you want to make a comment and we can turn it over for sure I think I'm sitting here pondering what scale means so there's system scale which let's say University of California system there's statewide scale and then there's national scale so I think from my view of this I'll use a small case study so we had some foundations Hewlett ourselves Gates Foundation fund 50 open source or five open source textbooks we built those for three are out two are coming I think Donny's talking about this a little later from Rice University but the point is we had three open source textbooks we went to Governor Brown and the governor saw the merits in this signed a bill that said California should create 50 open source textbooks for its undergraduates because we know 80% of those students are taking 25 to 50 lower division courses we know the outcomes of those particular courses what stats is stats you have some outcomes so they're well underway after that British Columbia sure funding that if you get to the question of legacy infrastructure so high-end cost in this is a very interesting way California did this so now California said we will put $5 million in to be matched by philanthropy and foundations and others to match the additional $5 million the goal there is to build in 25 to 50 open source textbooks with a matched type of system so everybody's in if you will but I think what's interesting about scale is and then you found British Columbia deciding to do this you found other states now introducing bills this year like what legislator doesn't want to introduce a bill to lower the cost of textbooks for undergraduates so you find now states beginning to talk about let's all build 25 to 50 books the bigger question from a federal government perspective I think is really why should every state build the same books you know why wouldn't we have some if you will some guidance from our federal government to say look you know everyone shouldn't build 50 books if every state built 10 or 5 you know maybe this is a way to build the true repository just pause before you finish that statement to clarify we're in Virginia yes Virginia has as its law right that guidance to the schools that if there's funding from the government the resources are creative commons licensed right in the $2 billion labor grants yeah similarly a requirement for creative commons licensure so just want to be clear that would not mean you'd start from scratch in each book you'd basically remix and reuse right exactly right exactly I mean I think so that is policy but I want to make sure I want to make the point is how do we scale that in a way that everybody is talking to each other and sharing and I'll give you an example I think if the federal government were to put out a you know let's just use a number of $500 million allocation to get this coordination going to allow states to not necessarily look to philanthropy and others but to be matched to build these books it seems and to have these constant updates make them better it seems that that's the kind of scale that really is would be far reaching would have positive impacts it builds on technology because obviously you mentioned CK 12 and flex books and others I mean these are the kinds of things I think that offer tremendous amounts of potential for scale but the hard part is coordinating I think everyone got together in Canada just a week or go and talked about how everybody could coordinate but I think the hard part for us is in this particular movement is how do we have that type of coordination and guidance I think that would be a real positive good question anyone from the audience want to bring forward a question for our panel before we wrap to get you back to campus if you will please stand up and tell us your name Jason might be partly based on the way you started the conversation and each kind of like what's the government's role but the question I have for you guys is it has ended up being a very much how can government drive open resources how can government drive innovation when in fact you know taking the MOOCs and maybe their hype but that there was no government grant involved in creating the MOOCs and all of that there was no government grant created and starting Blackboard which became a billion dollar company and doesn't necessarily have the best learning management system in the world but you know it was a very successful company lots of the innovations in every other industry have come out of private sector venture backed or even just bootstrap backed companies how important should the government role be here or should the government back off in some way and let the innovators do it themselves it's a phenomenal question would love to get any reaction but just to clarify and make sure we we're on the same wavelength the question about government role was on the question of scale and whether policies today an inhibitor or a barrier or an enabler just to clarify a lot of the experimentation in testing you're describing is all clearly funded from the private sector please my perspective is whether the question is about scale or not I think the policy environment has the ability to open up and we started this conversation about what has changed in the market and the answer from all of us was the number of people at the table right and that we wanted this to be demand driven and if this is going to be demand driven education that fills the pipeline of jobs my sense is that we have to look at the education industry with the private sector playing a huge role in it and that the policy opportunity is that collaboration is understanding what's working is allowing it so to your business model question and my answer that those can scale to actually serve more students and the barrier at least in K-12 to scaling is this very franchise model of K-12 where a business has to go out to 15,000 school districts to find scale to students and so I think about you writing five textbooks and I say where does the market come in where does the demand driven market come in that says there's lots of content providers and how do we get the best in our districts how do we get the best in the university so I think the private sector has a huge role to play in this I think it's not mutually exclusive I think there's a role for both some innovations have come out of our institutions either public or private and some of them have come through venture backed and some of them are government supported but I think at the end of the day in higher education if you're in a accredited institution I mean one of the barriers to getting some of this stuff out there is the accrediting system and the process and being able to get innovative ideas through your creditors so I think there is a role there to remove government can play a role whether it's working probably at a federal level working with the creditors across the board especially the regional creditors in terms of how to become more adaptive to the innovative models that are there we have a grant from the next generation learning challenge through the Bill and Melinda Gates Foundation working on some innovative things and they have some other grantees who have come up with these great ideas and innovations that have been supported that they can't really launch because the accreditor won't let them so I think there's where a role for the government to that was the question but keep going on this thread is there a pure libertarian view of the world that suggests we could completely disrupt this market without any role I'd love to get that just please engage Dean yeah well I think that's a very interesting way to put that but let's start with some real case studies here if anyone here in this room thinks that MOOCs don't want access to the credit raise your hand I mean they're all private sector they're all private sector driven at the beginning but they will all run to government to be protected within some fortress and whether it's MOOCs whether it's the publishing industry with textbooks let's just face it the reason that the open textbook movement is flourishing today is textbooks have increased three times in inflation students are being held to a standard that professors pick for them professors pick books students have to buy it's kind of like the doctor in the prescription and government will find itself stepping into market failures if you will in this case I think to scale up on open source textbooks governments have a rightful role to build competition where there is none to in many cases kind of a monopoly of just a few publishers in the case of MOOCs I think what you'll find is that MOOCs will very much be interested in trying to figure out how to be an outlet for students when they cannot move through to a path to completion they will offer lower cost courses if you will to those students answering the question of course affordability you're going to find that when institutions themselves fail not just markets they'll find the private sector moving in but at some point those protections will be sought for sought by those particular entities so I think there's a dual interest in each government is never innovative enough sometimes when you were with Obama but I think what happens is this interplay of let us flourish then support us then protect us is always going to be a cycle that I think all of us should be aware of and I think there is always going to be that dynamic other questions before we wrap up I can maybe see one in the back yes sir tell us your name Hi Anisha my name is Paul Lagan oh what's up Paul how are you I wonder if we have a speed problem if you look at the speed of innovation the speed of technology it's very very fast if you look at the speed of policy making the speed of accreditation the speed of proving if something really works in the classroom that's very slow so do the technologists and innovators slow it on or do the rest of the stakeholders get faster can I just jump in I think that's really the answer by our first opening speaker what you've just mentioned is risk so if you are a policy maker if you're a politician if you are someone who's going to be making a governor you want to minimize as much risk as possible in that speed of innovation speed is not your friend unless it can be proven in many cases by mitigating risk now risk also in many cases to some policy makers could be a good thing to be the first out to capitalize on the idea whose future might come there are policy makers like senator steinberg in california that might challenge and say maybe our institution should let third party providers provide an outlet and give the credit big bold moves sometimes do occur on the government political side cc by licenses in virginia I mean these were early early types of actions placing a big bet on where I think this space may go there may be some frustration because I think they want to in essence get ingrained into if you will a policy making protection but they're also wary I always watched when I was past senate majority leader in california innovators come to talk to me about this great idea and then get really worried as I started to tell them I think I want to do a bill on this and they would go oh my gosh you know we came here for an aspirin not brain surgery I can't tell you what we're interested in but policy makers are interested in capturing ideas and trying to formulate them so those very fast advances sometimes could be in essence killers in some cases so I think slowing down even on the innovation side improving those pilots has been mentioned is extremely important so it's kind of between both anyone want to respond to this last question and then we can wrap up so we can get back to the piece judith any final questions on response to this pace of change this question of where is the role the private sector to drive where is policy any final remarks I think the only thing I would say is that I don't have an answer to should technology slow down or policy speed up because I don't know if either is possible but I do think that if you frame it in the context of when you're trying to make decisions in this area in education whether it's K to 12 or higher if you frame it in the context of what's in the best interest of our learners it might actually help us either slow down a little or speed up a little I think what will be really interesting to the future for me is if we can come up and the MOOCs maybe the model it may be something else but if you can deliver a $50 MOOC to a student that actually can get academic credit and then a credential for that I mean that would be a major breakthrough I'm going to take the chairs prerogative to just summarize what I think may be a way to digest a number of these questions and the reactions just on the fly if I may so let me see if I can wrap the following manner first I want to just dispel the notion that it's an either or proposition what we're hearing today about the challenges and the barriers and the need for some type of collaboration public and private it's not engineered as a one size fits all the beauty of today and the moment we're in is that there's a lot of experimentation whether it be West Hills for good or ill or others there are institutions that are engaged in trying new ideas and they're going to demonstrate I hope some efficacy that we'll see will hopefully lead to some scale but let me just describe for you what's happening right now as a way to close this piece out we started the conversation about workforce that I think your thread Dean of the career path business model innovation maybe the one just to use as the closing the tech industry has said mathematically we have high demand for software testing we heard this I heard this in the White House a lot we launched an initiative last summer summer camp spelled QAMP for quality assurance to get kids basically underprivileged kids internships and exposure where they could potentially see a career path that has a high earning potential so in the spirit of understanding the credentials that are needed to be a quality assurance technician we saw that the association for software testing had open sourced its assessment framework and its credentialing documents we saw last summer and going into this year the production of an open source textbook again at CK12 that allows folks to access that information not yet but possible is the publication or the development of an online course or a hybrid course that could provide that textbook life and to educate kids to see what's possible for them to enter the software testing arena given what Judith said one could design that course with competency based models at heart so that the actual price of that course could be reimbursed through financial aid which is what you were describing earlier with the dear colleague letter and we could have theoretically a model for scale because the software industry association for software testing has said folks that have earned these credentials if they're badged I think Richard someone mentioned the badges movement could conceivably award industry credentials associated with a paid for competency based model course built on open education resources in an area where there is high demand and a high lucrative job as one little microcosm to test and it seems to me from the conversation we're having there's a brood of everybody in there the entrepreneur who's building the for-profit course material built off of some open education resources with their secret sauce maybe the big data analytics folks saying what works for kids and what doesn't maybe it's got some federal funds because it's part of a labor grant either in all of these scenarios are possible right now without a change in law without any real significant change in budget that's the possibility so I think if we've tried to stitch these pieces together my conclusion from all of this is we're actually in a good place technological advances are strong there's a desire for reimbursement models and business models so that these things can scale and with Dear Colleague Letter and the competency based there's at least a possibility so I leave this conversation hopeful I hope you enjoyed this discussion thank you so much and let's move on to the next thing who's going to take us to the next thing you will alright thank you panel