 My name is Andres Martinez. I'm the editorial director here at New America, and I direct our Future Tense initiative here. Future Tense, as many of you probably know, is a partnership between the New America Foundation, Arizona State University, and Slate Magazine. We do a range of events on different topics involving emerging technologies and their impact on society and policy making. Our events range from movie nights, happy hour, salon dinners, day-long conferences. You can follow us on Twitter at at Future Tense now, and we have a hashtag for this event, which is Hack Higher Ed altogether, and you can also read articles and blogs on Slate, the Future Tense, by going to Slate.com slash Future Tense. A housekeeping reminder that today's event is being webcast, so in the comments and question periods, please wait for a microphone and identify yourself, and everything is obviously on the record. I'm very excited about today's talent that we've been able to coax to come speak with us, speak to us, and also of course the subject could not be timelier. I was also thinking on the on my way in this morning as I was fairing the elements that it had Future Tense been around long enough. We might have had an event once upon a time on whether, on the impact of the Blackboard on education, and how this technology was going to revolutionize education, and we probably have heard from voices that would have postulated that this, that the Blackboard changes everything, and now we're you know it's going to solve every problem in education and isn't this exciting, and we might have had you know a few voices saying wait a minute this is a bit of a gimmick, it's going to distract us from the basics, and by the way it's going to enrich some private companies, you know we need to be a little bit more skeptical, and that you know that's just kind of a reminder of the cyclicality of some of these debates that we go through in terms of every time we have new technologies to get excited about, and clearly this is a moment for that, and whether your relationship with technology and your worldview tends to veer towards the more optimistic side of things, or whether you are a little bit more on the skeptical end of the spectrum. I think we can all agree that there's cause for some unease about the present higher ed model. I think as a society we've never been more reliant on the importance of education, and our expectations of it have never been greater. Our need for an educated workforce and citizenry has never been greater, but some you know there's a sense that the model has gone awry at some level, certainly when it comes to accessibility and affordability, and you know to what extent can technology help improve the model. To what extent do we need to hack higher ed? That is why we're here today to explore these questions, and I can think of no better person to sort of frame the remainder of the day, and set the table than the director of our education program here at New America, Kevin Kerry, who's going to get us started. Thank you. Thanks Andres, and good morning to all of you, and thank you for braving the spring rain to come be with us this morning. This is an event sponsored by Future Tents, and as New America we certainly like to face forward in our discussions and thinking about these issues, but I'd like to just start this morning by taking a little time, not too much, to offer a few historical perspectives on what we're talking today, and perhaps also offer a few clarifying terms that we can use as part of our conversation. And I'd ask you to start by thinking about what, in your mind, perfect education looks like. For a lot of people we have, I think, consciously or otherwise, a vision of perfect education, and it's two people sitting together, a master and a student, talking to one another, and this is the classic Aristotelian tutorial model, when the king of Macedon wanted to educate his not-yet-great son Alexander, he simply hired the smartest person in the world to come and educate him. And the first information technology to really alter that model in any substantial way was the written word. And like all ed tech innovations, it was controversial at the time. Socrates, in fact, was deeply skeptical of writing. And here's what he said about it, in the Phaedrus, which he imagined as a dialogue between two gods. Socrates believed that written words will create forgetfulness in learner's souls because they will not use their memories. They will trust to the external written characters and not remember of themselves. Words he believed are an aid to, not to memory, but to reminiscence, that give people not truth, but only a semblance of truth. They will be hearers of many things and will have learned nothing. They will appear to be omniscient and will generally know nothing. They will be tiresome company, having the show of wisdom without the reality. And as was usually the case, Socrates was kind of on to something. There, he was a smart man. Tiresome company, having the show of wisdom without the reality, that sounds like most of the book-read, college-educated pedants that you run into from time to time in your life. Compared to real-life people, words are static, standardized, and unresponsive. Or to use a modern education policy insults, written words, or a one-size-fits-all solution. But in focusing on what was lost in translation with technology, Socrates failed to fully appreciate what might be gained by it. And this turns out to be a pretty consistent mistake that people have made over time. Writing extended the distance words could travel from within earshot of a speaker to anywhere that pieces of paper could be carried. Writing became a storage medium, making words like Socrates potentially immortal. Writing also brought the educational value of words to more people and thus reduced the cost of words for each of them. Only the king of Macedon could hire the smartest man in the world as a tutor because he was expensive and there was only one of him. So the distance, the scale, and the cost of education were changed for the better by that particular technology. And writing did one more thing. It actually improved the quality of education. We tend to think again of those two people, master and student, talking as the ideal educational environment. But it isn't always through revision, collaboration and extended work over time. Writing allowed people to create and communicate structures of thought that weren't possible in oral conversation. Writing made education better than it was arguably the last technology to do so until now. As time passed, every new information technology was adapted to education, never without controversy. The printing press, for example, several thousand years later, greatly accelerated these benefits of distance, scale, and cost. More people in more places for less money. And this actually greatly alarmed the faculty at the University of Paris, which was the second major university to emerge in the pre-Renaissance era after the University of Bologna. At that time, it was expensive to make copies of words before the printing press. Paper was not cheap. And so to save cost on paper, scribes would bunch all the letters of the words together without spaces in between them. And learning to read such a text was a skill unto itself and students would practice that by reading out loud under the supervision of a teacher. So printed books with these new spaces between words technology allowed students to read silently, which closed a window under the student's thought process. So the University of, like Socrates before then, the University of Paris professors, they did have a point in being upset about this. The central challenge of education, of teaching, is not imparting information to people. That's the easy part. That's what I'm doing now. The hard part about education is making learning visible in each person because every person is different. And so the way that they manage and contend with your ideas differs depending on who they are and what they brought to that conversation. So printing technology actually made it harder for the teachers to see what was going on in their student's mind, because now they were reading silently if they were reading at all. Who could tell? But those professors, too, failed to appreciate the compensatory advantages of more abundant books. Fast forward a few more hundred years, we come to the Postal Service, which was an open, egalitarian, publicly financed communication network. It was the Internet of its time. And this, the Postal Service, more or less solved forever the problem of moving words over long distance. In 1639, the very same year that America's first college was named after its benefactor, John Harvard, a nearby tavern in Boston was designated as the official repository for mail. Coming in and out of the British colonies. And so colonial readers of the Boston Gazette would read advertisements claiming that, quote, persons in the country, the rural areas, desirous to learn the art of shorthand may, by having the several lessons sent weekly to them, be as perfectly instructed as those that live in Boston. Now this claim of as perfectly instructed might seem suspect on its face, but then again, different subjects require very different methods of educational interaction, and shorthand is unusually text dependent. And in fact, there is to this day an extensive research literature demonstrating how students learn just as well at a distance as in person. Over time, the advances of information technology would increase the kinds of educational interactions that could be offered to more people in more places for less money. Radio brought spoken words, television and film brought moving pictures, and now the internet has essentially collapsed all of the remaining barriers of space, time, and cost to the point of nothingness. Any words, any sounds, any pictures to anyone, anywhere, instantaneously, at a marginal cost that is indistinguishable from zero, its breathtaking. Most of the online courses that currently exist, including the MOOCs that have received so much attention over the last couple of years, largely take advantage simply of the recent developments that I've described, broadcasting educational words, sounds and images at a scale and a cost without precedent. And I have to say we should not diminish that accomplishment in any way. It represents an amazing step forward in and of itself. Yet what most of those courses haven't managed to do yet is use information technology to improve the quality of education, something that hasn't been done arguably since the invention of the written word. But I believe they will and for two reasons. First, because modern information technology allows for unprecedented interactivity and interpersonal communication at scale, the formation of communities of learners of a global size. Second, because the ability of computers to process information means that we can for the first time start to replicate and improve upon fundamental processes of learning. Let's go back again to our supposedly ideal model, two people sitting together talking. It really is not as perfect as it seems, even the smartest person in the world knows far less than he doesn't know. And the implicit process of diagnosis in the tutorial model in which a teacher listens to the student, make judgments about what they've learned and responds accordingly is bound by essentially human limitations. It's very difficult to see inside another person's mind to provoke and inspire their learning in just the right way. We value people who are great at this precisely because they are always so rare. And it's no coincidence that all of the people now leading the major MOOC providers, Sebastian Thrun and Peter Norvig at Udacity, Andrew Ng at Coursera and not Agarwal at edX, come from or near the academic disciplines of artificial intelligence and machine learning. Or it's no coincidence that some of the best designed existing online courses whose effectiveness has been proven through rigorous research come from institutions like Carnegie Mellon and Stanford. Based on decades of research into cognitive psychology, neuroscience, human-computer interaction and AI. The amount of data that will be generated by millions of people engaged in digital learning environments will yield insights that no single educator, no matter how good he or she may be, can obtain. The AI-based cognitive tutors that are already being used now in online courses will only get better over time. This is not science fiction. It is the unevenly distributed future rapidly approaching the universal present. Now, traditional colleges and universities predictably really inevitably are following in the footsteps of their predecessors by badly underestimating the net educational benefits of new education technology. These are organizations that were essentially designed in the 19th century under conditions of information scarcity that simply do not exist today. And they are in a profound state of denial about all of this. To start, they grossly underestimate how much of the education they currently provide is already wholly replaceable by a simple broadcast model. Every aspect of the standard lower division college course, lecture course, which happens to be, those courses happens to be hugely profitable for colleges, every aspect of those course can now be perfectly replicated online today and distributed at no marginal cost. It's basically a trivial problem to solve. It's already been done. Any of you can, for example, take MIT's entire mandatory undergraduate science and math curriculum exactly as the students in Cambridge take it on edX for free today. Now, if you can get colleges to admit this, which is hard, they will then sort of fall back on assertions that are unsurprisingly rooted in the intangible, the ineffable, the unprovable, and the, you just kind of have to be here to understand. And I would just say a few things to keep in mind about that. First, whatever the benefits of things like being on the campus and interpersonal interaction with professors may be, and to be clear that those are real benefits that people have. Colleges have absolutely no evidence that would meet their own standards of scholarship, credibly estimating or quantifying the size of those benefits. None. If you don't believe me, try asking them sometime. Second, those remaining benefits have to be understood in relation to cost. As we all know, the cost of college is rising at an alarming rate. Even if colleges can prove how much value they do add on top of what information technology can now provide for no marginal cost, that still leaves open the question of whether that value is worth the increasingly large prices that they're charging. Third, artificial intelligence and machine learning will inevitably continue to improve and steadily eat away at whatever plausible remaining value proposition traditional colleges may currently possess. All of which means that we're in an exciting time to be discussing technology in the future of higher education, the most exciting time since antiquity perhaps. We have a great group of speakers and panelists assembled here today. And to close, I would encourage all of you to be clear about what dimensions of education you're talking about with respect to technology. Is it scale? Is it distance? Is it cost or is it quality? And to remember that we have in a very long and historical sense been here before. Thank you. Thank you, Kevin. That was a very authoritative overview and it makes me think that maybe our hashtag should have been Socrates was wrong. So next, we're going to drill down a little bit more on MOOCs and we're going to have Robert Wright talk about his first MOOC, which you can also read about on Slate. Bob is the author of Non-Zero and The Evolution of God among many books. And he's a senior future tense fellow here at the New America Foundation and a visiting lecturer at Princeton University. He's going to give a short talk and then he's segway into our first conversation, which he will moderate. Bob? Thanks, Andres. And thanks, Kevin, for that cosmic overview. It makes me feel I don't have anything very momentous to say by comparison. I'm not going to go all the way back to Socrates. So, yeah, I just taught the only MOOC I've ever taught. I'm just finishing it up right now. And I've been asked to say a few things about what I've learned. It's based on a seminar I've taught at Princeton for a couple of years. And the title of the MOOC version on the Coursera platform is Buddhism and Modern Psychology. And you can probably tell from the title of that that this is not kind of the type of MOOC course that gets talked about the most. I mean, what we've heard most about are courses with vocational value and hence economic implications. The idea is computer science accounting. You imagine some kid in Africa or something taking the course, plugging into the global workforce. Tom Friedman writes a column about him. And the rest is history. I don't think a lot of the students in my course aspire to be Buddhist monks and plan to use this as a credential at the monastery or anything. But in that sense, it's like a lot of humanities courses, right? It's to some extent a different kettle of fish and poses unique kind of pedagogical challenges. But I do think I learned some things that have relevance to the question of the viability of MOOCs, which has of course been called very much into question lately. And in fact, I would say the kind of MOOC bubble officially burst right about the time I was starting to teach the course. I was in December, I was preparing the lectures. I started taping them in January and it was in December that Penn released this study showing rigorously what insiders already knew, which is that the completion rates are incredibly low for these courses. I think the number in the study was 4% or something. The attrition rates are very high. And there was a headline in the Washington Post right about the time I was starting to tape my lectures. Our MOOCs already over. So I have concluded two things. One, the attrition rates really are high. I can attest to that now. The other thing I've concluded is that it just doesn't matter. If the question is about the viability of MOOCs, I don't think the attrition rate matters at all because I don't think it fundamentally affects the two key variables for the future, which are the supply and demand questions. Are there going to be a lot of students who want to take MOOCs and are there going to be professors and others who want to create a supply of MOOCs? Are the professors who are turned out to be best at teaching them going to be motivated to teach them? And I have concluded that the attrition rate does not affect either of these variables. So let me talk about that a little just from the perspective of a potential supplier. As somebody who might decide to, again, go to the trouble, and it's a lot of trouble, it's a lot of work, to teach a MOOC. And how I'm feeling about that prospect after witnessing a very steep falloff in participation, which everyone pretty much who teaches MOOCs experiences. And I don't deny that it was, it took a psychological toll for a while, right? Because you know, you see the numbers. And the first thing you notice is, wait a second, half of the students didn't show up. That's about the average. You know, I had, at the time I started, it was a little over 50,000 enrolled. The enrollment has gone up a little since then. But you know, you notice about half of them don't show up. And then you look at the viewership, you know, for the lectures. And each lecture is three or four segments. You have a bar graph for each one, and you, and it's down, down, down, down, down, down, down. It's all down. And I was really kind of bothered by this. But then one thing I realized is, the curve has to go down, right? I mean, this is sequentially presented information. People show up for the first lecture. Everyone starts at the first lecture, and then decides whether to move on to the second, or the second segment of the first lecture. So it's not like hosting the Tonight Show, where in theory, your audience can grow forever, because you don't have to have watched Monday show to watch Tuesday show, right? So the best case for a MOOC is horizontal. And in the real world, there's going to be attrition, right? There's going to be fall off. So the question is, how steep is so steep that you should, you know, you should just sink into a, you know, a state of depression and never want to teach a MOOC again. And first of all, the fact that only a little more than 50% of the people who enroll show up, when you think about it isn't surprising. What is enrollment for a MOOC? It means you clicked on something that says, sure, send me an email when the course starts. What the heck? That's what enrollment is. And people enroll in a lot of courses. And naturally, yeah, I've done this myself, you know. In fact, I'm worse than, my average is worse than 50%, actually showing up for the first lecture of courses I've, quote, enrolled in. So that part shouldn't surprise you. What about the, you know, how steep should the subsequent curve be before you get really alarmed? Well, you should expect it to fall off, you know, I think quite a bit. I mean, none of the incentives that keep people going to college, to courses at real colleges apply. At a real college, they've, you know, paid a lot of money. They have to finish a set number of courses to graduate. And if they're five weeks into your course and they bail out, that means they have to go back and do another five weeks of work. None of these incentives apply with a MOOC. With some courses, you have a strong vocational incentive, or they really want to master a skill, that's something. In my case, you didn't even have that. Okay. So, in theory, shouldn't, shouldn't bother you. The, the, the, the slope of the curve per se should not bother you that much. Again, it still did. And I realized the absurdity of caring about the curve when, you know, when I was looking at it, hoping it would, like, get flatter and flatter. And it was, it was getting less steep. But then I realized one reason it wasn't getting less steep faster is because although, yes, the viewership for the more recent lectures was rising, but the viewership for the earlier lectures was also rising, that was preventing the curve from getting, you know, flatter faster. And I found myself wishing that the, that people would quit showing up for the earlier lectures so that my slope would be, like, really a gratifying slope that I could brag about, right? And then I thought, no, that's crazy. You know, I should want as many people to enter the course as possible. And even if they only go to one lecture, that's good. You know, they, they, you know, they become a little acquainted with my work, a little with me. Who knows maybe the reason they didn't show up for the second is they went to Amazon, bought one of my books, got engrossed in that. I doubt it. But it does, that, but the subject books remind you that, you know, one statistic I'm glad I've never seen, you know, a book is sequentially presented content. I've never seen the, the, the chapter by chapter bar graph. But I'm, I'm sure I would be alarmed if I had that data. Thankfully, I don't. So, you know, in the end, I got over all of this, you know, I worked, worked through this psychologically. The course is wrapping up. And I'm realizing that in the end, you go through all this, what you care about are the absolute numbers. So it looks like, I mean, the numbers are still going up and but, but it looks like I can safely say more than 10,000 people will have gone through all the lectures. They will not have done all the assignments. Mine, you know, one of the challenges of humanity's course is like, is multiple choices, that really what you do with a human? So we're doing peer reviewed essays. And that means that if you're going to agree to do the midterm essay, you're going to assess five others so that everybody gets five assessments. That's a lot of work. There's no motivation whatsoever. In fact, they don't even get, not only they not get college credit, they don't get a certificate of completion, because that varies depending on the university, the policy of the university where the MOOC originates. My students don't even get that. Still, we had more than 2000 do the midterm. To me, that's amazing. There's just, I mean, the idea of 2000 college students doing an assignment because they want to do the assignment. I don't, you know, most professors go through their entire careers without encountering 2000 students who would just do the assignment for the gratification, the intrinsic gratification doing the assignment. So I think as for the way professors wind up feeling about a MOOC, it's all about absolute numbers. You know, how many people encountered how much content in the end. And I personally don't feel that strongly about doing the assignments. I mainly want them to encounter the material as I prepared it. Now as for, you know, there are intangibles aside from raw numbers, you know, and those tend to be good. It's like at the beginning of the course, all these students from all these countries weigh in. Coursera has a little map where they can chart themselves and you see like this explosion of dots on this map. They're from all over the world. It feels great. And then at the end, there are, you know, you get people from various parts of the world. You know, just yesterday on, I set up a Facebook page, which is something I kind of recommend. It's a form of interaction with a professor that's a little more direct in the discussion form. I did some other things, and I don't know that I'll have time to talk about them. But a sense of interaction with a professor is important. Anyway, just yesterday on the Facebook page, a student from Helsinki says, I just want to, you know, thank everybody who's been in the Sangha. Sangha is the Buddhist term for the community of Buddhists. And this is a big part of it for a lot of students is the interaction with the other students online. And then somebody else on Facebook said, I think we should do a, this is somebody from Vienna, you know, after the person from Helsinki weighed in saying, I think we should do a simultaneous meditation, global meditation session. And the person from Helsinki said, count me in. I don't know if this is going to happen. I'm not actually don't consider myself the type to lead a global meditation session. But it's the kind of intangible that keeps, that I think will keep professors coming back. There is a real sense of engagement as for what keeps students coming back. Again, it's a lot of things. And it includes a sense of interaction with the other students and with the professor. So like, I did a weekly office hours thing. The lectures were pretty much pre taped. But every week I would read the discussion form or to the extent that I could keep up with it, the Facebook comments, then I would tape a thing in my house where I respond to these. And I think that means a lot to the students, the sense that that aside from the intrinsic interest in interacting with the students in the forum, the idea that the professor is watching and responding as best he or she can, I think is important to them. But in any event, I don't think they're not put off by, if you look at their end of the high attrition rate, it's not a downside for them. And, you know, I mean, if they sign up for three, wind up showing up for one or sign up for 10, show up for three, stick with it for one, whatever, that's no big cost from their point of view. And in fact, just the final thing I'll say is that one thing that occurred to me while listening to Kevin's introduction is, you know, as I said, one of the challenges with the humanities course especially is individual interaction. This is something that in general MOOCs, where MOOCs cannot provide the substitute for the, you know, Aristotle, Alexander, the great models to two people sitting together and interacting, you know, normally in a course you have these students and you sense which ones don't understand what, like in a seminar, and you try to address the lack of understanding. Well, in a way, a high attrition rate is a substitute for that. Because what's happening is students, there are students who show up for your first lecture, they don't get it. And you can't, I can't do anything about that at that point. I cannot respond on an individual basis. What they can do is go find somebody teaching a course on Buddhism that's just for whatever reason, more on their wavelength, presents things more in a way that they can assimilate, or it's more the part of Buddhism they're interested in, or whatever. So I think, in a way, a high attrition rate, it's just inner, it's a form of interaction that's allowing students to find the course that works for them. And if the attrition rate is so high that you wind up with zero students, that's a serious problem. But I do think in the end, for professors, the question is how many students do they wind up reaching? And a lot of other factors will go into the viability question. I don't know whether MOOCs are going to work. I'm not particularly predicting that they will. I just don't know. But I don't think the attrition rate is anything to worry about. And I think that's all I have to say. Now I'm actually going to be moderating a panel that will look into some of this stuff a little more deeply. Okay, so we have a great panel here to continue the discussion. We're going to talk for a little while, and then we're going to open it up to you for questions. Starting at the other end, Jeff Solingo is currently professor at Arizona State University. Am I okay sound wise? And also as the author, we're a contributing editor at the Chronicle for higher education and the author of a book called College Unbound, which has the subtitled the future of higher education and what it means for students. Robin Goldberg is chief marketing officer of Minerva, which is a very interesting business model that we're going to talk about. Previously, senior vice president in a couple of places, including Lonely Planet, the travel guide series. And finally, Adrian Sainier is the chief academic technology officer at Arizona State University. Certainly relevant to this question. And you're also a professor of practice. Both of your professors are professors of practice, and that means what? You're good at, you practice a lot. What is a professor of practice? You eventually get to be a real professor. Really? After you practice? For a really long time. Well, listen, let us know how that works out. When you're a real professor, come back and we'll have you on a panel again. You hear the term unbundling a lot these days in general. So my main vocation has been writing or had been back when that was a viable business model. And the reason it's increasingly challenged as a business model is because of this thing called unbundling, partly at least. And what that refers to is, you know, the institutions that used to provide certain services are no longer required to provide the services. So the institutions I used to work for were things called magazines. Some of them still exist. But you know, magazines, broadcast networks, newspapers, more and more because of technology, you don't necessarily need that institution as an intermediary to receive the service of news or commentary or whatever. You can just go to somebody's blog, you can watch video of some uprising in Ukraine on YouTube, whatever. So that is unbundling. And colleges are said to face the unbundling threat as well. They traditionally have been the institutions that were required for the provision of this service called higher education. No one had a better way to do it. And now there are all of these other ways to get the service provided for you. And I think it's safe to say that this term unbundling explains why Jeff titled his book College Unbound. So I wanted to start out, Jeff, asking you to just explain a little bit about what's driving that, maybe a little bit about the variety of things that are driving it, even beyond, you know, MOOCs as we know them. Well, in some ways, to go back to Kevin's talk earlier, in some ways, the unbundling of higher ed is not really that new, right? In some ways, we have always had research universities, we've always had teaching institutions, we've had transfer colleges, we've had two-year colleges and four-year colleges. So we've had colleges that have decided in their missions to focus on one or two things rather than try to do it all. Now, obviously, we have very comprehensive research universities that do try to try to do it all. So this is not exactly a new concept on focusing on one or two things. But I think what has really propelled this is the idea that everything, you know, universities are essentially mini cities, they try to do a lot of stuff for a lot of students. And the fact of the matter is that that's becoming incredibly expensive for both the institutions and for the students. And what we're trying to do, I think, is provide a very one-size-fits-all system to a larger diversity of students. You know, in higher ed, we used to think of, and we still think of, students in really two buckets, traditional and non-traditional. And the fact of the matter is that even within those buckets, the motivations of students vary greatly. So there are folks who go to college because it's coming of age experience. At 18, there are others who go because they're very career oriented. And in some ways, they actually match much more with the career switchers who are older and are looking to get a new career out of higher education. And so the future, to me, is one where institutions, and I think this is what many of the startups are doing, right, they've decided instead of trying to serve a market that has motivations of six different types of students, we're only going to focus on one or two, very much like most companies only focus on one or two customer segments. I think the future for colleges is to decide, you know what, we're only going to focus on one or two of those customer experiences. And as a result, I think their offerings will look very different than they do today. Now, Robin, Minerva is a really interesting thing. I think it's an example of unbundling. It's not doing everything traditional universities do. What I kind of gather is that it's saying that one thing it's saying is, unlike the assumption underlying a lot of MOOCs, a lot of these things, it's saying there actually is value in students being physically together. That's one thing that happens with Minerva. But there are a lot of other things. There's some other things that happen in ordinarily universities that don't. So can you kind of flesh that out? How exactly does this work? Sure. Well, and it really follows directly from some of the things Kevin said and some of the things Jeff just said. And that is, you know, what do we want to think about the experience should be for an undergraduate to get an incredible degree that's going to prepare them to go off and do some things later in life. And when we look at the experience, there should be an academic component of it and there should be a student life component. And we look at how do we bring the best of those together. But where we can unbundle is things that are available for free, you shouldn't be paying for as part of your undergraduate experience. So as Kevin just said, you can get your entire first year curriculum for free. Why go and pay for that? So at Minerva, what we've done is really define how do we develop curriculum to really prepare global leaders and innovators in a global context? What are the ways that we can help prepare them to think, to think critically, to think creatively, to communicate? Because those are skills they will need. And those are skills that will take them regardless of what discipline they go into. So we think about learning in the classroom, being all about active learning, not passive. MOOCs are fabulous for information dissemination, for gathering information. They're not great for really digging in and being actively engaged so that you can apply what you're learning. So the classroom is all about that. And then there's the student life component, which is a residential-based component. So the students do have an opportunity to live together and to immerse themselves in the places they live. Okay, and so you're gonna, I gather these students, they're actually gonna, in some cases, the whole class, the whole cohort, will move from city to city from year to year. That's San Francisco one year, Beijing the next. That's part of the education, is encountering the local environment. But can you flesh out the teaching experience a little more? I mean, are you harnessing a lot of, so to speak, free content out there? I mean, there are all these MOOCs and so on. Or are there professors physically with the students? Or where is the kind of instruction coming from? So we have developed our own curriculum that is really designed around how to teach students how to think. So very often students will go, they'll take their freshman curriculum, they'll take a spattering of different classes so they get exposure to things. But it's not really teaching how to think. We've developed a series of habits of mind and foundational concepts, habits of mind, things that should become automatic in nature, foundational concepts, concepts to build on. And that is what the cornerstone curriculum has been developed to teach. That curriculum is all being developed by our own deans. Our founding dean, Dr. Stephen Kosselin, is one of the preeminent neuropsychologists. Absolutely passionate about thinking about how do we prepare our students, how do we give them the fundamental skills that, again, will be critical as a foundation for whatever they go to do. Okay. Now, Adrienne, you're at ASU. I assume ASU's hope is not to get completely unbundled and rendered irrelevant. And it sounds like your job is associated with this aspiration. What's your plan? There is this peril, and Kevin spelled it out. I gather the idea is that you're going to actually harness technology as a way that, in some sense, strengthens the institution. But what's the plan here? So I think harnessing technology is the exact right metaphor. When we think about the way that technology's disrupt industries, everybody thinks of Clayton Christensen. And when you think about Clayton Christensen, you think about coming up from the bottom, reaching underserved populations. Wait, actually, I don't think of him because I don't know who he is. Oh, well, Clayton Christensen from Harvard, founder of Disruptive Innovation Theory. The guy who sort of describes how it is that these technologies emerge first to serve people who aren't being served, like the transistor radio served teenagers, and then gradually climbing the value chain, taking over more and more of the capabilities as the technology's become more robust. So think about MOOCs as an example. They're clearly entering in a place that's underserved. They're entering in a place where people are, I don't really want to invest all this time in getting a degree necessarily or getting a master's, but that topic's interesting to me. I'll attend the first few lectures and then perhaps continue. There's an underserved population, but it's also beginning to show what's going to be possible at the next level of scale. Now in Kevin's talk, he talked about the role that books played in disrupting the previous model of universities by making it possible for more people to get access to explanations than could attend a professor's lecture. And so in some sense, you unbundled a piece of what the professor provided and disseminated that at a lower level of cost. So the universities of the future that will be successful will be those that are able to climb the value chain and leverage these technologies as they emerge. Today we'll be talking about some educational hacks, and one of them that I'll talk about is the ability for universities and other institutions to use a technology that's emerging on the internet at scale today, the Khan Academy. And so I think these are examples of how the universities that will survive and flourish in this next century will be standing on top of technologies that make their professors as President Crow says, super professors, able to reach many more people at much greater depth with much greater levels of interactivity, a very different model than the one that we're familiar with now. You're going to use Khan Academy content? That's right. And what, okay, so what do you add to it? Well like you don't want to be taking away everything from my hack because I have to fill up the entire problem. But basically if we are now living in a world where explanations of complex concepts used to be scarce and now are quite plentiful, whether that's James Taylor explaining how to play that lick from Fire and Rain and actually showing you how he does it in his studio, to the Khan Academy explaining the squeeze theorem in a five-minute video that just was completely inaccessible 20 years ago, unless you had your equivalent of Calculus Alexander sitting next to you explaining how that worked. Okay, so but I'm a student at ASU and you say, in what form do I encounter the Khan Academy content? And what is there a professor there helping me out? I say, wait, I don't get this part. The professor helps or what? Why do I need you at all is my question. Well, I think the clear reason that we need universities at all is that most people are not autodidacts. Most people aren't Abraham Lincoln. They can't go down to borrow the books, go back and forth 30 miles each way, keep themselves with the discipline to learn by themselves. Most of us, especially for complicated subjects, need some form of learning community. And so at ASU what we provide, what other institutions provide, are those learning communities. A set of people who together are trying to discover a subject. And many times we learn as much from the learners we're learning with as we learn from the basic material. The notion behind technology mediated material is that it'll be more appropriate for me. It will be more personalized for me. It will let me be able to go faster and participate more vibrantly in my learning community. And that's absolutely what universities will have 100 years from now, just as they do today. So does a professor assign a Khan Academy video? Well again, I told you, I don't want to take away all of that. And the details for this hack are available at www.senior.net if you want to go up and look at them. But we've been experimenting at ASU for some years now in the use of adaptive technologies to address mathematics education. And we fundamentally changed what our model is there. Rather than individual teachers in a lone eagle fashion teaching one section of math in a lecture format, what we've done is begun to use adaptive technologies, not the Khan Academy at first, right? The first things that we've tried have been based on commercial products. We've learned many things about what it takes to teach as a team a set of students, each of whom is advancing individually along a personalized learning path that computer algorithms are mediating in order to determine what it is you know, what it is you don't know and how quickly you should progress through your material. So it sounds like one assumption you share with Robin is there is value in the interaction among students who are actually physically present in your environment. That's one thing that you think is a real value that you plan to continue to provide. Jeff, I'm wondering how all this looks to you as a relatively objective observer. You don't actually technically have a dog in this fight. You're not an institution trying to survive a hurricane or anything. Do you, you've been observing universities as they try to adapt. ASU has probably been more proactive than a large majority of universities. How do you think they're doing? I mean, do you think they're kind of whistling in the dark or? Some are, in fact, many are. And I think most, there's this great cover from Newsweek magazine back in 1976-77 which basically said who needs college, right? And it had two college graduates in caps and gowns, one with a jackhammer and the other with a shovel. And basically the article talked about the over-education of Americans and too many people were going to college and that, you know, hundreds of colleges will essentially go out of business. Well, you know, Newsweek is out of business. But we actually have more. Newsweek is back. I guess it is back, right? Well, it's not in print, though, right? It's not in print. No, actually, they were- Oh, I can't keep that story straight. And famously started off with a cover story that was subsequently called Widely Into Doubt. It's a great ability, so that was a bad launch. But they came back in print. Okay, well, there's actually more colleges and universities today in the U.S. than there were when they made this prediction back in the 1970s. And so what I think happens is that most college officials that I talk with, you know, especially these who've been around for a long time, they say, you know what, this is a temporary downturn and things are going to change. And I really do think, not just because I'm talking about this a lot, I really do think this time is different. And it's different for two reasons. One is technology. The stuff that Kevin laid out this morning and the stuff that we just laid out here, that technology allows us to do stuff at scale that we couldn't do before and allows us to adapt, allows us to build these learning communities, and allows us to really improve education in ways that haven't been done before. And the second is financial. The financial picture around higher education is completely different than it was even five years ago. You know, 75% of colleges today have either flat or declining revenue, flat or declining, 75% of colleges. I was talking to a CFO at a college the other day who said, you know, our net tuition revenue, that's the actual dollars that come in the door from students, is the same as it today as it was in 2007, even though our tuition's been going up 4% every year. So there's just not enough cash. The financial model of higher education is really broken up, a majority of colleges and universities, and they just can't continue. I don't think most presidents and boards and senior leadership teams and even faculty have come to terms with that yet. And the motivation for participating in MOOCs on the part of universities that do participate, do you sense a kind of a clear vision here or more of a sense that, well, let's try to figure out what they are. I actually have a feeling that there is a coming clash between the universities and the MOOC providers. I think, you know, there's a lot of reasons why COSERA might have hired Rick Levin, former president of Yale as its CEO. But I think one of the reasons is because they are now starting to see some tension between the universities. So I just took a MOOC from the University of Virginia through COSERA, and I don't know what your experience has been, but many of the students that I talked with, they love the professor. They love COSERA. They couldn't really remember the university that was offering the course. And these are name brand universities. I can't imagine them really wanting to give up their name brand. But the other thing that I think is really going to come to tension between these two groups is around kind of this always on education idea, right? So the problem with COSERA, of course at COSERA or even at X or others, is that if you start it and you decide to, well, I'm kind of busy this week and I want to drop out or I want to drop out and take this in a couple of months, you can't, right? So the course I took from the University of Virginia, a couple of months ago, all the course materials down. The course page is closed, right? And COSERA really wants to offer education all the time, right? So that I could take this course 24-7, 365 days a year, I could drop in, I could drop out. The universities don't really want to do that because their intellectual property is essentially always open for anybody. And these are courses that they're teaching, in the case of the University of Virginia, teaching on campus and charging thousands of dollars to business students to take. I think you do lose something when you leave it open 365. First of all, you lose the sense of urgency to get students to actually watch the first lecture, right? This is a limited opportunity, but there's also a conversation that goes on in the discussion forum that's not going to be very robust if everyone is watching lecture one in a different week. So I think you do lose something for, but you were nodding your head a lot during that. Do you, well, you agree, first of all, that most universities don't have a clear sense for what's going on and B are increasingly going to find themselves in tension with the MOOC providers that right now they're cooperating with. I do, I do think that's true. I think that some professors have a lot to gain by establishing brands. Other professors are participating in these MOOCs and describing that they're essentially diminishing their brand because they're teaching for free. I think Jeff is right. There's a lot of fluid motion and different institutions are involved for different reasons, but most I think in voyages of discovery. And so there are a lot of collisions coming, but I think the same things that we're saying now of MOOCs as an unbundling erosive force against universities could be thought of when textbooks were made too. Oh, hey, used to have to go to listen to the guy. That was the only way now you're going to put it in a book. Anybody will be able to read it anytime. It's always on. And so that will totally erode it. But you know when you went to school, if the professor wasn't assigning those things, if there's students in your study group weren't also expecting you to know and be able to contribute to the conversation, you didn't read that book. And so I think in the same way we see a lot of fall-off in MOOCs when the learning communities aren't strong. And so I think institutions like our most forward-looking universities are the ones grappling with what do these next-generation communities look like? Both on-ground, in-person, these active learning communities, and also online, where the tools that we use to keep connected when we work remotely are the same tools that we would use when we're going to try to learn remotely. And can you say whether at ASU the view is kind of like, well, eventually all universities will learn to do what we're doing and harness the technology. And so it all will end happily for universities and colleges. Or is it the case that still kind of like the ratio of resources required, even in this model, kind of the ratio of campus resources required, physical campuses resources required to other resources is going to change such that there aren't a lot of colleges are basically going to go out of business. Is that the sense? I make the argument that the way disruption will happen in our industry is very similar to the way disruption happened in the banking industry. When you think about the banking industry, as information technology entered that industry and completely transformed every aspect of that experience to the point where my mother is the only person left on the planet who still visits a teller in person. She's a- My father-in-law does that. Okay, well, you know, like there's that small sector, but everything else about it has changed because of information technology. That hasn't caused Mellon and Chase and Citi, hundred-year-old institutions to disappear. It has caused them to fundamentally change and it's also caused a tremendous amount of consolidation. There used to be a lot more banks and now there are a lot fewer banks. Okay. Now, Robin, is the premise of Minerva that actually there's just something that's not working with the various alternatives? Is something not working with MOOCs and we're going to require a different model or is it more the sense that a lot of this stuff is going to work and we're just like a boutique or something? Yeah, well, I think there's a really important point to bring up here and that is what MOOCs are. So MOOCs are taking a lecture and putting it online and that's a very efficient way to get a lecture. But let me just ask the audience, how many of you sit back and think about your educational experience and think about those lectures? There's 300, 500, 800-person lecture halls and how many of those were full the first class and then became more and more empty as the semester went on and people realized they could just get notes. So the concept of the lecture has a purpose but we don't think that that is an ideal way to educate the future leaders of this world. So when we think about what we need to do as an undergraduate experience, we need to put together a curriculum that can best prepare them and that is through active learning and it is leveraging technology because technology provides some really important things. One, you don't have to be in a physical classroom. So at Minerva, although our students live together, they are not in a physical classroom. It also enables us to track progress as the students are going through. So everything about Minerva is student focused and it's about student learning outcomes. So not only do we have to wait for the midterm or the final, but we can track after every class section progress that the students are making and that is enabled because of a technology platform. And because we can do it in a seminar format that is scalable, which is really different than MOOCs, classes no more than 15 to 19 students, you are actively engaged in every single class. And when you say it's scalable, I mean, most people think the opposite of small seminars. What sense do you mean scalable? Scalable meaning that we have two big things we're investing in at Minerva. Faculty, unbelievable faculty and developing curriculum to help these students. And because we can invest in those two things and we don't have to invest in a basketball coach, the performing arts center, buildings for classrooms, we can put all the money towards faculty and curriculum. So there is a faculty member present in every single seminar class, every single session. And that's where the tuition dollars are going to fund that and not to fund building a city, as Jeff said. And Jeff, you wanted to come in? Well, I just, I don't want to defend the MOOCs here, but I do think that MOOCs are the best MOOCs are slightly better than or actually a little bit much better than some of the worst lecture courses. The big general lecture courses that we've had on college campuses. I think that they're, it's not just a camera in the back of the room. I mean, the lectures are designed in a way that appeals much more to the way students learn. There's smaller chunks. There's regular assessments. There's much more of a community, as Adrian said, around, again, with the best MOOCs. So I don't think that they're just as bad. They're just as bad as some of the worst lecture courses that are taught on college campuses. I'm wondering if in the long run there isn't a little bit of a threat to the business model of like the MOOC platforms just in the following sense. I mean, I think that I assume the hope of places like Coursera is that they will provide this credential that will be of, in some sense, of value to, to high people who hire people. I mean, they, I mean, I think there's going to be a lot of change in the, in kind of how we credential students and that higher ed will no longer have the corner on that signaling market that it has now. I think right now where MOOCs are, I think they have proven that they could teach at scale. I don't think what they've proven yet is that they could assess at scale, which is really weird. That's the, that, it's not that we've known how to teach hundreds of students for, you know, a very long time. We just don't know how to necessarily grade hundreds of students in a great way. And I think, at least, the MOOCs I've taken, the grading is not really that rigorous. And I think that if you're going to give credit or give a credential from that, I think that it's going to have to change in a way to make it much more rigorous for people to trust it. And also, like, once they figure it out, what does the employer even need Coursera for? You know what I mean? I mean, if there's this test you can give that captures your ability to do this job, you don't care where they learned. And you don't need to know that they finished any particular. Yeah, and employers are really skeptical. I was just in Boston yesterday for a new survey that Northeastern put out of corporate CEOs and, you know, it's the usual stuff they always say and, like, 87% of them are something like that said. They essentially do not think today's college graduates are prepared for the workforce. You know, like 87% of corporate CEOs. So I don't disagree with those findings. But at the same time, I think it's important to sort of reality check for a second and think to yourself, okay, so this is all really interesting. All that money I was going to spend on my children to go to school, hey, I can go to Aruba because they're just going to go on the Internet and get their credit. And then the disconnect becomes completely apparent. And all this, all that we've been talking about that may happen, clearly has not happened. And the problem that's trying to be solved here is what are those things which the Internet can do at scale and what are those things which are yet, it is yet to be able to do at scale. And I think that many of those things balance on the knife edge between what machine attention is valuable and what human attention is valuable. And where our institutions of higher learning have tremendous advantages at the moment is that they have enormous concentrations of that human attention and they have methods by which that human attention can be brought to bear to help people to learn and that's why you send your children to these schools and not just to the Internet. The final question before we turn to the audience, Robin, I assume you are mindful of employers out there. What are the things that, how is your model preparing to kind of meet meet their needs and the students' vocational needs in ways that are specific to your model? So we are out talking to CEOs, we're talking to officials, we're talking to nonprofits, we're talking to try and understand what are people thinking about when they want to hire their future employers when they are employees and when they want to think about those to bring into the organization. And what they come back and consistently say is we are looking for students who know how to think. We are looking for students who know how to take a problem they've never seen before, deconstruct it, think about it creatively, innovate around it, develop the communication skills to work in teams to present all of those things. And I think that's consistent across study after study. So what we're learning is it's not just about specific depth of any single discipline but it is really also being able to sort of absorb anything that might come their way. So when we think about preparing students for that, that's the critical piece for employment of the future. Part of the problem with employers is that sometimes they don't know what they want and the thing that frustrates me about these surveys of employers is that the corporate CEO always says they essentially want the classic liberal arts. They never hire entry level graduates or entry level workers. And so you go down to the HR people and they're hiring people with specific skills. They're looking, in fact, they're outsourcing most of this work to a computer that's looking for specific keywords, specific degrees. They don't really, they don't care as much about those longer range skills, the learn how to learn skills because the CEO is thinking much more long term than the person who has to fill a job tomorrow and doesn't really care if the person leaves in two years. Okay. Well, can I just interrupt for one second because I think what's really important is the jobs of the future, we don't even know what they are today. And I think that's the big thing right now. So people who are getting skills for a job for today are finding that in three years that job is gone. And so, you know, agreed that there are some very, very specific things that might get you employed today. And look, I think that's an important piece as well. But what we want to do is prepare for the future. And we do a lot of thinking about what will be the jobs in the next 20, 40, 80 years. And I think that's where we have to prepare for the unknown. Okay. Let's go to the audience. If there are people with questions, wait for the microphone, please. One of the things that has to be considered is, is the course have to be the container for learning? I think one of the mistakes that the universities are making and in some ways mimicked by the Coursera's and the MOOCs is that rather than learning community, which you have been sort of bringing up, there, we're assuming it has to be in a specific kind of container. And so I think what MOOCs give us is, could give us is, learning communities are on a topic that has a different kind of interaction that provides tutorials or whatever. But it's the interaction that becomes important. And I don't think the MOOCs do a good job at that quite frankly yet. But I've been concerned more and more about course as container, program as container. I don't think that's, I don't think we teach lifelong learning. That seems to me to be an excellent observation. When we talk about unbundling, that's another aspect of that unbundling is rather than thinking about any particular collection of concepts or understanding in the bounds of a single course. I think the reason we're still in the course metaphor is because it's a thing people understand. It's a lot of accreditation things and other things are built up around it. But evaporating those boundaries, moving more toward project-based courses. I have a colleague, Ariel Anbar, who has pioneered a course he calls Habitable Worlds. It's a course in the sense that it's an activity that a lot of students engage in. But it breaks down the barriers between astronomy, geology, probability theory, certain other kinds of statistics and mathematics, teaching sort of all of those skills in the context of a quest for finding a habitable planet. And so I agree with you. I think those modes are happening. What President Crow said last week at the Education and Innovation Summit, I think is right. Many of the technologies are here now. Many of the capabilities are here now. Investment is here. Desire is here. Culture change is the most complicated piece of the puzzle. So a lot of times, I think these culture changes follow the lines of least resistance. And that maybe accounts for why courses are still such an important part. And the same can be said of the bachelor's degree or the degrees in general, right? You know, what's so special about 120 credits to a bachelor's degree? And in many places, you have this bucket that, you know, engineering fills in one way and economics fills in another way and accounting fills in another way. Anything you want to say on that, Robin? No, we can go on to the next one. Other questions? I'm tall, so I'm going to stand over here. My name's Jackie. I'm the digital education coordinator for the Institute for Humane Studies at George Mason. Something that really struck me in Adrian's talk was when you discussed the attrition rates, because that's something we hear about a lot when we produce MOOCs. And I'm wondering, for those of us who don't work or teach MOOCs per se, but create the structure that the courses are hosted in, should we be focusing and appreciating completion rates for what they are and stop comparing them to all of the fall-off that happens on the way? Or should we create the incentives that you discussed in-person courses having, such as credible portfolios or something? Should we focus on creating those incentives to diminish that attrition rate? Because often with dedicated funding, you can't really do both. I think that my answer is going to be really simple. As MOOCs will get better as they get to be more interesting, and they'll get to be more interesting as more people interact with one another in their context or using them. Will that be because they're adopted by learning communities that are established in existing universities? Yes, it will. There'll be a way to do it that way. Will it be because those MOOCs themselves figure out how to titrate human attention in a more skilled way so that it feels much more vibrant, there's a better on-ramp, it's easier to build community? It seems to me that those MOOCs will be better. And I think that this false dichotomy that we're establishing, oh, are MOOCs good enough to completely wipe out universities? That's not really what the question is. The question is how do these emerging tools get better and where is it that we can use them to make people learn faster, make people learn deeper? Just to be clear, as far as what I said, I mean I would rather have a lower attrition rate. And if I taught the course again, I was going to redo the lectures, I'd try to figure out ways to make it more compelling or to add elements to it or whatever. But at the same time, of course, Sarah came to me and said, hey, we've got a whole new marketing dimension we're going to add, and it's going to bring you a thousand students who stay with you and love it, but it's also going to bring you 50,000 who sign up and eventually drop out. I'm just saying I should say sure, even though it makes my curve worse. And to a large extent, the steep curve is just a function of this whole mode of how students become aware of the content and find the content that's right for them. Any other thoughts on that? I mean I think that we make too big of a deal of the attrition rates in MOOCs. I think it's again applying the traditional higher education model of completion to MOOCs where most students are not as interested in finishing. They're interested in getting a piece of the content. So they might sign up because they want to get some of the readings. They want to get one lecture as they look down the syllabus. They're like, this is what I really need. So we don't measure intention as much as we should in MOOCs. Other questions? Thank you. This has been very interesting so far. Adrienne, you spoke of super professors, really good professors teaching really good MOOCs. I'm just a professor. I'm not a super professor yet, but I want to be. My department chairman comes to me and he says I want you to teach a MOOC. Well, all right. If then the MOOCs get better as they get more interesting, what kind of code switching do I need to do so that my numbers don't tank like Roberts did? The super professor who's got that MOOC. So I'm not necessarily going to equate super professor with MOOC though. It's pretty impressive. When Andrew Ng gets up at a panel and says in my first artificial intelligence course, I taught 250 years worth of students. You know, that's a pretty impressive super charging, right? So there is a scaling there. I think many of the things that we've talked about point to the idea that if you're going to be a super professor, it's not like you're just going to go to your Blackboard instance and start working a little harder and maybe get a couple of your graduate students to gather up some videos for you. The model that we've had for deploying technology at universities up until now has been very much based on a kind of cottage industry scale of development. And so I think it's very important for whoever aspires to be a super professor to aim for that level of scale. And so then, you know, who's going to be able to establish those kinds of brands and what kind of experiences are they going to create? I think that's an emerging thing. And I think MOOCs give us an idea of what they'll be like. But I think there are some other models too. The major thing will be the ability to concentrate a set of resources so that you can produce an artifact that makes a hit. And I think we are still yet to see something that's like a hit video game in this space, something that's that captivating. But I think there are some emerging examples and I'm going to talk about one of them later today. Any other questions? Question I have, and by the way, I think that's really an interesting point is that ultimately things that dominate the market are, you know, just as the Hollywood blockbusters that go global have immense resources put into them, I think you're going to find the killer app in MOOCs is not the creation of any single professor kind of working with the limited assistance of universities. You know, maybe more like a video game or something. Who knows? Did you have something you mentioned? I was just going to say, I think what's really important is that MOOCs are not a substitute for a university. They're not a substitute for an education. They are something new and something different. And I think what's really important, whether you are trying to be the super professor or whether you're trying to create a great engaging MOOC and you don't want people to drop off is you have to think about what is the objective? Why are you doing that online lecture? And I think that that is there's a need for it. It's important. It's not the end all be all. And I know that the talk of this entire session over the course of the day is about hacking the university. And I think we just want to keep in mind that MOOC does not equate to a university undergraduate or graduate or any other full comprehensive experience. Okay, I'm afraid our time is up. You guys have been great. I've learned a lot. Thank you.