 Our guest tonight is probably the most closely followed figure in higher education today. And I don't know why, he's prone to mealy mouth comments like, there have been no innovations in learning since the printing press. His ambitions are cautious and modest, such as, I aspire to educate a billion people around the world. So why everyone is paying such close attention is not clear to me. But I can say that, I can tell you that if you go home tonight and Google the now cliched and overused term MOOC, you'll find that it only appeared about three or four years ago, according to my Lexus search. But as soon as it did, it began to be linked to our guest's name tonight and rightly so, he can be seen as a founder of the global online education movement, which is now afoot. Now before his current notoriety, he was already a superstar in the world in which we live. He's a graduate of the world famous IIT in Madras, Stanford PhD, and for several years has led the computer science and artificial intelligence laboratory or CSAIL at MIT. In Purdue's view, he is a triple threat faculty member. He has won major awards for research, but also major awards for teaching. And on top of that is heavily, as we would say, engaged. He is a serial entrepreneur. I think it's five companies and counting that he has started. I read in one article that he dates his entrepreneurial career to his childhood in Mangalore, India when he made his first buck raising 40 chickens and selling their eggs. And now I want to tell you that if this MOOC gig doesn't work out too well, we have an excellent College of Agriculture where we'd be happy to help you start on yet another career. Dr. Agrawal founded edX and taught its very first course. There's an interesting anecdote in the literature about him. They offered that course, not really knowing where this was all going on circuits and electronics. He's reported to have hoped that perhaps 2,000 students would sign up. As the number passed 100,000 students on its way to an eventual 155,000 students, he had one, I guess, one of those what have I done moments, but the system didn't crash and the course was successful and the MOOC movement was launched. Dr. Agrawal, in a very gracious way, is challenging not just the mode of delivery that has prevailed in higher education, but things even more fundamental. He has spoken about his view that there will need to be change in, quote, stovet-pipe single-discipline departments. He's questioned why four years is meaningful in terms or necessary in terms of time to degree and implicitly has raised the question about how essential residential education is. When I somewhat playfully talk about the pajamas test and show the slide I mocked up of a student of the future sitting in his living room staring at a screen in his bathrobe. I'm worried that it's Dr. Agrawal on the other side of that screen and that's the challenge that he brings to us here at Purdue and at schools like us everywhere. So he wouldn't look at it this way, I know, but to listen to some people talk about the threat, the disruptive threat that this gentleman and his protégés and imitators generated that tonight would look a little like I've invited a lyrical visigoth into the Roman Senate right before he sacks the place, but I think you'll find out that's not his intent and you're about to find out how what a true visionary looks like. So please welcome enthusiastically Dr. Anand Agrawal. Thank you, thank you, Mitch. Those were extremely, extremely kind words and I promise you that a lot of the questions that I ask in terms of the future of the university and so on are meant to, really meant to challenges to think about, really meant to challenges to think about a system that really hasn't changed in a long time and I've actually seen pictures from the earliest days of the photograph of classrooms with the teacher teaching and a whole bunch of students sitting in rows like a neat little group from Indianapolis airport to here and reminded me of neat little rows like corn stalks right after a winter's thaw and that system hasn't changed in hundreds of years. Everything around us has changed. Health care has completely changed in the space of a few hundred years. Look at surgery, a few hundred years ago you'd be knocked on the head and they would operate on you before you would come to and that's the best that they did. And today with laparoscopic surgery it's in and out and everything has changed around us. I look at transportation, you can with jet engines and rockets and everything has changed but our education hasn't changed. It is absolutely shocking that the same institutions where incredible research is done and changing the world around us, we haven't changed ourselves, we haven't changed education. And so today I want to give you a sense of what is possible and really the questions are asked are provocative questions but fundamentally at X came out of universities got a non-profit and we're looking to transform ourselves from within and ask these questions to see how can we improve and reinvent education so this title is really about how do we rethink education, can we do or can we do a lot better on a number of fronts. So what you see here, this is not a rock concert and this person here is not Miley Cyrus swinging on a wrecking ball but this is actually a teacher and so this is actually a classroom at the Oba Femi Ovalova University in Nigeria and all of you have heard of distance education but if you look at the people back here, I would say they're undergoing long distance education and if you think that what we're doing in MOOCs and online education is new, talk to these people. When's the last time they've had any personal contact with the instructor. So our system is broken in terms of quality. I think in large parts of the world we don't have access to a good quality education either. So edX is a non-profit, we were formed with a $60 million investment by MIT and Harvard and really we came up with three parts to our mission. The first one is that we wanted to increase access. Let me just check the, for some reason the bottom of the screen is flipping off but that's okay, no big deal. So we want to increase access to education for students all over the world. A second big part of our mission was to improve research, is to create research and learn about learning. And third was to improve campus education. So really understand how we can improve campus education. So this is our website, so if you go to edX.org, edX.org, I encourage you to go out there. You can take some of these courses from some of the great universities of the world. The course highlighted here is a course on financial analysis and decision-making from Tsinghua. Tsinghua is among the top two universities in China. They are a partner of ours. And so you can go and take these online courses from just great universities in the world and the courses are free. And we'll talk more about these courses. So one reason why MOOCs, these massive open online courses, really caught the attention of the world was the sheer numbers and scale at which education could be offered. So for example, in the first course on edX that my colleagues and I taught, we had 155,000 students sign up to take the course. Now, this is a big number, certainly for MIT, it's not a big number for Purdue. Here you have 30,000 undergraduates, how do you, wow, it boggles my mind. MIT has, if you want to take a shout, how many undergraduates do you think MIT has? 5,000. Good guess, 4,500. 30,000, how do you, how do you, that's incredible. So MIT is 4,500, so the joke I'm about to crack won't make a lot of sense to you, but this number here, 155,000, is actually bigger than the total number of alumni of MIT in its 150-year history. At Purdue, you will blow past that number, so if it's four years and you have 30,000 students, then you'll blow past that number in about 16 years. Okay, about 150 years, then this number is bigger. So it really caught the attention of at least some people. So 155,000 students signed up for the course. And at the end, this was an MIT hard course. We advertised the fact that this had differential equations as prerequisites. It's a course on circuits and electronics. It's a really hard course. And 7,200 students passed the course. And these were from 162 countries. This is also a big number. And this is a big number because if I were to teach at MIT, I teach about 100 students each semester. I teach it twice a semester. And I've been teaching at MIT for 26 years. But to teach this many students, I would have to teach at MIT for 40 years. And in one fell swoop, you get a large number of students. So the access part is a really big aspect of these MOOCs. So to give you a quick sense of edX. So today, so we started about two years ago. And today we have 2.2 million students from all over the world. They come from 196 countries. And that is all the countries in the world. We have 4 million enrollments, which means that on average students are taking two courses, so this many courses have been taken. And we have courses in virtually all subjects, in music, in arts, history, engineering, computer science, business, math, the name it, and there are courses in those areas. We also work with a number of university partners. So we have about close to 50 university partners on our platform. And we also work with a lot of campuses where universities put up a course online and make it freely available to students around the world. It's completely online. And I'll give you a sense of what an online course looks like. And then they also bring it back to campus and create a blended model class where they combine in person with online. And so we have over 12,000 students that are taking these blended model classes across a number of campuses all over the world. So here are some of our partners. So we work with great universities of the world. We're actually coding Purdue and we would love to get Purdue to join edX as well as a partner and join other universities such as Cornell, of course, MIT and Harvard, Berkeley, Tsinghua from China, IIT Bombay, Luven from Belgium. There's the Australian National University of Australia. So many universities all over the world that offer courses on our platform. So even as we offer courses, students can take these courses for free. I forgot to mention that, these courses are free. So you can take these great courses for free. And so the students can audit the course, and they just audit the course for free. They can also sign up to take an honor code certificate where if they pass the course, they get a nice little certificate that looks like this, this is a course from Berkeley. We've also launched verified certificates where we use a webcam to take the picture and a picture of the photo ID. And make sure that they are who they say they are, the name and the face match, the photo ID. And you can get a verified certificate. Then we charge a small fee for a verified certificate service, $25, $50, something like that. And that becomes a revenue source for edX. I mentioned we had a non-profit. Non-profit does not mean money losing. So I want to be very clear. Like a non-profit does not mean slothful. Non-profit does not mean big government. Non-profit does not mean non-innovative. Non-profit does not mean slow moving. So somehow there's all of these things applied to non-profits. But I've done a number of startup companies. I only know how to run fast and startups. And so they're running edX like a startup company, but it's non-profit. So edX, it does happen to be my first non-profit. And so we are looking at various approaches to generating revenue. And our goal is to make sure that outflow matches input. We're not looking to do an IPO or get huge profits and so on. We want to break even. And that's our goal as a non-profit. So this is one revenue source that we have. And we share this revenue 50-50 with our university partners so that as they invest in courses, that they also get some money back so they can invest in campus education and online courses. So one of the things that edX did, so edX is the only non-profit MOOC provider. They're also for-profit MOOC providers. One of the things that edX has done uniquely is we've also made a platform available as open source software. So that open source is called open edX. In other words, anybody can take our platform, our software, and they can go off and build their own MOOC platform or system or whatever. And if they want, they can compete with edX. Again, we're a non-profit. We're looking to foster collaboration around the world and we made a platform available as well. So as an example, we made a platform open source and made it freely available to anybody six months ago and there's been huge adoption of that all over the world. So as an example, Tsinghua and the Chinese Ministry of Education launched Shui Tongx, which is a Chinese national platform. France launched a national platform in France. Stanford also has now adopted open edX and the Stanford platform called class.stanford.edu uses edX now, Stanford Online uses edX. This is France and also this is cut off here, but Queen Rania from the Middle East launched edRock, which is a Middle East Arabic platform using open edX. We're also working with the World Economic Forum. They launched, these are the Davos people, they launched Forum Academy, also working with edX as a partner. So one of the things that I do want to talk about is for those of you who've read some of these things in the paper, recently there's been some angst about what were the low completion rates? So for the course that I taught, I showed you the numbers, 155,000 signed up, 7,200 passed the course. So if you look at that as a completion rate, that's about 5%. So people are saying, oh my God, this is terrible. It's 5% completion rates. So therefore MOOCs must be terrible. And so I just want to point out that really looking at completion rates as a percentage of those people that sign up, it doesn't make any sense. And I'll give you a quick example why. So we really need to put these certification rates in context. So let's take, for example, a course from any selective university, right? Let's, I'll use my own example at MIT, for example, the course that I taught was the same rigor as the campus course that I teach. And notice that about, typically about 7% of learners pass these edX courses. For the course that I taught, 5% passed the course. But remember, anybody can take an edX course. You can go to edX, no matter what your background is and I encourage you to do so, and you can sign up to take a course. There is no admissions test. So part of our goal is to democratize education. It doesn't matter if you have the background, it doesn't matter if you have the money, it doesn't matter what your race is, your color, your geography, really nothing matters. You need, all you need to be happy is an internet connection and be able to click and sign up to take a course for free. There is no admissions test. So 7% passed the course, but anybody can come in a completely democratic fashion and take the course. So that's 7%. But then if you look at MIT admissions rate, MIT announced admission rate this year, and this year MIT admitted 7% of people that applied to MIT. So 20,000 applied and they made admission offers to 1,400 students this year. So then why is the press surprised that 7% passed the course and 7% get admitted to MIT? So it's not surprising that those numbers are similar. And so you just have to put this in perspective. The other way to look at it is that if you look at students, we also did some studies to see how many passed the course. So for those who audit, who just sign up to audit the course. So when you audit the course, you're saying, look, I just want to listen, I'm not looking to get a certificate. And 5% passed the course if they declared they're an auditor. So about 70, 75% of the people who sign up are auditors. For those who sign up for honor code certificate, that is also free. Again, 5% passed the honor code certificate. But those that pay a small fee, there's 25 bucks or 50 bucks, and sign up for a verified certificate, 60% pass. This means that those that even pay a small amount of money, indicating seriousness about completing the course, they pass at 60%. So therefore, when people go to university, they've gone through an admissions test, they are paying tuition, whether that's high or low, they're paying some tuition. And the pass rates are pretty high. So this pass rate certainly is consistent with the pass rates in many courses at universities. So what does an online course look like? So this is a artificial intelligence course from UC Berkeley. And notice this little icon here. So in a typical course on campus, you have a lecture. What we do is we replace lectures with what we call learning sequences. A learning sequence is a sequence of short videos, five, 10 minute videos, interleaved with interactive exercises. So you may watch a video and following a video, you may ask you a simple problem or a hard problem and you go and answer the problem. So really you interact with videos and then you do some exercises and so on. And so here's the video, the transcript next to the video so you can follow along if you like with the transcript and do these interactive exercises between videos and interactive exercises. And this form of learning where you watch a video and then interact and try to answer the question is a form of learning that is called active learning. And education researchers have known this for a long time just that we have never listened to them, right? So a lot of what MOOCs are doing is just not new. We've just found a way to apply what learning researchers have known for decades and finally we are listening to them and applying it and changing the way the university thinks. So what they found in this landmark paper by Craig and Lockhart in 72 that learning and retention relates to how deeply you process the material. So if you sit and just listen to a lecture, you're gonna forget. But if you use a Socratic method, you teach by asking questions, get the students to engage, they will, studies have proven that it lasts a much, much longer time. And so on edX, mechanically by making sure that we have these videos interleaved with exercises and having professors do that, really promotes this form of learning. The other thing with videos, I was in one of your classrooms today and I sat down with a number of students and that class was a blended class where students, they've replaced a lecture with having students watch videos before they come to class. And then in class they have discussions and projects and so on. And ask the students, what are they like, ask them how many of them like the lecture format versus this blended format. So remember what I'm showing here is a purely online version. You can also have a blended version in class where in a classroom setting on campus you can blend online with the best of in person. That's called a blended model. And ask them what are they like about the videos and the blended class versus a traditional lecture class. And one thing they all said was the flexibility that in a classroom, if you're like me, I remember most of my classes that on the fifth minute mark I would lose the professor and I would be sitting there scrambling writing notes. I suspect many of you had similar experiences and well all the students around me were perfectly capable of understanding what's going on but I lost, I would lose them around five minutes. And then I would just be scrambling. But here you can pause the video of the professor. You can rewind it. Heck, you can even mute the professor. And in fact, in the blended model class that I taught at MIT, I took a poll. It turned out that two thirds of the students were actually muting me and reading the transcript on the side. So it's very flexible. Different people like to learn in different ways and the video gives just a lot of flexibility to learning. So one question people ask is, so how do you grade, if you have 100,000 students in your class, how do you grade these classes? How do you, there's no way you can sit down and do it in paper. So we use computing technology to do all of that stuff. So here's an example, I'll show you a quick little video where you're asked in the chemistry class, you're asked a question and we can assess all of these things by computer. So a student will enter an equation and they'll have a computer, check it and give them instant feedback. Okay, so let's take a look at a quick little video. So a student enters an answer, it's a chemical equation and oops, they're gonna get it wrong. Okay, they check it and they get it wrong again and then finally they get it right and get a little green check mark to say they got it right. So this is an example of instant feedback where most of us remember in classes where you submit a homework and get something back a week later but here you get a quick instant feedback and you can fix things if something was slightly wrong. And students absolutely love this and they're telling us that this little green check mark has become somewhat of a cult symbol at NX. The students telling us that they go to bed dreaming of the green check mark. And in fact, the green check mark has reached the meme status on the web. So we actually discovered this on the web. Someone posted a meme on the green check mark. So you know, as a tech knowledge, you know you've arrived when people have begun to meme what you've done. So the green check mark has become quite heroic. So the other thing that we do is how do you do labs online? So we have online laboratories. And here Purdue is quite a leader in this space in some of the work that Purdue is doing with Nano Hub and so on. Really, if we're looking at how do you do online labs and how do you do online interactives completely online? And so we have quite a bit of that. We focused quite a bit on that as well to bring much more of a gamification experience to students. So actually we have a little lab here. So this is from the science of cooking course from Harvard. And so here you'll see students are gonna cook some meat. They're gonna select what kind of meat, tuna, steak, choose the thickness, how long they cook it on each side. And then you'll see it cooking and don't miss the sound effect that you will hear. And then they can go around and play around with a piece of meat and see the temperature profile and a whole bunch of things. This is one example of a simulation lab for cooking. So the student picks the meat, picks the type of thickness and so on. And that's a grilling taking place. And then here they check the temperature profile and so on and so forth. And really just brings a gamified experience into the whole course. So that's kind of the whole access story and the online components of what we have. We also have a social component where you have a discussion forum where students can ask questions on the discussion forum and the students are answering each other's questions and really rather than getting rid of iPhones and iPads from the classroom, we embrace the social and bring the discussion into the online experience. And that is also really engaging the students. In fact, there's a study by Lorie Breslau who's a researcher at MIT. And what she found was that a student passing a course on the success rate was very highly correlated with the student collaborating and working with somebody else. And so the social can be a big part of the online experience. Next up, let me talk a little bit about the blended model of learning where you can bring online back to campus and combine the online with in-person. And this is how we think we can improve the whole campus experience. And here again, I visited a couple of classrooms today. I was really delighted that there's a lot of work going on at Purdue in the blended model already. So I went to a digital design class this afternoon, that's EE290, I believe. And there, what the professor had done was the students were asked to watch videos and so on before class. And they would come to class and the professor would cover any topics that had been particularly hard for the students. It spent maybe five, 10 minutes on the lecture. And then there's no lecture as such and for the remaining 40 minutes or whatever, students would be given problems and they're all sitting around in small tables in small groups. It's not like, you know, con drawers but small tables sitting around in little groups and collaborating with each other, solving the problems and working together in a group. You're learning all those great soft skills. How do you collaborate? How do you interact with each other? How do you solve problems together? And so I really engage with the material. So I really see that as the future of learning. We haven't quite figured out what the right blend between online and in-person is. So here's an example. So about, when we started out about a year and a half ago, we did a blended model experiment with San Jose State University. And the two experiments that happened simultaneously. One is to use a blended model in a classroom where students would watch the videos and in this particular case, we're using the edX platform on campus. And so they would also watch the videos and they would also do a lot of interactive exercises and online labs that I showed you before they came to class. And then, so this is a picture of the real classroom. They would come to class and they would work in small groups and do problem-solving exercises together. And the second experiment they did here was to see if the online content that the students watched, if that, how would it help if that came not from the local university, but came from somebody else? It's like a textbook. We're all comfortable using a textbook from somebody else. So why not use a new age textbook? Why not use a multi-media interactive online content much like a textbook and use that in your class? And in fact, many textbook vendors are not talking to edX and also looking at doing this themselves where they want to turn the textbooks into multi-media experiences and so selling a hundred dollar textbook to you, they'll sell you a multi-media experience for a hundred bucks or whatever else. In this case, it's free, but for some fee. And the professor in the classroom then has to decide how do they pass the pajama test. I really like the pajama test. I'm gonna keep using that. So as we all have to decide how do we pass the pajama test? If I can get a multi-media textbook, like a content like this very cheaply and it's much better than a flat textbook. So as a professor, how do I pass the pajama test? In other words, what value add do I provide? What value add do I provide as a university to those students who are paying in coming to campus? And so I think that's a question we have to ask ourselves. I think there is value add, but we need to really think hard about what is that value add? Can we focus on the value add? So in this class, they collaborated and so on and the results were staggeringly good. So here traditionally, that course was a circuits course had a failure rate of roughly 40%. So for about 10 years, this is a standard course that is offered, the failure rate was 40%. But when they moved to this blended model class, the failure rate fell to 9%. I have not seen such an amazing result in education in a long time. And then they repeated the experiment, this was in fall of 2012. They repeated the spring of 13 and then again in fall of 13. And they're doing it again in spring of 14 and the results are just kept getting better. In the most recent experiments, only 2% failed. And so now this experiment has spread to the entire California State University system and the course has been used in a number of campuses across the system. And we'll see how that continues to progress. We're also working with community colleges. So this is the picture from the Bunker Hill Community College where they used a course, a computer science course. This is Professor Jamie LaRue. So one worry that's already creeping up in your minds is that if I get this incredible interactive course, then what am I as a teacher, what am I, Jamie LaRue, gonna be doing in class? Am I even needed? Am I gonna be completely disenfranchised? Can students just take a pure online course and not need a teacher on the ground anymore? And these worries are creeping up and people are getting worried. So for things like this, I would again ask people to talk to people like Jamie LaRue who've done this now. Talk to Professor Kossler-Gardini from San Jose State as to what did the value the professor is bringing and this huge value the campus and the professor is bringing. So here's a quote from Jamie LaRue. So on being replaced, but I'll be using computers to replace teachers, no. She argues that if not for her presence in the classroom, helping the students work together, she's not lecturing anymore, but working with the students with this online content, much like you're doing in Purdue already where the instructor, as I went to the class today, the instructor was walking around asking the students questions, answering hard issues and questions and much more dynamic. There was a buzz in the class when the students were excited. I could feel the buzz. When you usually go to a classroom, what do you see? You see everybody asleep or if they've shown up to class, that is a sleeper on Facebook. But here the buzz, they're all interactive, engaged and talking, it's incredible, amazing energy. And so she said if not for the presence in the classroom, the students would not have passed the course. At MIT, they've really gone into a big way into blended learning as Purdue is getting in a big way as well. And at MIT, the space of one and a half years, 23 classes on campus were turned into MOOCs, offered for free, and the same content is being used in classrooms. At MIT, two out of three students are now accessing edX and using blended model on campus. Two out of three. And they've turned on a dime in the space of one and a half. The first course was a year and a half ago blended using the online content from a MOOC. And within the space of a year and a half, 2,800 students are using the content on campus. And there's a lot of good results. I won't go into a lot of the details. This is a blended classroom at Tsinghua University in China. So this effort is spreading. So in China, this looks very much like your class in EE290 in Purdue, lots of round tables. Look at this class, this doesn't look like a, this looks like a bazaar, doesn't look like a classroom anymore. Where are the constructs? It doesn't look like a drive from Indianapolis Airport to Lafayette, but this looks like, this is fun. Students want to show up to class, but they're engaged. The instructors, they're describing some hard concept. What they've done is the blackboard here, the blackboard here, a greenboard. They've got boards all around the classroom and little round tables and students kind of working together in groups. Well, very much like what you have at Purdue. But this is really how one aspect of how the future might unfold. Finally, I want to show you some fun results. This is my last slide. So I mentioned we have three parts to our mission at edX as a nonprofit. One is, increase access to education, where I really believe that everyone, anywhere in the world, should have access to a high quality education, much like the air we breathe, that's access. Second is, how do we improve campus education based on what we learned? And the blended model classroom is one example. The third is research. We are using the data that we collect on edX, along with the partner universities to research how we can do education better. So think of edX with the first course that we taught. You wouldn't believe it, but we had quarter of a billion data records. We track every mouse click, we track every piece of data. And Google does it, but in Google's case, they're doing it to maximize profits and eyeballs and how do we get more revenues. But edX, we use the same kind of data and we gather everything to improve learning. And I like to call edX the particle accelerator for learning just for that reason. So I'll give you one example. So one question we've always asked, how long should videos be? Should videos be one hour long like a lectures? Or heavens forbid an hour and 30 minutes like some lectures? Or should there be one minute long? I mean, who knows where the right number is? So Philip Goore did this study at looking at all the video links over a number of courses, a number of subjects, a number of universities at edX. And here was his finding. And again, all of this data was mined. This is the big data for learning. And he plotted on the y-axis the amount of time a student spent on a video. And on the x-axis, he plotted the length of the video. And from three minutes to six minutes to nine minutes to one hour, the length of the videos. And here, when he plotted the engagement, how long did the student watch the video? And what he found was six minutes is the ideal video length. Students watched a six minute video for almost six, for six minutes. But once he got to a video that was an hour long, students watched it for barely three minutes. So this really goes to show that after you show something to a student for about six minutes, it behooves you to really have some exercise. And what's interesting is the red curve is all the students, hundreds of thousands of students have signed up for the course. But then I told the researcher, yeah, but a lot of the students were not really interested. What about those that got a certificate? Maybe those that got a certificate and passed the course really watched the videos completely and they really engaged with the videos. But the blue curve is for the students that earned certificates. The same trend applies. Look at this. Even those that got certificates watched it for six minutes. And then they didn't watch the longer videos for much longer than the others did. In fact, they watched one hour videos for three minutes, they watched it for four minutes. So 33% improvement. But still four minutes. And so this is just one example of the kind of results we can get. And our hope is that, as we collect this huge amount of data, we can mine the data for good. We can use this data, unlike a for profit, we can use this to really study and understand how people learn, how students learn. And in a very short amount of time, maybe discover really new ways in which people learn. And really improve learning and on the world. Thank you. I'll formulate the one, let me ask one or two to get us started. Update us on one thing. Some of the early reports I saw indicated that of the completers, or those who are earning the certificates, degrees, or at least passing the final, they tended to be people who are already well educated. These were people pursuing continuing education, let's say, to a greater extent. Has that begun to change as edX has continued and grown? Because for those of us who were thrilled at the idea of a democratization of education, it looked as though at least early on, what you had was principally people who were already ahead of the education game getting further ahead. So let me describe some of the statistics and describe the rationale. So right now on edX, if you look at the statistics for the course that I showed you, the circuits course, 5% of the learners were younger than 18. So 5% were high schoolers and our youngest learner is eight years old. So high schoolers and elementary schoolers. That's 5%. 45% are between the ages of 18 and 25. And so you could think of them as college age or master students who haven't really completed a degree. So 45% are in that category. And 50% are above the age of 25. And so you can say that, so what we've seen is around two thirds of our students already have a degree. But one third are in college or younger. So we have two million learners and so a third of two million are college age or high school age. And so that's a big number. But that said, two thirds, a significant fraction, two thirds already have a degree. Now, is that a good thing or a bad thing? Certainly in terms of democratizing education, I would like to see more students who do not necessarily have a degree who are taking this to learn stuff. And there's really, I think, two impediments to that. One is just simply our fault. So I've talked to many students who are taking edX courses and I talked to a student at UT Austin. And he took two edX courses and he did not get a certificate in either course. So I asked him, what did you get a certificate? You know, what happened to you? He said, well, you know, first of all, your courses are offered in spring and fall. So our universities are our partners and our university partners and professors are used to creating courses in spring and fall. And so that's what we did, spring and fall courses. And when the semester was running in spring and fall, students are already taking it at Purdue. I took a poll in the class I was in. Several students that I talked to were taking six courses already. And so the student at UT Austin was taking six courses too. And he said I took on two more edX courses and when the midterm, so his campus courses began, he said he just couldn't keep up with the two extra edX courses and so he stopped out of those two courses. And so many of the students who are, who don't have a degree, they're already learning and studying and they can't take on even more than they are doing today. And so how are we fixing that? Well, we're encouraging our university partners to offer courses in summer. So that's one example. The second is that we give certificates but we need to, universities need to come around to saying we need to give some campus credit for students who have taken a MOOC course. That's something for you to think about is let's say for example, Purdue offers an online course and if you have a student in high school who's taken a EE290, a digital design course on from Purdue, offered by a Purdue instructor online and they've aced the course and they come to Purdue and they come to campus in the second year and they come and tell you, look I've already aced this course. Are you gonna make me take the same course again? But can you give me credit for the course I've taken online? And today universities have to answer these hard questions. MIT has to answer this almost immediately because the Batushig, who's a student, 15 year student from Mongolia, he came, he took an edX course, he took my course, he aced it, he got 100 in this hard course and he applied to MIT and he got into MIT at 15 and his acing the course had something to do with it, I think. And so he's on MIT. So next year he'll have to take my course as a required course and I can see him coming to me and saying, do you really want me to take this course again? I've aced it? And we will have to answer these hard questions. What do we do about credit? I love those stories and by now you must have a number of them where MIT, maybe some of your other partners are finding students, using this as a way to find students, I read about an Indian student, I believe in some fairly small community that would never have found his way to MIT before, but MIT found him, I think, because of the success he added. There are a lot of amazing stories like this where students, whether they're young kids in high school who are able to get to a university that couldn't otherwise have. And many universities, in Australia, for example, are using edX, not only to think about education, how to improve campus education, but also as a way of brand promotion to propagate the brand around the world and to seek out great students who are taking their courses and who do really well in their courses, seek them out and get them to come on campus. And so that is one of the, something we haven't planned for, but many universities are thinking about it in that manner. And we're seeing a lot of stories of students that take these MOOC courses that like the professor, like the way the thing is taught, they find a match and they go to the university where that course is from. There was a lot of speculation and you've experimented, I believe, with linking directly with employers to certificate students who pass a course of direct relevance to a given business or industry. And I think one of your, you've recently discontinued one of your adventures in that way, but it's still an intriguing concept that using your technology and your approach, you could prove to the full confidence of a potential employer someone's readiness. Absolutely, I think the students take these courses and I'll tell you the story of one student. He's from New York and that's a true story. So I won't tell you the name of the student or the company. And so this student took the software as a service course offered by Professor Armando Fox from Berkeley and he got a certificate in the software as a service course and then he posted that certificate on his LinkedIn profile, he just posted it there. He got an interview with, he got an interview with a company in New York. It's a huge media company. You can, a gigantic media company and he got an interview with that company and a week later he got the job on the strength of that course and now he's working at that company and there are many, many stories of students who are doing this kind of thing. We just, at edX we had an experiment that Mitch alluded to having to do with, so we said, oh, this is a good idea. Maybe it's a revenue model for edX. Let's connect students who do well in the course to employers and we discontinued the pilot because it did not work out too well because we were working with HR departments and HR departments are used to looking for degrees in traditional credentials and there's no way of measuring and judging non-traditional credentials and certificates. Heck, they haven't even heard of MOOCs and so we felt that it was gonna be a long road as we educated people about the value of these things and so we said, this wasn't a short-term thing. We would have to turn ourselves into an HR company and so we said, let's stick to our knitting and we discontinued the pilot. Audience questions, please. So the question in case you didn't hear it was, in the blended model, students are on campus and they work to discuss things with each other in the classroom and the question is, can you do that online by having students do the same thing in a virtual online chat room? And the answer is yes and Google was building their own online platform when edX open sourced. They've now adopted the open edX platform and they're collaborating with us. So it's part of that. How many of you, let's see a show of hands, how many of you have used Google Hangouts? So Google Hangouts is a virtual chat room with video and so on. And so Google integrated instant hangouts into edX. So if you go into, we have a demo course on edX I encourage you to go to edX.org, sign up and check out the demo 101 course and as part of that, at one point in the course there's an instant hangout. You click on the instant hangout button and boom, you're thrown into a virtual chat room where you can have a chat session and discuss something with a few of the students. So good idea and it's something we are experimenting with. The question was in the past quarter alone, quarter of a billion dollars have been invested in education and so what's your sense, why is this so and so on? So first of all, I think it's fantastic. When's the last time you've seen for profit industry putting money into education? Education used to be a lost leader and I've been quoted as saying, we celebrate tall men and women that can take this spherical object and dunk it in another spherical device and we turn them into heroes and pay them 20 million bucks and so on. Why can't we celebrate teachers? And so now money is going into education and education is hard and it's fantastic. I think it's a good thing. And so the question was, why are you set up as a non-profit? I really think that the for profit has really picked it up and they're smelling money and it's easy to make money in education. If Purdue wanted to make tons of money, it's not hard to make money but there's a mission. There's a mission aspect to it which is education is a basic human right and to do it right and the mission aspect, it really has to be a non-profit and we felt that what we're doing, it's transformative, it's gonna be disruptive and we felt very strongly and despite my own for profit roots, strong for profit roots and I come from a strong business community in India that making money is like religion but we felt that non-profit was the way to go because this is really, really disruptive and we wanted to be sure that our board and us, we made decisions based on principle not profit and I can give you example after example after example of decisions we've made that I would have made differently had we been a for profit. I'll give you an example. Here into edX we made a platform open source which means we gave away a platform for free and giving things away from free is not something for profits do. We are the only MOOC provider that has given away a platform for free because it's the right thing to do so we do things based on it is right, not that it's gonna make a lot of money and that's just one example of a decision and I can give you a number of decisions. Another example where we really asked the universities to create rigorous courses that are matched the rigor of campus courses. Now that lowers pass rates. That's not good for MOOCs. So if you look at the for profit providers they will be talking to universities about dumbing down the courses, about watering down the courses, make it easier upping the pass rate but edX we said no. We want, we focusing on quality education. Forget the eyeballs, let's be getting the eyeballs nonetheless but let's focus on quality and eventually we will get there but let's do the right thing. So I can give you a decision upon decision that are made differently as a non-profit and it's not surprising that in the US virtually every university is a non-profit and the two or three universities, I won't name them, that are for profit, I'll let you think about the reputation. Can someone name a for profit university in the US? Phoenix University, you said it. Yeah. So now let me pursue that question one short step further. There was a very interesting pair of announcements, I think maybe on the same day or at least in proximity recently in which you folks hired a well-regarded woman whose background is in for profit, very successful leader of a for profit entity and on the same day I believe, Coursera, maybe the most prominent for profit MOOC entrant hired Richard Levin, the former president of Yale University. So there's a little blending going on there too or how would you describe that? Hey, I think it is quite interesting and in our speak for edX where we are a non-profit and so we're a university's trust us, I think we're a non-profit, I think we are very similar to universities and we're not viewed as a threat, we are very collaborative, we're not VC backed, we have an academic board for better or for worse. And so it's run by academics, although in my case the press and others view me as an academic, a little do they know I'm a serial entrepreneur, but that said we're viewed as academic. So we brought in Bindi Sabula as president and CEO of edX and I'm the CEO of edX and the reason is that she's gonna run internal execution and we really believe in very efficient execution and we want to be sustainable and we really use very good business practices and so that's where we brought in Wendy and I'll let you think about why a for profit had to bring in a academic to run the organization. I'll let you think about why they had to do that. Couple, we have time just for one or two more, Andy. There have been studies and edX, we are interested in doing studies ourselves and many, many, many studies have already been done, I'm not answering the question yet, comparing the blended to the pure on-campus version, many, many studies. And the reason we have many more studies of the blended is that on campus it is natural to offer a blended course and waiting for a campus to offer a pure online course on campus so we can compare it to the blended version because frankly I do believe that a purely online version is only gonna be slightly worse than the blended version while the traditional campus course is gonna be significantly poorer in quality to the blended version. However, no campus has done that and frankly part of it is I think of fear that if as a campus, first of all as a campus the academic committees don't allow a purely online course to say, oh if it's a purely online course so what are we doing as a university? And so it's been hard to get that experiment run and second is maybe we're worried about what the result is gonna look like. But that said, you know, your president just told me earlier that Purdue has run such an experiment with a on-campus traditional course, a blended course and an online course and they have the results and I'm looking forward to seeing that study and so you have done it yourselves. And so I mean again, you know I'm really delighted and excited coming here as to how far ahead you are in terms of the thinking in education. So edX and none of our partners have done that to my knowledge, but you have the results and so I'm looking forward to seeing the results. So next time I give a talk I'm gonna quote Purdue results if I can. It was interesting, Andy, in addition was interested in wondering how longitudinally I think you were graduates of your courses due in later years. Well they've only been around such a short time and it's gonna be a while, but I will point out that the Gallup Purdue Index which is in the field right now will produce exactly that sort of data and it may be a couple of three years before his graduates are out measurable but it will be fascinating to know. There was another question right behind Andy and then we'll, yes ma'am. Oh, Gabriel. You know, I planted this question in the audience. No, I didn't. But absolutely. But that's an obvious next question which is, you know, we have all this data. We have courses using all methodologies. So I encourage you to go to edX and you'll see some professors who tape themselves, you know, lecturing on a blackboard. Some of them, some people use Khan style videos. How many of you have looked at Khan Academy? Fantastic. So Sal Khan was my student and I've been inspired by him and the way he does videos with tablet capture, I call them Khan style KSVs or Khan style videos. And there are many, many ways of capturing video. And so natural question to ask is, you know, you have all of this data, all this big data and you know how long students have engaged with the material so you can check which one works. So I'm almost wondering whether I should give you the result but there's a paper. So on edX what we did again is a non-profit, believe it or not, we have a link on our, we have a portal on our site called Research and Pedagogy and there people post papers on research that they've done using MOOC data. They think as a non-profit we do crazy things like this, you know, post research results. When's the last time you've seen a for-profit do something like that? So we have that. So I would encourage you to go to edX, go to the Research and Pedagogy page and in there there's a paper by several authors, I forget the title, answering exactly the question you asked. They looked at all different types of videos and to look at which are the ones that are most engaging. I'll tell you the result. They actually found that the ones that recorded the professor teaching the way back in the classroom are not the least popular. The second ones are a professor recorded close-up in a informal setting. The second most popular and the most popular are what I call the Khan style, you know, very informal, most scribbling on a piece of paper, the most popular. And the reason is students feel as if the professor sitting next to them teaching on a piece of paper, much like a parent teaches a child. Very good. Last word. We're lucky to have with us tonight the Chancellor of Indiana's Community College, single community college and all its assistant, Tom Snyder's here and the Chancellor of WGU Indiana, Indiana's eight state university, I always said, fully online, competency-based, very innovative education, primarily aimed at older students. So please, the last question. The question is, what excites me about the future and what's next in the space? I think what excites me about the future is that I've absolutely no idea what the future is gonna look like and that's why it's exciting. Now, once you know what it's gonna look like, it's boring. But imagine going to a film where you know what the ending is, it's not fun. But in this case, frankly, now I've been teaching for 26 years and I have no idea what the future is gonna look like. I think the university, as we know it, is gonna transform itself, it's gonna be different and I think some of the classrooms I saw today, presages what classrooms are gonna look like. I've written in the Huffington Post about university unbundled, where we need to figure out, anything we do in university has to pass, has to pass President Daniels's pajama test. Although, why do it? And so we need, and what are those things? We don't know the answer. So I think the most exciting part of the future is the fact that we have no idea what the future is gonna look like. What's next for edX? I think we are focusing on, I think one of the things we're working pretty hard on is how do we find a way to reward students? How do we work with universities and credentialing agencies to give them something meaningful of value so they'll be motivated and take the test? And you mentioned competency-based. I think that is a big one and I think we are collaborating with the college board and to see whether we can create exams or we can create courses but have some other agency create competency tests so that you can now take a full MOOC course and then go and take a competency test by giving by somebody. Heck, edX could give a competency test. Although that's not the business we are in but ideally we'd like to have someone like college board or someone such as yourself give a competency test and students can learn anywhere they want but go take a competency test and the credential they get from there is the valued credential. So that might be one thing that would be very interesting going into the future. Dr. Agrawal, you are at once a brilliant person and an idealistic person but on top of that you are highly practical and results oriented person and it's rare, almost singular to find that in one package. So it's been a great privilege to have you here at Purdue. We see you as both a teacher but also a very healthy competitor and a prod to our own action and we thank you for all of that. Thanks for being with us. Thank you for having me. Thank you.