 Hi, can you still say good morning? Good morning. Great, so it was a really exciting morning today listening to both Tim Shutton and Martha Cantor. And I'm going to whisk through some of the first slides that I've prepared because I know you folks are an audience that really gets this. Just a brief introduction of who we are to set the context. Open Study, that is the company I'm going to talk about, there's a small startup that was funded, that was started by two educators myself. I'm the Dean for Science at chemistry by training at every university and my husband, Ashwin, who's in computer science and cognitive science at Georgia Tech. And it's one of those things that come home groaning and moaning about teaching and then he says, technology is the answer, so that's kind of how Open Study got created. Our third co-founder was a student at Stanford, and he came into his master's with Ashwin. So we're all, it's all a family kind of thing right now. So that's who we are and I'll talk a little bit more about how we were funded when this slide comes. So the problem that we set out to address, which is very familiar to people in this audience is one of Apsis, you're never going to have enough universities, bricks and mortar to satisfy the needs of the millions of young people that are looking for education. So that's a really large problem and I'm so glad that we have in the room at conferences like this people such as yourselves that are creating the content. And we're talking now about the long tail of education you have at the head, the MITs for real and the Yale's and the Harvard's, and at the tail you have the OCW providers but also in this group you have the Khan Academies who are doing really well, our neighbors in Mountain View, the iTunes and all these folks that are not really challenging education and disrupting it in a fundamental way, really happy this problem is being addressed. The problem that we're looking at is one of the learning experience. And as educators in the room you all know, no matter how good your textbook it's never good enough. I can't think of a single engineer who read 15 textbooks and managed to go build a bridge just on that. So you are going to need help. You are going to need interaction, engagement and then the problem of retention which again Marga can't talk about today. This is where we are at. How can we construct that wonderful learning experience so that while you folks create the online universities and charter schools and online content, your students stay engaged, retained, motivated and succeed in graduating. So this is a no-brainer for anyone. Ask yourself, what are they doing now? What engages them? What are your students, your digital millennials engaged in? 24-7 sometimes? Absolutely. So there is a whole world of digital things out there. So as educators we know where they are spending their time and we know what's normally learning. So that's one thing. This is the other one which has come into a lot of attention thanks to President Obama as our quote. We know they are spending a lot of time on games. So how can we then address this challenge? How do we take the knowledge that we have that this is what motivates our kids? This is what keeps them engaged? This is where they are really far up and passionate about but then move them to a learning environment. And so that's the challenge that we set out to address. And so the answer for us was very simple. It has to be something that they can relate to. Something that looks and feels like a world of Warcraft, that looks and feels like Facebook, maybe even Twitter, something that they feel comfortable with because these are native natives. I mean these people have grown up thinking Facebook and thinking IMs. They have a different way of thumbs I believe. I can't do what my daughter does on that. So how do we do that? Our answer was open study. Open study, so there's my funding page. It grew out of research that I was doing with my husband and Chris Gray on collaborative learning environments, problem-based learning. And what we noticed was, what was really addictive for these kids was the ability to get together and collaborate with one another online. I mean you could put them in the room, but they'd rather talk to each other online. Great, that was a wonderful lesson there. And so we took all we had from that research and our program officer at that time said, go create a startup or a dot-com and we'll fund you through SVIR funding. So NSF, wonderful people have funded us for two rounds, NIH, Georgia Research Alliance. And that's because our work is really based on a lot of research, which I don't have on the slide here to talk to you about the learning experience. Okay, I hope you heard my plug for OCW this morning because what we do is the study groups we create wrap around content. We are content agnostic. Anybody that has any kind of content, whether it's OCW or OER, can access the learning community that we put. And the learning community of students and a number of folks as well, is focused on study. So the minute you log on to open study, you will see topics. So you know where I drive us, not about the bar you went to or the girl you're dating, but it's math, physics, computer science and things like that. We're pretty STEM heavy because of our funding. And then you'll also see right off the bat how many people are there. And this is really important for them because they want to go where the party is. They want to know that there's people out there and how do we integrate. So here's your MIT OCW site. And when you went to MIT OCW, they get, right now I think they're getting 12 million skaters in the room anywhere. They get 12 million hits. And their most popular course is MIT 6.0 Intro to CS figures. So they have, oh, I think about 40,000 people a month in there. But if you were taking the MIT 6.0 course by yourself and you had a question, the fact that there are 40,000 people also taking it with you is totally useless. Because if you raised your hand and said, I really want help with assignment 4.0 and here's this question I can't understand, who will you ask? So when we went to MIT and said, we'll build that community around so that when you ask a question, there'll be someone to answer it. That's what sold them. So we integrate with our partners in a manner. So there's that little get live help button you see, that's us. It gives a live feed of the questions that are being asked right there in that particular study group. Once again, it's one of those things our students want to see. They want to see action. So that's the action. And while you're there, you can click right into it and that'll then take you into open study. And for our partners, it's actually a good win-win situation because now whatever happens on our site, and you'll see that in a minute, is on our site. It's not on MIT site. So there's a little separation of where MIT and here's open study, which is a resource we provide. So what does it actually look like when you go into open study? This is just the feed. We have partners with all over people here in the U.S. and across the world. And we also partner with OCW and OER folks that are people that are just creating content. So this is what we call open social learning. It's open in the sense anybody, anytime, anywhere, any institution can get online. It's free. Thanks to our funding from the various organizations, and most recently Gates and Hewlett. And it's learning. It is about learning. The questions are about learning. The discussions are about learning. So when you look at this, for example, there's questions being asked on the left and on the right are a stream of groups that you could belong to. I currently belong to the MIT 6.0 chemistry and a couple of others and it's shown at the bottom. And you can see for every group you belong to, you can see who's there. The large squares are people that have been contributing a lot. It's like a leaderboard. And that's a very seductive concept for these folks. People are incentivized to grow from the little squares to the big rectangles, because now they're the leaderboard. You also see little numbers and you also, if you mouse over them, you'll see little interesting pieces of information, like what are the little profile, what groups are they following, how many levels have they achieved, how many badges and things like that. I'll come to that in just a second. The green bars at the bottom are chats. So these are off-topic chats, usually. While the on-topic chats are up here and they're really very focused, off-topic chats can be anything and that's kind of a social release. There's all kinds of exciting things quick in there. They're 15 to 24-year-olds. You can figure what kind of chats they're going to have. I'm talking about moderators in a minute. But the other thing that we have is these chats, for example, where you light up when someone's there. So while you're sitting there and you're answering a question about, I don't know, intro to Python and you see this little thing turned green to yellow, you go right there to see what's happening. So we again, we flesh out this notion of immediacy and presence and the fact that while you're studying, you can also chat but stay here because there's more studying to be done. So I just posted a question at some point so you can take a screenshot for your folks on Python, what text book can I use and right off the bat, three people came to answer that question and we see that a lot and we also see that this person was actually answered and right below him is someone that's in the process of answering. So if it was one of your learners and they had a question there that they were asking, they would then stick around to see what was being said. Of course, they might get distracted but they'd certainly come back to see what was being said and the value of this is that it's not a post and pre sort of a scenario where you post a question and you have to come back tomorrow and see if there's an answer. There's a just in time kind of a help and second, it's not just which of the following is true, that a yes or no kind of a thing where you get an answer and you go away. There's a dialogue. Because there's people in there, we get this give and take and quite often you get a wrong answer then the next guy will come along and say, by the way, I think this is not right and then the next guy will respond and you get a real rich dialogue and we think that conversation is very, very important. Even if there's a wrong answer, as an educator, we believe there's a value to having that conversation because the longer you spend thinking about these things, articulating their ideas, evaluating it if it's right or wrong, the greater gains for learning. So we're really excited that we get multiple people going right in. In fact, we sort of laughingly call this the piranha effect. So I wanna post a question. People dive in. I'm gonna be able to answer that because of our reward system. So what does our reward system look like? We've really worked hard on a couple of things. The first is we know that kids want just in time help. So let's provide that by building a community and having enough people in there so you come to ask for help. But the second part of it is we want them to stay and stick around and engage and then we wanna incent them with the right rewards for the right behavior. So our game-like mechanisms allows us to do that. If you're in World of Warcraft, how many of you have kids or who have, why should I say kids? How many of you have encountered World of Warcraft? Either by paying for it or having kids that do this. But you know that when you're in there, it's actually a pretty good system. And when you're in there, you are in quests together. You look for trophies. You do things and you socialize. Well, our team that's built this is a very young World of Warcraft sort of a team. So they've built some of these ideas and you'll see you get medals for helpfulness. You get rewards for answering questions. Which also get rewards for asking questions because it's really important to articulate a question. And then you get rewards for bio-peer by being recognized as a generally good person by accumulating fans. A fan is the highest compliment your peer group can give you because you can only become a fan once. So we really worked hard on building the kinds of rewards so that we can get the kind of behavior that we would want to see in our young people. And that's some of the gamification. We've got good feedback. One day just when I was born and I had nothing to do, I went and I counted the number of times people had used the word addictive. It was 82 times when it was used in the last couple of weeks or so. And that's exciting. Can you imagine a kid going back and saying I found a math quite exciting or addictive? I think that's mind-blowing. To me, I teach chemistry to freshmen year after year. And to get that kind of engagement with the topic, I think it's a great way. So there we are. This is our global community. Again, following on what Martha said, I changed my slide to say, we're building world peace because we have 70,000 people from 170 countries that are getting together, talking about stuff in a meaningful way if they don't behave to get kicked out by the moderators. So they have to behave. They learn to behave. We get 180,000 uniques per month. Average users, 100. Users online every now and then and we get a... I know someone will ask this. Of the questions that are asked, about 80% of them get answered. 70% are actually answered within five minutes. So it gives you that just-in-time feel. We go back and keep asking our users with surveys, how is it, why you're doing what you're doing. So we know all kinds of cute things. The one that I like is 84% agree that Open Study is fun. And why is that important to us? We think it should be fun so that they can keep coming back. You can learn on the learning, but if it's not fun and if they're not there, you can't do anything. 70% of them also said that they get the help they want, which is good, which means they'll keep coming back. And of course, they're recommended to their friends. We also listen to them and we ask them, what are you looking for? And some of them want to, and we ask them, are you coming here for homework? Because, you know, that's a big thing. Are they cheating on their homework? It turns out that a bunch of them are more than the homework group. There are people that are coming because they're on online courses. This is what we had predicted. And also, people that are using it to learn better or have a deeper understanding. And then this is the part, this is the one question I was, I said, oh, interesting. We asked them, do you want to connect with people you know or people you don't know? Do you like to find other people from your class and this and that? And 36% of them said, they really want to connect with people you don't know. So go figure, this is that generation that's bold and out there and really wants to grow their social circle in what we would think in a very high risk way, but it's great. So that's the value you probably bring to them. Just in time, fun and engaging, being like, and it helps them grow their peer growth. I've got stuff about demographics as you'd expect. Most of them are male. 60% of them are full-time students and 70% of them are from the US. So we've got the engagement. We've got the interaction. And the next thing we're looking at now is retention. And this is in the environment and the ecosystem right now. Arne Duncan and OP2PU and Bosila Foundation and we went back and talked to the teachers and our students and when we talked to some of our OCW learners that are on our site we actually found that 44% of them said they would love some kind of recognition for the amount of time they're spending in there. So we're working on badges or certificates, whatever it's going to be called, we don't know yet. But the idea would be here are people that are spending months on our system demonstrating to us that they are helpful, that they articulate, that they can demonstrate subject matter expertise. How can we find a way to recognize that and their engagement with the community as well as with the topic? Whatever form this is going to take it'll be a joint initiative with our OCW and OER partners and ourselves. So if you're interested in talking to me about that I'd love for us to do that. And I wanted to leave plenty of time for questions because I know this is always ready for questions. So, I thank you for your time. Subject matter is mostly STEM based or, you know, science math. But does your site support mathematical notation? Yes. Or and how does it can you talk about a little? Yes, yes. We have a screen shot but yeah we've got a letec style equation editor that's inside. When you you know can't see that there. But if you go down there it allows you to do absolutely and put in equations of the whatever you want to of fancy you want to do. So yeah, absolutely. And we while you're focused on STEM we have writing for example they're partnered with Purdue's online writing lab we have arts they're partnered with ProcessArts which is in UK so we do have that but the bulk of our what's really hard is math for most people so we get a lot of research from there. I think there was a yes. This is really exciting I have no idea about it. I'm curious I mean you're such a large user base seems like people are finding it how do you how do you make that happen? That's a huge A partnerships work you know we get 30% of our traffic from partnerships so people go to MIT and say oh I want to see this particular aspect of Python and then they see our link and they come to us 30% of it just comes from Google searches because each and every question that's ever asked and I can address this is archived and it's searchable so when you go to Google you ask say what's Taylor's series or something like that and open study will be one of the heads so we get 30% of our traffic just for that we don't do anything fancy for a CEO or anything like that and then the rest of it comes from things like stumble upon or people just knowing who they are so 60% comes from these two sources yes so this is content like technostics yes it's actually possible for somebody else to take their content and place it inside your system right right absolutely so if you wish to partner with us what we would do is we would just give you a JavaScript service page if you go to our website and see a partner with us it's three lines of code and when you put that in your HTML pages then that pops up this widget and it links it to the right study group the topic study group and open study so it's a sub-service yes and then who does the you mentioned moderation so who does that and how much work good I'm glad you asked so moderation is very important for us because of the age group actually the demographics what we decided to do was if we were to really scale we can't hire moderators enough or teachers for that matter so we select people from the community these are people in the community who have proven themselves as subject matter experts over time and who also have proven themselves to have the right attributes of personality and everything so that they can be chosen and then we train them and we ask them you know would you like to do this so they are allowed and because we go we have moderators 24-7 from different countries and they have the ability to do two very important things one is to remove delete inaccurate contents and also inappropriate content so they're probably around but more than that you know when you build a good community you know it's successful when the community itself reaches out and says this is wrong you shouldn't be doing this so along with the moderators we now have a very strong sense of what's right and what's not right on the system and and it's marvelous so so we do have moderators but the community is doing it themselves yes I have kind of a a two part question my first is kind of a question about the cheating that you mentioned and the format of moderation I I love the like user interface on defense study and when I logged on I found it very addictive but I I got really frustrated right away because it seemed to me like all the questions I saw initially it just screamed cheating to me it seemed like people asking for you know answers to their homework um and so I guess I'm curious like how you do with that and if you've tweaked the format at all um around that or kind of what the progression around that is so I'm going to have to go back and ask just because it's homework is it cheating uh I mean I don't know that if it's someone asking a question getting someone else to do the work for them and then representing it as their own okay so we are we are both educators for good institutions we have academic integrity um built into our terms of agreement if for any reason it really is cheating um a couple of things happen as I said the community itself and well it will come and report it to us if it goes beyond that and an instructor has some doubt that it is cheating then they come to us um our strongest or our strength isn't the fact that everything is archived everything is transparent should there be any doubt in your mind if you go to an individual's um site you can see all their questions all their answers and there's no they can't delete it so so that's if anyone any time can go and see that so we believe that's our our strength and we approach it that way over and about that um institutions have different policies on academic and integrity and we wouldn't really be able to police it for everybody but we take it very seriously and if the community or a moderator or a teacher were to come and report this to us we would take that but go back and think about my first question just because it's I think it makes a lot of sense because I think you could easily say back to me well people could do that offline absolutely but there's a tool there's a way to misuse it so um and can I just the second part of my question was if you have any plans especially along with the people who look at outcomes around like how people who are participating in the study groups like if they're participation in the um in the courses that they're in changes or if they are if they're um absolutely we have great plans this is the challenge in devising a gold standard study as you'll appreciate and sympathize with me is that we don't work with teachers in that sense so we have 70,000 students and 16,000 schools so why would you love to do this gold standard where and any volunteers to work with me on this we'd love that but we'd ideally like to take a cohort on a say an online course with open study and without open study run the experiment through so it's been a challenge coming up with that but over and about that what we can do is do like a living-life study and when we ask them um how many of you can report and this is self-reported learning gains which Elaine Seymour has been you know a proponent of how many of you have experienced an improved learning outcome or a learning gain so we have was it somewhere 68% of them actually report an improvement in their learning gains and that's pretty good you know this is a study that Georgia Tech does for us on our the job students as an experiment so any of you educators that want to do any experiments and you know our data is fairly available and we work with all kinds of institutions for that I think I've got the one minute so a really quick minute I take the lady in the front do you consider having one that if a question can't get answered if a question can't get answered for those 20% of questions on the state don't get answered I think it's an interesting idea if we we would probably call them something different but yeah I think we could certainly work on that right we could directly to an expert is what you're saying I think that we are certainly do that have a minimal fee it's a free service but we can talk but thank you so much