 Dyma yn ymddych chi i fynd a rwy'n edrych yn arwain yn Dablin. Wrth gwrth, Arizona Siaradau Unifersodyd yn ymddych chi'n bwysig beth oeddaeth i'r bwysig sy'n cymryd i'r hynny'n gweithio'r bydd ymwysig o'r unrhyw hwnnw. Felly dyma'r clwb yma. Rwy'n dod i'r gweithio'r unrhyw ar ynno. Dyma'r hwnnw. The largest university in the UK, the 14th largest in the world, 180,000 students, 1,100 academic staff, is important to say that right from the start we've been a research active university and we do as you know research into online learning but also research in many other areas. You've probably heard about the Rosetta space mission that landed on the comet, there was equipment there that was designed by the open university space sciences group so we do a wide range of research and our approach is supported open learning so all of the learners who take degree courses at the open university get a local tutor who in small groups they can work together with that local tutor either online with them or face-to-face meetings with them so it's the approach of not just learning online but also having tutorial support and there are 7,000 associate lecturers, tutors around the country and some of them abroad. But about two and a half years ago we were posed the question can we open learning at even more massive scale. And when notes came along the OU was somewhat complacent about this because we were doing that already. We've got the relationship with the BBC where there's 250 million views a year of open university branded BBC programs in conjunction with online materials and then leading on to OU courses. 66 million downloads from iTunes U and we have our own open learning site called Open Learn with 5 million registered learners. So when MOOCs came along we were pretty complacent that we're doing that already and the UK Minister for Higher Education went off to Stanford and came back and immediately got off the plane, got on the phone to Martin Bean the Vice Chancellor and said okay you may be doing it but you're not doing it the same way as they are. You haven't got this thing called MOOCs and so the Vice Chancellor was stung into action and to his credit he immediately set up an operation which was originally called Project Kylo for obscure reasons a Kylo was a sort of highland cattle as in MOOCs. And Project Kylo morphed into this thing called Future Learn which is a separate company so it was set up as a for profit company by the Open University but with the profits being reinvested back into the company. For very good reasons the Open University has got a lot of strengths but being agile isn't one of them and it would have taken a long time to within the OU to set up that operation so he's set it up as a separate company. And within six months of being set up we had staff and we had a platform and we were starting to develop our first courses so that was quite an achievement. So that's where it is now so Future Learn now has 130 courses from 51 partners with a broad range of business, health, science and arts so it's not science focus to some of the other platforms. We developed from scratch a new online responsive platform so that it provides access not just on desktop but also on tablets and on mobile phones and 25% of people who access Future Learn courses now do it on mobile devices. One of the big debates we had early on was should we put Future Learn on Moodle because Moodle was the Open University platform to the IT department in the Open University Moodle was the obvious thing. We were the biggest user of Moodle in the world so you just extend Moodle for Future Learn courses. And there was a great debate about whether we should, it didn't seem sensible to IT in the university that we would go off and build another platform but there were a number of reasons to do it. One is that Moodle just doesn't scale up to million or more people and Future Learn now has just over a million registered learners and it just doesn't scale. With Future Learn the platform we've built as we get more people we just spin up more servers. It's a cloud based platform in the way that Moodle wasn't. And it's responsive that it was built from the start for mobile access and particularly, and this is what I'm going to be talking about, it was designed around a different pedagogy, a different approach to teaching and learning. And the other thing to say is that we're particularly pleased about the demographics that 60% female learners on the Future Learn platform which is again different demographics to some of the other US platforms. Those are the partners currently in the UK and those are the overseas partners so we now have 19 academic partners outside the UK and that's growing rapidly. There's been a new course just started this week from Yonsei University on Korean and Chinese history which is really excellent from an American who's based in Yonsei University and is a kind of star academic and he has a really compelling history of China and Korea. So it's become an international operation. And the cultural partners Future Learn is based in the British Library which is a great place to work. Twice a week I go into that beautiful building and then the British Council who put on the most successful course so far with 120,000 learners on exploring English and the British Museum. So those are some of the cultural partners of Future Learn. So what I want to talk about primarily is how do we design successful massive open courses. So if you go back two years ago we had this huge opportunity and as a learning technologist it's a kind of once in a lifetime opportunity. If you've been for the last 30 years as I have designing technology that if you're lucky gets used by a class of 30 children or possibly 100 people in the museum. The opportunity to design a new platform for a million people. It's a kind of learning technologist's dream. But then the reality hits you which is okay, so how do you design it? What do you base it on? And that was what we were confronted with about two years ago. So these were the sort of backgrounds. We knew it probably wasn't going to be Moodle. There was the initial C MOOCs, the connectivist MOOCs and then the X MOOCs. So the MOOCs based on instructional design and delivery of video material. We wanted to develop a design that was informed by the best theory and practice of pedagogy of teaching learning and assessment. But it had to work at massive scale. It had to be scalable up to a million or more people learning online. So you couldn't do the open university supported open learning. You couldn't give 20 people a personal tutor. There was a team from BBC Digital. So that's the other aspect of Future Learn which is that the CEO of Future Learn came from the BBC and he brought understandably a number of his BBC friends with him. So there was this BBC culture. So there was an interesting sort of clash of cultures between the OU Open Access and the BBC particularly in terms of the production value. The BBC is committed to the highest quality production value. Now the OU's teaching materials are pretty good quality but we weren't making BBC TV programmes. So there was this and still is this interesting tension and it had to be implemented in six months. So we had to not just design a platform but we actually had to build it and get the courses started within six months. So it was quite a challenge. So where'd you start? Well, here's one starting point. You may know of the work of John Hattie who produced an amazing achievement. It's a synthesis of over 800 meta-analyses related to achievement in learning. And if you haven't read his book Visible Learning, I really do suggest you look at it. So he did a survey, not just of 800 studies of learning but 800 meta-studies of learning and looked at what is it that makes learning successful. Mainly at school level, things like class sizes for example or the charisma of the teacher but also different pedagogies. What he concluded and why it's called Visible Learning is that one of the main aspects that make for successful learning is making the learning visible. What's most important is that teaching is visible to the student and that learning is visible to the teacher. The more the student becomes a teacher and the more the teacher becomes a learner, the more successful are the outcomes. So that was one basic principle to try and make the learning visible both to the educator and to the learner. Then there's this paper in 2000 in the journal magazine Science called Foundations for a New Science of Learning. And what that seminal paper concluded was a key component is the role of the social in learning. What makes social interaction such a powerful catalyst for learning? That paper ended with a question, what is it that makes social such a powerful catalyst for learning? And so these were some of the principles that we started with when we were developing Future Learn. Visible learning and goal setting. So making clear the educator's goals, so what is it that the course is about? What's the big question that the course is trying to address? And also help the learners to make clear their goals. Provide reputation and reward so that as you go through the course you can see how well you're doing and get rewarded for that. Collaborative learning enabling people to work along with other people and to see the other people who are collaborators who are co-learners. And to try and get away from online learning being a rather lonely experience. So, again, if you go on to some of the other platforms, you go on there and you watch the video, you do the quiz, but you don't get the feel that you're learning along with thousands of other people. To provide review and feedback so that you can give timely feedback on a learner's progress through formative assessment and mastery learning. So learning towards mastery, towards developing your competencies and empowering educators. So allowing the people who are running the course to have not just analytics at the end of the course but continuing analytics as to how the course is progressing. And do that at a massive scale. So scaling learning. Some educational methods get worse when you try to scale them. So you think of sports coaching, for example. That if your tennis coaching, then it works for two people. Not much better when you go larger than that. Football coaching, 11, 15 people. But they don't scale. Other sorts of educational methods are pretty much the same depending on scale. So I'm lecturing to you now. It's being videoed. There will be people who will be watching later online. It's pretty much the same experience. This is why X MOOCs worked, which is that it's pretty impervious to scale. Lecturing and testing. But which educational methods get better with scale? So one way to start with addressing that question is Metcalfe's law. So Metcalfe in 2007 produced a paper in Forbes based on other work, which is basically quite simple, that there are some network systems which the larger they get, the better they get. So the telephone system is an obvious one. When you had the first two telephones in the world, it was great to talk to one other person, but it wasn't much use beyond that. As the telephone system expanded, particularly as you had international subscriber dialing, it got better. There were technical challenges, as to how you could manage this complexity, which is why dial codes and international codes and subscriber dialing came in and so on. But from the user, the more phones there were, the better the experience or the better the possibility. So for some network systems, the value of a product or service increases with the number of people using it. But, of course, we're not just points in a network. We're not just passing data to each other. We're human beings trying to learn along with other people. So networks enable learning if they support productive conversations, conversations that are new, important, timely, understandable, appropriate and trusted. In other words, we want to develop effective social networks for learning, human networks for learning, not just passing data. And also one other wrinkle on that is that if you've got a two-sided network, then both sides need to benefit. In this case, we've got learners and we've got educators, and both of them need to benefit. Learners need an opportunity to learn. Educators need an opportunity to teach. Learners need ease of use. Educators need to manage that complexity of a million people learning together. And learners need value that increases with scale, educators need to gain insight from that massive scale data. So you need to have a system that benefits both sides. And so what did we, how did we develop it? Well, I decided as academic lead to base it on a theory of learning at scale and learning as conversation. So the idea that all learning is a process of conversation. And that goes back to the work of Gordon Pask in the 1970s and more recently, Diana Laurellard, the idea of conversational framework. Now, I could say lots of things about Gordon Pask. He was a fascinating person. He came at learning education from a different perspective to people like Skinner and the sort of instructional scientists. He came at it from a cybernetic perspective. And he saw learning as a cybernetic system, a system that is continually interacting and evolving. And he said that all learning, and this is important, all learning is conversation. We converse with ourselves as we come to try and understand the world, making distinctions, black, white, big, small, democratic, not democratic. We, we, we understand the world. By holding internal conversations about the world that we're experiencing. But we also hold conversations with other people, with partners. That partner may be another learner, or it may be somebody more knowledgeable, or it may be a trained educator. So there are many different kinds of conversations. And they're conversations around things. They're not conversations in abstract. So you have conversations around things, and you have conversations around things, and you have conversations around things, yn abstract. So you have conversations at the level of actions where you are conversing about information that you've been given about models that you're playing with or exploring or problems that you're trying to solve. So you have these action directed conversations and you have conversations at the level of description where you're trying to understand not just what it is you're doing but also how and why you are learning. So this is a space of learning as conversations. What was interesting is, Pask was also an educational technologist. In fact, he was probably the first educational technologist and he built the first adaptive training system for example. And so this is an implementable model. It's not just a theoretical abstract model of learning. So when I showed this to the software team, many of whom had been recruited for the BBC, this was the first model of learning that they got, they understood. They said, okay, we've got to try and then build these things. We've got to try and build this here, this shared medium. We've got to try and support these sorts of conversations. So then how do you build that? Well, one of the things is that you don't want to just have conversations in the abstract. You want to have conversations that are linked to what it is that you're trying to do, conversations in context. And you want to separate out the more abstract conversations, the more reflective conversations from the conversations for action. And you want to make distinctions between learners and partners. And you want to be able to manage the complexity of that learning system. So in brief, how can you enable effective conversations for learning at levels of action about the things that you're doing and description about concepts for hundreds of thousands of people for many cultures? Because this is a scalable model that the more people who take part, the richer the conversations, the more partners you have from different perspectives, different backgrounds, the more opportunity you then have to come to know through sharing those different perspectives. So would that work in practice? That's a theory of learning. Would it actually work in practice? Well, this is a future-learn platform. And I was passing around some leaflets there. You can go on to it yourself. You can browse the courses. So that's what the front page looks like. And let me just give you one example of a course. So this is a course called Introduction to Forensic Science from the University of Strathclyde in Glasgow. And it's for people who have got a vague interest in forensic science. They've probably seen CSI or one of those other TV programmes and think they might want to be a forensic scientist because it's an exciting thing to do. You get to look at murder scenes and you get to use interesting forensic techniques. But they don't really know what it's about. So that's what this course is aimed at. Now, partly because of the involvement of people from the BBC, the other aspect that they were interested in was storytelling. So how can you create a course that's not just a series of disconnected videos, but how it can tell a story? How there can be an underlying narrative? And this is probably the best example of that. The whole course is based on a narrative. And the narrative is around a murder, a murder that took place, a real murder in the 1980s. Somebody was murdered in his car and each week you get a video which is a reenactment of that bit of the murder. And each week you have to try and use a different forensic technique to try and solve the murder. So you might use footprint analysis to look at the footprints around blood sample analysis and you're encouraged to take blood samples of your own blood and do the analysis. But you're not doing this on your own. You're doing this with thousands of other people who are also trying to solve this murder together. So this is what the to-do page, so this is your top level view. And again this goes back to John Hattie's work on visible learning, making the course visible at a glance. So you can see that this is a six-week course. The course is actually up to week five. But as a learner you can go back and look at earlier materials. So here I am in week two and I've actually only completed half of week one. So I'm well behind with a course. I've gone into week two and I've completed half of week one. And week two is about fingerprints and fingerprint analysis. So as I said part of it is based around this murder mystery but also part of it is around active learning. So actively doing things and one of the active things to do is taking your own fingerprints and here you are and then it shows you step by step how to take your own fingerprints. So you put it, rub a pencil, you put your finger onto the pencil, then on to some cellotape and you can get a lovely fingerprint on the cellotape. So you do that activity and then you get the video sequence. You watch the video. I don't know whether I can, can I show it to you? Let's see. And so on. So you get information about what happened in that real murder and then you get the opportunity to then discuss it with other people. So if you click on that icon there, you get the discussion and so we don't send learners off to a separate forum. Every discussion and if you think back to the conversational frame but every discussion is in the context of the activity that you're trying to do and it's a kind of water cooler discussion. It's a free flowing discussion and some of the learners get worried about this because if there's a thousand comments how can you read through them all? Well the answer is you don't. Any more than if you're having a chat at a conference for example in a bar that you would try to engage in every conversation. Typically what you would do is you look at the most recent ones to see what other people are saying and then you click on the most light and the most light brings up the discussions, the contributions that other people have rated as being interesting. And also typically the educator because the educator has a particular role in this, the educator will then comment on some of the most light contributions and so you will get the educator voice in there as well. So it's using social network techniques to try and manage the complexity of this massive scale interaction. Now when we designed the platform we really didn't know whether people were going to contribute to these discussions and it's been amazing in terms of the number of people who have taken part in these discussions and the richness of the discussions. So if you go down here and this was pretty much a random page that I took but just down at the bottom here I clicked on this person, reader Raymond, you can then go to that's the contribution that she made, Mr Duggan gave the wrong address which seems so contrary and so on. So she's trying to solve this murder, you click on her name then and you go to her profile and all of the people on future learn are the real names. We don't use aliases again in part for trying to open up education. So you don't hide behind an alias, you use people's real names and you get their real profiles as much as they want to give about which courses they take but also their background and you can see the other contributions that this person, reader Raymond has given, not only to that course interesting but to the other courses she's taken as well and then you can go on if you want to see a comment that she's made in some other course that she's taken. So you're following these threads but not threads of a threaded discussion but threads of interaction through people. So that's the conversational framework and to give just one other example here the most popular course so far exploring English for the British Council, 120,000 learners took the course just one video had over 17,000 comments on it so it's an amazing engagement of people with the course material and half of the learners who started that course contributed to the discussions so again a far greater involvement of people in the online conversations than in other MOOC platforms from 178 countries. So that's the basis of the learning as conversation that's embedded into the future learning platform. What other sorts of learning activities can work at massive scale well here are some of them in conversation the bigger the scale the more perspective you can get particularly as people come from other cultures other backgrounds social network learning so finding people to study with through following liking profiles again the more people who take part the more opportunities there are to find people who are like you to study with peer review I'll mention just that as one other example of the way we've tried to develop a scalable approach to learning with future learning so the more people who take part in the course the larger the pool of appropriate people to review your assignment so on future learning we made a distinction between peer assessment and peer review so where assessment the purpose is to rate an assignment to give it a rating with peer review it is to provide constructive comments constructive feedback and so the way that peer review works and this is another course on hamlet that you're given a structured assignment in this case to say what's your own preferred preferred version of the play hamlet and why and you have to discuss a number of different versions of the play you submit your assignment and that assignment goes into a pool then for other people to to review and immediately you get back another assignment from that pool for you to review and to do a structured review normally a three-part review on future learn each element which we call a step a learning step we try to keep to about 20 minutes 20 to 25 minutes so your assignment is designed to take about 20 25 minutes and then you get back another assignment which you're asked to do a review which again takes about 20 to 25 minutes and then after about three to four hours you start getting back reviews from other people and the more reviews you do the more reviews you get back and typically learners do one to two reviews so that's the way that peer review works as a an interactive process there's more that we actually want to do with this so that the review can then be itself an element that you can have a conversation around with the other reviewers and we haven't implemented that yet in fact there's quite a lot we haven't implemented on future learn and I guess this is the main issue so far that we've developed some very exciting approaches to pedagogy but there's a lot more to be done for instance around small group discussions around game-based learning around inquiry learning and looking at how each of these can be scaled up to larger scale so another area that I'm looking at is what's called citizen inquiry bringing aspects of citizen science where you've got thousands of people involved in citizen science along with inquiry learning so that you're not only doing data analysis for scientists but you're also initiating your own inquiries and recruiting other people in the kind of kickstarter way of setting up an investigation and then bringing other people in to join that investigation so there are other pedagogies that do work at massive scale and that's one of the things that the open university were actively exploring so that's the pedagogy that underlines future learn underpins future learn so we've implemented peer review we've implemented discussions linked to content both innovative and both successful but there are many other things that we still want to do I just want to say a little bit now about the other side of it so you know we're talking about this two-sided notion of networks so it's got to benefit the learner but it's also got to benefit the educator so I want to say a little bit about learning analytics um and in particular insight for analytics so what you want is that what you as an educator not just to get back masses of data you want to get back insight into what changes you can make to the course so how you can improve the learning design based on the analytics and in basically three sorts of analytics transactional who viewed what when did they view it on what sort of device interactional how did they interact with the learning design and conversational so what you know going back to the learnings conversation what did they talk about who are they connected to so this is a top level analytics from future learn so if you look at the people who sign up for a course well it's the same on every platform about half of the people who sign up for a course actually started and that's if you look at Coursera edX it's pretty much the same and the reason for that is you can often be signing up two or three months in advance of the course and when the time comes you're too busy to start so it's one of the reasons that some of the other platforms now are looking towards on-demand courses so that you start the course immediately you sign up the problem there is if you've got these rolling courses where you can just start the course whenever you want to you haven't got the notion of a cohort moving through you haven't got the notion of learning along with other people so it's a kind of it's a dilemma that we're just grappling with at the moment with future learners how can you have the benefits of signing up and starting immediately on a course but also working along with a cohort of other learners and it's something we're actively looking at them of those so all these other stats are in relation to learners so of the people who start a course 84% are active learners and at future learners we we can track who is an active learner because for each step of the course you have to mark you press a button to say that you've completed that step now it's up to you as a learner to decide what you mean by completed it could be you've just glanced at it and you're happy enough it could be that you've engaged with it in depth but you have to explicitly say that you've completed that step and 84% who start a course mark at least one step is complete 45% come back the next week so the other way of looking at it is you lose about half the learners in the first week and that's again something that all of the platforms but particularly future learn we really want to try and bring that numbers up and for some courses it is much higher particularly those courses that have a strong narrative drive because if you've got a reason to be coming back the second week for instance if you want to carry on solving that murder mystery then you you're more likely to return and we're looking at ways of taking people over that bridge from one week to the next with things like advance organisers for the next week 22% of those who start complete that's a pretty good number particularly again compared with some of the other platforms and it differs widely we've got some courses where over 50% have completed the course and what we're particularly pleased about is overall about a third of the people participate socially they join in the conversations now almost everybody views the conversations they learn vicariously from what other people are saying but 37% overall take part in those discussions some other analytics so we can look at what platforms people are working on and what time so this is time of day day of the week and platform so you get some interesting things like there there's a spike at about 10 o'clock at night of people who are accessing the future learn platform on tablets it's people taking their tablets to bed to carry on the course you can ask questions specific questions like how long should a video be and because it's at massive scale then you can get answers to those questions by doing the analytics so each of these dots is a different video on future learn and along the bottom here along the x axis is a video the length of the video and some of them are 30 minutes or longer some of them are very short and up the y axis is the percentage of people not just who quit the video but who quit the platform altogether so that's bad news and up to about eight minutes it's it's around 20 30% soon as you go higher than that then you get people many more people high percentage who are leaving the platform so the answer to the question is about six to eight minutes is how long a video should be and then I've mentioned about peer review so these are some of the top level stats that you can get from peer reviews this is from a number of different peer reviews the average length of own assignment the average length of review number of hours to the first review but again you can drill down because you've got this analytics then you can look in greater depth so here this is for one course this is for one peer review so each of these dots is a person is a learner and this is the number of minutes between submitting an assignment sorry between first viewing an assignment and submitting it so it's up to 160 so it's up to about two and a half hours so there's a rough correlation but not a great one between the length of the assignment and the longer it takes but look at that person there they've submitted in a thousand word assignment in under a minute pretty good so that's when you start to get suspicious how did they manage to submit a thousand word assignment in a minute and well the answer is fairly obvious they copied it and all of these ones down here these are the suspicious ones because they're submitting long assignments in two or three minutes and these ones down here are just typing rubbish so we actually don't send these ones off to be reviewed because they're just you know a few word gobbledygook so you can start to do things on a platform to help the learning and help the reviewing but also you can look in more depth about the learners themselves so that's the sort of insights you can get from analytics now at massive scale and drilling right down to the individual learner another interesting one about the learning design after each peer review we encourage the course then to have a discussion if the learning is conversation for the learners to discuss their experience of doing that review so how did you get on with the assignment how did you get on with the review what was it like for you we find that 20% of the learners go to that discussion first before they do the assignment in other words they find out what other people are thinking about the assignment before they do it themselves so they're not going in a linear progression through the course so we're starting to find some things about the way in which people progress through the course as well so to conclude then how do we design successful open learning at massive scale the answer is to develop pedagogies that get better that improve with scale particularly learning as conversation social learning connected with insightful analytics now when we first develop the future learn platform all of the publicity was around both how great these moot courses were but also about how bad the pedagogy was how it's just pushing videos at people about the instructional design approach coming from the US of video led and so taking it taking a different approach of social learning was quite innovative at the time if we look now at the moot platforms diversity right in the centre now their front page is discussed and we know that diversity has learned a lot from the future learn platform in redesigning its learning platform nova wed now describes itself as the online learning environment and Coursera has just developed a completely new platform which interestingly enough is based around visible learning so that you can have an overview of your course and discussions associated with content so it's many of the features of future learn now are there in Coursera's new platform which is great it's a great compliment that the approach of social learning and learning as conversation is now appearing in some of the other platforms that's it thank you well thank you very much Mike we do have a little bit of time for questions because we are predictably sort of running a little bit over my little catch up at some stage you probably owe the lunch and my first thing I just want to make comment on is that when we were putting the program together we really did want to address the max pedagogy and the learnings that can come from MOOCs because of the hyphen some identity around MOOCs just being as you described right at the end some traditional forms of pedagogy being pumped down a pipe so I think you've really achieved the goal that we have was to help people to appreciate how future learning has led the way in being pedagogically driven around a framework very explicit framework that can't help but avoid this so I have to acknowledge of course my um that John Hattie's fellow Nyselodus. Yeah he is indeed. Now Melvin Melbourne knows so well we can't ignore that for a minute but with Diana Lorela it's conversational framework theory the other element I think that's important to note is of course the history of new technologies and education is littered with people taking traditional models traditional models of pedagogy and applying them to new technologies which is effectively what we've seen with many of the other platforms but future learning stands out for that reason and then the last point I'll just make before I open up for a few questions is I think the analytics that you shared in the last part of your presentation again underscores why we need to be engaging with the MOOC movement because the learnings the lessons are particularly took note of your data around the use of video I think there's a lot that can be taken from that so that's just my little observation we've got probably five minutes of questions I think we'd like to start off I've got that absolutely fascinating presentation last year I did a massive online learning course in my soul on learning to teach online they used a few of you as you turned out that there was some quite difficult um intercultural difficulties in the way that operation I wonder if this is the opportunity to visit you in part of that so that's interesting what what what sorts of difficulties in terms of misunderstandings between yeah it's a it's a big issue um that there are some cultures Chinese culture for example where it's not considered polite to criticise other people's work and particularly not the work of scholars but even the work of peers it's not considered polite to do that I think there's two things one is that we are trying to we're trying to engage people from different cultures and their different perspectives so for example in that exploring English course the first discussion was for people to discuss what their experience was of learning English rather than just saying this is how you learn English so getting people to start from bringing in their perspectives and as we've gone on developing more future learning courses we've wrought in more international partners and that's become even more important to try and understand and to not just understand but to celebrate and to engage people from their own cultural perspective I don't think all of the courses have done that necessarily well but that's certainly something that we've tried to do in terms of engagement of people from different cultures in things like the review process I think it's very difficult because on the one hand you can say okay we'll respect people's cultures so for instance we won't have a peer review because it will be difficult or embarrassing or culturally challenging for those people on the other hand we want to model we're not trying to model it international education practices we're trying to model we are a UK based platform and we're trying to model the sort of UK based European based educational practices and one of the reasons that many people from China for example are interested in taking future learning courses is they want to understand Western educational practices and we've known this explicitly I've got a a good colleague who's based in China and she's doing surveys of people who are taking future learning courses and one of the reasons they're taking them is that they want to understand how people in the West do education not so that they can then change Chinese education into Western education but so they can learn and understand and take the best practices from the West into Chinese education so I think providing some challenge to people to engage in those sorts of peer reviews or in those sorts of discussions I think that's quite legitimate to do but so long as you do it in a sensitive way that respects people's cultural differences which one okay great very good actually platform is so very interesting just wondering on the certificate of completion you can actually purchase or avoid it yes do you have percentage rates of people actually go down that line and do they continue on after getting their certificate um so about 10% of the people who finish a course get a certificate so go and buy a certificate although it varies a lot so more professional development courses we get higher percentage and then some of the courses are aligned with master's courses or an undergraduate courses and we track now people coming off future learning courses onto undergraduate or postgraduate courses and we explicitly ask them and future learn can provide that data to the universities of how many people are being recruited through it's obviously quite sensitive data but one way to look at it is to say that if you've got a master's course that you're teaching on campus with maybe 10 people 20 people you put it online and then you can use a MOOC as a recruiting way into that then if you get you know another 10 20 50 people that's good news if they're paying fees so some of the courses that's the sort of numbers that you're getting you know even if you can get 10 or 20 people onto a master's course out of 10 000 that's still good so it's particularly those universities that have got online masters like Edinburgh University which is moving all its master's courses apparently towards online delivery those are the ones that are really going to gain from the MOOCs as recruiting grounds yeah i've got a question with regard to the intellectual property so there's a lot of technology behind this stuff typically you need to have scale and i guess resources or whatever to develop it and the barriers to entry are quite large is that do you have any concerns with regard to like commercial commercial providers hate the thing or copywriting or whatever the the key technologies and therefore excluding or monopolising the kind of platform um so i mean one of the things is that commercial providers obviously can get into this space they could have got into this space for the last 10 years of providing free courses and they haven't for various reasons it's not that easy if you're a publishing company like Pearson's for example to produce a successful massive scale um online learning operation it's not just a matter of having the platform but you've got to have all the partners as well willing to contribute and at the bottom of it academics are giving up their their labour or um you're getting academic input um into MOOC courses um as as a partnership now it's i think it's harder for commercial purely commercial companies to get that partnership so it's not just a matter of building the platform you've also got to have the partnership good thing is that there's a variety of different platforms you've got open edx for example which is an open platform and there are universes now that are starting to use open edx and forming consortia and one of the things i don't quite understand is why there haven't been more consortia that are using open edx as um a shared MOOC platform um so future learn is a in terms of its technology it's a closed platform and that was a decision that the OU made for right or wrong um way back at the beginning but the content is all open um and it's um becoming one of the things we're doing with the future learn content is making it uh searchable from the web now so that you can search using google search terms for individual items um of content on the platform and also all of the interestingly the learners comments they are all published under a creative commons license so the learners own the rights to their comments um so the learners own the right to the material that they contribute so we've been looking at um openness in terms of the the content but the platform itself is a proprietary platform and one of the reasons for that is future learners now starting to develop closed in-house courses for companies uh which are paid for courses and we're using that platform um to develop a suite of courses that bring in revenue to future learn which you don't know about because they are in-house courses for companies so future learners trying to get into that market and because we've got the benefit of testing the course on a million people now then we've been able to do some rapid innovations so that now as we've got the courses for the companies we've learned quite a lot about how to design successful courses. I'm going to have to stop on that note right Mike is uh going to be here for most of the day in actual fact it's a great minute you know we'll do that so we'll have some opportunities over morning tea and much time for you continue. Thank you Mike if I could just ask people to join with me and that.