 Mae'r newid yn fwrdd ac mae'r ffordd o'r Serfamdd. Mae'n gweld o'r ddweud o'r Llyfrgell o'r Llyfrgell, ond mae'n ein bod'n gweithio i'r rhain i'r lluniau yma, i'i ei wneud am ymlaen i chi. Rwy'n meddwl, mwyaf am yr Ymgyrch. Felly mae'r ffordd o'r Llyfrgell yn ffyrdd, maen nhw'n iawn i'r ymgyrch o'r cerddau o'r maen nhw, a'r cyfnodd, oherwydd yn gwneud o'r ymgyrch. I know that because we're maybe in a slightly different state where we've kind of got different layers of courses and things going on that at any point I'm saying something you'd like more information about if you've got a question as we go along, very, very happy to take shout outs and hands up or whatever. This isn't a lecture of any means, this is more of a kind of a discussion because we still have a lot to learn from everybody else as well so your questions are really useful for us. So just kind of putting it a little bit in context to start, we had six courses go live this year in January, we've just announced a further course on future and so we started off with Coursera and we've now just launched a partnership with FutureLearn as well and we've got a further six courses have been announced on Coursera with about ten or so in development behind the scenes on top of that. So it's busy, it's exciting and our subject areas are very broad and we started off with our first six courses where two courses from each of our colleges at the university so we had two from medicine, two from humanities and two from science. And we're kind of seeing that trend kind of continue, it's very much an even piece across the board, academics are coming from far and wide to be part of things and enthusiasm is still very high. One thing that we've always been keen to ensure is that all of our courses are short, these are by no means just taking what we've done online or taking the on-cams experience, videoing it, putting it online. These are purpose made for MOOC experiences rather than for taking over what we're already doing so very similar to how many of the other presenters so far have kind of said is developing new courses so it was a real opportunity to be saying well what do we want to do in this space and let's get excited about learning and teaching again and see what we can do and challenge things, do things a little bit differently. We were obviously fully online free to take but something I'll talk about a little bit later is kind of open resources. It's not only where we're taking a sphere that we should be doing good practising where there are resources already out there in the web and using creative commons wherever we possibly can because it makes it easier for sharing but actually the most exciting thing is everything we do wherever we can give licence for it we make creative commons. So we're really keen for others to be using the content in different ways to be repurposing it, recycling it etc and just without stifling the innovation as much as we can say put it out into the world and see what happens and if we put too many licences and barriers over that then it kind of stifled some of the things that could potentially happen. Just at the bottom was just kind of a quick summary of our scheduling how long it took to go from start to finish on the Coursera initiative and actually it was very similar for Futureline as well. So again just to kind of give you a little bit of background on some of our subject areas this was our first six so pretty broad then we launched the Higgs boson with Futureline and then we launched a further six with Coursera but I can give you more details if anyone has any better feeling about that. So yeah why did we do this? It's funny because we hear quite often especially across the piece about this being about making my return on investment having marketing or was it marketing as a point, yes it is brand awareness but actually this is research and development, this is bread and butter for institutions. It gives us an opportunity to be exploring new spaces, challenging how we're teaching learning but also to say well this is a research project for itself, it yields data, there's learning analytics so we could be looking at this. The way that people are using content is going to change because we're just putting it out there and saying what happens if? Trailing dots and then trying to measure the impact in whatever means that could be and actually realising that this isn't something that we're very used to, we're not used to kind of saying here's an unknown entity and engage with it in an unknown way and then let's try and deal with that on the other side. We're used to saying here's a defined course and you sign up to that course and if you do not complete the course then you are a dropout or you are a failure in some respects and this is flipping out on its head and saying you could be coming in with any reason whatsoever. That's really exciting and we're really keen to have a look at that and to work with the insurance or trying to ensure that you get the experience out of it that you wish to sign up for rather than something that we have prescribed on to you. Yes, there's obviously the universal strength but we also feel this is really good fun and it still is a lot of huge fun given though it's a bit taxing, it can be exhausting but it's worth it. It's been incredible and we'll talk about it a little bit later but we also knew it was never going to make us money. Yes, we can get some return on whether there are some things that we can be doing but that's not the point of it and also we were never expecting to suddenly break even because well there's a little bit of investment up front that's needed in order to get these things going and we're keen as well for the schools to be reinvesting any revenue that comes from the courses say for example on assessments or certificates and things to be reinvested directly back into the courses themselves so either developing new content or investment in teaching and assistance etc. So it's not about making back to the centre it's about developing those courses further in ways that people respond to and just again saying what can we do if we invest in these things in different ways and let them grow and nurture them in different ways I guess. So how did we do it? Well to begin with we didn't impose a template it was asking academics to come to us and say well what would you like to do? You've got this blank canvas how would you like to develop it? And we ended up with six very different courses and it was wonderful actually to see how each academic was viewing it in different ways each team brought their own personality and it's something that we're continuing on throughout our new phases as well. So although on one side it would be a lot easier to say this is the Edinburgh approach to MOOCs and everybody has to do it in the same way and you're only given this amount of time in front of the camera so on and so forth and good for scheduling. So in many ways that goes against the ethos of us saying what do you want this space to be and let's try and work it out. Get us excited about your course and make us want to put something a little bit different into it and respond to it accordingly with the different kind of resources that we can put in. We were also keen for them to be experimenting. So when we said it's a blank canvas we said you've got this framework and that's great and you can do that but you could also break it. If you want to do something different try it give it a go. It's fine how having something that's so big and so open and so visible to the world seem to be a less risky space to experiment in. Whereas doing things on campus for credit seem to have a lot more risks involved so perhaps it was the nature of it being not credit bearing gave more flexibility for this in fact it was short and it wasn't connected to any of our normal activities that it felt less of a risk to try something out, something new that perhaps people have been thinking about for a while but hadn't necessarily had the space to develop further. And as I said already we were keen for people to be using content that was sitting in other places to signpost to other things to essentially do what we hope to see in the future of people reusing our content so we reused other content and see what's sitting right. Most of us are sitting on huge archives in university libraries and digital media and things that people don't see unless we have a way of surfacing it and this moves an opportunity for us to do that to say look there's this space and actually the university is starting to join together and go around to each of the individual departments and say look you've got this great resource and we would really like to use it and they go oh that's fantastic can we put it in. Obviously we have to be concerned about licences and things but wherever we can it seems that people are really keen to be using this as an opportunity to hook their area, their discipline, their department onto the courses that are coming through. So a quick summary on one of our MOOCs the EDC MOOCs you may be aware of. Inland digital cultures they were a little bit different to the standard MOOCs space in that it wasn't like the traditional MOOC of having things all sitting within the course site. It said find your own spaces, talk about it wherever you wish to talk about it and bring it back into an aggregated hashtag feed and then the academics reflect on what's the conversations that are happening across the community. It's more of a connectivist approach rather than saying we have prescribed tasks for you that you must, activities that you must engage with at particular times. There was content that was seeded and then people went off and nurtured it in their own ways and then brought it back to talking to the community. Also it was quite nice that it ran in parallel with one of our online but for credit MSc programs, the MSc in digital education which had its participants or its students on that course were also participating in the MOOC as teaching assistants. So it was quite a nice community effort as well, getting more people involved and trying to encourage others to engage in different ways. Another thing that one of our groups played around with, if artificial intelligence planning, they took it out to Second Life. So they did a synchronous session on Second Life that was then kind of video captured and then put onto their MOOC space laser. Again another opportunity to kind of get the community to talk. So we had people doing things that were a little bit different going into other spaces where we could but we also had some very traditional like introduction to philosophy. We took a very, we have lectures and then parallel activities and questions at the end of each week. So we had the spectrum wherever we could. Overall in the first wave we had 310,000 enrollees on our courses. Now we're in the next stage and we haven't done as much marketing this time because it's not, I assume we're still waiting to do that next push but we've now got around 410,000 people enrolled on our course. So 310,000, the first wave added on another 100,000 or so who are currently enrolled on our 13 courses in total. The widest range of backgrounds as well and I can go into more details later if people are interested but much of what everyone else has been saying is that it's almost across the entire piece of countries. Wide age ranges, diverse intentions, everyone came with a different reason for being there. They had different backgrounds and that in itself was very rich and came out through the forum discussions. But what was most interesting, we said going back to the title of the experience is kind of beyond the hype. Mewks didn't really, they didn't just bubble up from after you know where. We've been doing many people in this room from institutions that have been doing online education for many, many years. And so it's not actually that new. The kind of like the nuggets of learning engagement are very similar if not identical to what we've been doing before just in a slightly different virtual learning environment. Obviously there are differences, it's mainly the scale that's different to what we've done before but the broad underlines are, let me say, one mix made of. So typical features, there's a virtual learning environment, there's some videos sometimes, interactive tools, discussion forums etc etc etc. So anyway, it wasn't really anything new but it is something that is enriching to that overall experience. What however has been very different is the way that the content is being used. So usually courses sit behind firewalls or chyblists or whatever that you can't see what's going on behind it. You can't be reusing it, you can't be sharing it in ways. Whereas Moot Street provides an opportunity for us to say, well here's some stuff and it's open to people and how can we be using it in different ways and be informing developments within the institution or out with the institution. So here are just a few examples of what we've got going on at the moment with our first six courses. On campus we have some of the learning materials, some of the videos are being used for on campus activities but that's just kind of to enrich at the moment. It's by no means a sign post them over and then say come back in five weeks time. It's embedding the videos in different ways on the virtual learning environment. The data sets themselves, so each of the Moots have learning analysis with you. You can trace the individual learners but that's a huge amount of data that's being kind of created. Those data sets are of real interest to academics across the institution for MSc dissertations etc. to get their students to be engaging with research of big data sets, big data mining that perhaps we haven't necessarily had access to before as readily as well. Every time we launch these courses we will be creating another data set that could be used within that mix and also by them being part of that conversation means that we then have a greater understanding of what's going on because they're exploring things we wouldn't necessarily be able to explore on our own if we're doing research activities ourselves. We're also seeing that people are starting to signpost their students to other people's Moots so just for again the enriching experience saying well there's content sitting over there with Harvard you might be interested in or there's another course that's coming up that you might want to enrol on that complements this course well. Some of the most interesting stuff is happening off campus so our content is currently being used in many different ways internationally. So for example University of Maryland have taken a copy of critical thinking and a copy of introduction to philosophy and are using that for credit bearing activities. So they've taken a version and they've treated it for their own delivery and then they've assigned credit to the university state system. We've also shared our content with an academic project called Generation Rwanda and they've taken again similar courses, taken coffees of them and are delivering them within secondary education within Rwanda to see what works, how does it work, whether students enjoy that experience, whether it's something that they would like to explore and that's a conversation I need to pick up probably when I get back to the office tomorrow is where that's going because they did a pilot and it was very well received but we're now saying well you can use all of our content if you want it but let's not just kind of say here's some stuff, we would really like to know what you're doing with it and how we can be developing things that maybe meet you beneficial of us then also gaining insight into how our content is being used because that in itself is a research initiative. On top of that we have local schools within Edinburgh have been using our content, we've had one example within introduction to philosophy where there was a teacher that enrolled onto the platform and then taught with the class, I think it was an extracurricular activity but they did the course together so there was an awful school I think where they sat kind of as a group based discussion which was lovely and something else that we really want to be hearing more about but that's another opportunity for us to pick up in the new year is how to be best engaged with schools because it's all well and good saying there's this content there but not every school is going to have a strong enough internet connection to get everyone to be simultaneously accessing it potentially or to have enough bits of kits or to have, you know, you might want to do it by CD, you might go old school and send them out to packages in the post or, you know, there's things that we could be doing but again there's no point us going in and saying here's our content, here you go instead it's a conversation of we have some content but how would it be best given to you would you prefer it online and it's successful and we can share it or would you prefer it in a hard copy form or, you know, again it's about discussions and not assuming that you know the best way to be engaging with these new audiences instead saying let's start building a little bit of a discussion together the nice thing we're seeing on course area is that all of our videos are being translated into other languages and there are a couple of initiatives that are going on through Passera to outreach in South America is Portuguese and Spanish is being put onto some of the very high, not high recruiting but high and roly courses and critical thinking and philosophy again have both been fully translated into Portuguese and Spanish and are being used for outreach initiatives there which is great and finally all of the courses on Passera are being streamed on a parallel server for Chinese engagement so usually there's a number of blocks that are present for people trying to access from a Chinese IP onto some of the data we have on the internet because of local firewalls and such and so by having parallel servers and streaming systems it means that people can be engaging with it over those usual barriers which is interesting because China is an area that there's a lot of potential but we're not necessarily engaging with as well as people say we have on campus markets for our MSE programmes for example that when it's put online the same demographic is replicated all by China and that's really interesting in itself of what are the online barriers and again it's a conversation and research project in itself to start asking are we doing things differently? Is there something that we could learn from this experience that maybe it's not necessarily that our content is not fit for purpose but maybe it's sitting in the wrong place or maybe it's not being serviced or maybe it's not fit for purpose, maybe we're pitching it wrong but we don't know, we just have to ask but it's also far more than just building courses and what we're seeing is that academics and the internal community are starting to look at this online movement and go oh I want to be part of that, I want to do a move or I want to be doing some online activities or maybe I could start reflecting on my on campus activities to involve more interactions, more online discussions more synchronous, asynchronous activities and it's not by any means an overnight change but it's getting there, the momentum is building and that's really exciting to see that we are seeing a step change and people wanting to engage in conversations that we thought the door was shut to people are now going I think we're ready now I think we're ready to start talking about online learning which is fantastic and if nothing else something that we should be building more on one nice example of this is philosophy again this was a school that actually had not done any online learning at all and we threw them right in at the deep end with the MOOCs in that the head of school was Cain and there was a team identified and they wanted to do something but they knew it was scary for them but they thought oh blast we've only got one shot let's give it a go and they loved it so much they're now doing a fully online MSc programme and they're doing more online activities with their on-campus students and their outreach in schools they've got book and research collaborations that have been established as a result of being in this online space that they've never been in and actually they've now become I think the first fully online philosophy programme it might be in the UK but they're one of the first in something and that makes them feel good about stuff but that's no bad thing the fact that we're saying there is no subject you can't put online give us a channel and let's do it think about it in different ways is exciting to see we have a strategy at the university for getting a fully online MSc programme in every school and we're not doing too badly but there are some schools that haven't crossed that line yet and we're really keen to know why and as I alluded to a minute ago it's predominantly because they just weren't ready at the time that other schools were ready and some are stepping into the MOOC space as they're testing the ground before they go into MSc provision and some are almost saying others can do the MOOC thing and that doesn't look too scary or they're doing online learning maybe I could do that too and we are seeing set changes of people and what we're also seeing at the moment is a space for collaboration and so with many of our courses yes we do have interdisciplinary online courses but what the MOOCs provide is a platform but it's quite easy actually to do collaborations schools can get together and say let's do something similar and saying let's do this one week that makes sense for us and that isn't something that they've necessarily felt that there was the legitimacy to engage with people so plans for the future for Edinburgh is that we're looking for more outreach we're thinking about community provisions looking to youth groups and schools and such to say how would you like us to engage we've got this stuff is it of use to you but also we can see that there's a potential connection here can we start a conversation and also getting our students to see that it's something that we acknowledge and we recognise and we would quite like it if they would like it to be on their transcript and we can facilitate that and trying to find a flexible portfolio approach to these things is potentially very nice but working with local needs as well local and national needs to say is there a particular move that Scotland needs a particular gap that the youth group could be filling with a move of some description and that's not necessarily for just Edinburgh that's for everyone that's kind of in the the move domain as it were for us to say well we've got this opportunity here let's again, let's not worry so moving out of the Edinburgh space and into kind of more of a national sphere we've seen that obviously the moves have had a lot of impact but how much impact have they actually had well we've seen it's had quite a lot of impact with university provosts and vice-chances etc it's definitely infused them with online learning that perhaps we haven't seen at the same level before Governments are talking about online learning and wanting to develop new courses which is pretty exciting and the media love it oh if you can talk about a course having 200,000 people on it oh wow they could not get enough of it well in some respects you can go oh but it's not all about that but in other ways again that's high level BBC media is a pretty, almost every week there's something about online learning and that's incredible as a domain that we've not been in before and then we're really interested in you saying no that's not the point but that's all right if it's going into mass media and there are people that are looking at these things that perhaps wouldn't have ever thought about it before wouldn't have thought that there would be an online course for free and branding well why not and although in some respects it's had a big impact it's also probably had more of an impact in the spaces that we're looking in so the next thing is getting the media out there for outreach to community groups that haven't necessarily had the engagement we can see there's lots coming through the Guardian education lots happening in BBC education but we don't necessarily see quite as much yet in local newspapers but maybe we will see more and maybe they'll get excited about it but what is interesting is the loop the connection on media and the impact that's having on students and as we've seen that many people have seen that their students are engaging the moog activities and are keen to do more of it and are starting to kind of you're doing this thing over there and that's really fun, why is it not coming over here they are becoming more vocal and more empowered to kind of speed up and etc that rate of change that we see within the on campus and the online classroom to say if things happening that we could be doing maybe we need to start having an internal reflection and kind of development on our learning provisions but also what's most noticeable is that there's been most impact in the areas that we would expect where the avatimes have been happening of the US, the UK, Australia etc and the regions do vary and I think that's where the future plans are to make sure that we are starting to outreach and you know your point earlier around internet is a limiting factor here not everyone has has broadband access that's always going to be a difficulty for streaming videos so we do need to start this has been a fantastic opportunity we've seen a lot of impact but if we're going to really take it to the next level we do need to start thinking critically of how does it make greatest impact how do we start reaching to those areas that we hope by teaching the world we could teach the world but you can only do that when everybody has access I think that's pretty much me done obviously just quickly, I'll finish on this we've got at the moment over 5 million people are talking about online learning and if that's not an opportunity for us to be capitalising on if that's not something we've got all these institutions we're talking about online learning we've got hundreds and thousands of academics that are involved in this stuff I think the topic of this whole conference is should we turn back why would we turn back on that surely we just need to keep running with it and seeing what happens next but I've got other bits where I'll stop there questions I wonder if we can take a few questions for Amy now we do have to panel later on while we're setting up for the online connection oh yeah Mike from your experience so far this sharing of experimentation has everybody shown that it could be seen as quite a competitive advantage to know how to do things well is the approach to share all this analytics is approaching all the participants in different universities to share or is it seems competitive so there's a lot to be said for doing what you wish to see so everything that we're doing is being shared is our content Creative Commons all the research that we're doing all the scripts that we're developing they will be Creative Commons open access all of our materials that all our handbooks for example that we give to our academic stuff they're all Creative Commons they're freely open to anyone having a future loan version of that as well which we will be circulating to the masses our reports are open access everything we do we're trying to share and we are starting to see noticeable changing because actually what's quite nice is if you do something, you share it and say it's not finished, it's not perfect tweak it, do it well but let's learn from this process we've seen our surveys have been used now that's the European standard for the first kind of questionnaires that are given with the Coursera now that's great because it makes it really easy for us to then share and compare datasets if we know that there's a commonality within the question sets that are being asked at the beginning and the demographics that are being taken we are seeing, especially within Europe there's a real enthusiasm to be doing this but not everybody has something to share yet many of the partners are still very new so we're trying to do a bit of a fit something out there first and hopefully others will join in but if they don't then that's fine, hopefully they found some of the stuff we put out useful and if they didn't well that doesn't matter at least we tried so yeah it would be great to see more people sharing we'd love to see it happening people are keen to share but I don't think that's necessarily because Coursera are doing a lot to nurture that development but it's not necessary therefore you must share it so it kind of has to happen so Coursera is saying they've got more data and more knowledge about learning from their platform the rest of humanity put together I mean they are very open-minded things if you ask them for information they do share within the platform and then we share the outcomes of that analysis for example so we are trying as much as we can so yeah they're good guys likewise with all the mute platforms everyone has it they've got their best interests at heart and so yeah just a detail I love the way that you make me in the opening there was a clause in Coursera when it was changed to say that all the posts and all the resources posted by learners were the copy of the Coursera exclusive has that changed? so there's a big difference between the content that's created that's then populated within Coursera and that Coursera instance so Coursera do not own any of the copyright of materials that are developed by the institution the institution holds for like these but the information that's put into forum discussions and things because they sit on Coursera's instance on the platform then Coursera you could say own but it's really it's just in that they have they sit on that data and they hold that data I don't think it's changed but they also have no rights over the content that's put on so they have access to a version of our materials that are uploaded to the Coursera platform for the intention of that particular course but we can for that at any time so it's it depends what people don't have to write anything on the forums if they don't want to I guess it's an interesting one I don't think that I haven't heard any grumblings of it and it's something that lots of people are engaging with and don't seem to have a problem with at the moment maybe we'll see something in the future but hopefully not, hopefully everyone's for the right reasons and not necessarily feeling too bad about sharing and asking questions and things on the forums for IP and patients or whatever we'll see I guess it's an experiment Any more questions? I was just going to follow that up I was just going to follow that up my question is if you're taking a screenshot of a forum post or a set of forum posts if you wanted to use that in an academic publication would you need to use the screenshot? If you're taking a screenshot yes, only because you would then be able to trace the individual if you found them on the Coursera platform So you could have normalised all of the Yes, you can definitely use the data within the platform itself for research potential there's no buzz there it just needs to be normalised taking a screenshot if it's traceable back to that individual learner and not to the best of my knowledge but that's also because it advances them for them to have us doing research and things so I think there's a fine line that it seems to be working in everyone's best interest but we'll see We're still waiting for our connection in America so are there any more questions? How do you license your materials? The materials you use because they said they're being used on the other basis so are they open realised so they can change them to make them relevant to the context and to use them to make them pay what? So all of our content is their watermark but they are open access and they are we use Creative Commons images wherever we can if we can't find it there so everything we do can be shared and can be repurposed wherever possible if we've used a particular image that has a license associated to it that we bought that particular image for that particular instance when we share it we would make it clear that that particular image we need to have if it was being used in a particular way so it's not quite as simple as here's a full copy of it but we have in each of our courses a copyright section that gives traces of each of the individual resources that are used and what their license means and how we can use it etc so hopefully it's as clear as it can be for sharing and repurposing later but we haven't heard many of the instances that have been taken for further use and development have been for educational, private purpose rather than for mook spear use it's not then taking Maryland have taken a version and said oh this is the Maryland version of the Edinburgh mook being used like a mook it's being used as a private course and there are certain affordances that come with doing education behind most people use images so if it were open they probably would be able to just take something from Google on their signs for a private cohort of students for example but we do get away with a lot in education because it's being used for those particular reasons rather than it being kind of an open space whereas the minute we put it out to the world and say it's our content we kind of need to make sure that we've got a trace for everything that is being used I mean Matt most of the action being your student body in terms of do you know any students that are doing any of your mooks or indeed have they been incorporated into any programmes on an official level so we actually haven't heard any grammar everything that we've heard has been positive but that's also because we haven't done the whole integration into formally integration into our provisions it's been more of a oh there's some nice videos over there or there's a good course over there we also have been very very clear that we have our mooks learners that sit on this platform over there and if you are you could be a student of us and happen to be a learner of the mook but you are by no means a student with us if you do the mooks so we've been keeping them very very clear in every kind of document or whatever that goes out it's always to go through our formal some of the enrolment processes and learners of people that sit on different servers by Coursera or Futureland so I think because of that we probably haven't we're going something I guess ambiguity that comes with others talking about students collectively or students or students maybe seeing more of that merging agenda just maybe because we haven't explored it as much yet but I guess watch the space I know some institutions have had more problems with that than others but we haven't seen it yet Erin, thank you so much for connecting to Doug in Vanderbilt University so my pleasure now to introduce you to Doug Fischer he's actually a professor on machine learning mook he used machine learning mook with his students on campus and he wrote a paper about wrapping the mook around that course with Coursera and he's very kindly agreed to talk about that with us today so thank you very much Doug and I hope to hear from you soon great, can you hear me? yeah alright fantastic so I'm going to be talking to you today about much of my personal experience wrapping a mook in my on-campus courses I'm not sure how I can control the slide deck there who would I be interacting with to do that I'm not going to do that I can just forward it so if you can forward from the title slide to are you seeing the slides up there? no I haven't got the slides you see he does it so has Martin happened to be there? yeah are you just sharing your slides? let me see next slide can you share your screen with us and just show us your yes I probably can hover over to the left towards the top it's the green icon a green icon? ok so you should be seeing something now yep we can see it great thanks Martin so can you see now the title slide? yep fantastic I'm going to try and put it in a slideshow everything good? yep so I think I'm going to skip past the big picture here I'm pretty sure you all know the big picture but once we are running behind I'll simply go to my experience and then we'll return to the big picture later probably at this point looking at a slide announcing the Stanford online courses there's a robot in front on the machine learning course could you just maybe instead of doing a slideshow can you just do it from your deck sure ok thanks so this is the announcement of the Stanford online courses back in November 2011 and there were three courses being offered online one was in machine learning one was in database and one was in artificial intelligence and I was teaching those three courses the following semester in fall 2000 or spring 2012 at Vanderbilt and for various reasons I had been I think I was prepared to use other professors content in my courses given some earlier experience at the National Science Foundation but I decided if this material these lecture materials were available when I did my courses in spring spring 2000 I would use them and in fact the material was available in spring 2012 when I taught my undergraduate database course and my graduate machine learning course so you all think I'm sure familiar with the idea of flipping the class so here is a week 2 schedule for an example of what I would do if the students watch the videos from Jennifer Whiteham this is in database she was the Stanford faculty member teaching the MOOC this is after the MOOC was done this was actually the first time I did this this was before Coursera was founded and so there were no terms of service to worry about the videos were just up and online I had students watch those I adopted Jennifer Whiteham's textbook in database to make for a smoother experience for the students because they were both reading and her videos you can see in the middle I would also quiz students on the materials so they would watch the videos the week before they would walk into class it was a Tuesday class they would take a quiz on the videos in the reading that they were supposed to do and then we would do some kind of in class exercise and for this example well, here's an example of Jennifer Whiteham's screenshot of Jennifer Whiteham's video in database this is a good nuts and bolts lecture I think I will always want to do the inspirational lectures in person but for nuts and bolts I'm a big believer in video so you can see Jennifer Whiteham there in the bottom right she's interacting with the screen displaying PowerPoint students would watch these kinds of videos for the most part of the nuts and bolts variety and then they would walk into class following the quiz and they would do some other exercise and in this case this other exercise was to watch a video by Hans Rosling on visualizing data and you should be seeing Hans here the year is 1948 and he's talking about a visualization of economics and health in 200 countries over 200 years just do a sanity check were we all there? great so they would watch a video like this a very short video in class and then they would spend the class mirroring a database that would be sufficient to support Hans Rosling's visualization here and it was a much more interactive experience than just listening to me lecture giving the same kinds of nuts and bolts that Jennifer Whiteham had already done now you might imagine if you were a student and you're watching Hans Rosling lecture and then you're engaged with your colleagues in small groups working on a problem engaged in active learning that things might improve things did improve this was when I first started doing this I was worried quite frankly what other people would think of my using other professors lectures I was worried about what Vanderbilt would think I was worried about what my students would think but it's been a great success and everyone seems to have liked what I did which is one reason I'm director of Vanderbilt Institute for Digital Learning right now but these are class ratings and you'll notice that I guess the surprise for me was having my students watch other professors lectures my instructor rating has increased across virtually all classes in my field a special interest artificial intelligence it's held steady so these are ratings that are taken at the end of the semester and they're on five point scales and on a five point scale and I've done this now in three classes I've highlighted in green green box the database progression at the bottom there of that green box you can see in my pre mook use of materials my ratings were three is average four is very good five is excellent they were decent ratings but they were nothing to write home about but those ratings have increased the means have increased as I've adopted mook use and as I have adjusted to it so that now at the top in spring 2013 a semester ago things are quite good I think if you look at the bottom observation just that summary observation in the red box again the experience was that the instructor rating was typically going up or at least holding steady and as importantly I think the standard deviations are going down now there are some confounds to all of this I was really excited about the course I don't think quite frankly that the fact that the students were watching another professor's lectures even though they were very good was primarily responsible for this increase although I think it was responsible for some of it because they could go back they could watch the lectures over and over again they were very good lectures but I think in large part this is due to the fact that they are engaged in active learning inside the classroom the confounds here are probably several I was a lot more excited to go in and work with a class in smaller groups than I was to what lecture from PowerPoint quite frankly, lecturing from PowerPoint has become very old so that's an example of a confound I was just a lot more enthusiastic about some of these classes than I had been before but you know as a good academic I think if you're going to be using other people's content you start producing your own and I started producing my own for my AI courses first then for my database courses and I would simply do voice over PowerPoint what I noticed after putting up producing some of this content putting it up online because I have been impressed by the very good nuts and bolts lectures of some of my colleagues if you don't know what a nuts and bolts lecture is it's describing an algorithm or a mathematical proof in some detail it's not a kind of inspirational lecture or a big picture lecture it's sort of getting into the trench and working on something technical and again I think these are much better online and I started doing my own nuts and bolts lecture and I was happy to find that I could do outstanding nuts and bolts lectures too as long as I was going to put the time into them but what happened after I did this is other students started finding my lectures online and started using it and this was you know this was quite a revelation when this first started happening you can see the student comments here they were coming to my youtube channel to look at my ai videos these were students in a UC Berkeley MOOC and artificial intelligence so MOOC students coming to my youtube station to to you know remediate their understanding of some of the content that had been described in that in that MOOC so this was when I went back and verified this but this is sort of an example updates statistics on my on my youtube channel and at the top left there you can see you can get a glimpse of the pattern of performance over the number of views 37,000 views is small by MOOC number it's a lot more views than I've ever gotten before and elsewhere on the far left see the subscribers 121 this is the first time I've ever had followers of any kind in my life so this is kind of a kick as well but and this is going to be something that I come back to is you look at a pattern like this and it impressed upon me to think that much of our educational finding work so far as MOOCs are concerned is looking at patterns of student behavior within the MOOC but really I think where we will go I hope where we will go is understanding that a MOOC is I like the sun and the solar system and there's a much larger ad hoc community that grows up around the MOOC so what we will be looking at eventually is that mining the data from that larger community so my hypothesis about my YouTube channel performance is that is that these high points are synced with the MOOC views and to some extent I verified that now something that I have done since this experience of students using my MOOC is that edX has just opened the way for what they call closed instances and I am now using that 188 MOOC a material from that 188 from UC Berkeley as part of my current artificial intelligence course and I don't show you a screenshot here but I can go in and they see this closed instance with the UC Berkeley lecture material and quiz material test material and I can go in and I have gone in and swapped out some of their material and substituted some of mine I've augmented the material the lecture material with some of my own I've augmented it with lectures from University of British Columbia so I've gone in and I've customized my course UC Berkeley content Vanderbilt content University of British Columbia content probably down the road other content as well this is something I'll second that something that I think is really exciting to me is that this MOOC activity has made me feel for the first time as a teacher that I am a member of a community I mean as a scholar I've been a member of a research community for a very long time but I have never felt like I was a member of a community as a teacher when I walked into the class it was my class and quite frankly as you get older the idea that you are now a member of a teaching community is a lot more exciting than the whole lone wolf idea that you walk into your class and it's your class that's just an old concept forming out something that I don't enjoy nearly as much as being a large or scholarly light community of teachers just to show you that we can do other kinds of content it's not all a video this is a wiki book that I started that's community driven the intent here is to use well we're running short on time so I won't explain it but community driven content in the form of wiki books open source materials the AI course uses a text which is freely available online as well that I can adopt and use in my lectures because of the licensing the next time that I teach a machine learning course online I really expect that I'm going to be customising it even to a greater extent than I did before and these are the two main themes that I want to stress right now which I've already talked about but also customisation I overheard the last speaker talking about University of Edinburgh say I have a planning course but if I ever teach a machine learning course again I think I will customise it by drawing on if I can material from different sources not just the machine learning course from Stanford on the Coursera platform that I've used before but another machine learning move that may be offered from Washington and some of the learning content that's in the AI planning courses natural language courses as well as some of my own content so I'm now getting pretty comfortable with the idea that I can customise courses I can draw from different sources given my edX experience and students aren't freaked out about this they like it they like looking at different professors for different content that's not something I had thought that they might like but they seem to so educational data mining just some questions suggested by this experience I think we want to look beyond individual MOOCs and mine data from what I call the MOOCs solar system how do MOOCs interact with YouTube Wikipedia, other sources there's going to be a challenger how do we get that data Google going to hand us the data for this kind of thing how do we link users in Coursera course or edX course or Udacity course with users in some of these other open source environments so some real challenges there about how you get that data how do you link it up and of course how do you mine it I think one thing that I've been tempted to do is start creating content intentionally so not just putting when I put up material when I produce material now I tend to be busy right now but when I do I intend to be pretty strategic about it and I'm going to put up content that fills gaps in MOOCs I as an expert in a MOOC know where students are going to have a problem with content in a MOOC and I can fill that content in I can bridge that those points of difficulty in advance with content of my own and I think we'll see other examples of that I want to have my graduate students do this I've had graduate students in my advanced courses produce and upload educational content as a requirement of the graduate course and I think I will do that more and I will have them look to MOOCs to figure out exactly what content will be most useful given the current resources on the web if we go below to benefits of the on MOOCs I think there are some real so flipping the classrooms an obvious one I think there's real first design issues the previous speaker mentioned that online learning education has been around for a long time and that's true but how do you take as an on-campus instructor I think this is a new question and incorporate a MOOC in full or in part into an existing on-campus course in a way that's ideal in some optimal way in a way that maintains the faculty engaged with the second bullet there and which is not a no-brainer at all I mean we can imagine faculty members incorporating MOOCs and then sort of becoming very disengaged unengaged with their courses these are things we're going to want to better understand and then with these local learning communities and just the learning with the material how do students interact as a local learning community in their own class with the global learning community that's part of the MOOC if in fact the MOOC in the on-campus course is synced I'm going to ask Fiona for a time check how am I looking okay we can spare you a few more minutes how much longer do you have to go if you want to ask you some questions I can go a very long time I will get a couple more points then I will close brilliant okay so this is even in October 2012 you could complete essentially a computer science degree online for free given the MOOCs from different places and I think that you will with all this plethora of resources not just MOOCs resources on YouTube Cullen Academy elsewhere well we know students are already customizing curricula if you will they're looking at different ways of working their way through a sequence of courses so they might take the intro course at X in programming they might take a second course in data structures from Udacity and then move on to some Coursera courses from different universities and so they create their own sequence certainly do this but once start once students start creating their own sequences and start embedding these within larger social networks we now open the possibility of crowd sourcing and that some of these trajectories through existing online courses and other remedial material are endorsed or not endorsed by the community and then become consensus favorites and so I think crowd sourcing curricula is going to be something that's going to emerge in the near future Vanderbilt and the University of Maryland have just joined forces to offer not the first MOOC sequence so these are back to back MOOCs with a kind of loose intended to have a loose coupling and so the students will get something added by doing the sequence above and beyond simply taking two independent MOOCs again I think you will see university partnerships that create these curricula constructs that then are vetted by the community I'm going to go to the very it's not the very end but this is my last slide design strategies for MOOCs we want to start designing MOOCs with local learning communities in mind not just so much of the MOOC landscape that was opportunistic we get we use MOOCs local learning communities but how can we design what are the design criteria when we actually start deliberately designing MOOCs with local learning communities as well as global learning communities in mind design MOOCs for remixing again I think customization is going to be a big thing and if you can create your MOOCs so that I can swap in certain things and take out other things I think it will be a benefit and finally design MOOCs with research opportunities in mind so we can use MOOCs for research questions right now much of the data mining is again opportunistic let's look at the data and see what happens but let's say that midway through a class I ask half the students the following question if you finish with distinction would you be willing to become a community TA for the next offering of this class I ask that question of half the students I don't ask it of the other half and I look to see how such a question as simple as it may seem affects retention rates affects completion rates affects the rates at which students obtain distinction another example that we might ask is we're interested in looking at MOOCs and other kinds of social media we want to have students tweet some online lectures and then take quizzes half the students tweet half the students don't in real time look to see how this improves or doesn't comprehension in quizzes and whatnot and so moving from opportunistic research to deliberative planned hypothesis testing is something that I think certainly a Vanderbilt you'll see more of and but I can't help but to think you'll see it everywhere if it's not happening already I guess I'll just stop right there because as I said I could go on a long time no no that's lovely thank you so much thank you very much it's really weird talking to a screen so it's really odd that I can't see you but I'm just going to ask if there are any questions for you does anyone have any questions for Doug oh yeah we do I just get them out of the phone so you can hear I don't this is Helena Glesby from U.A. in Norwich and I was just wondering if we could transport you back a few years to your beginnings of dabbling with MOOCs can you hear me clearly please Fiona can you repeat that we're just going to come up to the front okay this is Helen Glesby from U.A. and I just wanted to ask you if we could transport you back in time a few years to the beginnings of your dabbling in the world of MOOCs is there anything that you would do differently because that's kind of where we're at at the moment and I think we'd kind of like to learn from your experience and if there are some tracks out there we'd quite like not to fall into them so is there anything that you would change or do differently or is the fact I really like the way you just sort of describe it as sort of serendipitous is the serendipitous thing just part of what we should embrace thanks well I think to some extent the serendipitous is something we should embrace we should try certainly new things I am sort of naturally conservative in doing these kinds of things and I think that that conservatism was an advantage one thing I was told by the higher level administration here is you are in your current position as director because you did some interesting things and you did them by the rules so for example if you have MOOCs in your courses this is something to think about if you use MOOC materials in your classes at least it was the case that some of the terms of service for some of these groups for Sarah for example said that you needed explicit written permission if you were going to use this materials part of a tuition bearing course now the problem is if an instructor requires that students turn in assignments from the MOOC as part of their on campus course the instructor is not in violation of those terms of service the students are in violation of those terms of service for turning it in and you don't want to as an instructor put your students in violation of terms of service so those kinds of sort of legalistic traps are things to think about as you move forward you might start off slowly I sort of naturally started off slowly I did not rely on any of the materializing of the lectures I quizzed the students myself I had them do the different assignments than the MOOC assignments so they were watching lectures but I was still remaining very engaged in the course again I think a possible trap is to some extent something I have felt this year for the first time but I think it's probably just due to my new job rather than the way I structured the course per se I think we want to be aware of getting to faculty becoming disengaged from their students because they are relying heavily or they start relying too heavily on the MOOC material there is also I think you want to think about how is the online material going to relate to the in class experience creating in class activity takes time and it won't take the time to create that in class activity I think you will fall down as an instructor the students won't like it so there has to be some relationship between them time for one more short question there's no more questions thank you so much that was excellent thank you very much thank you we'll just go straight into coventry now have you ever heard of coventry that lovely but we can always cut out a little time from the break what we want to do is it's okay to be the current generation this idea of using our material has always been available and possible but the claim was to start with higher education it was interesting giving the fact that there is also research to focus on universities the argument that staff have got time for teaching so one question is why should they do it now because in a way it's in the public I do think there's more of a sense of community around this I do really feel that I feel that more than everyone says this is not new we've been doing this for years but what we haven't had is a collective being so it's the culture of remixing so there's been a cultural change and that cultural change has happened that's sorry talking about the next door of this person it's the culture we've got a culture of remixing and it's emerging and we're understanding it and as our understanding is defined and as our individual practice becomes more familiar with that then we're more likely for that to happen and there are some people for whom it would be too much for other people for them to understand and the same thing that don't say about having the students contributing to students producers and students as creators the point is that might be more likely to happen when the students are already doing that it's something that they've just grown up doing maybe sitting, visiting maybe just mixing remixing, it's just the thing that you do I don't think the remixing thing is a new thing at all I think we've found out through three years being sold at a high price but now I've made images and images are abundant they can be reproduced at no cost and shipped at no cost and so my business model had to change I mean I've come to recognise this in retrospect now but that was a really important moment understanding that in fact the photograph does one thing it's fixed in time and it's location specific whereas the image the image is abundant and it's distributed and it can connect you to other people and so in 2008 a friend of mine asked me to write some classes for a new course a new university photography course and I said I would do it but I couldn't write the same classes that I've learnt as a student because they've been written in the 70s or whatever and it would be entirely disingenuous to hold up these famous people that I have photographed that represented a business model that changed completely and then to try and sell that so I said I'll write the classes but it has to address these issues it has to address the fact that a century photographer is when everyone has a camera and the means to publish what is it that makes a professional different and so that was the basis for writing the classes now I had to learn how to teach pretty quickly and I had to learn what to teach pretty quickly and so it made sense to stick the class on a blog to ask the big difficult questions publicly and then seek out answers from whoever had them for instance this idea about rethinking your product that's what I had to do as a photographer what was my product when it couldn't be photographs anymore when I made images now that was something the science fiction writer Corey Doctoro helped me to come to an understanding of when he shared his business model one of his business models he was giving away e-books e-versions of his book for free and still selling hard copy books and he was making a healthy living in this he shared that idea of versioning of what we do rethinking what we do with me and I trialled it as a photographer and it was quite successful so we even picked things like that in the classes that was the beginnings of a class called picture in the body known as picbod now by its twitter hashtag and it was something we went on to develop and explore even further in another class called photography and narrative which is known now by its hashtag phono just the sunrise that you took your experiences as a photographer where the business model was about allowing more openness to what you were doing as a photographer gave you greater reach and greater impact to mirror that in an education setting yes absolutely yes that was it what I did was I realised that the images would connect me to people using the internet enabling me to use the image to connect to people who I could reach it was a very discerning way of reaching people who would be most interested in other versions of my product and so I could use the image to reach out to distributed geeks let's say who would then come to me to buy this other version of the product let's say the print for instance so a superfan of Heath Ledger I would come to me to buy another version of the product buy the sign print so yes that was something that enabled me to do when I put it on the blog the first thing that people said was if you give this away for free then no one's going to buy it and I had to sort of then say but this is similar to my photography my journey is a photography in fact the product isn't the knowledge you can give that away for free the product is the learning experience and what giving away the knowledge does if you will is it connects all those people and you have this very networked experience with a huge amount of added value and added opportunities for the people who bought the version which is to sit in the room the premium version so just when thinking about you've got a vision of what you want to achieve in terms of the teaching design what was your starting point was the university VLE ever in the picture or was it you wanted to start a clean piece of paper from scratch and how did you get from that to where you are now I realised very quickly that any barrier to entry would stop most people engaging then there are a number of barriers to entry so with regard to the university VLE one was that you had to pay £27,000 to get access to that so that narrowed down the class quite quickly and substantially to whatever it's 40, 17 year olds or 40, 18 year olds in the room so that really wasn't wasn't appropriate and similarly every time we considered a space or a different way of teaching or speaking I had to think is this going to be a barrier to entry so it's not using academic language so that anyone can engage with it not using discipline specific language so that people could engage with it using a blog that people didn't have to sign up to to engage with that was important so that people would drop by and drop in and of course there are other reasons as well for using I was using blogger at the time that's the first one I did I had no money had no money at all and no resources and so everything had to be free and I realised one of the strengths was that I had no money I couldn't afford to build an entirely new network I had to go to the networks that already established and those networks happened to be really well populated and so it meant going to Twitter to Flickr to Vineyard, going to Soundcloud all those spaces where the fish were already swimming as it were and go in there and say who's interested in this class who's interested in this subject area having them to change their existing social media behaviour behaviours in order to do something entirely different I mean that's I've come to understand now that's largely unsustainable don't build a new version of Facebook at university use Facebook it's really good at being Facebook so with these spaces as a photographer were you already present on Flickr present on Vineyard were any of these new spaces for you? it was all new spaces I hadn't even, I'd never had a blog I had to learn from scratch and so I go back now and I was quite open about it I didn't know how to teach I needed to learn as quickly as possible and I asked for help I didn't know what to teach there were no answers the book's not been written yet on what it is to be a 21st century visual storyteller so if you go back to that first it is one blog post for the entire course the entire course just gets the entire post longer and longer and longer with the comments just getting longer it's acutely embarrassing but then that's just the nature of learning openly it is people have been nothing but forgiving to me every time I've asked for help people have come out and helped no one's ever said that's rubbish what you did there people have always been very generous and always sort of sought help sitting next to Alan Ovee continues to help me constantly and I hope I feed back to him the point was no I'd never blogged I'd never used any space before so I came as a learner I came as a peer learner and I learned from my students who were using these spaces and they could tell me how to navigate them that wasn't actually a bad thing in itself it's probably at this point that we're pulling in Alan so around 2008 2009 was that when DS106 started? actually it was 2010 2010 that was Jim Groom's creation at the University of Mary Washington I had an interest in this idea of digital storytelling I went to a conference first in the 90s when it was really a film based being popularized very well by the Center for Digital Storytelling a lot about personal narrative and people learning how to craft a story and then present it in digital form and so I was always interested because I've long had an interest in the platform of the web as means of expression and publishing and the idea that a story on the web was innately different from a story in sound or in video because of the affordances of the web to link to cross media there's people like Henry Jenkins who's been researching the idea of trans media storytelling and Jim Groom took this course at the University of Mary Washington it is on the books as a computer science class as digital storytelling had been taught in a classroom by a teacher with lectures at a book and students doing video and when Jim took it on in January of 2010 he had this idea of putting the students in the web to do their work but also with the idea of importance of this notion about students learning to be able to create their own digital space so they would have to register web domains not use anything the university provided and learn not only the things about publishing their work but also what it took to install WordPress to manage things to have control over that digital space and I was interested among other people I followed what they were doing because the students were publishing their work in the open Jim was publishing his class in the open and you could be part of it as an observer and that was very interesting and in October of 2010 someone asked him when this was in the context of the first MOOC the CCKO8 that George Siemens and Steven founded the idea about having a network that participated in the course so we were aware influenced by that but the idea of DS106 in January 2010 being something that people not registered in the course could participate in they would set up their own blogs or content feed into the central course and the parallels I think that are very important and why I got very interested in what Jonathan was doing different sort of approaches the platforms but similar in the idea that the students who paid for the class and had the classroom experience you weren't trying to replicate that same experience for the open participants and at the same time what the open participants did was not the same as students were doing the overlap so you benefit by having people from the outside participate, react, communicate maybe even collaborate with these students but they definitely had a different experience and to me that's what makes what we're doing not really the same thing where you're trying to have everybody go through and get this same carbon copy experience something that's quite interesting that's kind of afraid between these is the the literacies that people are developing as they go on to Jonathan you were saying you were picking this, you were learning as you were going along I imagine a lot of your students are learning as they're going along it's a course after all they should be learning but they're learning about the kind of digital presence being online publishing stuff online is that all kind of integrated into the course in terms of support in terms of support it's kind of fundamental and it's grown to be fundamental we've talked about digital photography for years but we haven't actually thought about what that means we've always talked about what it does it means a digital camera but to actually discuss what the image means we've never really done that and so yes now the course is much more now about visual literacy digital fluency and so yes it's about having something to say being able to say it very clearly but then amongst all the digital noise it's about being heard and that's the bit about the digital fluency so is it supported in a sort of academic sense yes it's now part of the fabric of the course we have to learn how to use these spaces and how to use and navigate through these spaces these digital spaces we also sort of think about what it is to be a sort of digital citizen as well I suppose you get this question quite often how much of an administration overhead is it to run the course in the way you have had administration overhead okay so yeah we should clarify exactly me opening up by saying I don't want to move so mine is a regular class which sits within a closed undergraduate course photography course it lasts for 10 weeks but as Alan described there is a version of the experience that you can have by joining via the blog submitting your pictures, listening to the lectures one of the crucial things is tweeting your notes so if people are listening now and I'm not making sense if you tweet your questions we haven't established a hashtag so I guess you're maybe going to establish a hashtag afterwards because I'm listening in now just as the contributors are to phonar after we've recorded a lecture they listen in very often and they'll actually respond to the notes and the comments that people put as they listen to the lectures and so I'll do that now, I'll try and answer questions now if people have them but I have gone and lost my thread but if I could jump in I mean Jonathan Downplay has not been an academic but his intuition is so perfect for this about how people engage and the idea the unmoog idea that it's not this box that's a course, that is an entire experience that there's this whole cultural and social framework that builds around it that means human to human connection and the way we interact with each other which you can't package you can't plan, you can set up so in DS 106 we don't prescribe so much the digital classes and learning ends up like Jonathan describes focusing on the tools and it becomes so minute and uninteresting and the end product and students come in with I refer to it as like an assignment mindset they need to produce something that's polished and that's what they chuck in the box and what we really want to encourage is the people to sort of share their process and share the thinking behind the media things that they create and create stuff that's bad so we have sort of the same philosophy about we don't come off as teachers of this class as being perfect so we mess up in public and students see that and they realize I don't have to be perfect and I can try something new that doesn't have to be perfect and it's okay and then they get the support of the community around them so that community building thing and even this idea we always talk about building community you don't really build communities they happen and they don't happen accidentally they happen in some nexus of place whether it's a cafe or a park but they don't always happen where you build your nice shiny course mook box and those communities as well that have become probably the most valuable aspect of our entire course currently there are still only two open classes in the degree program but you know the most people we've had come to one ten week iterations over 35,000 now that's 35 compound network opportunities from that many people coming to look at your work and listen to the class with you over 10 weeks it's phenomenal and as you say you can build those you can put the things in place I hope those networks can build but one of the most valuable things that people can now draw from some of the stuff that we've done is to adapt or adopt some of the things that we've done because they can make less mistakes don't want people to go away thinking that what we do even though I've described it as learning how to teach and learning what to teach that it likes rigor or any sort of sustained academic engagement because it doesn't I think it is very rigorous and there is we work really really hard to make sure the quality is the very best that we can make it we should rethink what the product is as an education institution as educators I can't see a reason for not teaching this way it's so valuable it enriches the learning experience so much it repositions what we do and it actually it doesn't actually cost that much at all it just kind of makes sense we're talking about open and connected teaching and learning here I'm not going to use the word MOOC because mine isn't a mine isn't mine isn't distance learning on an industrial scale which is what I see when I see most MOOCs that actually works out to D Laus I think if we're going to start that kind of but what I'm saying is to actually think that the class isn't already connected is the kind of mistake because the class is connected they're all on Facebook they're all on Twitter so the class is connected the only person at this point who isn't connected is the teacher and so where's the sense in that if the teacher just connects to this network then they can augment and enrich the experience not only of learning but of teaching as well I've learnt how to teach and I'm learning how to teach from people that are really into this and they're really good teachers you know it's accelerated and I've learnt to do what I do I don't know 100 fold it must be and so it seems to me that there kind of isn't an option there isn't an option for photographers anymore the old business world was broken the product is very very different now if you want to operate as a visual storyteller in the 21st century then you have to rethink what your skillsets are and what your values are as a supplier and so I think it's much the same for educators that we have to rethink what we are and we're in a great position to do this but the bricks and mortar experience is very expensive to run but it's marvellous, it's great it's what students want to go to college for there is no reason why you can't have this virtual version as well enriching that on-site experience it really is a win-win virtual circle what are we going to describe it as I mean Jonathan's parallel example of his recognition of how his business model for photography is not adaptable to the way technology and culture are going MOOCs get expensive largely because of this reliance on high production video that's where those big cost are that's why they're getting into these hundreds of thousands of dollars there's a lot being put into these high production videos and that's not really changing much of the methodology of the teaching that's one of those transference of what has gone inside the classroom to going on the web and if people are sparking up there on Twitter tweet decks or whatever they use if you search on the hashtag phonear then the reason I'm not with you today is because I had to teach my phonear class this morning and Alan Carney flew over from America just to attend our phonear class in person that is the only reason I'm in the UK Jonathan you know that what's you doing here then I'm soaking up the sunshine and if you do do a search on phonear right now then you should see the echoes of today's class so today we will have listened to a lecture to an interview that I did this morning with an artist with Sarah Davidman and with the founder of the Path Charlotte Institute in Bangladesh which I did last week he's gone back to Bangladesh and so you should see echoes of the tweets the comments and the discussion there if you scroll through that I hope so anyway but if you tweet now or have been tweeting and hopefully I'm responding to and Alan is responding to your comments and questions that was a bit meta we've asked people to tweet some questions so we do have some questions but has anyone got any questions right now that they like to ask we should can you guys hear us can you hear Jonathan can Jonathan hear please put thumbs up but we can unhear you that's the problem there so are there any questions stunned silence I've got a question actually and that is I don't know from Adam Warren from Southampton my question is does your model only work for particular disciplines so for example how would it work with hotel management or surgery is perhaps more realistic example hi can you hear me yep you know that's that's something we have been thinking about constantly that's where the really interesting stuff future of this lies I think is pushing it out beyond digital storytelling into into other areas I mean surgery no I'd still go with a dentist or a doctor that had attended bricks and mortar institution and I knew you know that had gone through the regular system but hotel management I'll be honest I haven't thought about it I haven't thought about it so asking me to do it on the fly I'm probably going to do a bad job of it I have begun to think about how it might apply to a maths course or an English literature course or an English language course or perhaps even a science course then I think that's really interesting where you start to push out into the audience stroke community that are attending the class and ask questions of them and sort of as both a learner and as both a teacher and a learner so the short answer is I think it's absolutely not only applicable to photography if we just started this out I think if we just started out wanting to write an open and connected class I don't think we would have started with photography we might have looked at a storytelling class but I don't think we would have started with photography so yeah I think there are lots of things that other people can adapt and adopt and I hope they do Does that answer that question or is that just a rith? No that's fine We have a question via Twitter and that was Who owns the content? Is it you or is it the students? Yes I try to answer that or I try to answer one question so well I'm very clear about this I write everything that I write is licensed to CCBY so that it's there specifically to be adapted and adopted but over the course of the class and each iteration the students feed in or the lecturers chip in and it moves beyond anything that I originally authored and so who owns it and I guess we all own it is there another question that's buried within that ownership question one of one who if you own it do you get to sell it or something like that? I don't know I guess that's my question I guess there is something probably hidden within there I guess I'm asking you if there have been any problems about that because my campus based students we do various things and they're blogging at the moment and one of the things that they're getting concerned about is who's going to read their blog and what I'm going to do with it and I think it's partly unfamiliar these are first years with the medium that's making them nervous but because there is this kind of hype in the media about young people and digital content and digital identities and somebody out there on the internet is watching you I just wondered whether those two things had kind of come up against each other in kind of unhelpful ways or whether students feel comfortable in sharing and sharing ownership of materials in the same way that you and I might do as educators Does that make sense? It does make sense, yes There definitely isn't a silver-body's answer and I think one has to take each case by case some students really are reluctant to share stuff and you can always make blogs closed or they can choose to not blog if that's the case but on this idea of navigating these online social spaces if we will it's not a better place to learn about that than at school with a teacher I want my kids to learn this as a part of their learning experience this was my point which I made really badly about people saying they teach digital photography where they're not teaching about making photographs in the image in digital spaces I see it as part of my job my role to take them by the hand and take them through these digital spaces or isn't appropriate to share to discuss in a safe relatively safe environment why perhaps it isn't appropriate to share that in this space or how to deal with that very very negative response that you received to that work or the attack that I got in the press we had one class where the students became very territorial when someone wrote an article that was critical of me and I said you know what you can choose to jump on the comments section start attacking these people back or someone gave me some great advice some time ago and they said if you are attacked there are only two things that you do one of them is that you completely ignore this person you do not shine your spotlight on them or the other thing is that you befriend them and you are completely nice to them now I passed that lesson on because it was really valuable to me to learn that I was really struggling to cope with the attacks that I was receiving and it turned into a really valuable lesson to the students so we wouldn't have had that had I not been criticised publicly and had the students not been in that social space thank you very much so are there any more questions no okay lovely okay well thank you very much thank you very much thank you quick break now we've got some tea, coffee and cake out there just in case you have a love cake at lunch okay okay and then we'll just come back in just to get the coffee okay okay you're welcome enjoying it it feels like like a mashup it feels like but it feels like it's more cut but I think it's really important that you have this and you have to have a tutor okay okay okay okay okay okay okay okay okay okay okay great thank you very much I hope you enjoyed the tea and cake with more cake out there if you need it for later on this is your session innovation technology manager show me the data this is really interesting what can we do with the data no pressure I'm sure you all want to I need to get back up to Edinburgh as well I'm being really safe with this one slides so we'll see how that goes and did have another joke but I've forgotten it so I'll just move on oh yeah I remember now I'm not just the AV guy for those that don't know me I do a lot of things lots of different things and one of those things is I like dabbling with data and analytics in particular learning analytics which I hope all of you have that word buzzing in your ears and PVCs asking you questions like what should we be doing about learning analytics in the context of MOOCs this was a slide I got from Simon Buckingham who's doing a lot of the future learning learning analytics just as Google made their fortune off data and Amazon have made their fortune off data this is where a lot of people see MOOCs making their money off data so it's not a surprise when Corsera announced that they're tying in their whole platform with a career service because they want to use the data that you're generating constantly to come up with something but there's before I go a bit more into this it's probably worth just considering what analytics it is this is a definition that my former colleague Adam Cooper put forward if you haven't seen it there's a series of briefing papers published by Cetus on not just learning analytics the entire spectrum of analytics and it's a very good set of publications for you to have a look at and this is the definition that he put forward of actual insights so taking something and producing something that we can do as a result of it but breaking this down into a very simplistic model you've got some data, you do some analytics you get some insight but when you start looking at this one in more detail insight, who's this insight for there's various actors within this and it was quite nice to see Simon Nelson this morning talking about the self, the learner so the learner getting some insight about where they should take their own direction of study also you have the tutor who quite often at the tutor level it might be how do I make this better how do I make this design better where the things the points where the students are struggling the institution the insight for them might be who are students that we want to convert onto our courses and so we have this mix of educational commercial drivers as well to get to the insight you need an analytic how you know the top section of that list comes from a paper that Rebecca Berkson and Simon on five different types of learning analytics for social you've got social network analysis, discourse content disposition content one of the reasons I'm quite interested in the field of learning analytics is because it draws in so many different disciplines there's so many different ways that you might want to analyse data and I think it's time to keep reminding ourselves it's not just about learning analytics in this context administration that is another aspect of analysis that's quite important and then the data I was thinking about what data there is and we're in the current climate of future learn, Coursera, edX it's very a platform as a service model so there's data being collected by these platforms but in the wider context if you start pulling in connectivist style courses you have service service data so data from Twitter, Facebook the open web so these can all be sources that pull in and there's I think something that's quite often overlooked is access and availability of that data or perhaps it's not being overlooked now I'm sure quite a few of you now running courses are scratching your heads about where's the data where's the useful data where's the data I can actually use and there's lots of threads from data to an insight so what I thought I'd do is to show you some existing examples of how analytics has been applied to open courses this is one that really caught my eye and I was quite early interested in analytics and I saw this if it works hopefully I said there was no tech I lied, there's a video this is definitely cruel so here's an example of that from Andrew's machine learning class again this is a distribution of wrong answers to one of the programming assignments the answers happen to be pairs of numbers which is why we can graph them on this two dimensional plot each little x is a different wrong answer now you can see that some of the x's are small representing one-off answers where students just have their own particular misunderstanding but for example that big x at the top left is where 2,000 students had the exact same wrong answer now if you have 2 students in a class of 100 that have the exact same wrong answer you'd never notice but when you have 2,000 students it kind of jumps out at you and so Andrew and his DA's went in and looked at that assignment and understood the misconception and then they constructed a targeted error message that any other student whose answer fell into that particular bucket would get that targeted error message as opposed to just the wrong and that gave a much more personalized and useful experience to the students because it put them on the right track in terms of what they needed to fix in order to get the right answer and so this effectively is a much more personalized experience that you can do by utilizing this large amount of data that we have the questions I'd like you to consider are where did that data come from? was it in a wall log? obviously Coursera collecting it so that's the other thing they have to be collecting this data so that they can query it and then someone's got to do that query it and come up with a graph do your platforms that you're using for open courses provide that for you? so data access and data shape are all quite important one of the other examples of this was they were actually using it for error detection to find the questions that they had posed incorrectly so a tutor had created a question and he himself had got the wrong answer and so they're using these techniques to identify bad tutors bad questions which is another way to look at it it shows a post it shows that there's a disconnect there and then you look into the reasons but by the way it's joining them and also thinking about this is a very simple XY plot but if you were wanting to query that data on demographics was it gender biased location age how would you do that with your data? this is quite a recent one I should have probably paid more attention to this before I did the video with Jonathan it's looking at the length of videos in open courses that are most effective most effective being people got to the end of it so they didn't drop out so in the 6 to 9 minute range so these are groupings of video length I should say this is from edX this is edX data and this is the median time spent watching the video so in the 6 to 9 minute you've got a key how you know they watch 6 to minutes of it so the conclusion is 6 minutes make your videos 6 minutes not a minute longer, not a minute less 6 minutes think about the data that's behind this graph so they've recorded student activity in terms of when did they click the play button when did they stop the play button and then this is a a summary of I think it's about 20,000 videos or 20,000 students so how would you get to this graph I have answers for these later you'll be pleased to hear this one's quite interesting this is getting more advanced this is Coursera data it's quite interesting this analysis is looking at engagement and disengagement patterns within Coursera so we have they're using the assignments as basically an indicator of whether or not a student has is still active within the course they came up with a categorisation so a auditing I think we would use terms like learning you've got the German peripheral participation d behind they are still doing the assignments but out of sequence T is on track and O is out and this was a summary of I think about three or six classes what's also quite interesting about this analysis is produced by students Stanford had the litics lab where some of their students have been given access to Coursera data and this is what they're coming up with interesting analysis I think so to come up with these clusterings they're using K-means which is a fairly standard analytical tool and one of the reasons that they were interested in this was they wanted to make comparisons between courses so was there characteristics of a particular course that suited another one so by having a way then to pull out the data analyse it, allow them to start identifying differences but again, they've had to pull out the data they've had to do some you can find the paper online they've had to do several iterations to come up with this point and there's still work they say to be done to refine this further so I've kind of been prodding you about this whole data thing data data data which hopefully is the question that you keep asking future learners Coursera, data data data data but it has to be the right data so if you're wanting to do analysis of videos you need that data so that data needs to be recorded in your system it's not just about data it's getting access to it even within an institution there can be terrible issues with getting access to the data that you need so if you're, even if you're using Blackboard getting the data out there is a nightmare who has permissions to give that data Google Analytics which I'll come to in a second is a fantastic tool for getting data and summarising and analysing data but usually it's just a web admin that has access to that being distributed through your institution so we have, you have to get the data you have to have the data accessible the other thing is the shape I was fortunate to be given access to data from one of the new platforms I can't say which I'm under a non-disclosure agreement but it was just the mySQL table dump a database so I'm talking to people from is Amy still here is that what you get from Coursera just a database dump when you're kind of left to your own devices to work out I think you can provide data in more useful means not quite like CSV future learners will be my data as a big scary spreadsheet but also in a nice tiny pink Ruben and Custodd play with the power point for the end data I find that very helpful but I guess when I probably get better at this they're not asking questions that I would want to ask that's it so that's what helps which hopefully if I remember my slides flows, data flows so it's creating these recipes or you can get a dump of data out of your provider but you're able to quickly turn that into site that's useful to you that provides the particular insight that you need and there's a whole list of tools that you can do to do that but I think within open courses there's a great opportunity to actually share some of the expertise around whether or not the people that have the dollar signs in their eyes will permit that hopefully they will so I just wanted to highlight some kind of quick wins easy alternatives that you might not be aware of but might stimulate some of your thoughts in this area so here we have again the longest video blah blah blah 20,000 students several weeks work this is YouTube have you done the I don't know if you can see this this is the audience attention graph on a YouTube clip so this is one of my videos this is about where we are that's what I gave well no one liked it we got just about 20% of people got to the ends what was that about so this is this is there on YouTube and YouTube do provide the export to a degree so we can see it nice and gradually get up but that was so this video is kind of a nice fluffy visualisation video this is a different video this is an instructional video look at these peaks we got people and this is the nice thing about this when I play this band I can see the video so I can see the point when people are hanging on so this is an interesting point in terms of cognition, misconception maybe this video has I think 2,000 views so you got quite a decent data set there of what's going on and there's other things within the YouTube analytics about demographics, location gender and which might help you there's a you can you can specify the date range oh so yeah it's not one I've dug deeply into you can see the thunder so you can very quickly go through a jump how do you have an issue with going through a jump so don't know would be the honest answer and whether or not that information is public is another question which is another consideration of you to go down this route so the MCQ test what you can do in Google Analytics is you can actually do event tracking so if you've got a MCQ and you've got Google Analytics you can track which response people made to that MCQ you can target is correct or wrong within Google Analytics itself you can get a summary chart Google Analytics also allows you to do so we could do which Daphne didn't have I could do age gender I can run the experiment with Google Analytics, AV testing so I could be tweaking the platform design running tests and getting data out of Google Analytics so early I mentioned the issue of, quite often Google Analytics is held by your web team and they don't like can you admin write or I admin access you can get out as CSV, it's a manual process but there are automated ways of doing it as well basically now Google have a bit code that lets you proxy the data so you can create a segment or an analysis or a slice of data and then just make that available on a server behind authentication so you can give people access to the raw data in a controlled way so the next one how am I doing on time 3 most so this is canvas, have you come across canvas VLE made in the States canvas have canvas network so basically it's a platform to provide open courses so you can say you have the standard kind of VLE tools, discussion forms announcements, assignments really nice thing about canvas is that it has an API APIs basically an API is a way for your platform will have lots of data locked away in it or processes or functions and you have something you want to do with that data processes functions so you write that code that talks to their server and our server gives it back to you the canvas one is very well documented so we can take a discussion forum within canvas and with a bit of API magic we get into a spreadsheet this was one I did for the learning analytics open course once it's in a spreadsheet you can just play around with the visualization tools and then this is a Google spreadsheet so I can share it with anyone so I shared it with the course so we could compare how we were doing performance wise the tutors could see how we were doing performance wise having access to an API really helps get the data out so that anyone who has access to it can slice it up, dice it try and find something interesting or useful there's a link up there for more information about that particular one you can go further so this is taking it into social network analysis so we're looking at the individual discussions one of the reasons to go into social network analysis is that it provides insight into how the group are doing particular characters within the group those were all dealing with other people's platforms at all we've actually experimented on our own platform we ran Octel which was an open course that ran early in the year not huge numbers 1400 students registered it was in Workpress so we had full control over the platform which was a godsend because we could create our own data APIs get the data out that we wanted which we found useful so this one it was a connectivist course so we wanted to identify all the students who had just created a blog made their first post and hadn't had any comments has a way to target some insight to the tutor team the target resource to say hey go and give this guy some love and affection go and tell him what he's doing he's good, wrong, right so it's another example of just taking the data out and doing something useful one of the issues within the open context a truly open context when you're not platform specific is that people have different profiles online and there's a danger that they become analytically cloaked but it was quite an eye-opener for me when I came across a site called full contact basically you provide a list of email addresses and it goes off and searches those email addresses against various databases and it comes back with a hit rate of, I think so I put in 250 email addresses and it's come back to 178 people it was recently kind of, you know, it brought who they were and it comes back all this information which Twitter profiles they have which blog, you know, they're on clouds which is quite scary so even within an open context you can actually find out quite a lot about someone and then use that data for good, for good people and we're going on at all to do science with the MOOC research initiative to look at some of this open data and analytically cloaked and this was the last slide so this was from Simon again this idea of sharing some of these recipes sharing some of these tools because the field is very broad and I think we could learn a lot together lovely, thank you very much we'll do the questions in the panel with Martin who can be on the panel and then Amy and Helen are gone so where's the microphone that's running, you've got the microphone okay it's okay but Amy so I'll check it I'll sit here for the end we were just sharing some actual data that's good to meet you anyway that's good to meet you I'm so going to go back to my pro by chance so are there any questions for the panel we've got a microphone roaming around if anyone has any questions there must be something Mike one of the problems with data is when people want to make conclusions and draw conclusions if they haven't got the data they really want then they'll try to draw conclusions from the data they've got so people say here's the data I wish I had that data because that's really important but this is all I've got so I'll draw the conclusions from now any comments? I don't know if we can fight human nature if we can measure the easy things to measure are the things you've got the data and therefore you statements about those some of the important things are harder to measure and so you say nothing about those it goes further than that as soon as you start measuring something the danger is that people just gravitating to that metric so it's Campbell's law I think how it's defined it so yeah there's a complexity of issues in this I think this is a really interesting question because it seems to me that at the moment we're still in a place where for at least UK higher education we could define what makes good for a MOOC we're still in a position where I mean Edinburgh are ahead of most of us but we're still in a position where we might be able to define that and own that a little bit but we could have lots of definitions of what makes good and therefore we could look for different sort of data and so my feeling is that we do actually try to own the kind of agenda that you're referring to and actually make something of that I always tell my students when we do learning technology 101 kind of thing with them and we talk about digital and I say you know when you get something for free online and they go yeah it's great and I said what's the product and they go and I say you're the product and they go oh I don't want to be well don't do it then because if you're not paying then you're the product and I think what we need to do is to try to make sure that we're owning the data here rather than the data owning us and I'm very keen that I present to the people who direct the project in my university the right kind of data to get the right kind of answers because otherwise they're going to get a hold of the wrong sort of data I hate to invoke it in this room but it's a bit like the National Student Survey I mean that data has disproportionate amount of power compared to its actual efficacy as a tool for quality but we need to make sure that doesn't happen in the MOOC world I think and that's why people are so valuable who actually understand big data because we need to actually own that there are questions via Twitter if anyone's interested so Katie Jordan has asked Edinburgh MOOCs as less risky area to try out innovations than campus to what extent is it about the money it's never been about the money there you go Katie yes I guess I don't really understand the question fully then so that meant quite a lot of further chat great on the Twitter stream looking at what were people saying well comments like if it's not about the money then what's happening is you are running exercises of some kind whether it's for marketing or increasing your own internal capacity whichever thing you're going for different people say different things and yet doing it out of the premium of 9000 quid ahead from the students that are currently attending so you're doing your and then okay so I think as well just because we're not necessarily looking at direct return on investment does not mean that we're not getting return on investment in other ways so we've had book deals have been signed as a result of the MOOC activity we've had research publications we've had collaborations, partnerships we've got a PhD student has just started in September as a direct result of being inspired by her EDC MOOC experience that she came to Edinburgh to study with those academics it's things like that that make it worthwhile it's not about the seeing that the money follow those direct learning so this wider impact agenda on things that actually often can't be measured and that's where it becomes a bit difficult I think that direct word is the word we've managed to get out of our kind of business model it's not we're not going to get direct return on investment but there's all sorts of ways that might get in direct return that's what I'm looking at I think it could but I don't think we should be shy if it is about the money that I didn't get to show when I did my presentation was about the MOOC Plus model so how you monetise the MOOC how you keep the teaching and learning bit free and then the stuff that you plug into it the enhanced assessment the face-to-face opportunities the publishing and so on how you monetise that part of it so although I'm aware that there's a kind of element of the MOOC movement that has something in common with the kind of open education movement which is sort of quite a strong philosophy of this is free for the public good I don't think we should say or in that case maybe we shouldn't make money out of it because I think that we probably can and if you look at some of the things that Coursera are doing if they're not making money already then they will be so I completely agree with all of the other things that can happen as a result but I think there's a big difference there between something being sustainable and something being profitable and that there's no harm in having direct income generated but it's never going to be the £100,000 per MOOC instance that suddenly they become very lucrative or it might be that some people may choose to put every plug in and it may make a lot of money but I think often we're concerned about kind of having enough for the overheads of teaching assistants and stuff more than it being making additional money to make the overall piece back but I was just quickly to that Doug Fisher mentioned that he asked his students if you've got a distinction would you like to be a teaching assistant and it's about reputation and it's about interaction and it's about community which is a drawing we had a really really nice instance in equine nutrition where the course was free but the community loved the course so much that they all put into a charity pot a large number of people contributed to this charitable donation as a community they donated that amount of money that had been raised to an equine charity I can't remember which one it was Donkey's Entry or something but it was really lovely to show that there is a design that there's a value within the experience but it doesn't necessarily need to be put back into the course but it could have been if they're choosing it was just that the collective said we want to put this towards We're all heading in the same direction here we all want to do this and we want to do it well and we all recognise that somehow we can't afford to do it for free because we have mortgages to pay everybody needs to be covering their costs at least if not making a little bit of money and we all want to do it really well and I don't think necessarily the two things conflict but there might be people from the OAR movement who would really like to argue with me about that I think Jonathan basically did everything he didn't do a move though but he did it for free he didn't cost them anything he still does his face to face class he has this very rich 44,000 people participating helping his students so maybe you should flip it around and he's got 44,000 graduate teaching assistants already saved me it repurposed all its content and didn't have any videos in the first instance so it was actually free but if you're looking at academic time being already part of the mix and that had 45,000 so yes moog space not moog space much the same you had a question it's probably for Amy again we heard this morning about potential for moogs the positive disruption for the institution and that there would be investment in staff and courses and the innovations would happen there to what extent do you've got a feel for that happening in Edinburgh particularly I'm interested in things like enabling them to acquire the new skills are you seeing that happen is that something that's been a real benefit to engagement with me I think we were also in a slightly different domain of development and there was a big agenda for online learning engagement and investment in the distance education initiative and so when moogs came along it became part of that mix and so we see it as being an online learning agenda a lot of investment in staff development and training in the expansion of that community but it's been perhaps with a foundation that's already established rather than it an innovation within courses it's probably a little bit too early to tell we're seeing some things and it's slow we've had conversations started with schools who had been putting up barriers saying we don't think we can do online learning and they're now starting to say oh actually maybe we can do a moog and for us that's a huge step change that's what everybody knows you've got a catalyst there and what's interesting to hear from your story is the way that it's been such a positive impact on staff in terms of they want to get on board and do it very much so there's a lot of reference and it's understanding what story that we can tell at the university I think we were in a very fortunate position that our principal and vice principal were very keen on taking this forward there was also then a collective interest grass roots to be involved in it at the right time it kind of worked and so everyone could then push each other in a very positive manner but we also had started those conversations of being part of moogs about a year, two years or we then went into it so we were maybe at a slightly different stage and people are now almost doing the same we're seeing the same progress occur but it's just that it's a year later but actually the same things are happening of you can see slow building collective change and that's really positive I just wanted to quickly say that Katie's responded and said that her question was more about pedagogical innovation online on campus rather than making money and is on campus more risk adverse because students are paying for it that's what she meant but there's a gentleman just in the front and then Mike after that with most of it it's too early to tell we're starting to see some positive rumbles but it's been another game and I'm going to ask probably a question about even the video of how much it was edited by Nicola I think she mentioned that sometimes by selecting students we sometimes give out many students who have the potential to be good students but because we are probably selecting them and some of them cannot afford the courses which are very expensive anyway and I'm wondering with some business models what do you think of some business models starting seeing emerging with monks for example with masters of computer science with Georgia Institute of Technology where they enroll so many students but they pay less so they get the money even probably more than what they would get if they close the class to their students but it's not a huge burden for the students so they pay the money but they pay less because it is an economy of scale they are exporting so what do you think of that any kind about Georgia Institute of Technology and their masters I think they are very brave to do that well I think they are very brave I mean often in the courses too many people have the potential but all are left out of the system then they pay a little money they can afford the amount they can even get by closing the class it has to be interesting I think it's mainly the pedagogical scaling implications rather than I think if we could if we had all the stuff scale at the same time then the cost could be covered then great I guess the main kind of hesitation that many people have is having 200,000 people on a MSc programme when the MSc programme is meant to be tutored experiences, personal development can that happen in such a scale space can it be individual maybe it can, if it can then there is an opportunity but exactly as you say they are very brave to be entering into that space and someone has got to enter first the QAA were batted away earlier on today quite firmly but in my position was heading up this project I'm also Associate Dean for Learning, Teaching and Quality with the Q hats quite heavily on me at the moment because we've got an institutional audit and I can't bat the QAA away quite as easily and I would agree with you that there are some serious concerns about whether we can align the MOOC learning experience with a blended or campus-based learning experience sufficiently to put our hands on our hearts and say yes this is the same experience and there's stuff about study time and access to materials and that one to one development and all of those kind of things so I think it does become complex but I think the kind of root of your question is are MOOCs going to in some way providers with opportunities to extend the reach of higher education and I think all of the data that we have about MOOC demographics at the moment in the US and the UK very firmly says no they don't because they're just not doing that at the moment now whether they're going to do that in the future I don't know but the data is really clear at the moment that people signing up for MOOCs pretty much all already have an undergraduate degree that's not the demographic that we're hitting at the moment I just wanted to mention about the model of American institutions their model of education is different from ours and they haven't got the pressures of the QAA that we would have here so our institutions are used to the kind of rigour that we need to go through to put up a module online whereas the American institutions do have rigour but they don't have a body called the QAA on them all the time so there's not that the way that we do our education system is completely different and that's why the MOOCs could take off the way that they've done it's a competition that people have demonstrated that they have we have standards very high standards very high standards students come and demonstrate that they have developed those competition maybe they can even demonstrate that they have developed more competitive standards I wasn't saying that the institutions in America don't have standard but in MIT they do have very high standards but it's a different model of education Doug was talking about nuts and bolts education so everyone in America has to do calculus 101 and what the fear was in America was that if you could record those sessions and just give them to somebody then that covers that first year and that's fine and then all the administrators were saying do that course just do that one whereas over here we have a different model of education so we don't have 101 courses but we have a broader range like that so it's different models of education fitting into different holes and roots and shapes and things so how does the open university in the UK open university do you think that's interesting because students have a more open university but I think the open university does go through the same process as an institution here would do so just because it's a distance it should have no impact on a student studying on campus but that doesn't always happen in other countries that specific code is something that the QAA have put down that a student from a distance in the UK institution should not be disadvantaged they should have exactly the same experience as a student on campus so if we follow that role it's very very hard for us to do a MOOC as a degree when we have so many other experiences in the institutions we do still do distance programmes I'm a master's student at Edinburgh so I'm having the same but I'm very aware of what my rights are as a student online if you're a learner on a MOOC you may not necessarily have those same rights it's very difficult to get library provisions for 200,000 concurrent users it's a lot easier when you know that you've only got 50 students on your class and that could be a big one so it's not quite as simple as just chuck everything online there are other things that need to be put into consideration one interesting development in Scotland is that they're developing a national framework for recognition of prior learning so how that would fit within the MOOC context where you have nationally agreed guidelines on if someone comes to you and says I've done these courses I've got certificates even if they're unofficial just completion record of completion will they get into the second year or the third year as a consequence of that and will people make market places just for that Scotland's obviously quite a small country testbed in terms of enhancing the learning for students at a particular university might the money and resources and time that is spent on a few MOOCs with very high production values be better spent in terms of enhancing the learning teaching in the university by spreading the money and resources and doing small improvements on lots of courses for lots of students I think this is a great if Jonathan was on this panel he'd be saying he had no money he did this he had no money and as a consequence of it he'll say that his students have got a much better enhanced experience because of it so it's like the money away he didn't say that a lot of the decisions he took was because he had no money which I think is quite interesting I think the whole idea of combining face-to-face provision with something else that allows the world to come and participate to join I think is very interesting but it was quite interesting Stephen Downs was talking at Alt C and one of the biggest mistakes he made was calling MOOCs courses his original vision wasn't that they were courses they were models they were opportunities for people to come together and connect conversations, communities not courses but I know what you're saying Mike but I think the MOOCs have done has opened up the door to academics that would have no interest in going online at all or changing the model that they teach and that's encouraged them to look wider so the impact for those MOOCs that we're spending lots of money on has been for the long term I think opening up the door to experimenting with innovative ways of teaching that's what I'd say to you Mike for James, do you want to say? James Little University in Leeds I've got a question really about the platform based approach to MOOCs that seems to be developing is it an opportunity or is it a complicated threat that we could end up with a lot of different platforms all doing the same thing? Opportunity we couldn't have got as far as we go without future learning in so many ways that for us it's that if you've got more resources behind you and you've got somebody you've got people who can code and stuff then maybe you could do something more bespoke and you could keep more of the money or you could do your own things but actually for us there was no choice really but often it's about choosing the right laboratory for your experiment there are some things that are going to make a lot more sense to have on a bespoke platform that really the outreach is going to be a lot smaller than going on to something that already has an established user base of 4.5 million there will be certain things that having lots of options available to you in itself is a real opportunity to be exploring new spaces for new purposes and taking that opportunity and running with it rather than saying oh there are all these things and maybe we should try and we must go for that particular platform because actually sending school kids for example over to Coursera and it's gauged with the course isn't as that's not all we would want to do we would probably want to put a local moodle instance with content populated on a similar thing because we know that we can manage it and it will give it the right experience so I think it's about looking at the audience looking at the purpose and then choosing something accordingly so I would agree it's an opportunity that's not to say that we need millions of them I think there will come a critical mass point that will help new developments and then it just becomes localising but who knows It's quarter past four now so I think, as I say, thank you very much to the panel, thank you very much to say thank you as well and close the day I wanted to say thank you to Fiona in particular because she's really with her colleagues at the MOOC make this day possible within two months pulling together what I thought was a really exciting programme of speakers lots of different media I think everybody who is involved and that deserves credit for that as well so I think we should say thank you often when we have events like this and I hope you've enjoyed today there is sort of a feeling of this was a great day but what next and this time there is an answer because what next is join the alt MOOC sick and continue the conversation one of the hopes that we have is that the group will be quite active face to face events over the next period of time and we want to support that so we really hope that you will join the group become involved give us your feedback and continue the conversation on Twitter but I also have a few more people to thank before we all hurry off our DigiChems there if you want to maybe stand up and say hello I think there have been snipping videos of us throughout the day and so we are looking forward to having lots of wavy videos coming our way after the event and the university has been fantastic to us a really hospitable environment here and I also wanted to thank Caroline Greaves our events manager who has organised the event with Martin who has done a lot of the technology wizardry today and it's been great to have remote participants as well so I wish you all a safe journey home we hope you enjoyed today and then we shall see you again soon hopefully in the next Morgsig event thanks again for your help thank you thank you do you know on the dish slide? yes I think I know more than I think she's on it but I'm just kind of ashamed that it's because we're all in the same kind of circles that's why so why did you take this? because I am so I am I am I am I am I am I am I am I am I am I am I am my arm is on my neck it hasn't been neck operation because