 So I work for the National STEM Learning Centre, I'm their online CPD coordinator. The centre actually provides training, its professional development for teachers, ongoing professional development for teachers in a whole range of different ways, in our centre in York but also nationally across many of our partners. And I've come from the University of York with a fair bit ei dweudio a Owl. Ac rheswm ychydig eich Llywodraeth ac rheswpeth mych dمةillent y profesiydd. Ac rheswpeth arall na byddio gyffredinol nhw i fy nghymrestodd. Rwy'n gwneud i fy yw ei ddweud i ffwrdd o'r lluniau hyn am y byddwch ar hyn o bryd. Rhyw bryd ond esbygoges o'r eich rheswm fel ei dweudio. Rydyn ni'n gweithio ar gyfer yma. Trwy'n gwneudio'n ydych chi'n gweithio'r erbyn a this is quantitative data from five different types of courses that we offer to explore the design issues we might need to consider when developing MOOCs. But more crucially, of course, is how to best equip our online learners to utilize our courses and our content. So when we put a course or any open resource together, normally, as educators we want our learners to progress through it, Mae'n rhaid i ddweud eu bod yn cael ei gwybod i'n meddwl i'r uwch. Ond mae'n gweithio'n cysylltu'n ddiddel ac eiail i'u gwoith iawn, ac rwy'n meddwl i'n ffordd i'w ddweud, oherwydd'n ddiddel sy'n gwybod i chi eifydel, oherwydd ar hyn o'r ddweud y gweithredu'r cyfath, ac wedi bod nhw'n gweithiau chi oherwydd o'r pryd yn rhoi llwyddoedd mae'n cael ei gweithredu iawn. So怎麼 judge if a course is worki or not. Well we normally draw on a pretty graph like this and in this case it's a retention curve or attrition curve for a course and it gives us idea on how learners are progressing through it and retention is one of those metrics that's very easy to measure and gives us a sense that how learners are progressing through a course but those statistics aren't as helpful as you might have originally think. Yn y bydd y cyfle, mae'r cyfle yma ychydig yn gweithio'r cwrs. Yn y cwrs yn cyflwynt yma, yw'r cyflwynt cyflwynt o hynny ymwg. Mae'r cyflwynt yn cyflwynt, onlun fyddaeth, a'r gyflwynt mewn effeith. A'r cyflwynt yw'r cyflwynt yn cyd-dweithio'n cyflwynt. Mae'n atwm ymwg o'r cyflwynt a'r cyflwynt. Mae'r cyflwynt o'r cyflwynt, yw'r cyflwynt yn cyflwynt. Yn y cyflwynt, mae'n gwybod ei wneud ymlaen cuidado felly, yn ymgyrch am Cottage, cwrs, ond ychydig i ddysgu'r cymaint cymaint ymwysgol, a wedi cael ddysgu'r Ynmyneddol. Ond y cwrs yw arwefyni, wedi bod yn gwybod ni'n ziw ddorol y dyfodol. Yn gyfredd ymwysgol, mae mynd i dda ni'n o jealous yng Nghymru derbyn y cymysgol. Mae'n chyfodd iawn y ddechrau hyn o'r hoffod yma, ac mae'n bod yn cyfysgol y dyfodol a'r moed i gynnwysgol. That's why these graphs are represented here as a percentage of all the learners. Why importantly, even the first step doesn't have all the learners there, some learners will just dive in later. I thought, well, maybe that's something that's typical of MOOC, where it's not just a given to me, but it's not just a given to me. It's a given to me, because there's a big difference between what's going on in the course and what's not. Yn amlwysig i ddwylo'r mywg, ac yn fyddo'r ffodol yma i'r ddweud o ddwylo'r ein bod yn ymddangos, ac yn ymddangos, mae'n ddweud o'r ddwylo'r mywg, mae'n ddweud o'r ddwylo'r ddwylo, y ddwylo'r ddwylo'r gwneud o bwysig o ddwylo'r ddwylo'r ddwylo'r ymdweud o ddwylo'r ddwylo'r ymddangos. Rwy'n fyddech chi'n gweithio, mae'n amlwysig i'r ddwylo'r ddwylo'r ddwylo, that participants from industry or professions are likely to enrol on courses for professional gain by addressing a particular skills gap or knowledge gap rather than to get a piece of paper. That's really important to me if I'm developing resources for teachers in their CPD and not necessarily creating courses for them to come away with a piece of paper. We have learners with learning goals that might not align with other students, It might not even align with the educator perhaps. De Beyrran Colleys described this as the window shopping of open online courses. A characteristic which again comes through on the retention graphs. However, I'm not convinced that that window shopping activity is actually taking place across the whole course. So, if you filter just those learners who view less than a quarter of the course, which is the blue line here, you see that most of their interest still is only at the very start. Not much is happening along the rest of the course. The start of the course is still the focal point for these types of learners. Learners who access more than 75% of the course have a more even distribution, which is what you'd expect. So, there's something going on at the very start of the course is informing learners' decisions to either engage or not. And I think that's something we need to pay attention to as food designers. Now, it wasn't till a change in the way that courses were advertised on future learn, many of you might be aware of, that I decided to start considering the significance of course start dates. I got quite upset somebody's come from online distance education about the idea that the course start date is no longer really advertised. The idea of the course is an event is being challenged, and those that see a course on future might not know when the course actually started, when they joined. So there's a bit of attention there when you look at the typical model of a student thinking they're going to do a course but might be arriving late. Bonaparte a the Cloud's use-of-infestification of engagement over time, on a number of MOOCs, provides part of the rationale for this. Patterns of behaviour were categorised with one category, Samplers, being particularly common amongst late starters. The concern for late starters is the core structure of this weekly pattern might actually discourage participation by late arrivals, leading them to become Samplers rather than completers. So, the new, future approach is actually quite clever if you think about the weeks are now adjusted to where the learners starts, If the course is four weeks in, a new learner joins to them, it's still week one, and they get all the week one emails and things like that so they think it's still at the beginning of the course I still want to explore whether there is a difference in learning experience for those who signed up in advance compared to those who joined while the course is already running or even supported periods in what has been known unsupported periods and what has been known as extended availability without facilitate involvement This graph starts to show how two groups, those enrolled before the course and after the course, they visited the course pages is the retention graph, but split out into those two groups. Now this was, of course, with an advertised start date, immediately you see a difference that those enrolled after the start date, the orange line, drop off at a much higher attrition rate than those who joined and rolled the course before the start of the course itself. So if you go back to the idea of window shopping, this seems to suggest to me that learners who have an immediate access to the course are not browsing the whole course of Rellen Bits, ac sy'n gwneud o'r fforddemaeth o'r cyffredinol ytersiwn, yn ei bod yn gwneud at y cwrs'r cyffredinol yn fuddedig yn gwheithio. Felly yn gynnalen i'n gweld. Gwrs fod o'n eu defnyddio'r ddechrau'n cwrsysgau, wedi'i bod wedi'u cyfriffyrdd eu cofwyrur yma'n i thrall. Gwiaill mor gyfyrdd ar y cwrsysgau i chi'n ymgwale. Mae'r pasoedd ymwileb yn ymdwynt ar osgood y cwrsysgau yn cofิchio'n cyfrifwyrdd, ddweud a gwahaniaeth i'rzwod y gwasb yn gweithio yn gweithio, ac mae'r ddweud y cwfysgau'r gwasb yn yn dweud. Jog, wrth gwybod, unrhyw'r gwasb yn gweithio'r gwasb, ond roedd yng Nghymru'r gwahanol ydyn ni'r gwasb o gyfyfyr o'r cyflogau'r gwaith. Rwy'r gwasb y pattern gan gweithio'r gwasb o'r cyflogau, fyddwn y gwasb 6 o hanes gan cyflogaeth wrth gweithio'r gwasb o gyflogau'r gwasb o'r roedd o'r cwlau bydd iawn o bobl yn ymgyrchol i ddweud i'r gyfnodol, a'r tyfu'. Oni'n ddweud ym mwynfodol, ydych chi'n ddweud y cwlau bydd ymgyrchol, oherwydd mae'r cyflawni'r cyflawni, yw'r cyllidegau i'r cyfeirio, yn ymgyrchol, allan o'r ymgyrchol yn deilladu o'r llunio'r cyflawni, ond mae'n ddweud o'r cyflawni'r cyflawni. y ffocos yw y cohoedd cyflwyno a'r cyflwyno yw'r arlwyddyn nhw'n ffordd ydy oedd y ymddangos yw'r ysgrifennu ymddangos yw'r gweithio. Ond ydych chi'n amser yn ddigon i gyd yn ystod y wgeiddiad. Yr ysgrifennu yw'r ffrif ac i'n ddegwyd o'r cyflwyno'n ddigon, rwy'n gwybod hynny, mae'n gweithio i'r cyflwyno'n gyfrifennu sy'n gwybod i'r cyflwyno'r ddweud. Rwy'n meddwl ceisio ar roi ddweud o gymhau i dŵr gyfathodd ffarnidg. Nghaer hwnnw, y gweithio'r cynyddiad bwysig a ddaf yn meddwl cychwyn gwirioneddau gyda'r gweithig. Ieithwch chi'n hwbl ydi ddod yn gweithio'r gweithio, gyda gлизio'r cyfansiynau i ddweud. Rwy'n golygar a'r gweithio'r gweithio'r gweithio, cyfan i ddweud drwy'r cyfansiynau. Again, not surprising. But aside from whether late learners joined before working out whether the course is right for them or not, there are other issues at play. Swinerton and colleagues reported that commenters, those learners who make posts who contribute, are more likely to have online courses before, more likely to have taken online courses before in a sense that they brought some experience to that engagement. So when we look at the way that different groups of learners have started to comment, it's quite interesting to see that those who joined before the course started, those who perhaps might have done online courses before and aware of how the online courses work, they've engaged with that type of learning, some assumptions in there, definitely more likely to be actively contributing. And again, we have that problem of whether you've got a load of people joining together, there's a bit of a buzz going, coming in late, you're going to get perhaps a lesser of an experience. But is that actually the case? You can still learn vocariously from the experience shared by learners who may have posted several weeks ago and are now long gone. So whether these late starters have prior experience or not, we're asking them to engage with the course in a very different way. And the same is true of whether they're social learners, whether they reply to other learners. So our design challenge is therefore how can we make the course flexible, open to those with different learning needs at different times, whilst also aiming to provide support and deliver the benefits and richness of social learning. Particularly as we're having more and more people join the course late. Now we've tried a few approaches in our courses. The way that these later learners are learning is different, but not necessarily less valuable. We still receive positive feedback about the impact of the course on their practice. We incorporate a mixture of design elements to help respond to a growing number of learners without trying to take the course without our educator support. And that educator support is actually something we value quite a lot, having our academic expertise and our professional development expertise to help our participants. So we've tried a few things out. I'm still not sure which of these is the most valuable for those late starters. I'd be very keen to hear your thoughts and if you've explored this issue yourself, how we make sure that late starters get the most out of the course. But one way I wanted to consider what is happening in an online course is to use the student-teacher content interaction model. And certainly this has informed my thinking. Myso and Addison applied that student content teacher interaction models three classic MOOC categories of X, C and S, with content being more significant X and interaction with other learners and educators being more applicable in the connectivist, constructivist or social formats. Bain and Ross in the 2014 review of MOOCs in the UK came to the conclusion that there are many different types of MOOCs out there. And that definition of X and C just is completely irrelevant. And I want to argue a similar but slightly different point. Cos from what I've shown, I've come to the conclusion that MOOCs are metamorphic and that might be a bit of a contrived alliteration there. But MOOCs don't just have one format or one mode of learning or teaching as the designer intended. What is happening within certain online courses and certainly these courses which have extended enrolment, extended availability periods without support is the very nature of the MOOC itself changes. The levels of potential interaction are different. Learners and possibly thinking is the course facilitating teams still available. Have I missed the date for any time-based activities? Are there learners making comments? Is anyone actually going to read my contributions? Now, I'm not sure whether all these questions actually go through the heads of MOOC learners but if our courses are designed to encourage interaction in some way, then our courses not only have to be flexible for different forms of participation, for different learning aims, flexible enough that they work in both transmissive and collaborative pedagogies. Dare I say it then? Is MOOC pedagogy itself open to interpretation? Thank you. Thank you, Matt, and a very provocative thought-provoking ending. Let's take a question from the back first. Hi, I'm Doug Balshaw. That was really interesting. So I'm interested in, and this might sound like a facetious question, but why do we even do courses? Like why don't we start from where people are with granular content and then give them a bunch of pathways and then it doesn't matter where they start and stuff? I'm just interested in why we persist in courses where we don't necessarily do that offline sometimes. I think that's a very valid point and I think that's why I was sort of suggesting that maybe course structure is something that we as educators put in because that helps us order our thinking and we have that picture in our head of where a learner needs to go through certain key points in order to build their development. We have in our courses a self-audit task where learners identify the points that are against 10 statements of the things they want to address in the course and they have to identify what their professional development objective is at the start of the course and I think that's actually an incredibly valuable experience for them to do because whether you're joining the course with all the other learners at the beginning or if you're joining it late, you can still do that activity to help you direct what you want to get out of the course. So, I think there's a scaffolding structure there that we put in as educators but how people engage with MOOCs is certainly up to them and I think that's one of the tensions I'm still trying to grapple with. Another question from the front here. Hi, have you considered giving participants alternatives like downloading the entire course and doing it offline at their own pace or including themes of gamifications like achievements and accomplishments? Yes, it's an interesting point and that's not something that we've looked at so far. I'd be interested to hear a little bit more about gamification in the professional educational sphere. That could actually be quite interesting to look at but I don't know how much is out there on that. In terms of downloading content, well, all the videos are downloadable but future learners, as you know, has got to cut off now unless you pay to upgrade and I think that is an issue but we include with our courses certain takeaways so there are handouts and things that they will take on and we have a lot of online resources that we link to on our own website which are freely available and are open. So yes, I think that's a very valid point to consider is the portability of the course content and how the participants can return to it because within professional development actually that's incredibly important is being able to go back and change and review your thinking over a longer period of time which is actually probably why we see quite a number of participants take our courses multiple times. They might get a little bit out of the first run and a little bit out of the second run but they do come back which is very interesting to think about. Thank you. Question from the middle here. Hi, thanks. That was great and I love your alliteration. I've already gone off and stolen it. Oh, you are creative commons, whatever. And so, hi, I'm Andrew Smith from the Open University. With MOOCs, do you think there's a mythology there? Do you think actually students know that they're doing a MOOC? No. We don't use the word MOOC anywhere. I think, I don't think so. They're doing an online course. They're engaging online but as soon as you start telling them it's a MOOC or anything like that you may as well start beating your chest and speaking cling on at them because it's pointless. It's strange languages, whatever. But it's interesting your data for your MOOC because I've been engaging because I'm quite different in a completely different community and it's a few percent out but it's behaving more or less the same. And I'm teaching teachers network engineering. So whatever you're teaching is behaving the same way and in my community where I'm using a completely different platform the data is very similar. So is this another thing about human nature? I couldn't possibly answer that very high level question but I think that there is certainly we've looked at retention data across a number of different partners and it comes out very similar and our ous seem to be noticeably higher because we must be doing something right and I think it's that extra design elements that we put in which keep that retention going better. But the consistency of the shape of the curves in the difference between those two groups between all of the different courses that we ran was a thing that surprised me the most because yes you can look at retention across every learner but those learners are divided in so many different groupings and future learners just done some work on archetypes where they're looking at learners who are doing it for professional reasons versus those who are just doing it for the weekend as a hobby and the different behaviors that are exhibited by those two different types of learners. So we can start to break that down in many different ways but I think that overall picture is the long tail. It's a long tail that we've been talking about for years now in terms of the way the internet is used and the resources that are out there. Everyone's looking for a niche and that might be something, a couple of steps might be addressing their niche. Thank you, Matt. I'm going to have to stop you there but thank you very much.