 Gwis ymlaen i'r dda, i'r Sydney yng Nghymru, ac mae'rodaeth lawr hanfach sydd hefyd yn ddiwedd iawn. Rydw i'r dderbyniad yng Nghymru, oherwydd y cy intersectaig yn y ddiwedd iawn, oherwydd mae'r gyda eu cyfnod a'r cyntaf mikroedd ar y paradymau agorod gyda Gwis Wyrddigau. Fydd ymlaen i gondol i'r ddodd iawn – y dyfodol a'r ddyddai yng nghymru ac mae'n bydd ymlaen i gondol i'r ddodd iawn – ddodd iawn, That means I'm responsible for the learning design of the online programme and also the programme manager So I spend all my day thinking about how we can improve learning design for our online participants And our participants, they are teachers They are teachers in schools and colleges around the UK and our international audience because all of our online courses are available on the FutureLearn platform Mae unrhyw Rhaglen Gwanaethus yn cynnig oed. Rydyn ni'n ddweud o gwrs a gyrs ymyr bwrdd o wneud, a wnaeth gyda'r bwrdd o gwmwysig ar creu cyfnodol ac mae'r bobol a'r rhaid. Rydyn ni'n ymdangos i chi'n gyrs yma. Mae gennym ni'n 5 gwyrs yma ziwn i 5 âser, a wnaeth i gael eich cyfarwydd i gael a ni'n bwrdd yn holl hwn. Mae unrhyw gyda'r gwn cwrs mewn gwrs a gwyrs yma. Mae'n gael hwn i 연teru cyfnodol a gwahanol. Ac mae'n cael ei ddweud ar gyfer ei ffordd o'r ysgrifennu online yn oed yn ysgrifennu, ac y gallwn y cyfnod y cyfnod yn ei ddiweddol, ond mae'r ddweud yn gallu ei ddweud yn ôl i gyd i'r gwybod. Yn y cyfnod yma, yma yma, yma yma 50 cyfnod, a 57,000 o ddweud, 109,000 o ddweud yn cyfnod o'r cyfnod yma. Ac mae'n cael ei ddweud o gyfnod yma, ac mae'r gael yma i gyfnod yma i'r ysgrifennu online, wrth gweithio allu gweithio â'r cwrs i gynnalwch arall, ond mae'r ddechrau'r llwythau fydd yn eu ddweud. Rwy'r sefyllfa yn gyffredigol. Mae'r sefyllfa yn lleolio'n gallu'n gweithio'r dweud ar y ddweud ei ddweud eich hun o'r cwrs. Mae'n gweithio'n gweithio ar ddweud. gallwch am y cwrs online oherwydd bod y cwrs ond-dwell yn y dweud o syniadau ymgyrchu am y maen nhw'n yn cael eu cyfeiriad, mae'n ffordd o ddiddordeb o 100 cyflau a'n ffordd o 5,000 cyfoes, ac mae'r rhan o'r amser yn y cyfrannu Mae'n ymgyrchewch ar rai llais o'r phouiau â'r ardal wedi felch chi, a byddych yn y pwyllgor yw cynpridd, lle yn y gallu sydd weidio ar y mewnlineg, ac yn mynd ymgyrchewch y cyd-di gymrysiau ar gyfer y gweinbeth mewn lineg. Felly am ysgrifogaeth lywodraeth yma i'r pajamasol, mae'r llais ymgyrchion maen nhw yn ystod yn y byddol. Mae'n oes pwynt o'r pwynt, ac mae'n eu cy Arsenal a'r llais yn yôl yn grei agurio. Felly rydyn ni'n gwneud o'r casodd. Maen nhw'n mynd i ddweud bod y mynd i'r gweithio ymlaen o'r technologiant o'r rhoed o'r bydethol, y byd credu y bydd, y bydd y bydd o'r bydd, y bydd y bydd y bydd. Dyna'r pwyllgor y llythdoedd, ychydig i'r rhaid i ddeunydd o'r ddweud o'r corpus iawn, ac mae'r cyfnodau yn cael ei gael y gallu'r pedagogi sydd yn bwysig yn ymddangos o'i eu mewn edrych. Felly, dyna'r mynd i'n gweithio'r mynd i'r lleoli ar y tîm, yn ymddangos ar y tîm ymddangos, dda'n y Llorolars, yn 2016, dwi'n bwysig i'r tîm ymddangos, yn ymddangos ar gyfer coleirni. Maen nhw'n meddwl, yn ymgyrchol yn 2011, o'i cael ysgolion yn eu hwn o gweithio a'r ysgolion ar y ysgolion, o'n mynd i'r unedoddau, os ysgolion, o'r cyflawn, o'r projekteis, o'r cyflawn o'r eich môn, oherwydd fel ymgyrchol, oherwydd fel ymgyrchol, oherwydd fel ysgolion. Yn ystod y maen nhw'n ysgolion, mae'n meddwl o'r parmysau mewn gwirionedd, yn ymgyrchol ond rydyn nhw'n mynd i'r llyfrnod yma, One of the big things for me at the moment is to make sure that online and face-to-face are blended together and really draw upon the value of that. And that's where that network and interchanges among schools are taking place. But are those, is that networking, are those interchanges actually taking place on online courses as well? We'll have a look at what might be happening there. And Laurelard relates the idea of co-learning to communities of practice through MOOCs that have curated digital resources and orchestrated collaboration. And that idea of orchestration of collaboration, that people are working together in an orchestra to come up with a great output, I think is incredibly important and valuable. Monica Lewis in 2017 and her colleagues looked to a study in the Netherlands, not necessarily for online courses but for teachers' professional development choices about how they are self-directed in their learning. And the range of aims that these teachers are trying to achieve. And the same applies to HE lecturers as well. So when I'm talking about teachers, you can also think about the colleagues if you're working in university about who you're working with. Whether they're newly qualified, they're mid-career or they're very, very experienced. The types of things they're looking for are activities that help them to reflect on their practice, to keep up to date, to experiment, technical things like managing the classroom, getting up thick into grips with the subject matter that they're teaching and student care. So these are all traits of professional learning. And in the teaching sector in the UK, there's DFE, Department for Education Guidance, on what the professional's learning standards should be. And key to that is sustained and embedded practice, which is another reason why we can't just have one-hit-wonder workshops or anything like that. There has to be a journey, a longer period of development and online is perfectly situated for that because you get courses where you can try something out, put it back into your practice that week, get a bit of feedback on it from your mentor online, do a bit of reflection, try something out next week, and do that across a whole term. You sustain that in a journey by linking these courses together. So these are the traits of professional learning, but how are professionals actually supported in adopting those traits? Our approach at STEM learning is to have structure points of interaction in all of our online courses. We have the sharing eye of ideas as key to the learning activities. All of our content is evidence-based, whereas academic research, or whether it is through practice itself, the idea of action research, ongoing reflection, which we have reflection grids each week for our learners to complete, and action planning to sustain the impact. What are they going to do? What are they going to take away from our course and what are they going to do next week, next month, and next year? This is what we do on future learning. We have that weekly structure. We break the structure down into sequences of steps, pages of the course that meet many learning outcomes, if you will. We have really high-quality video. We go out to real schools, film their real classrooms, interview teachers, talk about what they're doing. We use quizzes not as a mechanism for assessment, but as a mechanism for delivering content to challenge our learners' way of thinking. We have expert Q&As, asynchronous Q&As, learners post their questions and we get a response. Sharing through Padlet for anything that can't be summarised in a short 500-character text box on future learning. So things like lesson plans could be videos with some excellent videos of teachers and technicians demonstrating what they're doing with their classes and their practical work in science. We have our own high-quality demonstrations from the National STEM Learning Centre, and we also have fundamental to all of this, of course, is the social learning pedagogy, those discussions, those constructed activities, those orchestrated collaborations. But we're trying to address here a range of learning needs. This is just from one course. These are the different learning needs that these teachers have. Want to know more about practical work in science, want to be able to meet the assessment criteria, help their students meet the assessment criteria. Want to be able to evaluate their own practice, know how to evaluate their own practice. Want to be able to relate their content to the real world so that students have that relevance of what they're being taught. And if they're new, there might be a non-specialist, an increasing proportion of the teaching workforce, a teaching subject they don't have a degree in. It's not the same in academia. And that is a key area for us to support. And that all wraps up with confidence. And confidence is the tricky one. We always aim for our courses to build confidence in our participants. But how do we measure that? How do we make that a learning outcome that we can actually measure? So I think there are some contradictions in open online course design. And that goes along the lines of trying to meet everyone's needs, but also having a very clear structure to a sequence of activity. All of our courses have a weekly structure. They're linear in that respect. But we have designed them specifically that you can, if you know how I think is the question I'm going to come to, you can choose the different activities that you want to meet your own development needs. But how do we do that as a design process? That's not a design process that I know of very well. And we've got ideas of personalisation of the only way you can have these crazy maps and find your way through. But there's still a big challenge. It was made by Kate Lindsay earlier in the session here about supporting our learners to be better learners, essentially. We've also got individual times. If you join before the course starts, you're more likely to complete. Because you might not be part of the cohort if you join later on that's already had the discussions. So how do we account for the openness of that timeline? And how do we still build in that socialisation? How do we make openness of access whilst also making sure that all of our learners have that self-efficacy to be able to learn with us? So what do we normally see in all the literature? Retention graphs. And these are for retention graphs from a selection of courses that we run. And we might see, okay, we know there's a dip in week one. Oh, that course is slightly different. The bottom course there that I've just highlighted is a five week course which has a researched story behind it, essentially. How we learn science of learning. And so we take people through a model. And so they have that hook very much early on week one. The one at the top, managing behaviour through learning. It's about practical approaches for managing behaviour in the classroom. You could cherry pick if you wish. So it's naturally going to be a drop off. Three week subjects, especially this course. A few bumps here. A bit of a bump at the end when you've got a Q&A. What does that actually tell me about what my learners are doing? The decisions they're making. Not a lot. I don't think we can take much retention curves. Now, I was really grateful for the input from Sebastian Miller and David Jennings who said, well, why don't you look at retention on a week by week basis? And actually, this is more interesting because it tells me here that once we get people to week two, they're in. We're okay. And so the people have committed to week two are going to be the ones that are going to complete more of the course. But again, I have to ask myself, do I really want them to do that? Do I want them to be able to complete the whole course? Do I need them to be able to complete the whole course? And that focus on retention is still driving a lot of the discussion. So again, we can look at the blobs on the graph. So open online courses, access measures need to focus on outcomes. And I believe we need to let go of retention because the outcomes are more important, I think. What a teacher is going to do with the course is more important. One of our learners is going to do with the course is more important. And this is summed up beautifully by Deborah and colleagues in their 2014 paper that MOOCs open online courses have loads of data. But it's the diversity of user intentions, their backgrounds, the unconstrained asynchronicity of their activity. What a wonderful phrase. I mean, that's just telling me that there is no way that I as a learning designer can control when and how and why a learner is engaging with my course. But there's still quite a lot of faith in the data. And it's a need to seek out patterns to explain the learning that's going on. And I'm questioning that quite a lot at the moment. I'd also sort of challenge perhaps that some of this has come from changes in the platform that we use. So FutureLearn has an upgrade model. So if you want a certificate, you pay. But if you want that certificate, you also have to mark the steps on a course that's complete. So you have to go through and tick a little box for our courses to say, yes, I've done that. And to the extent that some course designers will put this instruction at the bottom of every step that says don't forget to complete because it ticks a metric somewhere down the line. For me, I tell my learners right at the start of the course, use the completion measure because that will help you keep track of where you are. I don't care if you complete or not. I want you to make the most of the course. But the platform and the process behind that is slightly different. Success then does not necessarily mean engagement in my view. And I want to move beyond the view that you must complete the course according to my design. We need to allow learners to be able to choose the steps that are appropriate for them. We shouldn't be chasing for correct responses to tests and quizzes. We should be thinking about, well, how do those test and quiz responses help us understand our learners better? So non-engaging in a course, non-engagement in an open online course does not mean a lack of design at success at all. We use the normal model of activity design. We have an activity in the middle that we're designing to lead towards an outcome. We think about how the relationship between learners, the content, the educate and the cohort all mixed up in that different form of activity. We use the ABC design model. But for me, I would like to suggest that it's not necessarily that activity that's determining the outcome, but the context in which that activity is sitting that determines the outcome for that learner, which means you could have a whole host of unintended learning outcomes, which are more important to that learner than they are perhaps the learning designer, which makes the design challenge incredibly difficult. But we know that learning cannot solely exist online, particularly in professional development, it sits within a practice context. So, our learners should be supported to learn and develop, and we get the feedback at our Rental Course Surveys that they have been able to take the course flexibly around work, they've enjoyed sharing, they've been able to try out new ideas, they've been able to rethink about their practice. And I love that quote because that came from a teacher who said, I've been teaching for 20 years, and I've had my practice challenged and changed through this open online course. So we can look at some of the data, though, to maybe think about, well, can we maybe infer something from the way that these types of learnings are taking place? And I've looked here at what I'm going to call course wobble, which is the deviation of a learner from the intended trajectory of a course. So normally you go step 1, 2, 3, 4, 5, 6, 7, 8. If you're a wobbler, you might go 1, 4, 11, 12, 2, 3, and so on, jumping all over the course. And I find that interesting because that really shows me our learners making decisions about missing steps, our learners making decisions about, OK, this isn't relevant for me, I don't need it, or perhaps they're saying, oh, this might be interesting, this might be an unintended learning outcome. And we see a weekly spike, so learners jump to the first part of each week, and that's probably triggered by the weekly emails that the Future Learn platform sends out. But in a three-week course, those spikes aren't as pronounced, but we do see some patterns happening at the start and at the end of this particular course, again, due to the interventions we've put in to do with synchronicity. I don't mean synchronicity in terms of people being online at the same time, I mean things that are stuck at a particular point in the calendar, like our Q&A with an educator. And that forces people to skip ahead. So some interesting ideas are about rhythms and learning routines, but most participants will take the course in a linear pattern. So if we're trying to encourage our learners to identify their own needs and use the course to meet their own learning needs, why are they taking the course in a linear pattern? Very few participants will also start elsewhere than at the beginning. I had a look at four courses last night, 10% to 13% of people will start the course anywhere else than at the very beginning. That's a very, very small percentage when thinking about people making their own decisions about what needs they're trying to address. Social learning, the platform is all about social learning, but it doesn't necessarily mean it has to be restricted to social learning activities. We know that maybe 20% to 30% of learners are commenting, and commenting perhaps has a relationship to completion, as it does responding to others, as does receiving a response to others. But of course, as Swinerton and Hodgkis and Morris said in their paper on this exact issue, that completion and commenting are intrinsically connected. You cannot use measures like this to measure the success of a course. You cannot use measures like this to measure success of social learning, because the more you complete, the more opportunity you have for commenting. So the dependent variables begin with. Most comments are not replies, but does that matter? Anecdotally, the quality of the comments on our courses has improved over the last 18 months, and our mentors have said this. We may change the course to facilitate this. And the comments are becoming more thoughtful, they're longer, which the mentors delight in, because it gives them a lot more rich data to play with, to extract themes and ideas, and we're getting fewer of the, oh, that's a nice step, or great video, getting fewer of those, and more thoughtful comments, which is great. They're reflections on that teacher's practice, and they're using the platform, not necessarily to talk to someone else, but as a way of capturing their development. So in terms of planning to engage with a professional development as well, the data shows us that if you, as I said before, if you join the course in advance of the start date, you're more likely to complete. And I'm also looking now at whether there might be something to do with the time difference between steps. So if you engage in all the course in one day, how does that change your completion, perhaps, or of metrics that we can think of, performance on the course, compared to somebody who takes over much longer period? And the outputs we have from our model, our approach, and our challenging of sticking rigidly to a single pedagogy are shown through the outcome surveys. And I have to admit now these are obviously people who have completed the course, or have completed enough of the course to get to the survey point. But we see here that 98% of our participants say there's been a positive impact on themselves, 80% on their students, 63% on their colleagues back in school, which when you consider most of our participants are probably taking these courses individually, with a lot of work we can do about that blended model of embedding these courses as a structure for schools to use. So impact on their practice, but also improved understanding of the subject matter of the course, changing their practice, the course is relevant to them, and it's a good use of their time. How have I met all those differing learning needs through a course that we've designed six months prior? These are questions I don't yet have the answer to. But we know that they like the structure, they like the discussion, they feel more confident. That's a feeling. How can we design learning for a feeling? Seeing the practical aspects and being feeling part of a community. So I'd like to conclude by saying that I think pedagogy should be open to. When I first started this role I said I had to rethink everything I knew about online learning. Social constructivism, communities of practice. They just don't work the same way in a course that is only three to five weeks long and is essentially got an anonymity cloak for our learners. The data shows by the absence of engagement as much as evidence learning paths that pedagogy is open. We shouldn't read too much in the data. We should think about the outcomes and the outputs from that learning experience. So to conclude I'm very much steered by my own professional viewpoint. What I believe about education, I believe it's very much situated within a context. I believe that social learning is important because it helps you to relate ideas to your context. I believe that discussion is important. But then my own prejudice is about what I believe is right. I always say that our online courses are not online textbooks. Absolutely not. That is not the type of online learning I want to deliver. But how can I convey my prejudices about pedagogy to our learners? I have to acknowledge that my way isn't the only way of dipping in just for five minutes and being able to take something out of that. So I think where we are coming to then is a situation where... It's interesting I've been talking to a long, long slide. Coming to a situation then where we really do need to think about as learning designers letting go perhaps of our own prejudices, acknowledging that we have a professional viewpoint, conveying that to our learners, helping them to understand that, helping them to challenge not just their practice in the form of professional development, but also helping them to challenge what they think about online learning. And through that, hopefully they'll be able to sustain their professional development as well. Thank you very much. Thanks Matt, that was a really interesting talk. I think we might hear as well. So please do either post your questions online or raise your hand if you wish to ask a question in person. Matt, one of the things I'm really curious about, I'm just going to use my Tearsborough to ask you to pass that question. Go on, Tearsborough. What is the next step in this development? You've talked a lot about what's been achieved. Where do you see this in two to three years' time? Where would you like it to go? Within our own programme, the course maps, which actually relates to the question there as well, and I know that the course maps approach has been explored by Grania Cymull as well in terms of helping learners think about the guidance they need about how to learn, but the types of activities that are available to them, what type of activity they want to be engaging, how they're going to be supported through reflection. And we've put in our courses already extracts from Dynallolard's conversational framework as well to help them think about the types of comments that they're using. So that was one of the changes we made recently over the last 18 months is to think, well, can we encourage our learners to be more thoughtful in their engagement? And they have embraced that. In terms of where online learning and professional development needs to go, it is definitely a blended model because we want to be able to equip our online learners with the capability to understand how to use the online courses to support their departmental needs. What are the needs of your context? How can you cherry pick what you need? So there's almost a level of metacognition that needs to wrap around all this as well. And I think the same with applying university context, where I have worked at university before quite extensively, so I know the challenges that you face. And I think that the most valuable part of working in a university in a learning technology role is having those conversations with the academic staff. And you need to make time for that because you understand what they're trying to achieve through their learning. So you need to wrap your professional development around that particular context or enable them to make those links. Thank you very much. Anybody raising their hand? No, we have some questions online. So how do you encourage to explore different pathways through the courses or do you encourage that? So that's what we're going to be looking at at the moment with the course maps. But as I've put them together with some colleagues, I was debating, well, should we do course maps based on activity? Should we say, okay, this is a reflection step, here's a discussion step, or should we say is it a video, is it a video classroom footage? What's more likely going to be useful for those learners? So it's still something I need to unpick. And I think that there's an interesting point there about encouragement of your learners as well related to that because you want to encourage your learners to be able to work their way through a pathway and see the links between courses but also buy into that idea of sustained practice as well. So I presume that's the question on how do we encourage learners? It comes to looking at the outcomes showing what people have achieved previously. But one of the key parts of our online courses is the mental facilitation. All of our courses have it and they do a video diary in some of our courses where they pick up on some of the key points that that cohort has raised from the content and that's been incredibly valuable for the learners to see the links between the content and their context. But we also encourage our learners through not necessarily reward mechanisms because the reward is actually what changes in practice in the classroom. So that's how you encourage your learners by saying, and by learners we mean teachers, we mean academics is by saying, well look what's changing in your practice, what's changing in your classroom. So having something that's small and easy win to begin with before you go into more depth as the course progresses that's how we're trying to do it. Thank you very much. I think we have time for one more question if there is one or maybe you've met everybody is sufficiently hungry that it's nearly time for lunch. So an opportunity now to ask any final questions. There's one more. So in your opinion or experience what can't you teach through online learning? I think that's a nice way to finish before lunch. I think you can teach anything through online learning but it is the activity that doesn't always work online. We had this wonderful debate when we first started on this journey at STEM Learning with my colleagues and the subject expert team. What works face to face, what doesn't work face to face, what works online what doesn't work online. And so you come up with a whole plethora of different types of activity that will work in these different situations but I think you can probably teach anything probably apart from no I won't be very specific here but there are some relationship building approaches that like human to human development skills that I think you can deliver something online but you have to link it back to the face to face environment. You have to have that face to face practice opportunity. You can't build relationships with your students just by reading or completing activities in online course. You have to have that link between the two spaces. Thank you very much and talking of links between people and in spaces. We hope that you will all join us for lunch downstairs now. We're just five minutes ahead of time but please put your hands together once more for Sarah, Neil and Matt. Teachers and students can develop and share coding skills with multiple or Jupyter Notebook service. Our DigiMap services deliver high quality mapping data for all stages of education. Future developments include a text and data mining service, working with satellite data and machine learning and smart campus technology.