 OK, ddweud, ddweud, ddweud, Jessica Grant here from University College London speaking on behalf of Matt Smith and Dr Sarah Warns and you can see what it is, active flipped learning using Moodle Quiz, which is another exciting one for me so I've picked the best place here this morning, thank you. OK, so I'm going to start with an apology. I only got these slides very recently so I'm going to do my best. Don't ask me too many tricky questions. Matt and Sarah are very happy to answer questions by email as well and their addresses will be up towards the end of this presentation. So I wasn't directly involved in this but I've been briefed by Matt very quickly on Friday and I'm going to be talking about flipped learning using Moodle Quiz. So Matt Smith is the learning technologist in the School of, the Faculty of Engineering at UCL and Sarah Warns is a senior teaching fellow in the UCL School of Management and for the past four years they've been working in partnership to redevelop Sarah's undergraduate teaching. So today I'm going to talk to you about what they believe makes good flipped learning and how it can be facilitated using a Moodle Quiz. So, start with a question. Which of these do you think you can do effectively on your own outside the classroom? So thinking from a student's perspective, if you just pop up your hand so I can get a rough idea of what you think. So outside the classroom do you think acquiring knowledge, yep, so most of you, applying learnt knowledge and skills development, quite a few as well, developing professional and subject based attitudes and values, so a few less this time. And now looking at this from the other perspective, which of these do you think are best achieved in class? So again, acquiring knowledge, not too many hands at all, applying learnt knowledge and skills, a few more hands and developing professional and subject based attitudes and values. So quite a few of you there. So the research that Matt and Sarah have done have found that they generally find for the first question. So what's good for outside the classroom is acquiring knowledge and for inside the classroom it's more appropriate to look at students applying the learnt knowledge and developing their professional attitudes and values. And they feel that this is a good introduction to the idea of flipping. So I'm going to have a look at a couple of activities here that relate to flipping. So a live debate, this would generally require students to be in the classroom and to apply their learnt knowledge. So they've decided in remodelling this module to keep that within the class setting in face to face time during lectures. However, for things like reading articles, this is more appropriate to be done out of class. And this is exactly what they did when redesigning Sarah's lectures, or Sarah's course. So this is a breakdown of the activities in one of her modules. And you can see here that they've decided that six of these activities are appropriate for out of class activities. And two, really, they felt needs to be done in the class. Although there are many different definitions of flipping out there, there are some consensus about what it means. So the commonalities with flipped learning is that it's about the information transmission, the knowledge acquisition being moved outside of the class, and class time being used more for collaborative activities and active learning. And there's also an idea that students have to prepare before they come to class in order to fully engage in that class. And of course it's facilitated by web technologies, in this case the Moodle Quiz, which is why we're here today. And they looked at the research and the earliest mention they could find of flipping came from someone called Baker in 2000. And he really talked about flipping in terms of students actively engaging in pre-class activities, not just passively watching a lecture that may have been recorded from the year before. That's not what he considers flipping. And so Sarah and Matt have used this definition, and they've built upon it. So they like to refer to it as active flipped learning, just to really make that point clear that it's about students participating in activities before they come to class. So the course that they looked at to read design has about 120 students and is an undergraduate module, and it's called understanding management. The assessment is 30% presentation and 70% coursework. And the flips happen for the first five weeks of term, and then after reading week all of the lectures are dedicated towards their project work and the student presentations. So I'm going to talk you through some examples that they actually developed for the first five weeks where they were flipping. So each of the pre-class activities is made up of some passive and some active tasks. And they made sure that these were brought together in a single interface. So as you can see on the screen here, we have some examples of quiz questions with the content that they're actually either watching or reading on the same page as where they're asking to engage with the activity. Or in an example I'll show in a moment there might be a link to that other activity when it's not possible to have that within the quiz itself. So this example here is a flip of one of the weeks. So every week has this big red box so students can see it really clearly on their course homepage. And there's a link here to the quiz and the submission box as well. And they're expected to go in and do this immediately after the lecture before. So as soon as the lecture ends these are set to automatically appear and students then have one week to prepare for the next week's lecture. So this example is where students have to answer a short form question. And you can see we're using templates here. So there's already a one, two, three in the box for students to answer. And they're given some stimulus material in the actual quiz and then they answer. In other cases they might be doing drag and drop answers where they're matching text to images. But this one here is freeform. And this example here is where Sarah is actually explaining what an ANSOF matrix is. I have no idea what one is but the students obviously learn about it. And then they come in and create their own ANSOF matrix and submit it via a submission inbox. And this example is where students are asked to go out and take a photo or a video as a treasure hunt style activity and then they submit that to Moodle. And what Sarah does is she goes into the quiz and she downloads all of the information that the students have submitted. And she then uses this to analyse where the students might be struggling. So which areas do they need more help with? And she then modifies her next lecture based on what she's hearing back from the students in their answers. So there's quite a bit of work for her involved but this really guides her activity and she knows then what to focus on in the next lecture. And she also is able to give general feedback. So one of the big problems here in the UK and elsewhere is that students are always saying they're not getting enough feedback. So this helps them understand as a whole cohort what areas they should be focusing on and what they might need to understand a little better. She also takes that same data and shows it in the lectures for the students. So here is a word cloud that is based on the text in the previous example, the Excel spreadsheet. And this is showing primary target markets for the Heinz range of beans and pulses. So you can clearly see what the key themes are here because they're coming out as really large text. And she also uses graphs and other ways of visualising the data. So another example is this student submitted this answer off matrix and she then takes that summariser in her slides. She used to use slides from the textbook. Now she can actually use students own work and they can then see the link between what they're doing in the pre-class activity and what they're actually covering in class. So they're motivated to actually participate. And I'll show you some figures later but you can see that there is some students who are actually going back and completing these activities after the lecture. So they see value in actually undertaking that work. And she's able to show examples from different students as well. So they can see their own work appearing. It's all anonymised. So here's the data showing completion rates. So in the first year they ran this they were very pleasantly surprised to see that about three quarters of the students were participating. It was completely optional. The students don't have to do this. They don't have to complete it before the class but it really does help with their engagement and with how much they already understand. And you can see that small difference between the pre-class completion rate and the total which is by the end of the course completion rates a little bit higher. So some are going back and using it for revision they found a lot as well. The second year they ran it they had a bit less engagement but they were still very happy with that result. And they're currently analysing the results for this year. Or for the last year that's just completed. So another course that Matt actually used this whole method with is in security and crime science and they had very, very good completion rates. They had much lower numbers of students there are only 22 but 92% actually completed the pre-task activities and 95% by the end of the course. So that's really impressive. So some of the student feedback that they were given. The positive feedback was saying things like it's very innovative. They liked the flipped activities because they're very short and very effective. It helped them to understand the topics and so when they got to the lecture they were better able to understand what was going on. There's a good build up of theoretical knowledge. It's extremely useful and also very interesting. And some said that it was the best part of the course and they found the videos helpful as well. On the flip side there were some areas for improvement or some students who didn't quite enjoy this model. Some said the videos took a significant amount of time to watch and some prefer to read rather than watch. So maybe considering having a transcript of the video might help those students. Flip lectures are a bit scattered in every week so some of them found it a bit disorientating it sounds like and the integration some felt could have been better in the lecture. And maybe a few too many flipped activities. So some key takeaways from this process is set expectations, explain the approach to the students and why it's useful for them at the outset and make sure they understand that it's about actively participating and not just watching or reading something. They're actually in there to contribute and to inform the lecture and that they will then get really good feedback from the lecturer about their gaps in knowledge. The linking, the learning is really vital so without that it doesn't really have a lot of benefit to the students necessarily. They might not really see why they should be bothered doing the pre-class activities especially when there's no grade attached to that work. And they found that sending reminders using the Moodle course participation reports really useful as well. And they also suggested to be consistent. If you start and you say this is going to happen every week make sure you do take that data and feed it back into your following lectures. Not skipping weeks. So this is the contact details for Matt and Sarah. If you do have questions I'm happy to try and answer them. And they'll be publishing a toolkit later in the year so if you'd like access to that toolkit I'm on that list. They will gladly take your emails and you'll be added to a list and they'll be letting everybody know. And just a few references at the end there. And I'll put those contact details back up for you though. So thank you very much. Thank you. Yeah, you can have my email for sure, definitely. Okay, do we have any questions please? I saw some feedback from the students about the videos and some of them they would say that they really enjoyed them and some others say that they were very lengthy. Do you have any data about what is the ideal length for these videos to get the right feedback? As far as I know the research I've done separately from this it tends to be just four or five minutes. I'm not actually sure how long these videos were but I doubt they'd be much longer than that because Matt's very conscious of that and it wouldn't have been a whole lecture. So I thought it was surprising that some students said that they liked how short the videos were and others said that they were too long. I thought the participation rates were quite impressive. What we find in whenever there's anything like this at the Open University is what actually happens is it starts high and drops off. Did you look at that kind of how it changed from week one to week five? I'm not sure actually. There would have to be a question for Matt and Sarah. I'm just going to go back and see if I can answer that question about the length of the video with that example screenshot. No, it doesn't actually say in here, sorry. Will it work on K-12 students as they're not doing their homework at all? I don't know. Maybe you should try and tell us. I think it's a nice idea because it makes them able to understand what they should be looking for in the materials. So if you're giving them readings and not giving them any guidance about what to do with that information then that can be really difficult for the students but the activity actually guides them to what the key points are in that activity and really can help them understand the key concepts. And then as someone said in the feedback before they come to class they've already grasped some of that and then they can ask the questions to the teacher of the parts they didn't understand. And we do this across UCL in other places as well and we find, I don't know how many people use the hot question type but that's really useful and it really helps. You wouldn't have this problem in young grades in school but in a large lecture theatre it's really hard to understand what the key questions are. You don't want to be inundated with questions and the great thing about hot question is it lets you vote and then you can just address the top five in the class and then the students are actively participating and it's not a huge load of effort on your part. I actually think it would work really well with K12 because that's my background and I have an ex-colleague who does a similar kind of thing with some computing students who are aged between 14 and 16 and it's helped the grades increase quite a lot. I keep tying him to come and do a presentation here maybe I'll get him next year. Is there a question? That's good, that suits me. You mentioned the lecture looked at some of the answers to provide evidence for the next class. Were there other methods of tracking student progress that the course used? I'm not sure, I think that was their primary method. I didn't hear of them analysing click rates on anything in the logs or anything like that but they may have done. Did they use some of the modal features of completion, progress checkers and things like that? Yeah, I'm not aware of that. Anybody else? I have a quick comment if you don't mind, Jess. We've done something similar in DCU and two things we found quite useful in the case of the video on YouTube and you can see how much of the video students have watched and if they had to rewind certain parts and so on. The second thing is we've developed an event reminders plugin so if they haven't completed the event just following on from your points, David, if they haven't completed the event they get an automatic reminder, an email sent to them so that could help with that too. Is that available on the computer? That sounds great. At the moment they're doing all that manually so that would be really useful. Okay, thank you. We will start again in a taff pass for our next presentation. Thank you again. People who are doing the next presentation there's one about stack and maths. Can you just make yourself known to me so that I know that you're here to introduce you.