 Hei, wrth gwrs. Mae'n Chris Nelson. I'm the Digital Assessment Development Manager at the Open University. And with me today is... I'm Tim Hunt. I've been doing Moodle development for about 13 years. I'm one of the senior developers on the OU's VLE team. And we're just going to start off with some context first. The OU, we've got a lot of online modules, and we like quizzes. So, just to give you some stats, we've got 180,000 active users on our VLE. It's one of the biggest in the UK, probably not the world. About 450 live module websites and about 2 million transactions every single day. That translates to about 650,000 quiz attempts on our module by students in the last 12 months, of which almost 7 million question attempts. So, that's probably about one quiz with 10 questions in, that which is out too. So, this is some of our latest reports. This is based on our Learn to Server, so our module websites. And this includes all Moodle quizzes that we've got on those sites. So, the reason why, although it's very impressive that we've got 685,000 quiz attempts, the reason why there's a 75% completion rate is because we've got the early diagnostic interactive multiple tries where the student doesn't actually have to submit the entire thing, as well as the formal quizzes, what we call the interactive computer marked assessments, where students have to submit and there's deferred feedback, et cetera. So, in this pie chart, what I've tried to do is show the questions attempted on modules by students, by the question type and then by the response type. So, the question type would be something like multiple choice, and then that's classified as a selective response type because students are just selecting options A, B or C, whereas constructive is where students are inputting their response, either a number or perhaps a short piece of text. And as we can see, 66% 2 thirds is selective and multiple choice is our most common and that's 2.5 million attempts by students in the last 12 months. And some of the other constructive, I mean it's very good to see actually because the amount of constructive response questions have increased over the last five years. Some of them are still round about 1%, like Code Runner, but there's still 55,000 attempts by students. So, I'm just going to hand over to Tim to talk about embedded questions. Right, so we really like quizzes and we've done a lot of it. This is what we're talking about, a question in a quiz. And as Chris says, we kind of particularly approve of, not to be judgmental or anything, but we approve of open-ended constructed response questions. I'm a mathematician, so I think this is cool. Apologies to anyone who finds this scary. But the point is here, students have been asked a fairly open-ended question. They've typed an equation and we've given them specific feedback. We've actually drawn them a graph of their answer, which is partially right and doesn't fit on the screen, but we've given them some particular feedback about what's right and wrong about it and then right off the bottom there's a button where they can click try again, use the feedback, try and get it right the second time. So this is really cool interaction, but it's kind of stuck in a quiz. I mean, if the student's actually learning about, in this case, it'd be about learning about something, something called splines, wouldn't it be better if this little bit of interactive learning was in the module material? And well, now you can do that thanks to some plug-ins we made last summer. So that's a standard Moodle book. That is exactly the same question. So how did I do that? What does this plug-in we've made do? Well, this is the page you get to if you click to edit that page of book content. You've got the HTML editor where you put the page content and there's a new button that you click to, say, insert a question. Of course, you only see that button if you are a teacher with access to the question bank. So you click that button and it pops up the little dialogue where you can select questions from the question bank. Now, obviously, if you're letting people embed questions almost anywhere, you then start to worry, well, it'd be a bit embarrassing if someone published the final exam question just in the middle of the learning materials. So in order to be able to embed a question, you need to have marked, you need to give it a particular ID specifically for embedding. So what's shelling up in this dialogue are just the questions and categories that have been given an ID. And then once you've selected the question you want, you click the button and what's inserted in the HTML editor is that very long cryptic code that you don't particularly need to understand. But again, the thing to notice is that particularly long cryptic string at the end, which looks like the kind of URL you get if you share a document on Google Docs or Office 365. And again, that's part of the security. Only the teacher using this tool can generate that unique code that makes the question embedding work. If someone tries to fake that and types the wrong number, the embedding just won't embed the question. So basically, we've considered security quite a lot when implementing this, so you can be sure that teachers are only embedding the questions you want to allow to embed. But you've got this capability and then it's a question of what can you do with it? And really that's up to the creativity of teachers. So here we've shown a question embedded in a label on the main course page. That's quite good. Here we've got a mock-up from a forum. Students asked about, you know, how do I do this thing? And the teachers told them how to do this thing. And then said, and actually here's a question that you could have a little practice. And obviously there's a computer science example that would work really well in languages as well I expect. Just to prove the point, I did try. You can, of course, you create quiz questions using the HTML editor. So you could embed a question in the text or the feedback of another question. You can try it. It works. I really don't recommend it. And basically at the moment this is as simple as it could possibly be. The first thing, we first made it work with single questions. The next addition is what you're seeing here. We have randomisation. So instead of embedding one specific question, you can embed a category and then it will randomly be a question from that category. And when you've done one question, you can say give me another question and you move on to the next one. So that's randomisation ticked off the to-do list. The next thing is obviously if students are interacting with stuff, it'd be quite good if that was kept permanently so students could look back on it and then teachers can see how their students are getting on. It currently doesn't do that, but kind of top of our thing to do next is to make that work. I just need time to think it through. And also I want to make the point. This is a really, we think, very nice way to add interaction to your Moodle learning materials. Of course it is not the only way, as Martin said in his keynote this morning. H5P is really cool. It does very similar things. This is not a competitor to H5P. It's just yet another tool in the toolbox. And if you happen to know Moodle questions and if you happen to have some of the impressive question types installed like Code Runner and Stack, then you can embed them. So that's what I wanted to say in this section. Now over to Chris to talk about a different thing. Thanks, Tim. So sorry. Tell you at the end. Yeah. So two years ago at Moodle Mood London 2017, I was sitting in the audience and there was a presentation about student quiz and I'll show that shortly. And I was so impressed with it that as soon as I got back next day, I rang up the crater. And we've been collaborating ever since. So student quiz is a community activity by Professor Frank Caw at the University of Applied Sciences in Rapswil, Switzerland. And I hear often described as a collaborative cuisine activity, but I prefer peer cuisine simply because if you look at the literature, collaborative cuisine tends to mean that a group of students have got together and then they're responded to a quiz as a group. So I prefer peer cuisine for that and also because it also makes sense if you look at Moodle Workshop as well. I think it's a great activity. It really aligns with the social constructions heart of Moodle. I believe it might become core at some point in the next year as well, which is fantastic. And Frank has this screencast, which he updates every six months as well. So we're worth having a look at that later. You can easily find it on Google. So explain student quiz to be forums but with questions. And this is an example of what the students see the table. And when we first started working with Frank, we had to fix a few of the permissions. So student access to the question bank. And another big thing that we did was limit the question types available to the student because originally it just showed everything. So we've now got settings at the server and also at the website level. So I'm afraid there's a theory slide here. I think that everyone should be aware of Bloom's Tax on Me and also deep learning theory. But I think the Einstein quote summarises it best really. If you can't explain something simply, then you don't understand it well enough. And student quiz delivers here on two points. The first one is that the student has to think of a good question. So they've got to go and engage with their learning material. But then they also have to think of a really good way of explaining why it is right and if the student's selected a wrong response, why that is wrong. So it actually hits for two. So this is just an example question that I've authored. And I've gone in as a different user. And now I'm feeding back. So you've got the little rating activity. And basically in the comments section I'm saying that it's a good use of the question type. But there's no feedback. So because of that, I'm only going to give you 3 out of 5. And there's two other blocks. And just to mention the gamification section that came up earlier, Mark Glenn was talking about it. You've got the ranking block in the top left, which he can set to be anonymous or not. And basically you get scores for the amount of questions that you create if a tutor approves it, the ratings and comments that students respond to. And then you've also got the My Progress block, also known as a Personal Learning Assistant. And the little more link takes you to a statistics page so it allows you to see how you are doing in context with the rest of the community. So currently in the just about piloted stage at the OU, we know that there's good value to be had for using it for student-led exams for our modules that do have exams. We have the individual little courses at the OU and they're against the backdrop of what we call a qualification website. So the little courses might be three, six, nine months. Qualification website is obviously for the entire time of the qualification. So we want to introduce it on qualification websites maybe for language revision. And that just basically means that students can practice their knowledge throughout. They don't have to wait until the next course starts. We're also thinking that maybe level three languages students can create content for our level one undergraduate language students, stuff like that. We've just finished a new phase option, a bit like workshop, so that you can have in week one a question creation stage and then in week two students can take those questions and feedback. And it's basically just to separate those stages out. And we're also working on better moderation and administration of create questions as well with Frank at the moment. Thanks, Chris. So before we wrap up and reveal the answers to Martin's questions of what these plugins are called, we just wanted to make the point that we've told you about just two things, new things that use the Moodle Question Bank. It's not just us, lots of other people out there have been very creative. And indeed Richard Jones actually did something very like our embed question plugin in the past. The reason we chose to make our own was really this thing, this focus on security and making sure only the right questions are embedded by the right people. And actually Richard knows about our plugins and is quite happy we made them. Also, before student quiz came along, there were actually two separate plugins that each did one half of it. There was question creation activity where students create questions but then they were just rated by the teacher. And there was question practice where students practice questions from a pool. Student quiz brings those together and I think combining them is a particularly powerful combination. But then there's all sorts of other stuff. There's all sorts of basically variants on a traditional quiz, more or less traditional when you get to quiz venture made by one of the devs at Moodle HQ as a side project which turns your multiple choice questions into space invaders games. And then there's other stuff. I mean, I just went to look in the plugins database to prepare the stalk and there's all this stuff that's just amazing. I don't even know what some of it does. Must take time to find out. But the plugins we specifically told you about which you can get right now from the plugins database, there's the embed question which is both a text filter and an Atto plugin and there is the student quiz activity that Frank Koch made and we contributed to. So thank you very much for your attention. Good. Do we have any questions? Yes, one straight ahead. Does the embedded question in the book appear in the downloaded book on Moodle mobile or Moodle desktop? That would be a no right now. Partly, as I said, because we started by implementing the simplest thing that could possibly work and getting it out there. And partly also, I think I don't yet understand how the mobile app supports filters and I think the support is a bit limited and is something Jordan is planning to work on. So at the moment it would be difficult to do, hopefully it will become easier in time and then we can make it work. So yeah, basically no at the moment, sorry. Question right at the back. Keep Mary fit. Just wondering, how does it work with the grade book? So obviously when you got embedded questions other places outside of quizzes, how does the grade book know where those questions are and how is it tagged in the grade book? Again, basically it doesn't. As far as we're concerned, this is more about using questions as a little nugget of interactive learning, not about collecting grades. But again, as soon as we made this, our users at the OU started asking about, can we capture the grade? So again, it's kind of on our list to think about, can we make it work? Okay, thank you. Can you embed more than one question at a time? Select two questions in one book page. At the moment in, it's just one question, one iframe. You can have one iframe followed by another but obviously then you start to hit the limits of the device's rendering capabilities. So it's fine for pretty powerful desktops, but when I was trying it on a, I won't say a brand, a generic tablet shall we say, it really hit problems there. We can with two or three on a page. Yeah, yeah, you can have about three maximum. We'd recommend no more than that. That is another thing that we've got on our sort of to-do list. That we're interested in investigating. Can we make it more a thing? Yeah. One more thing. So you said it only works with the Ato editor. Well, it's an Ato button at the moment, yeah. So not with... We no longer use TinyMCE at all. If someone is using TinyMCE, they could very easily take, I think very easily take the code of what we did for Ato and make a TinyMCE plug-in. It's just someone would have to do the work. We're probably not going to get the time. Sorry. Simon Press. Martin wanted to ask something. So, yeah, Student Quiz is really amazing, and we were looking at getting it to core. I wanted to try and integrate it with Quiz, but that's turned out to be too hard, I think. So it'll probably remain a separate thing. The second thing was that Space Invaders thing you saw. There's also a virtual reality version of that where you can shoot down the spaceships in VR if anyone's interested. Timon Chris, thank you so much. Thank you.