 Hi, I'm Kelly Fleming. I'm from Future Generations University, and I can look down here, which makes it a lot easier for my neck. And I'm going to talk to you today about getting comfortable with quiz. So you've probably never heard of Future Generations University. We're based in West Virginia. We're a very boutique institution, meaning teeny tiny. We do basically exclusively international, in-country practitioners of community development and help upskill them. In 2018, so we started, again, a really weird thing. We started in the beginning of the year, so February. We decided to transition from this immersion learning style to a blended learning model. So in the immersion learning, we've been using Moodle for 10 years. They get their materials on Moodle, but we weren't using it as an interactive place for learning. But in 2018, we got rid of these immersion sessions where you'd go off on residential to India and learn from Indian community developers. You'd go to Uganda, learn from Ugandan community developers, and went straight to a blended learning platform or model. So just because we're here at a Moodle conference, they had a lot of trouble. We brought in Mahara. And there were some major headaches, mainly because they had nobody in the institution managing the LMS at all. And so they were sort of relying on it all to kind of work itself and be magical. And it just didn't go very well. So the first thing they did was they shopped around for another LMS. But they decided to stay with Moodle partly because of the open access things and also just because it had a lot of great functionality. And they decided to hire me as a director of learning management just to make sure that it didn't all fall over. And so far, so good. So we're hopeful that everything's going well. So we've got a faculty summer, some few very early adopters of using lesson. They were using all these sorts of competencies and all these sorts of features of Moodle that I hadn't even used yet. But mostly highly skeptical faculty, people that were really invested in the immersion design and weren't that excited about going to a blended model. So that's who I'm with. And then the director threw this at me. All courses now must include at least 30% automated grading. So he realized that automated grading existed in Moodle, which I think was probably my fault, and was like, yep, 30%. Now everyone has to do this. And I was like, OK. Now, so audience participation time. I don't know. Is 30% automated grading in the courses that you guys support or teach? I mean, if you aggregate them all, because I know we all have different things, is it only going to do this the right way? So if we all put our hands up, is 30% much less than what you have? Or is it much more? So I really want to see everyone's hands up. Hands up is 30% like a normal amount? Is it much less than you used to? Or is it much more? OK, OK, interesting. All right, great. So we're all going through this. So I thought, you know what, I think Moodle Quiz would be a good solution here. It's not that it's kind of an easy place to start along with some other course on automation. I think we'll get up to 30%, hopefully. But how do I get faculty who are already not that excited about it comfortable with using Moodle Quiz? Because it's not just drag and drop your materials in. It's got features to it. So this is my approach. Why? What? How? Well, equal success. So I started with why. Why use Quiz? And I started with the pedagogy, because these are faculty that are really committed to the students. They really want students to be getting the most out of the course. And like I said, we've got international students. So I have students that English is not most of them. English is not their first language, but they're learning it in English. So I was selling them on, you know what? Quiz is a really good way to do a knowledge check. It's instant. It's right there. It's individualized feedback based on the readings that they've done. It's quick and easy. And it's a great check for them to see the quiz and then maybe go back in and look at the materials and see that they haven't missed something. So it allows for this direct, instant, individualized feedback, if you set it up that way. The other thing that I was trying, and so this comes to the blended side, we had these live two hour Zoom sessions using video conferencing. And I was saying, if you write a quiz that finishes on a Sunday, you've got your session on Wednesday, you can actually look at the data as to how people did on the quiz, which questions were tricky, which things do you not need to review, and actually change up your live sessions so that it focuses on the things that people weren't really getting. And I thought that was a really cool feature. What is quiz? Well, it's a feature of Moodle or it's a plug-in, I guess, is the correct way to say it. I think the name is really misleading. I got a lot of faculty, the minute I said the word quiz, we're like, these are middle, these are people that are in their careers, we can't treat them like little elementary kids doing multiple choice questions. And I was like, that's not all that quiz does and that's not really a good way of thinking about it, but that was kind of the guttural reaction I got from faculty. And I tried to encourage that the name is just a tag, like we can call it whatever you want. And the settings are endless, you can make it gradable, not gradable, it can be taken a number of times until they get it all right, you can give the answers right away, you can give the answers at your live session, you can do all of those things. So they're like, okay, fine, now, how do we do quiz was the third thing. So I approached it in a various different ways of training on quiz, I use synchronous sessions, both small group, large, well, small group is my largest group. In real life, via the video conferencing, I wrote a Moodle course for how to do quiz, I made some Google docs, I video recorded it, did it all. And the other thing I really did was, look, don't think about the Moodle part of it, like you write the questions, you have to make 10 questions, and I will build them in Moodle. So don't even give me, I don't know how it works in Moodle because I'm not gonna listen to that. And really when you're writing these questions, and so I think that saying that I would build it in Moodle meant that I could emphasize what I wanted as the question. So it needs to be good question development with good feedback, both for correct and incorrect responses, it can be multimedia, all of that stuff. So, did we make it? We had two semesters, we had 100% compliance with some amount of quiz, we only got to 20% of course assessment by quiz, which is actually quite a lot if you think about it, I was hoping to put in more peer assessment as far as the automated assessment as well. But we had plenty of user error, a lot of faculty saw what I built and tried to replicate it on their own and things went awry, of course, and so we did a lot of correcting on the fly. But it also, I felt really moved the needle for faculty from Moodle is a repository where I put my materials and then I create a question that then I get an essay answer for. And this faculty member just at the beginning of October, he was like, I really didn't think I wanted to do it, but it really does a good job of, especially if it's a well-written question, of having the same effect of check-ins that I were doing in other ways. And so the other thing is, did you know that in my work in online learning, I always find that building a rapport is a real challenge, like how do we build rapport between students and faculty, students and students? Moodle quiz can actually build rapport, partly because if a quiz question is really badly written, the students come to class and are like, that question was awful, but it starts a dialogue and they've all all over the world, but they've all experienced this quiz together. So they all have this language they can talk to. And the faculty member also has something that they can be like, you know what, your interpretation of this question might be a little off or yeah guys, that was a really like middle of the night question and I really shouldn't have put it in the quiz. May Kelly take it out and change all the grades. So yeah, so wow, I'm right on time. So thanks, that's what I did at my institution when it came to quiz. I'm happy to take questions, but again, you can't really see anything and I can talk to you about what we do and quiz later too, if you want. Right, so why did the director want 30% and not 50%? That's kind of why I asked the audience participation question. I think it was a sort of a thin air sort of a situation. And so I'm glad he didn't go like 70% because I would have been, so he was trying to come up with a number that wouldn't upset faculty too much, but was moving us in the direction of giving faculty time to focus on the things that I think faculty really need to use their energies on like the development of projects and that kind of thing, not checking to make sure that students understood the content, that can be done in other ways. Are we using other gradeable items as such as assignment? Yes, we're using, I mean again, there were some faculty who weren't using the gradebook at all, so we moved them into using gradebook, we grade forums, we grade assignments, I use the workshop feature. Trying to think, we're trying to get into lesson, I have some questions about how lesson works, but yeah, we're really pushing the envelopes, so I'm really excited about it. Okay. Great, you mentioned gradebook, actually before quiz, before assignment, gradebook seems to be the big monster to really understanding how to set up their gradebook and use gradebook. Could you put that, could you do it all for them? Okay, yeah, so again, back to, I'm just gonna do it for you this time so that we don't have like a huge, I have a very small staff and a very small course load, so I can manage the gradebook, but I also, I think going through the process of managing gradebook, they had these crazy strategies of like these lanes and like you could do these options and then you had like 1,000 points that you could choose from and you only needed to do 100 or something, so talking through how this would look to the student on the gradebook really helped people be like, you know, maybe all that variety doesn't help because I think in online learning, that just adds to the chaos, so they were trying to be understanding and give lots of variety, but I think it actually just makes life a lot more difficult for the students sitting in Botswana, trying to figure out, am I supposed to do all 1,000 of these? Am I supposed to do, it's too hard, so I really help thinking about gradebook, it's really actually streamlined their thinking as opposed to the other way around. Thank you.