 Hello and welcome! We would like to speak to you about our Moodle Quiz and Questionment project. We have ongoing scenes Moodle 4.0. Some of you might have heard of it, some of them won't. Anyhow, along this presentation we have prepared some questions ongoing we would like you to look at and maybe answer during your speech and we will look up the answers at the end. It's a simple VooClap, those six letters. Here's our agenda and we'll start right away with the project overview. Since we spoke to you last in September 22 a year ago Moodle 4.1 and Moodle 4.2 has been released. We talked shortly on what of our developments did make it into those versions but we wanted to state clearly that the 4.2 developments were smaller than the other steps we have been able to accomplish. And this due to the fact that we reorganized internally and we switched our development partner from catalyst AU to catalyst EU which took us some time resources during winter and spring. But who's that aforementioned we that's a lot of institution backing us with funds and the initiators precisely those four people. Tim, Thomas, myself and Antonia who does lead the project lead. In Moodle 4.3 we will be able to land one of our architectural base issue. The filtering conditions in question banks now do use the same user interface as participant filtering does. Then we are allowing for some more flexibility to customize the question bank for teachers and some other technical improvements. But let me show you what the first two bring to the user. Here you see that the question bank filtering condition can concatenate now and multiple one of these as well and use the same user interface that you know from the participant page. Thus it is possible to drill down to the questions you want in even more detail than before. Bonus point, if you have drilled down in all that detail as you wish the filters are persisted within the URL and you can share that link so that the other person receiving it and opening the link has got the same filter criteria in front of them. The same user interface elements are also there where you add a question to a quiz. Teacher can now rearrange the columns of a question bank, remove columns or resize them in width based on their proper needs. This is implemented and integrated in Moodle as we speak which makes us the most actual speech which is the reason why you should for us for a speech of course. In this slide you see some more aspect that we have been able to get into Moodle 4.3 but there's still one main part we are working hard to get done and that is the possibility to make questions which are in a question bank in a course shareable to other places and with that I would like to hand over to Tim. Thank you very much Luca. Yes when we started this project all these various ideas we had for improving the question bank we thought we might achieve in Moodle 4.0 and it's taken a little bit longer than that but I think as we worked on it our ideas have improved but as Luca says one of our key goals right from the start was improving the ability for teachers to share questions between courses and while we've done a lot of other features that will make that easier to implement we still haven't actually done that key part and what I'm going to explain now is what we think is the right way to implement that part and the reason I'm talking to you now before it's done is we think we have a good idea but we're not absolutely sure so after we've explained what we're currently thinking we would very much like your feedback are we on the right track or do we need to think again do we need to improve it so before we talk about what we're thinking about for the future it's probably best to make sure we are all aware of what currently exists in Moodle today and what currently exists in Moodle is what has been there since the time of Moodle 1.9 which is more than 10 years ago and it just no one's worked on this so what exists for sharing questions between courses in Moodle 1.9 is only available if you have an elevated role like admin or manager and if you have one of these roles or a custom role and go into the question bank then what you will see is as well as the questions for this quiz and this course you will also be able to see shared areas of question bank at the system level and the course category level and as I say one problem with this it's only accessible to people with elevated roles and if you have elevated roles to access these areas suppose you have permissions to access questions in the system area then because of how Moodle permissions work that means you have permission to access every question in every course in the Moodle site which may not be exactly what you want unless you really really trust all your users there's another problem that there is only one system area so if you have different groups of people who would like to be able to share questions throughout the site you don't really have a way to do that so that's what exists today and it has these problems which are basically what I just said by the way that the slides are not appearing on this screen down here I don't know if there's anything we can do about that the other big problem with the current system is that if you use it there are some nasty bugs which happen when you back up and restore courses and within the current framework various people have tried to fix this and they have not succeeded so it's kind of an inherent limitation of the current system which makes it a bit dangerous to use the sharing features which is a problem so this is where we are today and thinking back to Marie's talk about the Moodle development process I mean in proposing a new feature there are several steps they're sort of identifying the problem and what the requirements are and then there's ideating possible solutions and only then do you build it so we're looking at both these first two pages sort of identify what the problems are oh I've got some diagrams so yes at the moment sharing you can put your questions in the system question bank and then use them in different courses but for a lot of institutions this doesn't actually work for them so what a lot of institutions actually end up doing is they just keep questions in the course question bank of their various courses and where they need the same questions in a different course they will export them and import them and then you have two copies of your questions and as I as a software developer I know that whenever you have a copy of things you need to maintain that is a big headache and causes problems so if you spot an error you have to remember all the places you copied that question and go and fix it that's the picture of today so to summarize what we think the requirements for a new question sharing system is we think that a shared question bank needs to be a kind of thing that you can create the ones you want not sort of tied to any specific part of Moodle and they're having created a question bank you can make a particular question bank available to a particular group of users and then when a user is building a quiz they should be able to add questions from any area of shared question bank that they can access and add them to the quiz if we're saying shared question banks are a kind of thing you should be able to create one give people access there are other things we would expect in Moodle for example if you can create these things you probably want to be able to back them up and restore them and so on well we do want to enable powerful sharing features it's also not everyone needs this quite a lot of the time we're just in that simple a teacher will be in a simple situation where a teacher just wants a quiz in their course independent of everything else they just want to be able to go say new quiz add a new question make a question add a new question make a question so well some people want sharing other people just want simple quizzes and we must remember that and must go on supporting that and there are the existing backup and restore bugs we need to fix those so that is our understanding of what the problem is we are trying to solve I mean how do people do people feel we have correctly captured the problem or do you think oh I see a few positive bits of body language coming back right so this is the problem we think we're trying to go on and now then we've been ideating well sorry keep forgetting I did these slides this is my loose diagram of what it is we think we need so sort of continuing the previous example I sort of indicated that within Moodle you need to have these we want to have in this case in this example these two different question banks but I kind of gray boxes it's not quite clear what they are and the rest of the talk is me explaining what I think these things need to be so let me take you through my thought process as I contemplated these requirements and how we might solve them now I mentioned that there's the requirement it should always be able to build a simple standalone quiz and that suggests that there's the potential we have today that questions can be stored inside a quiz we need to keep that and there are other activities within Moodle that use the quiz for example at the Open University we use the student quiz plug-in it's a very nice little add-on if you're worth looking at if you don't know about it and the student quiz is basically a bit of question bank with tools around it for students to do peer learning and simply as I was just saying the quiz is like a bit of question bank with some quiz functionality so I thought to myself we you know activities already have bits of question bank and we need to keep this and also Moodle activities get you a lot of the other things we want if you have an activity you can create activities where you want them in your courses you can control which users have permissions in activities for example in a forum you could give a user moderator rights by assigning them the non-editing teacher so that kind of flexibility of permissions exists and activities you can back them up and restore them so activities give us a lot of the requirements we're trying to achieve and there are things that already exist in Moodle and we know we need activities with questions in so do we need anything else you know I said quizzes like question bank with quizzing features what if we made an activity that was question bank plus nothing else do you need anything beyond that well you do need this bit that when you're building a quiz you can add from any question bank you have access to not just the current limited range of places you can add questions from so that was a conclusion that doing this was the simplest possible way to meet all the requirements but well anyway let's yeah sorry here is a mock-up of how it might look in the simplest possible way it might be implemented so here is a quiz with a couple of course with a couple of quizzes and we've also added a question bank and looks like any other activity with a particular icon probably students will never see it so it is a bit weird you have something on your course page that students will never see and when you go into it it's kind of a bit like a quiz you know when you're in the quiz interface you have question bank here in this you have question bank you don't really have anything else and then when you go to add to a quiz you're very familiar dialogue if you've used to building quizzes it's showing you currently the questions from one question bank the course question bank and then there's a button there you could go and select a different question bank and add questions from there instead I haven't mocked up what happens when you click that button but it be some sort of list of all the question banks you're allowed to access and then you can change so that summarizes what I've already explained will make a new type of activity called shared question bank will change add to quiz so you can add from any shared question bank obviously there's a migration but that should be a completely automated upgrade as moodle upgrades always are and once we've done that we can clean up a lot of the old mess in the code and fix the backup and restore bugs that's the basic outline of the plan and here are some of these schematic diagrams so this is a direct translation of the previous example the only thing is to have a place to put these question bank activities we've had to create some extra courses for like language resources that are shared between different sites now at least at the open university this is something we're already doing with other things and I'm seeing some enthusiastic nodding there and I've spoken to other people this seems to be a pattern people are already using I don't know if everyone's already using this so this aspect doesn't really worry me and of course you don't have to do this if you're currently in the situation where you only ever store questions in your course question banks that will just continue to work pretty much the same it'll just the question bank will now be an activity rather than a special built-in thing that's part of moodle and you may think for a course you only never need one question bank but once you realize that these are things you can create and remove well you can start thinking so maybe at the moment you can only have one question bank in your course but maybe in future it would be helpful to have two perhaps if you have a final exam you just want to keep questions related to the final exam separate from the main course question bank so someone doesn't accidentally use them maybe you want to have different permissions there so this is all about give this this brings a lot of additional flexibility that could be used in various ways and moodle users tend to be very creative in how they use the features even in ways that the original developers didn't think of but there is this issue suddenly this thing that's in being in moodle for a long time suddenly it's going to start appearing as an activity on the course page and normally things on the course page are things we're using to teach students this is something only for teachers well as I said there is this just too weird is this an insurmountable UX weirdness that means we actually need to think again and find a different plan that is that is the fundamental question and I switch between thinking this is a neat and elegant solution to all our problems or it's just a UI disaster UX disaster waiting to happen and this is the thing we would most like your feedback about because of course at the moment in moodle's navigation we're familiar with finding the question bank here we're thinking that probably will keep that link in the navigation and that will take you to a page that will list all the question banks in the course and probably in future when you create a new course you won't immediately have a question bank you but if you want one will probably have a single button there just create me a default course question bank but if you're not going to use that you don't ever need to create it so that again simplifies things for people who don't need question banks so yeah that is the sort of fundamentals of what we are proposing and as I say we would like your feedback we'll have a chance any minute now right in this room to start the discussion and Luke and I are here till the end of the mood so come and grab us and of course there are the forums on moodle.org and actually a week or so before this conference a discussion spontaneously broke out someone asking anyway this has turned into a discussion of these proposals so please you can also continue the discussion on moodle.org or reach out to us in any other way so just before we go to the discussion I think Luke is going to show us the results of the WooClap thank you much yeah yeah I'm feel free to answer along that already a first part and maybe yeah this is to WooClap code so it's on the sheets I try to read out the results if you'd like it's not possible to switch to it and switch it back we will then share that with you so far 42 answers so 59% say yes to the question does the organizers share questions between courses using the system and course category question bank it's a bit surprising to me it's good yeah so it's sharing is seems to be something that is a use case and a requirement 55% say yes to the question do you understand that feature course do you understand that feature could you explain to someone how to set it up so 45% say you know and don't really understand you couldn't explain it and is sharing questions between courses a feature your organization needs at the third questions there 88% say yes that's the case yeah thoughts on what Tim myself the team that is thinking about this is some has shared here with you yes please okay thank you for the talk I have a suggestion and a question I'll start with a suggestion with also sort of a comment I'm not a fan of the question bank appearing directly on the course page I think that's gonna be incredibly confusing I can already hear the professors in my office telling me why is this here why is it happening have you considered or maybe you should consider having it as a block in the course because my professors already kind of use the block drawer as they're like go to course management thing and so if the question bank appears over there they can expand it or contract it so it's still it's it's not something that's going to be directly confusing to see but those professors that do really want to use it can sort of optionally have it right so that's my suggestion you are in that that's too weird yeah can I also just check which version of you still on Moodle three-point something we're on four point two yeah we're on four point two as of August with blocks yeah yeah okay yeah thanks and then my question so something that seems a little a potential for some sort of for some stickiness if you have these question banks they're shared between many different courses right ideally with many different professors right and this will especially be an issue with larger universities what happens when you've got five quizzes with the same question in it and then one of those professors goes in and says oh I want the wording to be different here and then you've got active quizzes that now we're changing the text of those questions while students are taking them so what's what's the thoughts there so I think the overriding thought is that any time you have a group of people using a shared resource they need to actually be functioning as a group and communicating with each other because it is a shared resource so you kind of need collective ownership although we haven't got the key sharing requirement yet we have delivered a lot of other features in our ongoing project so for example now in the question bank there is a column that shows you how many quizzes use this question and let you see where this question is used which helps you understand the implications of changing it there are comments in the question bank so there's a some technical I mean conversation is really a people problem but there's some technology to support it and I was thinking that was the third thing but I can't remember anyway so that's that's my sort of oh sorry yes when you're building a quiz you can the default option is that if the question is edited students will get the new version but you can actually fix a quiz to that I want this version of this question to be used in this quiz so thank you some of the tools support to help with that situation yeah so lots more questions so thanks for you talk and thanks for the initiative I'm definitely one of these people that found the idea of having a question bank activity very weird and it was totally opposed to it but the more thought about it the more liked it and figured out that lots of the problems that we have current projects would be solved by that even though it feels weird at the first time to actually the first months I thought about that yeah my question would be I totally agree with with you that it is weird from a UX perspective but have you thought and figured out whether having a section separate for teachers like a teachers area would could be navigated to differently like this we at section zero yeah whether that could be a UX solution to your problem again that is a good idea and you're not the first person to suggest that and actually an additional situation talking about what people already do I've known courses like at the university with groups of teachers where they will have a forum that is hidden from students to let the various teachers on the course communicate so I think maybe and I think maybe having a special section in the course to hold these what you might call teacher tools because there are lots of tool Moodle has lots and lots of tools and they're really useful for education but they can also be used for teams of teachers so that is an idea we're thinking of yeah that's a good suggestion thank you yes so I'm also in the in the two weird camp and I know that it's probably extremely hard to do technically but I think that the best idea the most intuitive thing for teachers was always these are my questions why are they not bound to my user in the way but I guess it's impossible at this point to to change the overall architecture in a way that was reflected and if that's not possible then maybe the idea with activities is not so bad but I'm not sure because the whole concept doesn't fit into my head right now but could you have like central courses with a lot of these activities and just give the permission on the activities to certain teachers and have these at the course that sits in the root category would that work at the moment that would certainly work with this proposal I mean almost I mean the thing about this proposal is then it's like anything else in Moodle you can set up what you need for your organization we're not forcing you to follow any one pattern so if that pattern that what you suggest would certainly be possible and if that's what works for you you can set that up but if someone else needs something different they don't have to have it tied to individuals do you know what that work at the moment like with the current architecture if I make a center you can't do it well you know so with the current architecture when you're building a quiz you can only add questions from the course you're in or the course category or the system so I would never be able to add from this activity in a different course those would never show up those would never show up and that's one of the things we need to change that would be great maybe the barrier to change this is lower than to change it so that questions are bound to a user right yeah we only have time for one more question maybe to if it's quick so yeah just continuing on from that last point I was thinking along very much the same lines it being tied to the user and I think maybe useful paradigm is the idea of kind of Google Drive and you know they create resources there they then add collaborators to that also a corporate level if somebody leaves the company you can transfer ownership to somebody else I think that at that level for sharing across multiple courses maybe even multiple faculties it doesn't really fit nicely into courses or categories and things like that as as you're kind of acknowledging so yeah another vote there for maybe looking at the user context the issue with that is that it doesn't really fit with Moodle's model for managing permissions in Moodle you can give someone permissions on a course or an activity to do that we'd have to make like a category of questions in a question bank be a type of thing where you can assign permissions and that's quite a big change that's only used for our I mean it could be I mean anything can be done if you write enough code but that would be a relatively big change and a conceptual change from how Moodle currently works whereas activities already a concept in Moodle so yeah thank you has that been the last way I don't want to finish without credits never have done half very quick one minute before it's okay no so let me finish with the credit so we told you about the structure while we think about it who's the people behind we have our in close contact with HQ of course but we're our time has is funded by by our institutions but we are not funding the work by catalyst this is funded by this fine organization if you want to be a part of that this qualifies as a research initiative research cooperation collaboration pardon me so in that it is value-added tax exempt it's really easy to be to be contributing my institution does make the role as bank obviously we're Swiss and we pay then catalyst but if you want to contribute tell me get in contact with me get in touch you might have a fund for quality assurance there's maybe a special cashier for digitalization you sent me the amount I can send out an invoice we can do a contract even if you insist on doing it thanks to all these institutions and let's keep that funded and keep that going along please contact me thank you