 Hello everyone, and thank you for being here. Today we are going to run our third webinar from the assessment series, which is entitled, Assessment Exploring Forum, Exploring Glossaries, Not Forum. I am Anna Krasay, Education Advisor here at the Moodle HQ, and with me today is Meri Kucz, our Education Manager. I don't know if you have joined us or you have seen previous webinars about assessment, the exploring assignment and the exploring forums. If you do, you will notice that this webinar, the Exploring Glossaries, is also organized in the same way. You may follow the examples presented here through the course, live as learners. And remember, this is an interactive course, so feel free to use the chat to share your thoughts, and Meri will be actively watching it. Today we are going to see five assessing examples with glossaries, and these are available in the course assessment exploring glossaries. You can see the direct link in your screen. So if you may want, you may also take those glossaries as we go through them as examples, as students yourself. If you want to do this, please keep the course page open second tab. Now, even better, you will be able to recreate those forum, those glossaries, as we provide them all the full list of the activity settings. As I said before, the course is entitled Assessment Exploring Glossaries, and you can see the direct URL. We will start with an introduction about assessment in Moodle. So you might already know that Moodle can support different assessment methods and different grading methods, and has several types of activities to support assessment. So it has activities that can be auto-assessed, activities where a teacher needs to assess, and activities that are or can be set up for peer assessment. And in Moodle there are also two grading methods, the symbol grading, direct grading method, and the advanced grading. In the first, the grading can be done through points, like numerical scales, 0, 10, 0 to 100, and grading can also be done through descriptive scales, like pass, fail, and complete, a decade complete, or a five-star scale. Depending on the needs of your course, you might also set up an activity with no grade at all. That's also possible. When we use the advanced grading methods, we can assess activities either by a grading rule break or a marking guide. Let's just quickly see what these are. In rule breaks, a teacher creates a set of criteria with levels of achievement, like shown in this table, and grading rule breaks were first introduced in their workshop activity, then in assignments, and last in the forum. And marking guide is the other advanced grading method that allows teachers to add criteria and comments for learners and teachers about each criterion and marks up to a maximum. In assignment, where we first made this grading method, the marking guide looks as shown here. And now, marking guide is also available for forums too. As mentioned before, Moodle has a number of different activities that can be used for grading. The activities that require human grading, and I mean those that need to be graded by teachers or peers are the assignment, the forum, the glossary, the database, and the workshop. And the auto graded or system graded activities are the quiz, the lesson, the H5P, and the score. We have already explored assignments and the forums in previous webinar, and if you are interested, you may take those courses from our platform too. But today we are going to focus on glossary. And of course, you may expect that there will be some more webinars coming in this series. So, in short, glossaries allow participants to create and maintain a list of definitions like a dictionary. And its main function is to be used as a collaborative activity. Glossary is one of the easiest collaborative tools to be set up, but since it allows grading, it also can be used as an assessment tool. Keep in mind that glossary has its own block as well, the random glossary entry, and combining the glossary and its block, you may form some powerful combinations for online learning and gamification. Of course, you may use it for formative and summative assessment, just reminding that formative assessment is to monitor and improve learning, while summative assessment is to measure the outcome of the training. When we use a glossary for formative assessment, the goal is learning through review and reinforcement, contribution and evaluation of entries. In such cases, we usually set up glossaries to require teachers approve learners' entries so they can improve their submission before they become public and always allow editing to further improve even after publishing. Deplicated entries can be allowed so as not to discourage learners, and usually we allow comments on entries. Allow comments on entries can be really powerful, especially when it is combined with the always allow editing, as it gives learners the opportunity to continuously improve their entries as they get feedback from peers and teachers. When we use the glossary for summative assessment, things are different. The goal here is the assessment. So that's why we tend to set up the glossary so as no teacher approval is required. We may allow or note the duplication of editing depending the task, but there is not much value to allow comments on entries as contributions are considered as finals. Also, in a glossary that is set up for formative assessment, we certainly want to allow the automatically link of entries. This is a great way to reinforce knowledge throughout the whole course content. We also want to enable print view in order to offer learners another format to access knowledge. And we might use also the ratings to motivate learners or to engage them more in a collaboration. If you want to allow peer rating and the aggregation method, and the aggregation method that we might use would be probably the maximum rating. While in glossaries that have been set up for summative assessment, the automated link of entries and printing are disabled because the summative assessment would take place at the end of the course. The ratings would be certainly enabled to using probably an average aggregation and pass score, so as to evaluate the learners achievement. And in this case, ratings might be probably restricted to specific dates. Setting a date range would indicate actually the end of the activity assessment. So glossary is an easy yet powerful activity that allows learners to collaboratively build a list of entries. It supports communication and through comments and of course, course assessment. Depending on the choices that you will make in the settings, you may create completely different activities serving different needs. And together we are going to see most of these settings. Just before getting there and see the examples, let us meet each other a little bit and tell us a few things about you. Let me launch the first poll and I'm giving about one to two minutes to go through these three questions. Tell us a bit about your background. If you're a primary school teacher, secondary or professor, trainer or other, how long you have been using Moodle and if you have ever used glossary for assessment. I'm seeing some very interesting results coming up here, Anna, actually. Very interesting. Okay, so I'm ending the poll and let me show the results with you. So we have a lot of secondary school teachers and even more professors and many more trainers. And the other, I'm really wondering what this other can be. Feel free to share your affiliation and your role in the chat. It will be interesting to know. And I see that most of you are experienced Moodlers. Yet the majority of you haven't used glossary for assessment. Now, that's really interesting. That's really interesting because it means, hopefully, we can show some experienced Moodlers some new things about using glossary for assessment. As I said, typically in a Moodle glossary, that is, I said, we expect learners to submit entries that will be assessed by the teachers. But Moodle glossary due to its flexible nature can be adjusted to serve out of the box cases, such as learners assessing teachers' entries, peer evaluation and gamification. Let's take one thing at a time and get started with a formative assessment where teacher needs to rate learners' entries using a simple numerical scale. The example that we will see requires teacher to approve learners' entries. And learners can always edit their entries to improve them after teachers' feedback or their peers' comments. Dubligated entries are allowed so as not to discourage learners and the rating will be done with a simple numerical 05 scale. If you want to see the example into the course, please check section one, Formative Assessment in a Numerical Scale, and the glossary entitled, It's All Greek to Me, or Is It? Now, this glossary is a stimulating activity in a Learn Greek Linguistic course for American university students. So learners have to add a new entry from the passage provided by the teacher and write its definition. Since the entries require approval, learners will see the message that this entry is currently hidden until the teacher approves them. Now, when the teacher sees the message that there are entries waiting for approval, he will see the entry made and he will be able to rate or post a comment or approve. In this case, it seems that the teacher is not very satisfied with the entry made. So they prefer to write a comment without rating or approving it. And this gives learner the chance to improve their current submission. So, tell us, do you like numerical scales for glossary entries? Would you consider to use them? Or have you used them? Because just a few of them said that they have used glossary for assessment. Is this the way that you have used it? I think I tended to use numerical scales if it's a teacher rating, a teacher rating students. I'm not sure about if students rate each other, which I think we're going to discuss later. But I do also like custom scales, to be honest. We have a comment from Alex, who says, I use a star's scale. That's nice. Do you mean that you use an asterisk or you actually use an emoji, a star? That's interesting. Please do type in the chat if you have any other thoughts about numerical scales for rating glossary entries. Of course, you also have to consider your aggregation as well, if you're going to use, well, any scales, in particularly numerical scales. Let me move on to the next example. In the second example that we will see, it is also for formative assessment, only though this time teacher rates the entries using a custom scale. As Alex said, the example also requires that teacher's approval and learners also requires teacher's approval. And learners can always update their submissions, although comments are not allowed. The display format here is full without offer and you will see in a well wide. And for the ratings, it has been used a custom scale, an emojis custom scale, like the stars that Alex says. If you want to see the second example and understand better what this is about, you can take the section to formative assessment custom scale and the example, the glossary is named the animals. Now, in this example, this exam takes us to the first classes of a primary school, where the French language teacher use moodle glossary to engage the young learners by asking them to create their own vocabulary with animals. The learners must use images and make a voice recording directly into auto-editor reading the animal. So if the student says a fees, they should say poisson and record this into their entry. And then the teacher will have to approve the entry and also evaluate. But because we talk for young learners and formative assessment, the point is that teacher wants mostly to reward learners for their effort rather than hardly assess them. So they can create an emoji scale, as you can see them here. This scale is just a positive, fun feedback for the learners. It's not really an assessment, an evaluation. It's like, I don't know if you have ever experienced, but it's like the stickers some teachers are used in the face-to-face classes. I remember I loved them a lot when I was a kid. But if you're wondering, if you're not familiar with the setup of a custom scale, I have to tell you that it's a very easy process. All you have to do is go to the grade book, and find the scale step. And from there, the option to create a new scale. You give your scale a name, the scale levels starting from negative to positive, separated with comma. And if you want a description, in our case, of course, the order of the icons don't really have a value, but normally in a typical other scales, you should start always from negative to positive in order for grades to be correctly calculated. If you have admin rights in your site, you will be able to set the scale as a standard scale that will be used side-wide. If you have teacher rights, you will be able to create the scale and use it in your course. So what do you prefer? A numerical custom scale for glossaries, an American order custom scale for glossaries. What suits you best? Well, we already had an answer from Alex who said custom. I think I agree because it gives you more opportunities, for example, emojis or words and phrases and so on. But let's see what other people are thinking about as well. Also, I suppose it does depend on the glossary and your purpose or the needs of it, for sure. Do you think, Anna, or again, does it just depend? Well, I think it depends always the context. And, yeah, the purpose of the glossary, of the activity, to be honest, I love emojis and fun scales. And I would use them as much as I could. Right, yes. But I know they don't fit everywhere. No. And Elizabeth is agreeing, saying it depends, yes. Which one you use, it does depend. OK, let's move on to the next example. Now we're going to see a different example. It's the collaborative assessment. And here, the teacher has already set the glossary to allow learners to rate the entries the teacher has set. And the rating will be done by learners using a numerical scale. It's a different concept. Now, in this example that we will see, there are just four important settings. The ability for learners to make comments, the display format that is simple dictionary style, the ratings that use numerical scale 0 to 5, and the average aggregation, and, of course, the manual completion. The rest of the settings don't really affect the concept of this activity. Why? Let me be more specific. Because in this setup, learners are not allowed to add new entries. And also, in this setup, learners are allowed to assess entries and also to view the total ratings for each entry. And this is something beyond the default setup of the glossary. And it is something that can be achieved by role overriding, permissions overriding. Let's take a look in detail. If you are in the course, please check section 3, Collaborative Learner's Assessment and the Glossary Collaborative Bibliographic Review. Also, it's worth noting that if you're not able to add entries or look at it now because you want to focus on the slides, please do go and complete the course. Look at these entries later, and then you can get a badge for having completed the course and attended the webinar. So do explore them afterwards if you're not able to at the moment. Thank you, Mary. Now, in this glossary, it's taken by a university class. Here, the professor has provided a list of references. There are three references, three articles that need to be reviewed and evaluated by learners. So as you can see, as a learner, you don't see many things. You can see the all entries. So this means that the learners need to see all the articles provided by the teachers. And they don't have the ability to add the new entry. There is no such button. Well, they can comment, and also they can assess the entries and see also the total score in each entry. This is not how glossaries work by default. Now, as a teacher, well, the teacher has control over the entries. Teacher can add new entries and edit the existing ones. They can also leave comments. But they cannot rate. And that's because these entries are made by the teacher. So a user cannot rate its own entries. If you're not familiar with permissions over writing, let me show you quickly how easy it's to tweak a role and create something completely different. All you have to do is to go from the activity, click the Actions menu, and select the permissions. Now, at the first, you will see a huge list of capabilities that apply in your context. But you can easily filter to find the capability that controls the action you are interested in. So for example, here, teacher is looking for the rate entries capability. And by clicking the plus icon under the roles with permission column, you can select the learner in the Allow role window. And this is how you actually allow learners to rate the post. You don't have to worry about remembering this because you can find the detailed guidelines and direct links to the specific moodal documentation articles but also description of exactly of the way that the overriding is happening. And we are going into another example. But please keep in mind, the previous example, the learner assessment case that we have just seen. Because we are also going to see a summative peer assessment case where learners create entries and assess them, assess each other actually. So in this setup, entries are not approved by default, not because the teacher needs to approve them because this is a summative assessment, but because teacher wants to release all entries at the same time, we can say that this activity will be completed in two stages. In the first stage, learners need to make their own contribution. And in the second stage, they need to evaluate each other's work. Comments are enabled, and the rating will be done in a 0-themed scale with average aggregation and pass score set to 8. Again, we have permissions overriding so as to allow learners to assess entries, but only this one. Let's see, if you are in the course, please move on to section four, summative peer assessment and the glossary entitled customer horror story. I have to say out of all your examples, Anna, this is my favorite. I hadn't thought of using this like this before. And I really enjoyed trying this out when you were preparing for this webinar. Yeah, this is a great idea. Thank you. This example takes us into a corporate academy, and particularly in a customer service course. Employees are challenged to deal with an angry customer and write their answer as contributions to the glossary. From a technical perspective, we have already seen the entries approval process and the permission overriding. So there is nothing new here. What is different is the concept. While in the previous example, learners were assessing teacher's entries, now learners need to make submission and evaluate each other. And of course, while in the previous example, the teacher's approval was used to give teachers the ability to review and provide first feedback to the learners, this time it is used just to control the release of the entries. So all the entries will be released within seconds and not in different days. So what do you think? What would you prefer? What could be more suitable in your case? A collaborative rating of teacher's entries or a peer rating of learner's entries? Yes, please do type in the chat and let us know what you think. I suppose because glossary really is by default a collaborative activity, then peer rating is what I would possibly suggest. I've just noticed, yeah. Oh, Owen, Owen is it? I remember we had this discussion last time about your first name and I apologise a thousand times, but Owen is saying peer rating, okay. Elisabeth is also saying peer rating. Alex, both it depends. I'm interested to know that Anna is saying this is perfect for writing courses. So you've given Anna an idea for her writing courses there. Daniel is also saying peer rating. Stereos is saying peer rating seems more suitable for glossary. Okay, I think peer rating is winning here, Anna. Yes, it seems like. Okay, let's move on to our last example. Now we are going to see another example of summative assessment where teachers evaluate learner's entries but in a gamified activity. In this glossary, glossary as I said, is a very versatile tool and it is often used as a gamification element. To set up, the setup of the activity remains the same, but it can be based on a different concept and can be combined with blocks to make it more, to gamify it. In this example, the contributions don't need approval and entries cannot be always edited. Since we talk for summative assessment and the rating is set to 0.10 scale with average aggregation. Additional rating is restricted on a specific date range. For this case, the permission that has been overridden is the comments and learners have been forbid from commenting. To understand this gamified case, let's first the context. So we have 18 high school students that are invited to collaborate a debate for the destination of their major educational trip in a European historical city. The trip is a project in the context of the history class and the course is offered in a hybrid method using face-to-face and digital tools. Now, learners have to choose between three cities, Paris, Rome, and Athens. They have been already split into groups inside the class and as such, they need to make submissions to support the preferences. Now, we have actually three groups of six students and each group is going to support one of the three cities. What's the concept? Each group, one of the three groups needs to make six entries into the glossary. One team, one entry must be done by each of the member to cover the following topics. They have to cover the overview and history. They have to cover sightseeing. They have to do an entry about suggested programs, an entry about additional expenses, and an entry about food adventures and entertainment. Then, teachers will evaluate the contributions and the country with the highest score will win. So what do we have? We have a competition, a trip, a glossary with ratings, the activity results block, the random glossary entry. Now, if we see this from the gamification ante, the gamification ante, we will see that the competition is the challenge, the trip is the price, the ratings are the points. The activity results block is the leaderboard and the random glossary entry is the surprise element necessary in every game. So let's see this activity. If you are in the course, please check the last section in the course number five, summative assessment, gamified, and specifically the glossary choose the destination. As a learner, there is not much different to see. There is a glossary, nothing special. Learners can add entries and entries can be browsed by category for easier review. Simple setup. As teachers, they have the ability to review and comment and rate the entries, also very typical. And that's it. Let's see a little bit the blocks that support this gamification approach. We can see here the random glossary entry that is shared in the course main page. And as learners and teams are making submissions and to support their choice, their favorite destination, I think entries will randomly pop up in up there. So there will be a trigger to see what's coming next and what the other team has said and what's going on. And also the activity results block. This one is set inside the activity itself, at least by entry, so by user, the highest score. But everybody can see how the scores will change as new entries are coming in and how the score is organized. So I don't know if I have convinced you about how versatile a glossary can be done. I would like to hear from you if you have any comments for this gamification approach. Well, I had not thought of using a glossary. I have to confess with a random glossary entry as part of gamification. I've certainly done gamification and presented about gamification before, but this idea of doing something with a glossary is a new one to me. So I think that's really clever idea. Please, if you have something to say about this one or any others, please do. I mean, you convinced me, but then I'm easily convinced. I would like to know what our people participating think about it. And would you now, having seen these examples, would you now use glossary as part of your assessment, whether that's formative or summative? You know, I think we should launch a second poll. Go on then. Give me a second. While you're doing it, Elena is saying, it's amazing all the possibilities. I need time to process and imagine this working in adult education. Yes, that's why we have courses attached to all of our webinars where you can go in and revisit what we've done. And Bromi, could you please share the slides? Yes, the slides of this webinar will be available shortly after the webinar in the course, as for all of our webinars and courses. Okay, Anna, over to you. Well, meanwhile, I have managed to launch the second poll. Now that we have seen the examples, please tell us which one of the set-ups, glossary set-ups we have seen today, were completely new to you. And here you can select more than one. Very interesting results, I have to say, watching the people's polling results coming in, yes. And the second question is, which of these set-ups that you have seen today will you attempt with your students? Which one do you want to try? Bromi is asking, while we're waiting, is there any possibility to assess students with a linguistic variable like good, very good, et cetera? Yes, you can make a custom scale for that, starting at the lowest and working up to the highest, and then you can rate them using that with glossary entries, certainly. I think if you take the ideas and you contextualizing in your own context, I feel that the possibilities are endless. And I would love to hear from you, we would love to hear from you in the chat later, what we have tried, how you did it, how you implemented, and how it worked. I see Starius is saying that I would use glossary mostly for formative assessment, and it's similar, but much easier than a wiki indeed. The structure of a glossary is so handy, so easy to be used. That's why it's one of my favorite books, to be honest. Okay, I'm going to end the poll. So very interesting inputs. I think the most new concept was the gamified activity, but also the learners who evaluate teacher's entries and the peer assessment of entries are also very, very new. And it seems that a lot of people is fond of peer assessment. Yep, I will agree. I think it's the easiest activity for peer assessment to be set up. And I wish you a good luck with that. Come back to the forum and give us some inputs how it worked for you. Yes, please do, please do. I'm going to read a comment from Alex. I use glossary for assessment in a combination with forum or assignment. Analyzing a learner's comments and ratings gives me deeper understanding of how the learner thinks. So I have a clearer picture when assessing the learner's submissions or forum posts. That's very useful to know. Cheryl, hello, Cheryl. Cheryl says, I use glossary for fun, like asking my students to send a picture of where do they want to spend their next vacation? And then other students will make comments. It's really fun. Yep, I think one of the favorite setup is icebreakers with glossaries, like the one that Cheryl said, or your profile and image. So what do you see from your window and stuff like that to be asked for glossary? So this is, that was our presentation and our examples. I hope that it was interesting to you. And please, if you have any questions. Would you like to, would you like me to read this one out, Anna or Cheryl? Yes, please. Okay, Lucy's asking a question about a glossary in general, certainly. It's to do with backup and restore. When I backed up and restored a glossary, only the activity in its settings were restored, not the entries. The reason for that, Lucy, is that in order to get the entries, entries are classed as user data. And so as a regular teacher, you're not allowed the security reasons to backup user data. So what you can do is you back, you can, I think, is it exports the glossary entries and then you can import them into a glossary in a different course. So you can get the entries, although they won't say who made them originally, but they don't come automatically. It is something that has been asked for in Moodle, but they don't come automatically as backup and restore. You have to export and then import. And Brumi says, nice presentation, Professor Anna. So there you go. Thank you. Okay, are there any more questions or comments from people? We've only a few minutes left. We appreciate the thanks, yes? Well, Anna's done all the work, I have to say. I'm going to wrap up our session by saying a few information on how to get involved with the Academy and help us grow by contributing to its development. Being in the Academy, you can be an active member. You can suggest topic ideas to be covered in future webinars and courses. Please feel free to join our Get Involved course. You can contribute to webinars. You can be a presenter in a webinar. And remember that presenters will receive a presenter's badge from the Academy. And you can also contribute by setting your expertise and helping create a build course. And of course, you will gain a course builder badge from Moodle Academy. And also please help spread the word. Tell your friends and colleagues about Moodle Academy and the Moodle Educator Certificate. You can earn badges by completing the courses on Moodle Academy. And tell everyone what you have learned and what you have liked in Moodle Academy and help us grow this community. And if you are an experienced Moodleer with more than one year of teaching experience with Moodle you might want to consider taking the Moodle Educator Certificate. You can take the Are You Ready for MEC course and take the quiz to see if you are ready and get certified as educators, as Moodle educators. Do we have any other questions, Mary? No, I don't think so. I think just thanks and from Joanie saying having the glossary settings organized for us is very helpful, thank you. And I know we've discussed that and I do agree. It makes it easy for you to copy the settings and to try them out yourself. So no other questions, no. Thank you, thank you everyone for being here. Thank you for your participation and your time. I hope you enjoyed the session. And we will stay in touch in this course and in future webinars of the Moodle Academy.