 Hi everybody. My name is Rob Lanny and I am the Learning Technologist at Hibernia College. I'm here today to share our story, our journey in moving to using Moodle to host and manage all of our continuous assessments. Unfortunately my colleague Hilary Roach was unable to join me today but she has provided her voice to the presentation so we're going to hear from her first. Hello everyone. My name is Hilary Roach and I am head of teaching learning and assessment on the primary program in Hibernia College. Since the year 2000 Hibernia College has been providing both blended and online learning programs leading to professionally accredited qualifications. We provide initial teacher education for graduate students wishing to pursue a career in teaching. There are professional masters in education or PME programs for both primary and post primary levels. The programs are two years in duration. 45% of the PME program is delivered in an online environment via content and webinars. 55% of the program comprises face-to-face experiences at day-long seminars principally on Saturdays. There are also 24 weeks of school placement over the course of the program. Students are located in all areas of the country as you can see from the map. We provide online continuous professional development courses for practicing teachers. We have recently received the go-ahead to introduce a BSC program in general nursing and we also offer a PhD program. There are two intakes each year in each of the teaching programs in spring and in autumn. So we don't adhere to the traditional academic calendar. Our 1600 students are located throughout the country as I mentioned. And in addition, because of the nature of the program, we have some 300 assessors who are also located nationwide and beyond. Assessors can access student uploads from wherever they are in the world. There is one person located in the Middle East currently. We engage continually in continuous assessment. Assessors provide lengthy, specific, descriptive, qualitative feedback which is tightly aligned to module learning outcomes. So where have we come from? Students used to upload their completed assessments to Moodle. Assessors conducted a bulk download of the approximately 30 submissions that had been assigned to them and embarked on the grading process. They completed individual feedback forms in Word. The marks for individual sections of the submissions and any word count penalties to be applied were entered on Excel spreadsheets. The feedback forms and the spreadsheets were returned to the program administrator as attachments by email. The administrator then compiled the spreadsheets and manually cross-checked the feedback forms of which there could have been up to 400. Submissions were selected for moderation. These included the work of failing students as well as a representation of the variety of grade levels assigned by different assessors. These were then emailed to the moderator. The moderator completed the relevant report, amended student feedback as appropriate and completed another Excel spreadsheet. This was then returned to the program administrator. Lake penalties where appropriate were applied. The program administrator dragged and dropped the individual feedback forms to Moodle. The results were then released to students. There was a lot of manual work involved with the associated potential for errors, particularly when dealing with such large numbers. Thank you very much Hilary. So clearly our process was not as efficient as it could or should have been and therefore it needed to change. So we looked to Moodle for that change and in particular what we wanted to do was we wanted to make the grading process smoother for assessors, module leaders, moderators and program administrators, the key stakeholders in the process. Very importantly we wanted to reduce the reliance on the myriad of documents for capturing and storing feedback and marks. And then we also wanted to create a sense of a shared space, the shared ownership of the continuous assessment process so that all of those parties could see the role that they played in the process and so that the communication between them and the college could be strengthened because as we know the college is based in Dublin and our assessors are located around the country. So how did we do it? Firstly we mapped our own college assessment processes that stem from our academic policies and procedures to Moodle's own capabilities and as part of that we decided to use marker allocation in assignment to make it easy for the assessors to locate the submissions that they needed to grade and we also used marking workflow to control the various stages of the CEI process. On our old Excel spreadsheets on which we were very, very reliant it was quite easy for an assessor to apply penalties when a student exceeded a word count and they submitted late because they just used a formula in Excel to do that. So on Moodle we had the assessor alter the marks to apply a word count penalty by reducing the original mark they gave the student and then they would insert frequently used comments into the field to indicate that they had done so and then we had the program administrator at the end of the process apply an override in the grade book to indicate a penalty for late submission. We decided to use the marking guide advanced grading method with assignment because that allowed the assessors to write plenty of feedback as Hilary already mentioned they provide very lengthy qualitative feedback per student and the marking guide was perfect for that. We also converted all our old Excel spreadsheets and word feedback forms to marking guides. Providing training was and remains a huge part of the project providing training to 300 assessors located around the country was vital. We gave them step-by-step written instructions, various training webinars we offered one-on-one support sessions as well. We showed them how to use the various features of assignment and the marking guides and we also trained them in how to use the Erkan text matching tool which we also deployed to Moodle during this period. Similarly our program administrators are key people in this process and they needed step-by-step guides, they needed training workshops and ongoing support and troubleshooting in using assignment and grade book. Our academic, administrative, technical and finance staff members all came together in a working group to develop new SOPs and processes to document and agree a streamlined process for using Moodle to handle continuous assessments. Many of our modules contain multiple part assignments and we needed a way to see what a student's projected result would be where more than one Moodle assignment was involved. So we developed bespoke grading reports in Microsoft Power BI using data that was extracted from our nightly Moodle backup. We used the SQL code from the plugin Export Component Grades as a basis for the report because it could show the kind of the low-level individual marks that the assessor had awarded students. It was really, really useful plug and I think Marcus Green is talking about it at some stage as well. Grade moderators required ongoing training and training sessions and how to edit the marks and the feedback. And then lastly we decided to use the manual grade items in the grade book to make non-continuous assessment grades available to students so things like written exam results, school placement results and so on that didn't involve an uploading from the student's point of view. So what did we learn? We learned a lot. We learned a lot about what Moodle can do and what it can't do and we developed a few workarounds and some tips and tricks that I'd like to share with you. Developing clear instructions for our 300 strong team of assessors is incredibly important and in addition to the written instructions they received and the training webinars we also used the description for markers in the criterion not just to explain what that criterion was in the grading form but also just to signpost and signal to the assessors how they should complete the feedback for that particular criterion. As I've already mentioned our assessors complete lengthy descriptive qualitative feedback per student. They take them several hours to write all the required feedback into the marking guide grading form. So understandably they would want to take a break from time to time and return to their feedback at a later point in time but we discovered the marking guide grading form doesn't allow you to save your progress unless there's a number in each of the mark fields so we had to instruct our assessors to kind of enter a zero dummy placeholder in each mark field and then switch the marking workflow state to in marking so that they could return and they could see that they had not yet completed that particular student. The feedback, the lengthy feedback that our assessors provide is structured and they are required to use certain headings to break up their various blocks of text and we decided to use frequently used comments actually as headings not as comments and we required each of the assessors to use each one of the frequently used comments to give their relevant part of the grading form. Our academic policies require us to award a grade of 0% to students who do not submit an assessment so we don't leave their grade blank or empty we have to award them a zero. We discovered that Moodle doesn't accept a zero as a grade for some reason it wouldn't write it to the database so our work around was to give a student a mark of one in one of the mark fields save the change then go back in and change the one to a zero and then save that and then Moodle would accept the zero percent. There may possibly be a better way if anyone can help with that that would be fantastic but that was our work around for the time. You've gotten a sense of the number of different parties that touch the students' marks and feedback during the process an assessor, a moderator and a member of staff can all be involved at various stages but when grades are released the feedback table that the students see shows two fields it shows graded on and graded by and the graded on displays the date and time the feedback was last edited and the graded by displays the name of the last person who edited the feedback which might not necessarily be the assessor it could be a moderator, it could be a staff member etc. So as a temporary work around we had to ask the assessor to write their name in the general feedback comments area at the bottom of the grading form so that the student can actually see in there who their assessor was. And then lastly our academic policies allow for students to be granted submission extensions under certain circumstances so the existing extension feature works perfectly fine in assignment for that but students also have a right to make a late submission on top of an extension and in order to enable that we used a combination of user overrides and extensions to give students a cutoff date on top of an extension. So we'll just hear from Hilary one last time. We replicated the original feedback form online so as to minimize the change initially for the assessors. We were interested to hear that fears we may have had for the adoption of the new format did not materialize. Module leaders can now drop in as the work is in progress to consider how the assessors are doing. This is particularly useful where first time assessors are involved and can be provided with some ongoing feedback as they complete the first few assignments. Where an assessor may be having difficulty with an aspect of a student's work then both the module leader and assessor can look at the same work from their various locations and engage in a professional conversation. The role of the moderator is still the same as it was but emailing attachments to and fro no longer occurs. Moderators can access and amend feedback and marks where appropriate. The role of the program administrator has changed with more focus on the efficient progress of the work and less on the manual inputting of grades. And you can see there just a selection of some of the comments from the assessors which were generally very, very positive. They found it more efficient, user friendly, time saving and less likely to make a mistake because they weren't jumping between all these various different documents. The project obviously resulted in increased use of Moodle which was great to see. So we had 300 more active users among our faculty and it was great to see that Moodle was able to bring these different faculty members closer together. They were dispersed around the country but they were brought together in a shared workspace fulfilling different roles. And then this obviously had a related effect on their own digital capacity. So the digital capacity of our faculty improved. It became much more comfortable with using technology and therefore this also helped them develop their own skills. And then lastly, now that we're using Moodle to handle all of our continuous assessments and grading, it's kind of, it's a later foundation where we can do more with assessment on our modules and this is where we're looking to go now. So already we've planned to kind of divide up our assessments into smaller assignments and award different weightings and have the grade book handle all of that. We want to introduce some formative assessments that a student needs to complete before doing some sort of assessment. We're interested in looking at outcomes and bringing that into the grading as well. I'd be interested to hear from anyone who has experience of using outcomes. So this is kind of, it's really just the beginning. Now we're kind of looking to the future and looking at enhancing our assessments with Moodle. So we're Amila Mahogwilf-Galer and happy to take some questions now. Can I be greedy and ask two questions? Of course. So the first one's quite quick. That thing about not saving a zero grade immediately, that's just a bug. If you don't know how to report a bug to Moodle Core come and find me afterwards and I'll take you through the process because we need to make sure that's reported to get me fixed. That's the easy one. The other one, I may have missed this just because I was making notes. You said you were using additional custom grade items to show students grades earned outside Moodle. I missed how those grades were getting into the grade book. Was that an import or manual rekeying? Yeah, that was just an import. So they were already in an Excel spreadsheet from elsewhere and then they were just obviously formatted in the right way and imported straight into the grade book. Great. Thank you very much. Thank you. Hi Rob, if you have assessors around the country how do you do training for them? Do you just provide documentation or was there online training or did you bring them all into a room? It was online training mostly so they received various different documents and instructions. We held regular training webinars. Webinars are a huge feature in our college anyway being a blended learning program. So we scheduled a lot of webinars and then we had one-on-one sessions with anyone else that needed some additional assistance and that could have been over the phone or over Skype or something like that. Those combination of different things. Hi there. In the grade book are you just putting the final module mark or have you got room to put in component parts to make up a final module mark as well? Sorry, I didn't quite get that. Sorry, in the grade book are you just putting the final module mark like the overall mark or is there opportunities to put in multiple things that might make up a module mark if you see what I mean? I mean like individual ILO marks and things like that. Yes, so say in one of our modules for example four different assignments make up the module total and all different weightings applied so they're seeing all of the component marks in the grade book and then the final module mark as well. All that information lives in the grade book and then gets published to the students through that process as well. Exactly, and that's where students go. They don't go into an SIS to get their grades or something like that. They only do it in Moodle. Brilliant, thank you. Thank you. I think we're going to have to move on. So, Ross, thank you so much. Thank you very much Bob. Thank you.