 For at two o'clock, you would have heard Mike Wilson speaking about the My Feedback app. For a moment there, I thought he was going to do my presentation for me, but unfortunately no, it's something different. I'm here to talk about the My Feedback report, which is a plug-in in Moodle. And if you were at Moodle Mood last year, you would have heard me speak about the plug-in as we were developing it. So this is the update to that last presentation I did. So if you've never seen the plug-in before, what it does is it allows students and staff to easily view grades and feedback across Moodle courses in a single view, a single report. And it's available on the Moodle.org as an open source plug-in. We started development of this in late 2014, and the prototype is actually based on some code that was developed by ULCC as part of a GISC IOE Assessment Careers project. In 2015 to 2017, we actually got a developer in and ran this as a project internally at UCL, and we piloted it with, I think it's actually more like 850 engineering students for a year, starting October 2015. And a year later it launched into UCL Moodle, so it's now available across the institution. We evaluated it with our pilot students. So we ran a few focus groups throughout the project, and we've also been running questionnaires during the pilot, and also we've got one going at the moment as well. What the project aimed to do was to raise the visibility of feedback to students and staff so that they can see everything in one page. What we were finding was that students didn't really know how to access their feedback. And a quite depressing point was that we were often seeing students weren't viewing their feedback at all. So staff were perhaps adding a lot of feedback into Turnitin, and they were never going in and viewing that. And we wanted to make it easier for them to access that. We also want to help personal tutors to support students. So a lot of the time when personal tutors are in their meetings with their students, they're expected to meet with them a certain amount of times per year. They don't really know what to talk about, there's no conversation starter. And there's a number of other initiatives going on at UCL to support this as well, but this is one of the other tools that can help. We also want to encourage staff to start providing assessment feedback in Moodle electronically. There are numerous benefits which I won't go into here. But we're starting to see more and more staff adopting Moodle assignments. And some people are actually moving away from Turnitin to Moodle assignments. And I'll talk a bit more about why in a moment. So the feedback report shows Moodle assignments, Turnitin assignments, quizzes, workshops, and any manual grade items that are entered directly into the grade book. Now, with Turnitin assignments, we can only actually link to the assignment, which is still a big improvement on students having to dig around in their courses trying to find the links. But there's no API that we can use to draw in the feedback like we can access the other general feedback from the tools that are natively within Moodle. And with code adjustments, it's not that difficult to actually add further modules in. So some of the benefits of this is that I'll show you a screenshot in a moment where students can actually view their feedback across all of their modules in one place. And tutors can also see their students' feedback, but just for the courses they teach. They can also see whether or not the students have viewed it. This is what it looks like for a student. So I'll point out to you a few of the features. So up here, we've got the ability to export to Excel and to print the page. We can also, we were going to implement a link back to our archive versions of Moodle. We've actually had to roll back from that because there are a number of issues that are available in the Moodle tracker, which meant we weren't able to put that onto our production system. There are some problems with permissions and looping. So we're hoping that in future versions we'll be able to get that working, although we no longer have a developer working actively on this. So it's now out there in the community for the community to help develop, which is one of the reasons I'm here today. We've also got the ability to show and hide different columns. So this views quite well on a mobile device because you can really cut down to the information that you want to see and remove the columns that you don't need to look at. You can also filter. And the most important page actually is this page here where you can view all of the feedback. If you've got a rubric or a marking criteria, all of that comes in as well. So students can see the actual feedback that they've been given. And if they want to see it in context, then they can click through into the assignment. For personal tutors, they can't actually access the assignment itself unless they're already enrolled on the course. So that's something to bear in mind. And we've also enabled students to enter their own self-reflective notes. So there are a few students who have started doing this. Although a lot of students in our focus groups said that they probably wouldn't use this feature. We have been in discussions with some staff who are hoping to promote this and work with the careers service to get students reflecting on how what they're learning at university can feed into their future career prospects. As a staff member, you get my students tab. And on here, all of your personal tutors come in automatically into this report. Any other students that you're teaching, you can search for by email address. Initially we had a check box that you could tick and all of your students loaded. As you can imagine, some staff out there, we have category level enrollments. They've got tens of thousands of students. And this was really slowing down the page. So we've decided to implement a search feature instead of listing all of the students. When you click on a student name, you get taken to their report. And you're told which report you're viewing and you can go back to your own dashboard by clicking this orange button up here. So an important feature here is seeing when or if the student viewed the feedback at all. So if there's a cross there, the personal tutor might want to have a chat to that student and say, hey, how come you haven't looked at your feedback? And one thing to keep in mind is that I think with Turnitin, it doesn't necessarily register it as being viewed unless they're in there for a certain amount of time. As a personal tutor, you also get an overview of all of your personal tutors. So you can actually have a look at each student and whether or not they've submitted. Now keep in mind here that staff may have made assessments optional. They might be quizzes that are formative or a lot of staff don't know how to use the group's function and to hide things from students who aren't supposed to submit. So this may not be an accurate representation of how many non-submissions the students actually made. There's also how many times a student has submitted late, how many have been graded and how many are less than 50% where a numeric scale has been used. You can also break down each student into course by course and see how they're going course by course. As a module tutor, you get a slightly different view and you get a overall grade where you can see how the student is performing or how all of the students overall are performing. The lover's grade, the median grade and the highest grade. And then if you go down into the student breakdown, you'll see that for every student. We decided not to implement this for students themselves because some students said they didn't want to see this information. They felt that if they were not doing very well, it might demotivate them. So we decided that this would only be a view that staff could see. And subsequently, we don't have it on our personal tutor dashboard because it's likely that personal tutors are going to be in the report with the student sitting next to them in the meeting. So we don't want them seeing any of that information. And to get into here, you just have to select your course from the menu on the left hand, on the right hand side here. Okay, and as a departmental administrator, so we have category level enrollments as I mentioned, you can actually set up access across a whole category of courses for an entire department. And then you get a breakdown similar to the modular tutor view, but at a wider level, a higher level. And you can choose the categories from the top here, level one and two, and then specific courses that you want to access. Okay, so we released this in October, 2016. And by February, we had 14% of our users had actually viewed the report. But the biggest problem we've got is that my feedback is only as good as the assessment feedback that's inside Moodle. Without that feedback in there, the students can't use this and personal tutors can't. So we're working with staff now to explain to them how to put feedback into Moodle in a way that makes it visible in the report. And we make this available as a PDF. It's publicly accessible. One of the big things to keep in mind is that with Ternatin assignments, because we don't have that API link, a lot of our staff want to give their students letter grades and Ternatin doesn't support letter grades properly. So they've actually started writing those into the general feedback. So those grades are hidden inside Ternatin. So if they're there, they don't really have a workaround for that, but we can't get that into the report. So students have to click on the link to see their grade. One thing to note as well is that extensions that you grant for individuals or groups will come through into the report as well. So they won't flag as late if you've actually given a student an extension on a Moodle assignment or a quiz. Okay, so some takeaways from today. Communicating this to staff and students is really difficult. I keep meeting students and staff across the university who have no idea this exists. On the other hand, some students are saying, oh yeah, I just viewed my feedback from my feedback. That's where I figured that's how I get into my courses now to see all of my assessment feedback. And this is working especially well in engineering where they're putting a lot of their feedback electronically into Moodle. I've already mentioned Ternatin cannot be displayed directly in the report, but we're hoping that at some point in the future if there's an API, we'll be able to develop that. And just from a technical perspective, Moodle uses, my feedback report uses the Moodle parent role to map the personal tutors to their students. And we use a plugin called user role assignment from external database to automate this. So overnight, as people are set up and attached to their students in our SIS, it comes in and populates Moodle. The departmental admin access, as I mentioned already, requires the legacy category enrollments plugin. So if you're not using that, you can't implement departmental admin access. Another thing to note, a few people tried to attach the permissions to the course level roles like student and teacher. This didn't work because the report actually sits outside of a course. So the whole point of it is to give an overall view above the course level. So you need to make sure that the permissions are granted on a role that exists at site level. And as I mentioned already, if you're going to be implementing this yourself, don't activate the archive feature because it's not recommended production installations yet. And if you wanna see all the details about ideas for future developments and any bugs that are open, that's all on the Moodle tracker. So what are some ideas that we've got for future enhancements? We'd like the personal tutor dashboard to show courses that the student is studying. And on a mobile, we'd like the first row of the report to be expanded by default because we're finding that students and staff don't know to click on the plus to see the entire report on a small interface. We also want, it's possible to get to the report from the grade book. There's a tab. And what I'd like to see is when you click that tab, it automatically takes you to that module in the module tutor dashboard because that's the most likely thing you're wanting to be looking at if you're coming from a course grade book. And we'd also like to link directly to annotated Moodle assignment PDFs. That feature's not there at the moment. It tells you that they're there, but there's no direct link to them. If you'd like to contribute to my feedback, all the links are here. This presentation is available. And if you want to catch me later on at the conference, I'd be happy to talk to you about implementing it. The guys had one minute to spare and Jess had one minute, 10 seconds to spare. So thank you very much to both sets of speakers. Can I open the floor to questions to Jess, please? Hi, Mike Wilson, University of Portsmouth. Have you had any joy putting pressure on Turnitin to expand their API? We haven't really tried, actually. I've asked them a few times at Moodle Loot and it didn't seem like an API was in the pipeline. So we'll just wait and see. It is meaning that some of our staff are moving away from Turnitin to Moodle assignments because of that, but we're hoping to get the plug-in, plagiarism plug-in into Moodle assignments anyway because it does so many more things in terms of group assessments, marking offline, that type of thing. Thank you. Anybody, any other questions, comments? Nope, I just have one, Jess, if you don't mind. Do you have any idea how many institutions are using it? Oh, I should have had a look at how many downloads there were. I think it was 263 were coming up on the downloads, not that I look at it every day, but maybe I do. Excellent, well done. Well done, super. Oh, sorry. Sorry, it was the installation sites, yeah. Oh, OK, yeah. So it might be more. So you're better than that, basically, which thing. OK, if nobody else has any other questions, we will break for coffee, I think, now. Thank you very much, folks.