 Rwy'n credu'r gwybod am y gweithasol – gweithio'r guld-a-dweud ar gyfer a oedd oedd eu gael i'r gweithio'r sydd angen dri o'r hwnnw i ddechrau'n gweithio'n gweithio'n gweithio. The last year was Abeliric, this year it's the James Bond team, so anytime we mentioned a James Bond movie it counts as one scorer. Of course that's why I ended up giving Becky a high five during the panel discussions. We were not very mature, but that's neither here nor there. She did nail it, she's absolutely the legend of doing those sort of things, but I'll give it a good stab. I'll do those sort of things, but I'll give it a good stab anyway. So somebody needs to count James Bond references. Okay, so let's just see what we've got. Rethinking our approach to plagiarism, so I work for at least you, and along with a colleague of mine, Shadi Karazi, we evaluated how we manage plagiarism, not just technically how we manage it, but how pedagogically we manage plagiarism. And the reason why we did it was the unsatisfactory setup that we had technically. So it was the technology that drove us to it. And what I want to do over the next few slides is actually explain to you our approach and where we're modifying it to, or the direction we're modifying it to. It was very appropriate, and I'm sure Gavin did this on purpose, but a lot of the issues that Roger brought up there with Insolent, we had as well. And you may see some similar challenges that we had, but maybe a similar solution as well, I never know. So bit of background. So we have 16,000 students spread over three different campuses, and we are trying to move everybody towards the electronic submission. We have, like every institution, we have the issues where some staff are more competent with technology than others. And some think Moodle is the devil. So we're kind of in that zone where everybody else is as well. But a big thing for us, as is for you, with the internet and with the advances in technology, plagiarism is becoming so much easier for students. So we needed to have some form of text matching service at our disposal, but we didn't want to go at a half-hearted attempt. So we looked at the text matching service that we have, and I'll go through the criteria that we actually evaluated on. I'd be delighted to share the results with you. These slides are actually being presented. They're not just for your eyes only. They are up on LinkedIn. So that's two now, and I won't make a specter of myself anymore. But here we go. What were we trying to achieve? We were trying to get more electronic submissions. We were trying to get more consistency because very similar to some of the work that Jess was doing on with the feedback, we wanted to be able to use the data to our advantage. But of course, if the data is not in there in the first place, if they don't use electronic submissions, if they don't use electronic grading, we can't get that data. So what we were trying to achieve was to make sure we got more people moving towards electronic submissions, but also that at the same time they weren't being driven by the technology. We wanted to make sure we were educating our students on plagiarism. We weren't there to beat them across the head with a stick, although sometimes that was tempting. But what we were trying to do, as I say, is move towards electronic submissions. So I wanted to look at what we were currently getting. So as I was saying, we have 16,000 students in our institution. And if you don't already use it, if you are a customer of Turn It In, I would highly recommend looking at the statistics behind the system. They're very, very comprehensive. We had a small increase, and now I will say for 2017, that was up until Christmas, so it's only half the year. But the increases were going in the right direction in terms of number of submissions. However, they're still very, very low in my opinion, considering we have several, several thousand modules on our system, and you're going to have several assessments in each module. The other bit to bring across to your attention is grade mark. We didn't have that many people using grade mark, and the grade mark studio in, sorry, the feedback studio in Turn It In is actually very good, and as Roger was testifying to there earlier on, there's some really good advances to using it. However, we weren't getting the buy-in from our staff. We were getting our tech savvy staff absolutely no bother, and they loved it, and the audio feedback as well. They absolutely loved that side of things. But the numbers were very low, and we came to a decision saying, right, well, do we go in a hell for leather for Turn It In and just push everything through Turn It In or not? Do we re-tinker approach? And that's when we came across Urkin. So Urkin is another tech matching service, and we decided to compare and contrast the two of those and see where we go, look at the advantages of each one of them and compare them. So what we did was we compared them with cost, with functionality, matching ability, turnaround time, and this is an important feature which I want to spend a little bit of detail on, and support. And that's the support we get from them, but also the support we have to give to our students and to our staff, because they're the things that, Dad, where, well, it was just a fundamental for us to reduce the level of contact time that we have. And we developed, as part of the assignment tool to move towards electronic submissions, we developed a Moodle Tours, and we made the tours for each one of the assignment functionality, each one of the assignment pages, but we still had, I couldn't do that with Turn It In because it's an external tool. I couldn't, when we're pushing people out to an external system, they would complain, oh, well, I need to learn how to use another piece of software. Can Moodle not do it for us? So there, I'm going to go through them, not in any particular order, but I do want to start with support. So both systems produced a similarity score, which is great, and in Turn It In, we had, as I said, 23,000 reports generated from it, but with those 23,000 reports, we had 52 tickets. So that was a ticket ranging from where's my Turn It In score, it hasn't turned up yet, to how do I use it, to the PDF isn't uploaded, to whatever the case may be, a whole load of standard issues, nothing exception about 52 queries we had, and 52 queries we had to address. At the same time, we piloted Erkin and Turn It In at the same time, we ran on the same Moodle instance, and because Erkin is integrated into the Moodle assignment, we had 55,000 submissions. So every single submission that went through Moodle directly, 55,000, and we had four tickets, and three of those tickets were what is Erkin, right? So we realized there was an issue, but what we were able to do is because we were driving a true Moodle, the lecturers didn't even know that they were doing text messaging, they didn't even know that, oh, I need to do a Turn It In thing, and Turn It In in fairness to them, they are the Hoover to vacuums, everybody just calls the plagers themselves Turn It In, so we didn't want to upset lecturers by saying that they have to learn a new tool, we just embedded it in secretly. By default, we said it in every single submission goes through, and as I said, we had four tickets with 55,000 submissions. So what we had to do then is look at, that's a plus on our side, as you can appreciate. The turnaround time was a big issue because in those support tickets that I mentioned, we had, and we used a Turn It In stand-alone plug-in, and we put everything into Moodle, as I say, because we didn't want to drive our lecturers outside to another technology. But we used the plug-in, and we were getting inconsistent turnaround times, because I could submit an assignment on a Friday, and it would spit back the similarity report in an hour. And then, if I submit it on the Monday, or submit it on the Tuesday, it will take three or four days to do. And I understand the reason why that happens, obviously, the size of the essay, and all those sort of great stuff, and I was on to Turn It In on fairness to Jamie, I'm not sure if Jamie's here, assistance I was getting from Jamie was very, very good. But the difficulty was that the plug-in wasn't actually communicating with Moodle properly. So the score was actually turning up on Turn It In, if you went to turnitin.com, but it wasn't being passed back to Moodle. So I then was forced to tell my lecturers, oh, well, you need to log in to this external system. So then their attitude towards Moodle was, oh, well, there's something wrong with Moodle. It wasn't that, and I didn't want to turn that around. So it was definitely plug-in related, and it was, we identified that the plug-in issue needs to be fixed, but we couldn't rely on that. We were losing control, again, very similar to yourself. We wanted to bring all of our issues under our own control. So that was a negative, let's just say. And I'm not here to promote Orkan specifically, but you can talk to James about what his turnaround times are. But because of the turnaround times, what was happening was a lecturer couldn't actually correct the assignment until they could actually see it. When it turned in, whereas because it was built into Moodle, they could see the assignment correctly and then wait for the plagiarism score if there was any delay on it. But the other challenge that we had, and it's well documented, and again I would be complimenting, turn it in on this, is that they have a huge customer base. So they have a huge database record to check against. So we were immediately putting ourselves in our eyes without doing the research, but when our eyes putting ourselves at a disadvantage. Because if we move away from turn it in, we move away from all of those submissions. So what we did was we got into the figures. These are the matching abilities or similarity scores in turn it in. And we actually realized that most of our results, as you will see there, most of our plagiarism reports were less than 25%. So they weren't like major plagiarism cases. And if you were getting something that was maybe a 5% difference between one system and the next, it's not major. It's 15% or it's 10%. Whereas if it was a difference between 67% and 24%, that's when it becomes an issue. And I'm not saying that isn't the case or it doesn't actually happen, but the majority of our percentages, or the majority of our plagiarism in the first case are actually quite low. And that's when we're using the more comprehensive database of turn it in. So, yes it was an issue and actually we'll see there that 90% of the documents that we analyzed and actually just to explain the analysis, we put in 60 documents into both systems. And there was a difference and you can read there from the screen, but it wasn't a major difference. It wasn't a turning point for us. And the difference is down to the way the systems work, what they count as a match, how the algorithms work behind the scenes. And I'm not a techie person, so I can't even pretend to explain that to you. But it manifested itself in it's not an issue for us. So then functionality. I do appreciate this is a bit small for the people at the back of the screen, but what we did was we looked at all of the different features that we were concerned about as an institution for our assessment processes. And we looked at Moodle and the plug-in, and we looked at turn it in. Now we had built two plug-ins, specific reports to us for marking guides and marking guides and reports for advanced grading that allows us to analyse the feedback that we give to our students. Analyse and break it down on a criteria basis. So this was incredibly valuable to us and incredibly valuable to our external examiners, so we needed to have that in. And that was there because we used the loop, or sorry, loop is our local name for Moodle, because we used Moodle for our electronic submissions. We could still keep that criteria. Group work is an issue. Again, as you alluded to, individual extensions are an issue, but we could take advantage of all the excellent work that's done and available through standard Moodle. And now with the advances that's been done in the grade book, notwithstanding the problems that Roger alluded to, but the advantages with the annotated feedback, that really pushes everything more towards the favour of Moodle. The big thing we had was audio feedback, because audio feedback to students from a lecturer's point of view absolutely brilliant. The literature will tell you how good audio feedback is versus written, but because it's so easy, and because feedback studio allows you to do it so easy, this was an issue for us. But if I come back to the number of people that are actually using grade mark, if I come back to the number of people that are actually using grade mark, I was only affecting a small number of lecturers, so I had to come up with some sweet way of getting around those small number, but could I justify not having everything true in electronic submission just to satisfy what was 8% of our staff, 8% of our users, not even just 8% of our staff? And the answer was no. So I looked at the cost, and the cost that we had, the cost was the last thing that we looked at, and we started off on a trial licence for €15,000, and then went up to the full price for Turnitin, and I do accept that Turnitin has the feedback studio, it's not just a plagiarism, and Ercan is just a plagiarism, but Moodle has become so much better, those advantages are dropped down. But there's your difference, 9,000 versus 25,000. So what I did to address that is now in the course of making a plug-in to make audio feedback available for us. And we looked at the other plug-ins, and actually one of which there was a guy here yesterday where it's Justin Hunt, is he here? I can't see if he is, but Poodle I think is how you pronounce it, Poodle, Poodle, and that looks absolutely superb. So we're thinking, well, does plug-in already available so we could get away with that or still modify it? But we went a little step further, and we looked at the Atto editor that we have, and we decided what we're going to do, and we haven't done it yet, but it will be done over the summer, and we're just going to insert an audio button in there, because for those of you that don't know with the Moodle 3.3 that we have much stronger integration with Google Apps and Microsoft Office, or Google Suisse, it's now called. So what happens now is when you record the audio, it goes straight into the Google Drive. So you don't have to worry about sharing permissions, you don't have to worry about privacy, it becomes for your eyes only, and what I will say is that when you roll over your Moodle every time, the audio is kept there because it's a link to the file that's stored in the Google Drive. So it becomes a win-win for us. So instead of it being a disadvantage, and that to develop that plug-in cost me less than the annual charge that I'm paying for Ockon, so I still have a chunk of change left, right? So what we want to do, we terminated it, turned it in in June 2017, our task now is to move more people by making this stuff easy, and for those of you that have the eagle eye, you'll be thinking, well, this is everywhere where we have the editor, so now not only are we re-changing the way we manage plagiarism, we're re-changing the way we manage assessment, because a student could click that to submit an assessment, submit an audio assessment. Not only could the lecturer do audio feedback, students can do audio assessments, and when it comes to feedback, you can actually have that conversation, because in the editor text box, wherever that is, students can put in a comment, and the lecturer puts in a comment. So what I will say to you is I will definitely do a project like this again. In terms of if we took it on, it was a mammoth challenge. We are only at the start. I'll never say never again about taking on such a big challenge. How many am I on? How many am I on? I've lost it, right? Both, what we're doing now, and this is what I need, and where he's here, I'm going to put him on the spot, what we want to be able to do is to make that database better. And what I want to be able to do is for Urkan, no pressure, right? I want Urkan to come up with a system, a plug-in that will actually suck out all of our archived submission documents and put them straight into the database. Because if you develop that for us, it's also then available for everybody else within the system. So when they want to transfer, or if they want to transfer, that it makes it so much easier. Because by having what we have now in the Moodle assignment, it would make sure that we can, and we have the same concerns as Roger, I'm sure as everybody else, it will make sure that the Moodle community would benefit, and you're not driving stuff out to a third party all the time. And, of course, the more of you guys to join, the more I'm going to haggle with him for a cheaper price. Okay, so thank you very much, folks. Well, Roger, go on. Just a real comment, actually. The audio thing, we have Penocto, and Penocto has a plug-in for the HTML editor, so that's the same solution using a different tool, if that makes sense, but that's already available, or Poodle or something else. Excellent work. I'd like your view to kill. Turn it in. Hey, I'm impressed. Thanks very much. And it was the lady from Cranfield who works in the School of Defence, and she was on our Majesty's Secret Service. That's who the competition is with. Okay. That is? Have you guys?