 Good afternoon, everyone. Welcome to our second N.I. webinar in collaboration with the Center for Research Ethics and Bioethics from Uppsala University in Sweden. And my name is Rita Santos, I'm the N.I. Executive Director, and today we are going to learn on gamification of academic integrity. And for that, we welcome our three panelists from today who have dedicated their time and efforts in developing gamified tools to teach about academic integrity. We welcome Dr. Zina Khan, head of the N.I. Gamification of Academic Integrity Working Group. We welcome Dr. Mads Goddickson from the Rise in 2020 Integrity Project. And last but definitely not the least, we welcome Dr. Sonia Bialobaba, Coordinator of the Bridge Projects. Welcome, everyone. Thank you for your time and for being here with us. I'm sure your tools will be very relevant to our audience. For our session today, each of you will have 10, maximum 15 minutes to present your tools. And then we'll dedicate the last minutes of our session to a discussion and to answer any questions the audience may have. So if you have any questions, feel free to write them down in the chat. This is open now and it will be addressed later in the session. So without further delay, Zina, the floor is yours. I'll stop. Thank you. All right. I will quickly share my screen. Everyone can see my screen? It's coming. Yeah, perfect. Okay. All right. Thank you so much, Rita. And again, thank you, Sonia, for inviting me to this really exciting session. I think the NIA webinars generally are really great. They're covering such amazing topics across the board on academic and research integrity that, you know, it's really great to see one session dedicated to gamification. Gamification is like a really passionate project for me. I'm actually from IT. That's my background, but I'm always trying. I'm also teaching ethics. So, you know, I have to always find ways to kind of get people interested and, you know, engaged in the content that I'm teaching in a manner that talks to them in their language. Right. We don't get to choose the students that walk through the doors in our classroom. So we always have to be on our toes and trying to figure out what else is dragging their attention. We are from our class and kind of competing with those technologies and gamification is definitely one of those areas that is fantastic. If you're, if you know how to bring it into your classroom and use it, even technology aside, just the concept of gamification in a classroom is usually fantastic to have. I'm representing the wonderful gamification working group in NI, which started a few years ago as kind of like a Eureka moment that happened during a workshop that we were doing for students, where we actually did like a impromptu jeopardy competition for students and we saw suddenly students getting up and taking notice and running to the notice boards and writing their answers and trying to get there before the other persons. And the kind of the general enthusiasm and the excitement that we saw in students kind of worked our interest in looking at. Hmm. Could we look at gamifying academic integrity values and you know these modules and learning modules and stuff that's out there to make it more fun and more engaging for students. That's how the gamification, the working group kind of came into being just a disclaimer before I go ahead some of the parts of this presentation have been taken from the proposal that we as a team have put together for the guy project from the NI working group as well as from some of the outside collaborators and the and I conference abstract submission that we have made from the working group and the project, which is currently under review. So as I was saying about the gamification working group. The aim of the group basically is to explore gamification and game based learning and in a minute in a bit I will tell you guys the difference between those two to enhance engagement and commitment of everybody, not just students. Even staffs faculty members management and including even parents, because when we look at academic integrity we are not working in silos we're definitely working across the supply chain downstream and upstream so we're looking at schools students coming from schools we're looking at the employers. So you know we have to be able to always, as I mentioned you know stay on our toes about how best to approach the different stakeholders. The group really aims at looking at how can we bring gamification and game based learning into this fear. And you know, not just us building games but also reviewing games that exist currently exist. And what are the kind of games prepared talking about people are using how are they using it, giving advice on how best they could be using those tools if somebody's interested to take the tool and use it in their classroom or in their training. We have a number of people that make up this wonderful working group and as you can see we are from all over. We've of course got Sonya, we've got Dr Shiva, Sandra, Salim, Rita, Mike, Lorna, Laura, Jared, and of course myself. So it's a very diverse group of members from and I have come together who are really interested. And the group is quite active in terms of the kind of work that we're doing in we've, we've come up with a review tool that you could be using to review existing games. We've come up with basic steps on if you did want to develop a game game game based system for your, you know, any any kind of training or workshop that you're doing. And then of course we've gone ahead and applied for the particular the actual project that we're going to be talking about today. The projects have kind of gotten. We started off with a broad idea of academic integrity and decided we'll start small and then move upwards. And so the first first thing that we started looking at of course is plagiarism. Everybody here probably already knows what plagiarism is but I'll just put that in there just in case for clarity's sake that you know what when we were talking about plagiarism we are talking about any kind of intellectual property that is used by someone without somebody else the actual person's acknowledgement. We also of course have powerful text matching software already out there that helps faculty members in classrooms to monitor text match that happens and to you know investigate if there is a possible case of plagiarism. Just a point again to note here these are text matching softwares they're not plagiarism catching softwares or detection softwares. The maximum they can do is highlight the text that is being matched with some other source that they have within their extensive databases. And then it's of course up to us to decide whether that is that constitutes plagiarism or not. Other efforts of course across the world are things like honor codes that work traditional detection and punishment systems educational approaches such as training modules workshops referencing materials teaching students actually proactively how to do academic writing what are these skill sets what is referencing what is citations and so on and so forth. The bigger issue beyond plagiarism that we have also noticed of course is the access to the multi million dollar industry, which is the answer providing our companies and the academic support. And on purpose we put those on code because that's how they promote themselves we all know these are contract cheating sites. You know of course everybody's been talking about it artificial intelligence tools for content generation. And that's a wide range of tools, plus the predecessor kind of which is still existing to artificial tools which is the paraphrasing and translation. So all of these are constant push and pull that happens in our classrooms, when it comes to, you know, authenticity of students work. So, given this, what we did was we went back and specifically looked at what is it that everybody says that they're using and a lot of majority of the universities came back and said yes learning modules. Right having this proactive measure which is like a feedback led interactive learning module really acts as a deterrent for any of these above behaviors, but they don't always necessarily work either studies have also shown that a lot of the time. There is this perceived notion that they're inaccessible. They're very worthy. They're in, you know, they're encouraging rock learning. It's a lot of the times they've got these fixed five mcqs and you know you can just keep going back and do a trial and error. So it's not necessarily encouraging deep learning but rather just, you know, trial and error and just finishing it off when so there's a pro that definitely learning modules are the way to go because they seem to work, but majority of them do seem to kind of lag behind when it comes to really engaging students in the process of learning about academic integrity values and why they should not be engaging in plagiarism, etc. behaviors. So that's where we started looking at game based learning where basically we wanted to see how we could really use game based learning in this fear. And the things that we observed were of course that research itself has said that you know there's greater engagement with participants. There is definitely no knowledge retention that happens whenever you're using GBL. There's a transfer of knowledge and skills beyond the immediate course or content, which is exactly what we need and want. We don't want people to get stuck at just exactly what's just being taught and that's it and you know they're just able to apply it to that particular course. We want this to go beyond. We want this to become a second nature for people to be able to acknowledge sources, etc. Here I think I'll take a moment to quickly just talk about the difference between GBL and gamification. So GBL is what I just talked about where we are actually using games to achieve a defined set of learning outcomes. The example on this particular image that I've taken from Tech with Chick is about Minecraft. We know a lot of classrooms use Minecraft. Now with metaverse, for instance, I know for sure like with our university, we are looking at how we can incorporate metaverse and teaching ethics courses. So we are creating these little universes of dilemmas that students have to get into and then they have to make choices and then defend those choices and then come back out of it and then have that class discussion. So these are actual games that they are using. That's created and then they're using. Gamification isn't necessarily using a game but game-like elements into your non-gaming context. For instance, the example that I've given right at the beginning, I will give points to the first group that comes on the board and solves a problem absolutely correctly. 100% they have to show all the steps, all the workings and they have to get it right. So if it's something like that, that's where we are actually saying I'm gamifying my class or I'm gamifying the concepts. That I'm teaching because I'm using points or leaderboards. There is some kind of an element of competition in there. There's a reward system. I could be giving points, credits, marks, etc. So that's the basic difference between gamification and game-based learning. What we did want to do was we actually wanted to create a game-based learning system, not just have gamification. And that's where we kind of collaborated across the board. So University of Hulongong, I'm from University of Hulongong in Dubai, which is the Dubai campus of the mother ship, which is the University of Hulongong in Australia. The university has a lot of other campuses, for instance, the one in Malaysia as well. So some of our colleagues got together in partnership with an IS Gamification Working Group. And we decided we were going to look at how we could actually develop a game-based learning system. So this project, like I said, we aimed to look at completely everything to do with academic intensity values, but then we kind of narrowed it down and said we will start by focusing on one, the most prevalent one, that is plagiarism. The objective is to develop a GBS on, you know, the topic of plagiarism, to raise awareness on academic misconduct and integrity, and to demonstrate the effectiveness of GBS in deploying an academic integrity module. Given this, besides us, we also had Leone Michel, Tan Jin Ik, and Anne Rodgerson as part of this team. Of course, the rest of the team was the list of names that I've given you in the beginning from our amazing Gamification Group members. So once we had this grant that we received, we decided, of course, to really put our heads together and decide how best were we going to go ahead and develop this. And we realized we really needed the input from the stakeholders, which is the staff and students who would ultimately be using this module that we are going to be designing. So we actually went and did a round of focus groups with the, you know, with different campuses, staff and students, collected their expectations, put that down under, you know, we studied those. Then we put down all our expectations, the content was developed in the background, and then we submitted to the design team, the development team, who then went ahead and developed and tested the game-based module. Now that they've done that, we are in the process of determining its effectiveness and, of course, releasing the beta version. So that's the part that we are actually at. We put it as a cyclical process because we've, of course, gone back and forth a lot of time and we continue to work on the background information, looking at the target audience again. It wants that now that the game is done, going back and getting their feedback again, so on and so forth. This is just a small demo of what the game currently looks like. This, as I mentioned, we haven't released it yet because we are still in the process of doing the final, the second round of focus groups with our students and staff from the different campuses to find out if this is something that they would be interested to do. So as you can see, it's a little bit different than a normal learning module. It's not just having some kind of animation. It's a completely different system which has questions and scenarios, and then it gives you some background information, and then it gives you a choice, like what should this person do, that person do. It's got the little bit of game board going on there with the number of stars, et cetera, how many things have you got right and wrong. You can't just go back and keep clicking to get it right. You would actually have to move forward with the game. So it's a little bit different than the current existing, like I said, the game, the learning modules that we have seen that exist in different universities. So as I said, we don't really have the final game yet, but we are in that process in the last phase of that project. We are quite excited because from the little bit of focus groups we've already run this year, we've had very positive feedback from students and from staff. They were quite excited to see the little mascot that we have made that you can see Captain Integrity. The name of the game itself is Age of Integrity, so that in itself is calling out to students because it kind of meshes with the current naming culture that is existing. Things that we came across and what's really making us successful is the mutual expectations and how we went about defining the scope of this project. Because as you can imagine, it's a large group of people working on one project. So we really have to address these right at the beginning, creating an identity for the group with like logos and short forms and things like that. Assigning roles such as project manager, communicating regularly, working with technology to aid in communications. This group is fantastic at outputs. We don't meet that often, but we do a lot of our communication through emails. We do meet of course when we have to make crucial decisions, but a lot of the decisions kind of happen very much on the tech. We make recordings if we have to meetings and save data in common cloud folders for everybody to be able to access. Develop artifacts of milestones, achievements to record progress posters, videos, social media posts, just to keep everybody updated. At the same time, our sponsors like the funding group knows what we are up to and the teams know, yes, these are, this is where, how, where the project is. So this is in a nutshell, what is gamification, why we are so interested and what is the kind of work or project that we are looking at from the NI working group. A list of the references, there's a lot that I refer to for the slides, and I'm open to any question answers when we are done. Thank you so much. Thank you very much, Zina. So we'll save the questions later in the session. Matt, you are welcome to share your screen. So I will share my screen. And thank you very much, Rita for, whoops, this was not one I was about to show, this one. Hope you can all see it. So thank you very much for, to the previous speaker and Rita and Sonya for inviting. I will present on a tool called integrity games. It is a little bit further in development than the previous one presented. But a little different in the sense that we didn't, we didn't intentionally design a gamified tool, we just designed a tool and it turned out it was gamified. So, but integrity games is a tool that was an online learning platform that is developed by the integrity project that was funded by the EU. It is addressing mainly sort of gray zone issues that dilemmas that students face during their university education. So it's aimed at undergraduate students. We had Europe as a starting point. So, so that's sort of the core group, but I've talked to people from all around the world. And they've seen the game represented at the World Congress on research integrity back in the summer. And that seems to be a worldwide relevant for the, for the issues discussed in the, in the cases. It's a website. It's available in five different languages at the moment. English, French and Portuguese among others. So we're covering a very large area. And you can find on the website, you can find a lot of what I'm going to say now and a lot of additional tips and tricks for teachers about how to use this tool. The basis for integrity games is the integrity project, which started out by carrying out an extensive survey of what are the dilemmas, what are the issues that undergraduate students actually face during their studies. And we took some of these sort of most common dilemmas that the students face and turned them into a little simulation or sort of a gamified simulation that the students can play through before they reached, they encounter the dilemmas in real life, basically. And so, in that way, be able to reflect and be prepared for them when they meet them in actual practice. So let me show you the tool just quickly. So I'll just switch to my browser here. So this is the main page of integrity games. And as you can see, the core of the tool are these four cases. These are narrative cases where the student is going through three levels of dilemmas and each level, what you see in the next level depends on your choice in the previous level. So the, as you can see, there are four cases, two on data, one on plagiarism and one on collaboration. So the reason why we have two on data is because we wanted this to be relevant to a very broad range of students. So we have one that is on qualitative data for the social scientists and some people from the humanities. And we have one on quantitative data for more for the natural scientists and medicine. They all have the same structure. So I'll just show you one case really quickly. So the idea is that we, I mean, as you can see, it's not like a World of Warcraft game. It is, I think, what was the previous speaker would call a gamified tool. And so we have some elements of gamification, especially we have a level of immersion setting the student in a particular setting. You are doing this. You are facing a certain dilemma. So on the first page here, we see we have the sort of the stage being set. And then we move on to a dilemma. In this case, we're doing a group work, and we have a group member who's not really contributing. And we are asked what will we, how will we respond to this, and we have to make a choice. And when we make a choice, we get to the next gamified element, which is the immediate feedback. We are told not whether you did your choice was right or wrong, because we believe that these are ethical issues. So the point is not that it should be right or wrong. The point is that your choice has consequences. And you're told about these here. In addition, over here, you can see that there is a little text box that you can expand. So this gives you a sort of more abstract account of what you just seen. So you've just seen been immersed in a concrete situation. But actually, this concrete situation is an instance of a more general dilemma. In this case, it's about free writing and group work. And you're told about this in this little text. And you're also challenged to think about how the specific parameters of this context affects your choice. So in this case, the free writer was described as a close friend. But what if the person was not your close friend, but some more distant acquaintance, would that affect your choice? Okay, so now we move on to the next level. This is a new dilemma. So it continues in a narrative way. And this dilemma that we're seeing here is determined by the previous answer that I gave. So if I chose the other option, I would see different dilemma. Again, I make my choice. I am told what happened, immediate feedback. And then I move on to the third level. And again, this dilemma depends on the previous one. I make my choice. And I am given the final feedback, which is sort of the end of the story in this case. You've seen a number of times that when I curse over certain words, there are little boxes popping up. So these are explanations of words. These could be integrity words, like there isn't one on this page, but it could be something like plagiarism or ethics. But it could also just be terms that we all use, but we use them in a slightly different ways. So we need to explain them to make sure that everyone is agreeing about what we're talking about. So now this case is done. I go back to the main page, and now I can play one of the other cases, or I could play the same case again. And why would I want to play the same case again? Well, I could try and give a different answer at one of the levels because then I would see different dilemmas. So one thing I do is that I encourage sometimes my students to play it first the way they think is right. And then the second time I play, they try and get yourself kicked out of university to be as bad as you can, and then they will see different dilemmas. Up here at the top menu, you can see that we have the dictionary that I mentioned. We have a tab here which gives you access to particular specific dilemmas. So let's say I'm in a classroom and I want students to consider just one dilemma, not play a whole case. I can say, okay, go into this page and go to this particular dilemma on self-played journalism, for instance. And you can discuss that in the class. Finally, we have the teacher manual, as I mentioned before, where you find tips and tricks for how to use these games. And there's even a little video of me saying pretty much what I'm saying right now. So this is the tool. As I said, it's freely available to everyone. You can just go and use it. You don't have to sign up or anything. But how should you then use it? Well, the tool is designed to be intentionally flexible. You can use it in many different ways. The only way you can't use it is as a replacement of classroom teaching. So we strongly believe that students need information on sort of specific, the local setup, the local rules, local expectations of their institution. And integrity games does not provide that. So you can't use it as a replacement, but you can use it as a supplement. The way I use it often is as a tool for preparation. So I ask the students to play some of the cases before they come to class. Note down the things that they were in doubt about, things like that. You can also use it as an assignment during a class. So let's say you have two hour session and you want to have a break sometime and you ask them to just tell them to go to this website and play this case and see what you find out. Of course, you can vary this so you can have it as an individual assignment. You can also have it as a group discussion in a session where they play the cases in groups. Or as I said, you can go into the individual dilemmas if you want to do that. So you have to be aware that when you just ask them to play the cases, the students is likely to see different things. So if you can't assume that they've all seen the same, but you can do that through the dilemmas tab. So the tool has been through a number of iterations and testing. One thing we've asked, of course, is whether they like playing it. So we asked students whether they thought it was fun to play, whether they agree with the claim that it's fun to play and they score an average of four out of possible five, which would be the best. We also asked them if they would recommend this game to two teachers and they answer an average 4.1 out of five being the best. So students clearly like it. We also tried to see if they actually learned something from it. And of course, this is something that is difficult to test. What you're particularly interested in, of course, is knowledge retention, whether they change their behavior in the long term. And this was developed last year, so we can't measure that yet. But we can sort of try to measure if they become better at recognizing that there are certain things that are dilemmas, that there exists dilemmas in academic integrity that is not just sort of black and white. And we tried to measure that in a series of experiments that we did with some good colleagues. We had nearly 300 students from many different faculties involved in these tests. And the first part of the test is just a pre-test, post-test. So we do a pre-test, we have them play the games, do a little group work that sort of simulates a classroom, and then we did a post-test. And this part of the test went really well. We see a positive significant improvement for students. They get better at recognizing gray zone dilemmas. So if this was all we did, and this is what people usually do when they test these games, I would have a very easy paper to write because I could just write one similar to everyone else and say that it worked. But we did something stupid. We included a control group, which read a non-gamified teaching note on the same topics. And it turns out that this teaching, this control group learned exactly the same, or they made the same improvement. So our tool works, but it doesn't work better than the teaching note that I wrote, which I know is really boring. So this is a puzzling result that I think is particularly relevant to this kind of seminar where we discuss what is actually the value of having gamification in academic integrity training. So, and I'm happy to elaborate on this in the Q&A, but as time is short, I will end it here and say thank you very much. And looking forward to your questions later on. Thank you so much, Mads. Definitely, this is something that we will address later in the questionable answers. I think it will be a great discussion, especially with these panelists and the presentation that you are making. Sonja, we move to you. Thank you. I'm sorry for my voice. I have a bit of a flu. So, hello and welcome. I am going to talk about the bridge project in games we are developing within that project. The bridge project is Erasmus plus strategic partnership between six institutions. Uppsala University, Southeast European University, Mandel University in Bernal, Lithuania and Centre for Social Sciences, Kelso National Technical University and Office for the Founder's Purse and for Academic Ethics and Procedure in Lithuania. And the motivation for the project was basically that we teach ethics and integrity in several different places in the educational systems. We have perhaps courses in academic integrity, research integrity, people that study business do have courses in business ethics. Now we have also this emerging field of citizen science and in some cases students can get courses in citizen science ethics, but there is no connection between these four different fields. So what we wanted to do is basically to create three bridges from academic integrity towards research integrity, academic integrity and business ethics and academic integrity and citizen science ethics. And the target group for our project are master students, doctoral students and supervisors. So, you can find all of our outputs for free on this web page. You can find checklist guidelines, open educational resources, gamified cases, workshops, webinars, etc. One of the things that was developed during our projects are checklists for the academic and research integrity. The aim of those checklists is to make it easier for master students and PhD students and their supervisors to engage in ethical writing and have a lead checklist with different aspects needed to produce papers and their master thesis. In addition to these checklists, we have developed several different games and several different type of games because the aim of our project was also to see what kind of ethical games we might want to have. So, for instance, we developed small word-based games to learn the terminology in a fun way. Those are just perhaps not even for this level, we just thought it was fun to test it, so they're also on our website. Then we decided to develop a board game for bridging integrity in these four fields. This board game is actually a very good starter for discussions on academic integrity. Everyone likes board games, it's beautifully designed by our colleagues at Mandel University. Here you have a picture from one of our learning, teaching and training events in North Macedonia where people got really engaged in playing that game. The game is downloadable from our website, so you have this board and you have questions and the idea is to describe or to make pictures to explain different concepts, etc. It's a very good starter to see what works, what doesn't work, are we all on the same level? Do we define things in the same way? So, we think that board game might be a good starter just before in the beginning of a course in academic integrity or research ethics, but it also covers business ethics and citizen science ethics, so go check it out. Another type of game that we are developing are multiple choice storytelling games because while the first type of games I've shown didn't just teach you concepts and the second just showing that we are all on the same level. In multiple choice storytelling games you develop ethical thinking in order to see the consequences of the decisions you make. So, these are dilemma games a bit similar to integrity projects. Where you go, you have a role and there is a problem, you get different options, you choose next one and then you see what happens if you've chosen that path, etc. So, the story develops in different ways depending on what choice you make. During our learning, teaching and training events we also encourage our participants, which are master students, PhD students and supervisors to co-create games together with us. We think that this is a way of getting people creative, getting people to articulate the knowledge they already have about ethics. For instance, here you have barometer, integrity barometer that was developed during one of these entities in North Macedonia, where basically you had a scale on the floor and the questions were asked different types of dilemmas and then everyone would move to the point where they agree with something. So, you can see in a group, this is a very good game to see that the whole group has the same ethical standards, for instance. And it's also a good way of a good starting point for a discussion. That was just one of the games that was created during, so we have analogue games as well, not only digital games. And during the project we will have a description of this type of games for people to implement that in the classes. Another thing we are developing between the BRIDGE project are guidelines for research ethics and research integrity in citizen science. Citizen science is a concept that's not really widely known everywhere. So, first time we started to present guidelines. We presented them with a normal presentation and people were not really responding in a nice way. They were not sure what we are talking about. It took them some time, so it wasn't so fun. And then during one of our learning teaching and training events in Vilnius, after such presentation, basically we decided that we are going to play theatre instead. So, what happened? We created a role play in citizen science ethics. One person had a role of being a PhD student. He wanted to do a citizen science project and involve citizen scientists in order to get samples from last year's, which was not possible for him to do all over the place. We had his supervisors. We had a couple of citizen scientists that were from our group. And basically when the audience came, we engaged them as well. So, we had an advisory board for this supervisor. We had a funding agency. We had the institutional oversight agency. A couple of people were also asked to be citizen scientists. And then we described the project. I was leading the project and it was really fun. Everyone got really engaged. And we asked them to try to figure out what ethical challenges might be interesting in that kind of project. What type of ethical challenges you'd like to address in such a project. Basically, what our supervisors, which were in that particular case, they came up with a list of the same topics we covered in our guidelines. And after that, after presenting these themes, they were actually really engaged in reading what we had to say in our guidelines. For instance, going from consent, we had different guidelines and they thought it was suddenly very interesting, very relevant because we just had that type of game. So, not all games need to be something that you do digitally, but it's always a good starting point for a discussion. In addition to that role play when it comes to citizen science ethics, for all of these themes that are covered, we have small vignettes where you have a description of what's happening, then you have some questions. And people that are using these vignettes are asked to decide about something. They get points. They get feedback also in the text and then they can read more about it in the guidelines. It's also a way to connect the guidelines, exemplify them through the games. Now, what's next in the pipeline is developing educational modules and in these modules, we are connecting everything that we have done already. For instance, now we are creating modules for promoting integrity and authorship and publication, academic integrity and artificial intelligence is in the pipeline. We are creating videos. We are using our games that we have developed. We are going to have links, documents, etc. And package this as an educational model that can be used within other courses. I would also like to invite everyone who's interested in the BIDGE project on a second multiply BIDGE event. It's International Conference on Integrity and Higher Education, Business and Society on May 18th. It's going to be an online conference and call for abstracts. It's out. The deadline is now March 20th, so you still have some more days to create it. You can find projects outputs here and thank you. Thank you very much, Sonia, and thank you for the brilliant presentations from Zinat and Mads. As I said in the chat, the question and answers is open, so if you have any questions, please write them down. There was a question, I think it is for Mads, about a term that was written and if it was a translation from Danish, it's Sintz. So, Mads, is this a translation from Danish? Where was it written? It was in your presentation, Sintz. Was it in my presentation? Yeah, perhaps. I'm not the best speller in the world, so it might just be... Let me check. A list of scenarios in the game. I am not sure. It's probably a mistake. It's not a word in any of the languages that I know. It's a spelling mistake. My apologies. I hope it's not very confusing. I can share and show you where it is. Here, this one. Ah, okay. That's a good question. That's quite important. This is the level, so I don't know why it's at Sintz. So, number one is the first dilemma that you see. The one that is two is any of the two dilemmas that you will see, depending on what you choose in level one. Number three are the ones that you might see after your choice is two. So, it's basically the branching structure that is in the game. Yeah. And one thing that, given that question, one thing that I didn't talk about is that we have kept... When you get to the last page, we've kept the sort of little number that you can see. This is also in the teacher manual. But one thing I do when I use this tool is that I ask the students to note down which page they ended up on. So, this is the number of... There's a little code which is the page. Because that tells me what choices they made through the game. And I can... When you know the structure of the cases, you know... So, if they all end up in number 3-1 and 3-4, that means that they disagreed on level one. So, I take that dilemma in class and discuss that with them because I can see... If I make a little survey prior to the class, I can see, well, 60% of you answered this and 40% of you answered this. So, it's not just that people out there agree. You disagree on this. So, let's have a discussion about... So, it's a way to... You can also use this game to find out what the students in this particular class are disagreeing about and also what they are agreeing about. We often find that they... I also asked them often before a class whether they've been in doubt about good practice within these three areas. Collaboration, plagiarism and data. And they are very much in doubt about plagiarism because they've heard a lot about it and they've been told a lot about it. But they actually agree very much on what the right thing to do is. They always end up on the same page and they end up on the one that we want them to end up on. So, they're actually... So, there's some fear in the students that we don't... There's something about plagiarism that we don't understand and we're doing it wrong. But actually, they're not. They're actually quite good in thinking. And I think that's a challenge for this type of... I'm just... Please stop me if I'm going too far. But it's an interest. For me, it's... What working with this tool has led me to is really thinking about if we're going to gamify the teaching tools on academic integrity, we should gamify them in the right way. And the way it seems that what I can see in the literature is that the reason why gamification works is that it keeps students to the task, keeps them interested in what they're doing. And this works well if there's some sense of competition and there's some sense of reward. So, but the trouble for us in this kind of game that we're making is that you can't reward them for making the right answer because there is no right answer. These are dilemmas that you can't say they did it right or wrong. So, you have to reward their thinking. So, whatever you answered, you thought right. And this is, for me, it's really a challenge to developing games in academic integrity because if you want to have a game on interesting things, you can't have a right or wrong answer, but you can have right or wrong thinking. But how do you gamify that? That's a really... That's a thing that I would really like to learn how to do because I think that's the... If we're going to make good games in academic integrity, that's what we need to do, basically. So, that was a long blabber on one word, sorry. Yeah, but I agree. I think it's very hard to give credit for what the right thing is to do because sometimes people in real life get... They have some advantages if they behave unethically. So, if you have a game, should we show that or not? Because otherwise, you're creating an ecosystem within a game that's not really... That differs quite a lot from the real life. And, of course, you don't want to put the negative, don't want to give them awards for something negative, but you know, in real life, these type of things happen. Yeah. And we very intentionally did that in integrity games, so you can see that when... Sometimes, for instance, in the collaboration case, when you do what many students perceive to be the right thing, for instance, when you see someone cheating and you tell the teacher, then some of the feedback that you get is that, I mean, this person gets really mad at you and will not talk to you. And you get a little bit excluded from the social environment because that's what happens. So, we try to show them that there are consequences, even of doing what is often perceived to be the ethically right thing to do. And also the other way that sometimes... I think there is one of the cases where you actually get away with cheating. And that's because, I mean, in real life, people do. So... Yeah, that's why I think it's... Dalaimans are very important because we want to shape them to make ethical choices, even if it's going to be hard, basically, and it would be much easier to be able to be an ethically, not to be a whistleblower or... Yeah. Yeah. And how... Sorry, I was just thinking, and I would like to raise this discussion here with this panel, this is how often you mentioned about the dilemmas, how often the morality of with the friends, when this involves being a good student or being a good friend, when they see, when students see their fellow student doing something that is not ethical, they are conflicts of loyalty. How they play in terms of... And how you saw this when you were testing your tools. Was this something that was prevalent? So it's some... Well, we did, as I mentioned in the presentation, we did a lot of empirical work before we developed this tool. So we had interviews with 72 students from different countries about what experiences they have. We did a major survey, as you know, Rita, with 6,000 students participating about what they sort of feel as common. And one of the major findings in the qualitative stuff is exactly this tension between... We wrote a paper called Good Friend or Good Student. That's really the tension that they feel sometimes, that they perceive certain expectations from their institution to do certain things, and then they have a codex for being a good friend that they also want to try and adhere to. And they see a tension between these two. And we very much tried to build that into integrity games and we thought that, well, this is... Given that this is something your students sell us, this is something we should prepare them for. So that's why, as I mentioned, the situation with Kim, for instance, that this person is your friend and this does something to the social part. And then we asked them, for instance, in the About Your Choice text, what would happen if this person was not your friend, because then you take out the friend part of the equation and then they might actually act differently. So that's an interesting discussion to have with students in the class as well. And I tried to do that when I... So, as I said, I used it as a preparation and they just play the game and then they come to class and I'd say, well, how would you react differently if it was not your friend? And then there seems to be some sort of... I mean, I think it's... To some extent, it's also students' perception about what they think the institution wants, because in many cases, we actually want students to be good friends, because that's the way they learn best. So it's also about addressing that... If I could re-jump in here a bit. Please, the devil's advocate here. I'm looking at the free rider, the two choices that you have given the students. And I understand how you said that there is no right or wrong in this case, because given the two choices that you have given. But you know, there is a third choice that the student's just going to do the work for his friend. And I think that's where then the right or wrong comes in. So that's definitely not a gray area. That's a black and white choice. And these are the kind of things that you'll have to think about if you want to do any kind of gamification in terms of points or rewards. So where you're giving them two right choices, which is the more right or more wrong, then you don't have an answer. But there is an obvious wrong choice that students do take. Most of the time, which is that they'll just end up doing the work for their friends. Or if they're in a group, there are five people, two people are only working three or not. Those two people very rarely will come up and tell anybody. Sometimes they will try to talk to the friends in the group and they will not do the work. And what happens? The two people end up doing all the work, right? So that needs to be then reflected in the choices, because if you actually put it as a choice, that's the one that everybody's going to take, that I will end up doing the work for my friend, because the way you put the scenario, that he's having trouble, he's got problems, he doesn't understand, he's also got several subjects. So as a friend, I would understand what my friend's situation is. And a majority of students will come and say, of course, I'm going to help my friend and do the project for him. I don't know if they will do that. I mean, in my classes, they are divided on that one, roughly 60, 40, whether they will, what they answer. Another interesting twist on, I don't remember if we built that in, but that was in the interviews it turned out. So one of the things they perceive very differently is whether they find it difficult or whether there is some sort of external thing. For instance, if the person is very ill, if the family member is very ill, then the students will definitely do the work for the person because they want to help, but if they perceive the person is lazy, they don't. Because it's difficult, it's something in between. And it's, I don't know, I'm not going to say that there's a definite right or wrong to those answers, to those questions, but I think it's important to discuss with students what is it that the institution expects, what is it that they expect, and is there actually a tension between there? Because it's about integrity, right? I mean, in some way, academic integrity is a contradiction in terms because you can't have integrity in one area without having it in all areas. And that's what the students feel. I mean, I want personal integrity and personal integrity means being a good friend and a good student. And if I can't have both, then I'm in trouble. That's where the dilemma comes. But maybe it's because they are misunderstanding what the student, what the institution expects of them. And then that's my role as a teacher and well, this is what the institution expects. And I can't put that in the game because that will differ from institution to institution. But I can tell them what we'd be expecting Copenhagen and then they can see if that helps them develop integrity in a more general sense. But do we generally, for instance, want, in real life, outside of academics, do we want somebody to help somebody embezzle money? No, right? No. It doesn't matter how bad a situation that a person might be and that they desperately need that money. And I don't have the money to help my friend. But they need to embezzle this money. Would I help that friend? Of course not, right? It's integrity. It's just pure and simple integrity. I wouldn't help that person embezzle and do something wrong, right? So it's that understanding that we need to bring into students. It's not about I want to help a friend and be a good friend. What does that good friend mean? What does that look like? So maybe I disagree here a bit. I don't think there is a gray in this case. We need to also have that very united message to students saying, this is what it really means. What do I mean when I say this person has integrity? It's integrity. Like you said, it has to be in all aspects of life, not just academics. And I don't think academic and integrity clash, they don't. This is what's preparing them to be the better people once they've come out of our classrooms. So when they're outside and they've got that friend who needs that help, they need to know what is the right help to give and not the wrong type of help to give, just because it's a friend. But I think there is a difference between embezzlement and helping on an assignment. No, of course I've taken an extreme. But if you're looking at a student doing work and you're trying to assess that person's work and there's somebody else doing his or her work, I don't know what this person's work really is and they're still getting the mark, which is unethical. So that's why I was comparing it to something else outside of a classroom. Yeah. So I think there is what this tool does is that it focuses very much on the gray zones. So there are cases that are outright. I'm putting that because in my head I see these branching trees, right? So if you go to level three and go to the very bottom of that tree, there is something where you ask, would you do this, which would be outright cheating? And there you can say, if you end up there, didn't you do something wrong? You're just wrong. And so we take that because we know that in class we talk about, well, there are things that are just not allowed. There are things that are illegal. But to me, at least, they are boring to talk about because most students know. I mean, they know that they're not allowed to copy three pages from the internet into their assignment. They know that they're not allowed to pay someone to write their assignments for them. That's not interesting. What's interesting to talk about is all the gray stuff in between because those who do the very severe cheating, they will not stop doing that just because we tell them not to. The teaching doesn't work on them because it's the whole incentive, the race to get good grades, the pressure from family, whatever it is, that gets them there. So teaching won't work on them. But teaching will work on the gray zones because there we can help them navigate these. And to help them navigate gray zones, we need to give them gray zone examples where it's not clear because that's what they face in real life. And there I think we have this gamification challenge that we can't reward them or reward them for giving the right answer, but we can reward them for thinking in the right way. But that's a tricky one. The discussion is very interesting, but we need to be cautious on time. We are ready for the end. There's a great question from Emma about do any of the dilemmas include artificial intelligence? As I think the dilemmas around that are very interesting. Sonya already addressed it. What about you, Mads and Zenath? We are going to develop it in the bridge project. We don't have it in the... Directly at least in the integrity games. We do have it in the sense that we have a line of... There's a sort of a threat in the collaboration case where you're getting an increasing amount of help from someone on your assignment. So first the person corrects commas and then they start correcting the contents. And in a sense you could say... So you could have a discussion with the student. What is the difference between having your mom setting the commas and having chat GBTs? What is the difference between having chat GBT writing your assignment and your mother writing the assignment? So it can lead into that kind of discussion but it doesn't sort of have chat GBT in the text. Yes, we are developing the module so that's going to be addressed but we haven't done it yet. It's so pretty new. It wasn't actually included in the bridge project. We just wanted to develop it as a discussion. Our one was very narrow. It was just looking at plagiarism at this stage but we are hoping to be looking at next round of grants so we can continue that work. But very well. So just a final question and it will be for me, for all of you. You mentioned about your co-creation process with the teachers involving the students as well. And we've been discussing from the perspective of the student the importance of the gamification tools that you developed. I'm thinking from the perspective of the teacher how would you support the teacher if a teacher wants to implement your tools? You developed workshops as you mentioned about the manual for the teacher but often the curriculum for the teacher is very, very straight. It doesn't leave space to include those tools. So how would you recommend a teacher that wants to implement your tools? I don't know, wants to start first? Okay, I can go. So our module isn't necessarily for a classroom because we do also recognize exactly what you've said that teachers themselves have fixed curriculums that they're trying to get through a very tight schedule in any given setting, whether it's schools or universities. If you're targeting it as something either a student can try out themselves, a teacher can direct students to or a university can use as a learning module for their students. For instance, during orientations or any kind of restorative program that they may have for students who have maybe had an allegation against them on plagiarism, etc. So that's where we are looking at positioning our tool. Yeah, and like I said, we have developed a tool to be very flexible also in recognizing that if we just made our module that this is what you have to go through this module then no one would use it. What we made one that can be used outside of the class in preparation and we can use it inside the class in whatever way the teacher wants. There is a teacher manual with some suggestions about how to use it. And of course we are happy to, I mean people contact me from time to time and we talk about how they might use it in their class. So yeah, the teacher manual is basically the answer to your question, I think. Yeah. And the question, in terms of language, Mads, I know your tools are available in different language. Zina, Tensonia, I believe your tools will also be at least for the bridge part available in different language. No, we are mainly doing in English. But some of the things we are producing are going to be in other languages. And we encourage people, it's all open. And as I haven't answered for your last previous question you haven't asked, you jumped over me. I just wanted to say that whatever we are producing you can have games as separate entities. We have manuals for teachers, but also we have modules. So if teacher would like to incorporate the whole module it's okay. We are going to be in English mainly as well. Okay, so we are closing the session. Thank you very much. Thank you for your wonderful presentation and for the tools you've developed, they're all available. Thank you for the audience, for your questions. And I hope I see you for the next webinar on the 14th of April where we will be presenting about the NIH recommendations on the ethical use of artificial intelligence. A topic that we've been discussing now. And talk about the European conference on ethics and integrity that this year will be held in Derby, UK from the 12th until the 14th of July. Thank you very much for everyone for being here.