 So I'm going to sessions being recorded, so it's over to you, Helen. Lovely, thank you and hello everybody. So today it's on assessments. What to avoid with cheating and have to create new types of assessment using Generative AI. And as Ian was saying there, I mean, this is a huge topic that I get asked about constantly. You know, worried about cheating and things like that. So let's have a look. So my name is Helen Crompton. I'm the Executive Director of the Research Institute for Digital Innovation in Learning at ODU Global ODU. ODU is Old Dominion University and I'm also a full professor here on instructional technology. So I live my life on technology. But how to use it correctly and what we can do with it, as well as what we need to be aware of. So here is the outline of what we're going to be covering. So the overview of what we need to know, like setting the scene, chat GPT and other Generative AI tools, context why we need to change, strategies, how to spot cheating and how to avoid it. And then new assessments using these tools. So the first one is I have to show a picture of chat GPT. And everything I talk about and show today is just using the basic standard, you're not paying for the premium, nothing else. So this was chat GPT as it came out a little while ago. It has caused a stir globally. I present all over the world unfortunate to visit many places and pretty much to having the same issue across the globe. Now everybody has to sit up and pay attention because this is going to change our lives. And it already has so far, but we're literally right at the start. So there's been three major changes in history to information and that's changed education. The first one, the Gutenberg printing press, getting knowledge out to everybody. You might be able to guess the second one, the Internet. That caused a great stir. We always thought, oh, how do we continue education? You know, students can just go out to the Internet. They can check what we're saying. Maybe it was the first part of really critical thinking. Is that instructor telling us correctly? Let's go check. That scared many as well as the plagiarism. Oh, I can just get all that content and put it in as my assignment. So the Internet and the third major change here is called a chat GPT, Generative AI. So as I said a minute ago, we are going to go through great changes. This actually is a slide I showed a while ago. And this isn't, this isn't, this is actually just the start of it. So what happened was Google actually, that small little text on the top left, Google had this AI technology and open AI went to Google and said, oh, you're not really using this. Can we have it? And Google, big mistake, said, yes, yeah, go ahead. So open AI then had this technology, this AI and went to Microsoft and said, we've got an idea. And they agreed to give 1 billion. With 1 billion, they have chat GPT. So that's what it took. I'm going to 1 billion, but that created chat GPT, the chat bot. Then it went to 10 to 13 billion. So we're going to see great changes. So this was my slide a while ago. I've now had to update this slide. Are you ready for this? So remember 1 billion created chat GPT. We have now gone to 100 billion being invested by open AI who now can afford a lot more and Microsoft. So a little while ago, 10 to 13 billion, they were saying, let's invest more. And then they went, oh, no, we're going to invest so much more. We're going to invest 100 billion to make supercomputers. One of the major supercomputers that they're making is called Stargate. And these supercomputers, they're going to help process things a lot, lot quicker and just have much more powerful processing power to create a lot more things, chat GPT and so much more beyond. So like I said, we are right at the beginning of all of this, just everything that we have at the moment is just going to get better super quickly. In fact, 100 billion Stargate, they project as being finished in 2028. So this is going to be quick. So just reminding people what chat GPT is. So chat GPT is a chat bot. So if you remember the yellow there that you go into websites and it'll say, are you looking for a watch? What type of watch are you looking for? You're looking for a specific brand, color, whatever. And so you put that information in. And it has like just so much of information about products, about whatever they sell, about a banking system, whatever it is. Then we go to here on the right, chat GPT. Chat GPT blew that out of the water. It is so much more advanced. It's like talking to an expert person. And you can ask all sorts of questions. So you might think, well, Google, the home of ubiquitous answers can give those questions. For those again that have not used chat GPT. If you can use Google, you can use chat GPT. It's easier. Excuse me. What happens is you go into Google, you search and you get a list of websites where it's probably got your answer for what you're looking for. Chat GPT tells you directly. So you could ask a question or you could ask for it to provide something like here at the bottom. It's got just like Google there with the box in the middle that you type in. Chat GPT has got the box at the bottom. Now we've got various other programs like this. This isn't just chat GPT. We've got many others from Gemini to Perplexity to Claude, the same thing. And they all have just nice simple box. So I've asked it here, right, a thousand word essay on what regulates gene expression. And so this is what it produces. So it even gives you a title and the text here and it carries on. I show you this in real time, but I want to get through so much stuff. I want it to move a little bit quicker. So you can see here it carries on into the rest of the discussion about what it is. What's nice is I can carry on in that box at the bottom and say, oh, tell me more about point A that you have here or something like that. I could say transcription factors, we might be able to guess, but I can literally just say, oh, tell me more about point B. It's like having a conversation with somebody that it can literally carry on. And it knows what it's just said and answer the following as many times as you want back and forth. Now, like I said, it's various programs that do this. You've got chat GPT there. Bad, which went to Gemini, which we have various ones from them and you've got co-pilot. You've got Claude below. Claude is interesting. It's made by Anthrophic. So what happened was chat GPT. All the points I'm telling you now are really important for you in education. And what you need to know with the students using it. So chat GPT had some workers there that said, I don't think you've been ethical enough. We need to provide a tool that's more clear transparency and what it's doing regarding ethics. So they splintered off and they created Anthrophic, which is Claude on the bottom right. And that does give you a lot more information. It's a lot more ethically sound, a lot of younger schools with K-12, middle, high school, they're using it. Again, what's important to say about all of these is it's for students 13 and over with parental guidance. So technically a lot of these should be actually 21, but they brought it down to 13. So we're not talking young students using it directly. The teachers can, but not the young students. So Claude is often the choice of schools. Perplexity, very interesting one. I use this. It's kind of like between a Google and the generative AI, the chat GPT. It's a knowledge engine or answer engine as it calls itself. And it's interesting because you can probably see too close, but can you see in the box what it says ask anything? Just below it says focus. You can actually put an academic focus on where it just searches academic articles. And on the right there, it's attached. So you can actually put a document in with others you could copy and paste. With this, you can attach a document and say, okay, base your answer off this document. Like I could put my CV in resume and then say, make a bio. I've actually tried it. It was very interesting. It was fairly good, fairly good. But so you've got all those. And again, you hear all those names. If you've not tried all those, they are super easy to use. You just literally type in what you want. So here is the big question. Can chat GPT provide answers, students with answers to your assignment? So just think, stop and think, do you have a paragraph description of something asking the students they can literally get that paragraph and copy and paste it into one of those? So if you are, you need to pay extra attention here, which probably most of you are my assignments. And I work through generative AI and AI, but when chat GPT came out, I had to change my assignment because it could literally be pasted straight in and the answer be given to the students. And again, if you've not used it before, I could have a class of 500. Every time they've pasted exactly the same prompt, exactly the same question that I told them to do, put it into chat GPT, one of them. It will provide a different answer to all of those. It just kind of, some things are the same, but it provides text in a different way. It's like asking a student a question and then asking them the identical question after or after. They probably wouldn't say it with exactly the same words, especially if it was complex. So it's kind of that same thing is grabbing that information and pulling it in. So think about the purpose of the assessment. Do students know why they're doing the assessment? If they know why they're doing it, they're probably less likely to cheat. It's something like, okay, you're going to need to know this because when you go into your job, you're going to need to know it. You're going to need to have this information in your head, not just go and find it somewhere. So I always say that first thing, think about letting your students know why it's important. Also yourself, if you've not got very, you know, if it doesn't have any importance, maybe rethink about whether you need an assessment. Cheating will always happen. You can see here, there's a nice, easy way to do it. I've seen people do this, put it on the hand, all the important bits, you know, formulas or whatever else. It's never going to stop. As all these tools come out, as new things come out, people always find new ways of cheating. We just need to make ways to make it as hard as possible. We do have to be aware as well of false positives. There was a lot of those. What happened was students were put in assignments as normal and then the instructor was looking going, hmm, is that chat GPT? And they were getting trouble. And that can have a devastating long-term effect on that student. So we have to be very careful. There's also the idea that AI human detection tools were going to save us, but we don't have to change anything. We'll just get programs like Turnitin who actually declared that they had a 98% accuracy rate and brought the detection tool out. Well, it doesn't work. Students can super easy get around these tools. All they need to do is ask chat GPT to do it. And I'm going to say chat GPT every time, but you know that we've got all those tools. So they could ask chat GPT to write a paper. And then all they need to do is things like put some spelling mistakes in. And then the AI detection tool would say, oh, spelling mistakes clearly must be human. So there's all sorts of ways of getting around. They could even ask chat GPT write paper, but write it as if it was an undergraduate student or write it as if it was, you know, a 12 year old. So they can write, it can write in certain styles that can get around various things and look more, you know, easier. So here's the number one strategy. Educate students. Do they know what academic integrity is? Do they know what they're allowed to do with these tools? Because what we've found is the really good students are saying, oh, we don't use chat GPT. We just, we won't use it whatsoever. And I'm always saying, no, use it. That's super important because it can really help students. It can help students do many, many things. It can help them figure out study plans, study habits. It can help generate tests for them, not practice tests. And then they could put information and say, no, check that. We just need to remind them what they can do with it and what they can't. But I highly recommend the moment my students come to the class. I say, get using it, you know, use it for these things. It can really enhance what you can do. It's like the zone of proximal development that you can do so far on your own. And with a little bit of prompting, as if you had a lecturer, a teacher behind you saying, well, think about that. It can do many of those things to help our students learn much more. So it's important to use them. And just as a side note, you can see the image here. This was using Generative AI. Where I go to programs, this is Dalai 2, although my favorite at the moment is Firefly. So if you're going to Firefly, it has the same little prompt box. Now I just ask, can I have a picture of a student? And I just tell it exactly, sitting at a computer working and it generated this image. It wasn't from the internet. It created this image for me. So you'll see any images are using Generative AI. So same type of thing. So strategy two, explore what the programs can do. This is really important. I suggest you go through kind of them all, but mainly chat GPT, just to understand, like I said, what it really can do. And then you'll see the type of writing it puts out. You'll see patterns. And that's very important. We're going to go into those in more in a minute. So know what it can do. Understand also the limitations of the program. So chat GPT and many of these programs are drawing from the internet. So this one is focused actually on chat GPT got its information from the internet from Bing search. It was actually up to 2021, but now it goes to 2022 January. They put Wikipedia and things in human label and then outproduced chat GPT. And you can see the arrow carrying on there because it went much further. So all that bias, all that human information that some of it's accurate, some of it's not whatever all went in. So it's important to remember what information it's using. So here are the misuses and limitations of the tool. So as we know their first one cheating, it can do that. But again, in knowing the program, what's important out of all of these. And we'll leave you for a second because I'm going to shorten this. We've got harmful use bias data privacy. We've got things that it can do or not do very well. But what's really important for here for what we need to know about cheating is, okay, we've got cheating. But it does get things wrong. It has inaccuracies and plagiarism. So plagiarism, there's two ways there. But the way we're focusing on this one is it often will not tell you where it's got information from some will. Now, but if you ask it for citations are often wrong. Again, perplexity and a few others are getting better. But those are using chat GPT, they're still mostly wrong. So I'll show you what they look like. So knowing that information that I've just shared now, I'm going to pull those in now to specific ways of how to spot cheating. So the first one is the citations. Check to see if it's actually hard to spot until you know you're looking for it. So this actually, I just pulled out of one of my papers on AI and I've got various citations there. And if you're looking through as an expert, you look and go, oh, yeah, Wang to wing plus what all these. Oh, yeah, they're important in the field. That's a good date. So they must be accurate. But no, what's actually happening is the AI is going out and it's it's loosening and it's making these up. So it doesn't fully have the answer. Don't fully know how to do this. It just thinks, okay, get an expert from the field, get a date that match, you know, recently, then get a nice title of a paper that they might write. And the journal they might publish to and it makes them up completely. All you need to do is just run those citations out of the reference section at the bottom. Just grab a few citations and check the our actual papers. So that's a surefire way to spot immediately. I've had, I'm actually currently editing three books. And we unfortunately found a lot of professors, lecturers and things doing the same thing, grabbing chat GPT and asking it to write a paragraph with citations and we'd find a lot of the citations wrong. A lot of journals are actually now immediately first checking references to see whether their actual papers are not. And if they find they're not, they're just rejecting them and saying, look, you know, you've used Generative AI to write this. So we also need to check students work across assessments. If a one-pointing class say they wrote text and now they're producing something completely different. Sounds like somebody else is writing it possibly is. And the other way as well is I actually take a synchronously so I never have the students in class all the time, but still for those smaller pieces of text I try and look for, you know, see whether the kind of similar. Now here's a few. So indications in writing and I want to put underneath that remember these are possible indicators and not direct links to do so. If you see one of these happening, it doesn't automatically mean the cheating and, you know, to take them to on a court or whatever else. And so do be aware that it's kind of a, it'll help you have the conversations. So the first one on the left is analyzing those writing patterns. So look for come narratives across. But also, if once you've done your homework earlier on of investigating the programs, you'll see again like that have styles like chat GPT. The 3.5 will write three sentence paragraphs, always very organized. And in the last paragraph, it'll start. And finally, in conclusion, and it'll always submit up. It's always kind of that kind of very nice tidy writing, but you can see it to my love once you've been using the program a while. It's obvious, you know, someone's been using it. Watch for that uncommon vocabulary or phrases. Again, they use actually often same phrases or some things that you never expect your students to use. So if you find one of those, we'll get to the end the oral checks, but that could be a sign. Unusual complexity of sophistication that could be in the form of the text or the language. Then like a personal voice or reflection and the final one there conduct oral checks. So if you think cheating is happening. I actually, I do not send an email to a student. Actually, and I, like I said, I teach asynchronously. I asked to have a phone call with them or a Zoom call and just say, I'd like to just check in. And you don't have to tell them exactly what it's for you to sell. Just do a check in. Now notice you use this word in your text. Tell me about it. Or I noticed you did this. You might find as well that they suddenly put a section in of something that's not been covered in class. So you might say to the student, oh, tell me more about that. And when did you get that from? Because usually in the moment, it's harder to lie. They don't have time to kind of think about it. So there's ways to kind of catch them out and then figure out what plan of how to fix it. Last one there. Quick turnaround with complexity. Oh, I didn't get the assignment done. Don't worry. Just give me two minutes. Here it is. Yes, that's probably kind of away. So how to now avoid cheating. So we'd rather not do the other stuff where we're having to look for cheating. It's nicer instead to actually just try and avoid it to start off with. Now we've got the short fix and then the longer term solutions. So the short fix, shorter fix. Here are the things that work kind of nicely and kind of said like I said, nothing's foolproof and all of these is whatever we say cannot avoid all cheating because it's impossible. But these really help. So the first one. Ask for citations to back up the work. And you can say to the students beforehand, look, watch out for these programs because they don't do a good job at that. So now we'll find out immediately. So they actually have to look to back up the work on the right there. Ask students to connect directly with discussions in class or online. So what I do is I'll say, oh, you know, make to what's been put in flip. I use flip a lot to have discussions with students and across students. And I asked them to connect to that because it would be very hard for them to kind of download all those conversations and put them into chat upt as well as the question just makes it a lot more difficult. On the right there, ask students to connect directly with readings from the class. Yes, they can put one text in but let's make it harder by kind of connecting with various readings, not just one. Bottom there, ask students to connect directly with field experiences. So again, I teach asynchronous, but that doesn't mean we have to always be learning online. They could be doing many things where they live. I have a lot of people coming at distance and wherever they are, they can be having experiences with real person real time and not using technology. That connects with what they're doing. And so ask them to bring those in. Maybe I teach pre-service teachers and in-service teachers and I'll ask them telling me about an experience you had and what happened again. Chachi Petey and these programs can fake this a little bit but you can ask them very specifically to make that harder. And then encourage collaboration and peer feedback. So kind of have them working together to do some thinking and then be able to put that information out. So even if it's online, they can have the class discussions as part of the assignment. Now before I go on to the next part here, different types of assessment, I'm going to pause a second. I was going to have questions at the end but I've seen a lot of things popping up. Ian, are there any questions, burning questions before I move on to the different types of assessments we generate at AI? There has been a few questions about an AI citation checker to help people identify. Oh, so is there a citation checker? Funny enough, no. And if you ask these programs, you say, check these citations for me. It could make it up. It could go, yeah, that sounds right. So I don't recommend that. I believe there is something because the journals actually, there was one journal. I can't think whether it's British Journal of Education Technology, BJET or Computers Network. There's one that can actually check them through. So I'm guessing it's not human, but it could be. But I wouldn't rely on an AI for citation stuff. So I would do that manually. And the way I do it, nice and simple, I just wrap the reference copy and paste it straight into Google. Let it see if it can find it. And I should be able to say if it's a real one, it comes up straight away. First thing in my feed, I click on it and it's right. I do that with randomly kind of four citations throughout the paper. But yes, I wouldn't trust AI on citations just at the moment. That is changing. Right. So there's obviously a lot more questions in there. But I suggest that we sort of summarise a little bit more towards the end. And I'll make sure there's lots of time there for questions. We can have a discussion on these. So like I said before, we have the short term kind of fixes. But now let's go into different types of assessment. So we want to actually look at students being very creative, application based, real world tasks. These are great assessments. Regardless of generative AI, these are great ways to assess students anyway. But let's have them solving a problem in practice. So tell them, you know, like go out to school, interview someone. They don't have to not use technology. They could use the phones and record someone on an interview with the audio, with the video. They can do things like that, then come back and then analyse those things. Which obviously they can't put audio and video as easy at the moment into these programs. But these are great ways for them to be assessed. Practical components and then multi-stage activities where they're having to do multiple separate small parts as they go along. So these are ideas. I'm going to show you three what I do actually using generative AI. So I thought, well, this is a great tool to do things that we've never been able to do before to learn in new ways that are very, very, not just creative and fun and unique, but really sound as in educational practice. So I have three. And the way I do it actually is I give my students the choice of three, but they can just choose two of the three. And we've got a critique of chat GPT, a debate and interview. So what's nice is whatever learner levels, whatever disciplines you should hopefully be able to use all three probably of these things. And it greatly encourages critical thinking and active learning. So the first one, you've probably heard this, which is very common, the critique. So have chat GPT write a paper on whatever it is. You will then critique the paper for bias inaccuracies and alternative thinking. Remember to connect to the readings from the class and class discussions. You can see them in there. So I always put a suggested prompt for these. Write a paper of about 600 words on the importance of whatever. So now those spaces, it's not whatever, it's how it connects. And now what I do is you could just say, here's the topic. I actually like my students to choose the topic. Then they've got kind of ownership on it. Out of ABC and D, I'd like to go with a, I'd like to do this. So then what they do is they post the prompt into chat GPT and then they get the response then to submit. This is important, but I get them to copy and paste what chat GPT has done into a word document. Then what I have them doing in the same document is use track changes and comments, you know, so they have to actually write all the way through. So you see what can happen with this is you could have, they could have chat GPT write the paper and then they could respond after now critique this paper. So this stops it from happening. And even if they did that, they still have to, when they're doing the track changes, look and say, oh, it was that bit. And they're probably getting is just the same amount of learning being able to kind of write across. So, so that's important in the way that they hand in that they have for every little bit they want to comment on, they have to highlight it and then write across. So the alternative thinking that it's not just remember what I'm saying about the way it's been built, it's been through the internet. There's a lot of biases there are inaccuracies. But the alternative thinking in this is having them say, okay, they use chat GPT use this framework. Is that a good framework for them to use? Could it have been a different framework from class that they've learned about. So the alternative thinking is really important. But as they're doing this, can you imagine the thinking that's going on? So that's great. I must have met funny enough. Out of all my three, this isn't chosen as much by my students, this critique of chat GPT the next two are so the next one. Oh, let me quickly show you the rubric for this. So you can see on the left, let's just focus on exemplary. So for the critique problem was clear and there was an in-depth critique of the topic, at least five main critiques. But then can you see below they get the points if it shows strong connections to what was read in class or the discussions on flip. Literacy was accurately cited on various occasions to make the arguments and the materials were also got from out of class. So you can see all those extra pieces to make sure that it's as sound as possible. So now the next two. So this, the second one is the debate. And I think this is probably my all time favorite on how to use chat GPT for learning. And so have a debate with chat GPT on. And again, I often give them a choice of topics. And then here's an example prompt so they can copy and paste it in. I would like to have a debate with you on the topic of I would like to argue for this and you will argue against it. And so if you think about this, this is great. We actually do it to have been in whole class person in person and we've debated in real time as a whole class against chat GPT. And it's really interesting because it is as if you're talking to a person because you can even say, you know, please check your answer. I don't think that's accurate and it'll check. So, again, for submitting provide at least five responses to see debate. Remember to check the rubric for the quality of responses regarding your desired grade. So we're going to look at the rubric next. Once you've finished, you will highlight the chat GPT conversation and copy and paste into a world document and then submit it. I'm having it in two ways at the moment. I'm doing this and I'm also asking for screenshots when they can. I prefer screenshots, but sometimes it's just so long that it gets hard. So here's the rubric. So again on the far left and they did whatever they were supposed to do at the top and then looking again on the second one drew from content covering class and a lecture and content address by the arguments made by chat GPT while also adding new arguments to the other side. So I'm looking for specific things. Okay, right. The last one now. This last one has to use the and I'll show you my full prompt in a second on the next slide here, but this is having chat GPT interview the students. So this is a practice interview for a whatever position in a whatever that highly values so hopefully you get any idea of this because I'm going to jump to the next slide in a minute so you can see these. Your aim is to get this position so you will you show your understanding of these things. So think from your content expertise what you want in those places. But here, you can see below the, you know, the prompt. And again, like I said, they have to put this prompt exactly in. So I would like to practice having an interview for this in a place that highly values whatever I would like you to act as the interviewer and ask me questions about whatever there is. And you see this it's asking to move multiple steps here look at this next part it says ask me one question at a time. Let me respond and then ask me another question. Ask me a total of five questions with some of the questions following on from my prior answer. So this is having chat GPT like said not just doing one thing it's asking it to do multiple steps and guide the student through. So let me show you what it looks like from. So don't forget I teach pre service in service teachers and using technology for education. So the green is kind of my content that you can swap out. But this is what the prompts agreed like. I would like to practice having an interview for a teaching position in a school that highly values the use of technology and learning. I would like you to act as the interviewer and ask me questions about technology integration frameworks and how digital technologies can support aspects such as student collaboration and creation. Ask me one question at a time. And so what they do is they send it in to me as screenshots or copying and pasted in. So this one actually really enjoy doing. So there are the three points I have a rubric to go for this but it's the same as the others. I'm going to put up this slide here. This is the research institute that I lead. And you can scan the QR code you've also got it online there. If you want to know more about some of the sessions I've done before explaining more about chat GPT or even prompt engineering. There's going to be another one that's just about to be posted there probably today that explains how to create those prompts that you actually put in to get what you really need. Because like I said, these are very, very powerful tools, but you've got to think about what you're asking it to do to get the best benefits out. So I'm going to stop sharing so I can come back to the screen and I'm ready for questions. OK, now there's been obviously many, many questions in the right hand. Elizabeth has been looking through them all as well. So I think if I hand over to Richard or and then I've got a few questions as well. But let's get over to someone else first. So any questions from Richard at the moment? Hello. Sorry, I've just been video was not come on here. Hold on a second. It's not going to come on. Anyway, hello and you can't see me. I'm Richard Beggs from Ulster University in Northern Ireland. I was very interested. Thank you, Helen, going through that. And it's it's comforting to see everybody's going through the same thing. And we're all in the same place and helping one another. And I think this is really useful. You go through some of the kind of helps and tips that can help us identify things and redesign what we're looking for as well. I think there's so many different topics in the questions that are here. It's a hard to know where to start. Do you think it's one here from Rod Colin from Manchester Met? Do you think AI is is a significant challenge to the current model? The current what sorry model? Our education model. Our education model. Okay. Yes. Even more than schools, it's a great challenge for the fact that in higher ed, many people have not been trained in how to teach. They have the content knowledge, but the pedagogy or as it would be for adults and regogy. People just don't know as much because, you know, we've not all had the training and it takes a lot of training to kind of work through how to do these things. And so when some things come along that is messing with how we've all taught in higher ed, which is often lecture asking them to do papers and things like that is causing more disruption in higher ed than it is in the younger years. You know, the school age children. So it is causing problems that way and they kind of figuring out your assessments, but it's also in that students have got older and kind of getting more wise to ways of cheating. And something comes along like this. And as well as points that they're trying to do the best they can as students. And there's so much pressure on them, but it's not surprising that they want to go to this tool that makes it so much easier. It's kind of helping them figure out the right way of doing it. But then also I saw a question about policies. There are so many policies that don't mesh well with chat GPT. We're having to actually change the policies like academic integrity. We're having to rethink those because we can't just say to students, OK, no AI being used. Because, OK, that means you can't use Google because that is AI. Can't use the, obviously the Grammarly. You can't use pretty much anything because everything uses AI now. So it's, I don't know whether I answered that question well. So there's a lot going on in higher ed and it is. Excellent. I think Rod wants to come in, Rod. Do you want to get your hand up? Yeah, can you hear me OK Richard? Yes, perfectly. Yeah, Helen, thanks for that. Really, really fascinating and a fantastic summary of a lot of the problems that I think all of us collectively have been wrestling with. So it's really lovely synopsis of everything. I think what I would say is in terms of the solutions that you articulated towards the end, I'm in a really luxurious position that I have relatively small class sizes. And a lot of the kind of approaches that you've advocated there are things that are sort of even before the advent of generative AI are the kinds of things that they'd be working towards in terms of looking at authentic assessment. There's a couple of things mentioned in the chat there about the difference between assessing a product and assessing the process that students go through to get to that particular product. And that's the sort of things that I focused on. But I'm in a very luxurious position of having relatively small class sizes. And so I don't worry if it's going to take me an hour and a half or two hours to mark an individual student's assignment. But I'm increasingly working with and supporting colleagues who tell me I have 350 students in my class. There are two or three of us who are teaching on this. Whatever you suggest to me doesn't work for me. And so my shift is going to be back to time constrained, invigilated exams where I'm excluding the possibility of students using these tools. And I'm even going to go back to get them handwriting things. And that kind of fills me with dread. And so that was where my question came from, really. Is there a need for us to rethink the way that we resource our teaching so that we're in a better position to actually know our students better? And in that way, we can focus on some of the really lovely, elegant solutions that you've offered us there. Yeah, going to know your students. I mean, we're always, it's always tough. I work with students, lecturers that have 500 in the class. They are finding a lot of these solutions very useful. I work with many directly. And just having them making it very difficult, but having all those pieces in, of saying, OK, I'm telling the students, no, I'm aware of generative AI. There are strategies to figure out whether you're using it and you shouldn't be using it. Remember, academic integrity, that if something I believe you are cheating, you will go to one of cards and you possibly thrown out to the university. So kind of it's horrible doing that. But then saying also at the same time, these tools are great. But let's know how to use them. And going back to your points as well about, yeah, we all need to go in kind of face to face and just do it that way now. Many have gone that direction. They've gone, oh, no, this isn't working needs to go this way. But it's not sustainable. And it's not always better either because you are one person for however, like say you've got eight. That's still eight people waiting for your soul attention where you can use it on generative AI having discussions with that. Or, you know, so it's kind of using it strategically reminding students why the learning that that, you know, make sure you do it. Not something else because you're going to need to know it. So, so I did the answer your question and not whether answered fully, but it's just kind of the thinking round it. It's really, I mean, I guess there's, there's no easy solution at this stage is there. I mean, it's a, it's a, it's almost a political question, you know, when it when it comes to the scale of teaching and whether maintaining the current scale of things and not thinking about student ratios. Is that, is there are any viable options that we can think of in that to make it a little bit more manageable with the sort of the level of engagement that you're talking about talking about here between students around the use of these things. But yeah, great session. Oh, Ellen, really appreciate it. I will say I'm always hesitant of kind of going this direction, but also there are proctoring tools. And in fact, there's some AI proctoring tools that do things very, very well at stop students from using anything else on the computer at the same time. But also even looking at the eye contact of the students, it's able to do that, which again, you couldn't, you know, if you sat in front of the student watching them online, you know, they had to do exactly that. But the AI literally looks carefully about what's happening, any delays where they've looked off maybe a different computer that's letting them access. So there are tools out there, AI proctoring tools that can be used, but then there are other concerns about online proctoring. You've got to be careful as well. Again, you're seeing the students and space around them, a family member that shouldn't be on the video might come past or you know, this. So I'm always careful about saying that. But there are those options that people choose to go with. I have one question. Yes. So what about Google's notebook? Have you used it? Because that looks like a very interesting resource for education going forward. I know it's still an experimental mode and you can, you can access it, but we can't. What's it called again? It's notebook LM. And it's Google. Okay, so it's one for you then. I've got one on you. Google LM came out about nine months ago. It's an experimental mode and you can upload your own papers to this area. So you might have a list of 15 papers and then you use it to do the AI then interrogate your papers knowing that the information coming from that and all the citations are absolutely accurate. So that is quite an interesting game changer. I think it's still in its early stage, but I I've had a quick look at a couple of videos and so on. And I do think it's one thing that would be interesting. It can't do it yet, but you might be able to share the same area in the Google notebook LM to students. So they could criticize critic their own things. But then I think you're presenting them with reference material, which is your reference material and not stuff that they can get in the web. So it's coming into interesting context. Yeah, it was interesting. I'm also seeing that I always I like getting extra things to explore. I do and I've just been seeing some of the comments coming up that. Yes, it shows that there's major problems around major problems. But I highly recommend people use those strategies that I've mentioned just such as connecting back to the readings in class stating that even if you're a mathematics professor and you wanted to focus on that you can still say OK. Connect back to something we've done connect back to the discussion last Monday. You don't have to say specifically what it was or, you know, you can give them general, but you have to say pulling on the debates from different students. And so this has a really, really focused one another going to be asked that so when students are having debates they're taking notes and paying attention. But all these things make it actually really hard to cheat that I'm not seeing people doing it with some of these tools now in the way that I'm actually asking them to do it. So don't be disheartened. We're in a good place. And there are trying to figure out ways to AI is being used as well as it's being used to these things. Everybody across the world have got the cheating issue and things like that. So there are still working on it, but I don't want especially higher ed to think AI is going to fix it for us. They're going to have programs out soon that will be able to tell whether it's human or AI. We don't have to change our assignments because for a lot of us it's probably a good time to change and rethink what we're doing. I think some of the assessments they might well have been set by larger institutions that the students have to follow. And it's educating the higher people about the change, the need of change of assessments, knowing that the people who on the ground who are teaching have to follow. These are the assessments that are set by the governing body. We can't change them. And that is a big challenge. My wife's a biology teacher, so she goes through this and she's having the same problem spotting AI in the amount of time she spends. And yes, the references aren't right. It's all that extra time when you're marking a script. And you're quite right. The assessment criteria has to change somehow. And as we've been saying in this group, we're looking for more active learning type of assessments that could be created. And it's having ideas and then getting them approved one way or the other. I think that is the problem. You know what I think some of the problem is in having not changed at the moment what's taking time. A lot of them are struggling to what we changed them to. It's like this. So that's where the delays been. There's been a lot of us people. I noticed there are a few comments about policies and things like that. People are only just really getting the policies now for universities for the fact that no one again knew what to do. I mean, some were saying, oh, let's ban it. Oh, let's do this. And then we had to wait so long while we learn what this tool was doing. And see patterns to actually make better decisions. But a lot of it still now depends on what's going to happen in the courts and things depending on legal side of this. So we're going to see change, but they're struggling to I think change these assessments. But yes, yeah, they're going to have to do. Yeah, I think I said we're coming to the end now for the one hour. So I know that we try to keep within the hour for people who are on lunchtime. So if there's no any other burning question from anyone. Richard or Elizabeth. No questions just an observation that a lot of the comments in the chat are around identification of use of AI, which is probably the wrong way to look at it. Because it's probably eventually no way to identify. I think it will becomes harder and harder. It's about thinking about how we're asking students to demonstrate how they've been learning outcomes and how we can be creative. And I think that's something I'm taking away from this is try to think about different ways and things that students will be doing when they go to work and being more authentic. And it's the stuff we've been trying to do for years really. And this is really making it even more important for us to do those things. So I just want to thank you Helen for a very interesting and food for thought that has popped into my head. It'll be Bosnolnit with different things you talked about. So thank you for that. That's my PC and so thank you. That's fine. Thanks Richard. Richard's a chair of our little group. So I wanted to make sure he had a good word. And you know what that's funny because that's pretty much exactly my closing statement that I would typically give just before I left. If you had it exactly stop looking for cheating. Think about changing your assignments and think about doing that by hitting on all those points. So perfect. Thank you. Thank you very much Helen. Thank you. My pleasure.