 Hi everyone and welcome to today's session organized by the ALT Anti-Racism and Learning Technology CIG, Special Interest Group. My name is Matt, Matt Lingard and I'm facilitating today's session. It's great to see so many of you here. Please feel free to use the chat at any time during the session. If you've not found the chat then in the bottom right hand corner of your screen you'll see three white arrows that opens the side panel which is where you'll find chat and participants lists and also settings which you might want to look at if you're getting some annoying audio bings from your notifications you can turn them off there. It's great to have so many people here for today's session on achieving introduced education using AI. I didn't say what context I'm hearing actually so I'm one of the organizers or facilitators of the Special Interest Group and I'm just here to do the introduction and get the ball rolling for today's session. So just let me say a couple of words first then about the Special Interest Group before I pass over to Tunde for today's webinar topic. The first thing I wanted to do was just share the principles of the Special Interest Group. This is something that we do ahead of any event or meeting that we host. The group was formed in 2020 and one of the first things we did was come up with a set of principles and that's primarily because of the nature of the Special Interest Group and the topics that we touch on which can sometimes be both challenging and personal so we felt it was important to have a set of principles that we follow in meetings particularly the first one there. Number two is not relevant so much today. It's just around confidentiality in our smaller meetings and when people do share more personal experiences. But it's really important to us that people follow these principles when attending our events and being active members of the SIG as well. Just say a little bit more about the Special Interest Group for those of you who aren't familiar with either ALT, the Association for Learning Technology or the Special Interest Group itself. As I've just mentioned the group was formed in a more informal manner back in 2020 and then formally became a SIG at the start of 2022. And we've got a much more detailed remit on the website which you can get to through that short URL or that QR code. But we're essentially a community of practice bringing people together to talk around issues around anti-racism and learning technology and we're advocates and really seek to progress things through our activities which include events but other activities to projects that various people in the SIG have been involved in. So it's really great to have you here today. If you're not already a member of the Special Interest Group, being a member just means joining the mailing list and you can do that through the link on the page that's there. It's great to see so many of you as a SIG feel free to use the chat and it's now my pleasure to hand over to Tunde who I know will want to introduce himself a little bit more and you will have seen his bio on the event. Anyway, but Tunde's I should say is also the Vice-Chair, the recently elected Vice-Chair of the Special Interest Group. So it's great to have him here to give a presentation as one of his first activities on the group and I won't take up any more of his time and I'll hand over to Tunde. Thank you. Thank you very much Matt. Hello everyone. Thank you for making it to today's session. As Matt mentioned, yes I am the sort of current Vice-Chair but as opposed to this event has been in the making for quite some time because we started talking about it last year if I remember correctly. So it's a real pleasure to be here and I want to thank the Antiracism and Learning Technology Special Interest Group for sort of inviting me to talk on this I think very important topic. One that I think has not received the sort of attention it deserves which is achieving inclusive education and I suppose using the AI because AI is a current buzzword at the moment within higher education and it just makes sense to start to explore how we can use AI to enable that inclusive tendencies that we know that we can develop within higher education. So I'm here to really talk about some of my thoughts on this so I've organised some events around achieving inclusivity within AI and I've also organised some sort of round table discussions. One was organised two years ago at the Chatted Massaciation of Business School Learning, Teaching and Student Experience Conference that was held in Belfast I think. So what I'm going to be talking about really are just reflections from those discussions and from those events. Before I go any further I think it's very important to clarify how I perceive inclusivity or what I perceive to be inclusion and the picture you see right now is my own definition of inclusion and for me it's recognising a disadvantage and making adequate provisions to address that disadvantage, making sure that what we are measuring with regards to students is not what we call disadvantaged performance. So we want to be measuring the actual performance of students. So this picture is a very clear example of what it looks like. So that picture is a vertical jumping exercise and before you jump first it will measure your height and your reach and then you then proceed to jump and then they then look at how high you've jumped and that measurement is the actual jumping performance that you've been able to sort of convey. I think that in higher education at the moment many of what we assess and how we measure student performance for me I think there's a lot of disadvantage within that which needs to be accounted for and I think AI will be a very powerful tool that we can use to eliminate those sort of disadvantages. So just to introduce myself a little further and I know you already have this information I'm an inclusivity scholar or that's how I like to describe myself. I'm also in terms of my title I'm also a senior lecturer in operations and quality management at the Liverpool Business School. I'm also the programme leader for the large collaborative programme, the largest actually in the business school and I'm also the associate for diversity and inclusion for the faculty of business and law at Liverpool Children's University. A bit about myself growing up you know I grew up in Nigeria and of course if you're familiar with western Nigerian culture usually deference is given to the father figure in the family. The elders have that sort of provincial treatment and of course in my house it was no different. So as a little boy I would look at my father's plate because when you're looking up in the pot of soup there are two sizes of meat in there. There's a small size and there's a big size or what I consider to be large to my small mind at that point. So I always looked at my dad's plate and be like oh my goodness look at the size of that meat. You know what when I grow up I'm just going to have bigger size of meat on my plate. You just wait and see. So that was my vision was formed at that point but then of course fast forward several years I moved to the UK, I got married, I got kids and you look in our curry pot right same size of meat and I'm thinking okay so where is the provincial treatment when do I get mine? And you know sometimes I come back from work very tired of course we've got just one snack left in the cupboard and I think you know what I deserve it I've been walking all day I go to grab that and I'm told no leave it for the kids so the kids get professional treatment when I was little my parents got professional treatment and it's just like I can't win I just can't win and of course as it is in higher education I think that's also a sort of symbolic of how we perceive inclusivity alright so there are parallels to that story to inclusivity and that we correctly find in higher education alright so when you look at it really alright I the one always looking longingly at that big plate of meat I represent like the ethnically diverse you know student groups my father was always given the preferential treatment represents the white students counterparts and then of course the big plate of food or the big plate of meat represents that desirable you know good grades you know to one and distinctions you know the desirable good jobs the desirable good pay alright so this basically so there are parallels and this is a big challenge we currently face in higher education and I'm sure that most of you are not you're very aware of the awarding gap and that is an issue that many universities at the moment are trying to address okay there's also that sort of disparity in experience for example the National Student Survey in their report they showed that you know the portion of ethnically diverse students who didn't think that their assessment and the marking or the grades was fair was around you know 70% compared to the white counterparts the awarding gap also is an issue that we currently face within higher education where the proportion of ethnically diverse students and getting to one and above is around you know 9% lower than the white counterparts and when you start to sort of break that data down and you start to look at individual groups you will find that it's even higher for black students at the moment it stands at 19% for black students so there is an issue here that we need to address again when you look at the number and the portion of these groups going into full-time employment again you will find that there's also a disparity there where ethnically diverse student groups typically are 70% less than their white counterparts so this is an issue that we are all well aware of but then the question is why aren't we talking a lot about them in the context of AI because at the moment AI has been seen as a very powerful levelling up tool because it's sort of addressing some of our limitations as individuals for example in the medical world it's been used to diagnose problems medical conditions in the physics world for example in the physical sciences it's been used to solve hard physics problems in the creative industry we're also using that to very good effect in trying to sort of augment creativity of artists but for some reason one of these big challenges that we face in higher education which is inclusivity we're not seeing products that are dedicated towards you know addressing that issue of course in higher education we've got several applications at the moment we've got chatbots we've got assessment grading systems we've got adaptive learning systems and also in analytics these are all the various places that we've been using AI but of course we still find that the experiences of students are not equal students are still reporting disparity in how they experience higher education or whether it's in the general student experience or whether it's with guards or awarding gap or the assessment performance there are still disparities and these issues of course they need to be addressed and of course here's my provocation I think one of the reasons why we're still having this same issue is that many of the learning technologies that we currently have and also many of the AI systems that we currently have the intrinsic design lends itself naturally to the benefit of certain groups alright while it's sort of disadvantage in others and the real reason is because when we design we mostly design for the majority so every time we consider for example inclusion it's usually an afterthought after a technology has been designed to cater to a particular challenge and then we then use inclusion or inclusivity as an addon to say yeah it does this but then yeah it can also achieve inclusion if you think about it properly or if you handle it correctly so inclusion really has not been the primary reason for design of course that's not to say that at the moment the DEI industry technology industry is worth around 100 million pounds and of course there are technologies in place addressing specific DEI issues but when we come to higher education we still find that we're still lagging behind in the adoption of such technologies and that is a problem that we need to address so what you find is that like I said it's usually an afterthought and a very good example would be the AI systems that we currently have we all know that there's a wide application in assessment design for example but the question then is AI of course will always replicate whatever bias that is found within the AI the training sets which is used to train so the question is what are we doing at the moment to ensure that we eliminate racial bias because most of the time what designers do is they use those training sets maybe scripts that have been marked and they feed that into the AI system which then sort of learns and then replicates the same sort of behaviour and we sort of address that because the training set is one very clear example where we need to start looking at so if I was to sort of then talk about what's our potential solution is so we've established that there is a problem inclusivity is a huge challenge alright we need to think about it more closely so how do we go about addressing this problem and what we need to do is we need to understand our context now at LGMU we were worried that the attainment gap or the awarding gap was very very high and therefore we thought we needed to sort of address this issue so myself and two other colleagues led an LGMU wide project on bridging the attainment gap and so we did a number of surveys and we also did a number of interviews even with staff members and it was very very clear where disparity or disparities in experiences in terms of how students felt with regards to assessment, the feedback that they received and also we also did some statistical analysis of the performance of our students with regards to the different types of assessment for instance so we looked at exam time assessment, we looked at report type of assessment we looked at oral presentation type of assessment and different other types of assessment we looked at nine different types of assessment and we again very quickly found that our context in LGMU context for example exam was one assessment type where both student groups, ethically diverse students and also white students performed the least but when you start to then break that down we then found that there was a huge disparity between the performance of white students with regards to exam compared to ethnically diverse students so that level that level of inquiry into our data is required so we need to understand that context explain what that means very quickly and then of course we need to be able to respond so in terms of understanding your context we need to be able to collect and analyse our data what is our demographic data what does it look like, what does it look like in terms of gender split, what does it look like in terms of ethnicity, in terms of disability religion, sexual orientation area of deprivation as well and of course a very important intersectionality so what do we look like, what is our make up as a body because the the idea of I treat everybody equally or I don't see colour we all know that I think an irresponsible way of thinking, if I'm in the very bold so it's important for us to start to look at issues understand where those disadvantages are coming from and then we need to actively work at addressing those disadvantages also performance data how are our students performing student experience data is very important of course we don't wait until the national student survey to understand what our students are going through but then when you look at most universities the only time that we collect information from our students is at the end of the module so module evaluation so we need to think a bit more about how we collect that information and how we utilise those information so that's very very important moving on to the next bit which is how do we respond now there are two areas that I can sort of think of one is the operational response or the operational considerations and then the other one is the design response or the design considerations so when we think about operational considerations so things around how do we decolonise the curriculum or how do we decolonise the pedagogy in other words how do we ensure there isn't any disadvantage in what we teach and how we teach them and that's very very important and of course there are currently AI systems that can help us to bridge that gap for example GPT is one of the most popular one but of course there are other AI technologies out there that we can start to introduce into the classroom the second one of course is regards to the awarding gap because I believe that this also has a direct implication to finding good jobs because we know that many employers of course always look for good grades not necessarily the subject of study but whether because good honours degrees always an indication that the students might be intellectually capable to do the job so if the awarding gap is always going to be there then one can expect that there will always be a disparity in the number or the proportion of ethnically diverse students finding paid employment so that's very very important for us to address also in terms of engaging with students and one of the things we found during that inquiry at LGMU was that students tend not to go to detutors for you know any questions that they have what we found was that of all the five areas that the students can potentially go to for information their peers is where they go to mostly surprisingly followed by their family members so we have this idea within HE whereby we build this system we provide support services and then we find that students tend not to use those support services and you start to ask yourself why I mean for some people then it's a case of I put it up there's only so much I can do but we need to do better in terms of understanding why are students not coming to us and then instead of asking students to come to us is there a way we can take all this information to the students and I think chatbots is a very good way of doing that now it's not a silver bullet solution to the problem but it's something that we need to try and we need to be strategic about how we use how we use that and I'll give you a very good example during the induction we've got a lot of students coming into the university what we typically find especially for international students they are always late incoming but what we then do during the induction is that we provide all that information to the students but we don't make much provision for those late students incoming and I think that the chatbot would be a very good interactive way of getting that information to students so those are potential solutions now just put some examples in here and I know some of you are already doing it but for those who are not already doing it then it might be something that is what I'm talking about so for example you can use AI in the area of pedagogy for example you could be using the flipped classroom and this is also very good because you allow the students to understand the topic do some research on the topic and then come into the class and then discuss so you can use it in that sort of setting by topics to the students ask them to generate content using chatGPT for example before coming to class so what you then do is you then ask them to take that content from chatGPT and then compare it to established textbooks or leading textbooks on that particular topic and then you see where maybe differences are again it's just looking at things from different lenses also you can also use it in the inquiry based setting where you provide students with a problem scenario and then you ask them to produce maybe three different solutions to that problem using AI so these are some of the ways that we can potentially start to sort of close that sort of disadvantage gap because AI is hopefully available to most people also in terms of curriculum to develop and some people have even proposed that you can actually use chatGPT as part of the reading test or the reading list so for example you can ask students to maybe produce different reading lists from different countries for example Argentina but of course care needs to be taken to ensure that and they're able to access those texts using the available university resources of course we can also ask chatGPT to generate multiple perspectives on a topic and I'll try this several times for example I say explain inventory management in public procurement for example and then I can say can you look at it from the marketing perspective and you can start to see different angles or different lenses at looking at the same issue or the same topic and that again will enrich students' understanding of that subject so these are some of these very simple ways that I think that we can help our students again in terms of responding we talked about the operational considerations we also need to talk about the design considerations so we need to focus on how we design these technologies okay or how we sort of deliver these technologies so for example we need to think about those that are designing these technologies what is the diversity there how well are how well is that group representative of the community that they are serving and that's very very important to have that sort of diversity of thoughts also ensuring that the training set that we're using it's not just populated by the audiences or for example maybe white students or white staff members and I know some people have used the acronym weird which is western educated industrialized western educated industrialized are the democratized rich democratized okay so we need to ensure how we get a balance right in in the training set that we're using and of course when we're designing we also need to limit what I call the overt otherness in our design and this is what it looks like I've just used this as an illustrative piece so for example if you think of the escalator we always say okay anyone who's on a wheelchair you cannot go on the escalator you have to go you have to use a left all right and when we start to separate things like that they say you go there you go there we start to create that sense of disassociation from the very community that we're trying to build and thankfully I think in Japan I saw this technology whereby people using wheelchair can actually go on an escalator because they've sort of designed the escalator to allow people on a wheelchair to use it all right so in that case I see that as inclusive in that sort of sense so whatever it is we're designing all right we need to ensure that we are limiting that sort of overt otherness and that's imperative I'm nearly there question now is why do we need to bother why do we need to do all of this ensure that we're providing an inclusive environment for our students I believe that specifically as this group because this is a select number of people so as a special interest group I think it's our responsibility to champion the creation of AI and digital technologies that is designed specifically to address some of these inclusivity challenges that we're facing because they are there, they are challenges and we need to start to think about that normally I would play I would play this video of Shimaman Aditya she's a well-known Nigerian author and she talked about the danger of a single story and that effectively saying that as individuals we cannot just rely on our own lenses we have to consider the lenses of others and I think that's very fitting for what we need in our education as designers of technologies we need to have that multiple lenses and not just use our own lenses because inclusivity cannot be an afterthought it must be part of the grand design it must be part of that grand design so we need to adopt that multi-lens approach finally these are some of the practical ways that we can develop that multiple lenses one is engage with the equality and inclusion and belonging strategy within your organisation within your institutions also get involved with the various staff networks that you've got in there also if you can if there's a programme try and get yourself on that reciprocal mentoring okay I think that would also help build the empathy that I think that we all need and of course there are a lot of research that has been done in this area so please get those research articles and read a bit more and then finally of course there are existing toolkits many institutions have what they call the anti-racism toolkits and these are some of the things that we can sort of start using I think I'm going to stop there thank you very much for listening and I know some of you might have questions or even comments things that you've experienced that will be helpful for everyone to know that is also welcome in this setting so thank you very much for listening thank you for your patience and I think I'll hand over now to Matt Matt yes yes thanks Sunday for your talk this morning I encourage people to thank you in the digital way which is never quite the same as a round of applause in the room but thank you very much I'm just going to stop sharing the slide so you can see Tunde and anybody else who puts their camera on and we'll move to questions now I've got three lined up in the chat if you would like to ask a question using the mic and or video then feel free to put your hand up as well but I'll just go through the three in the chat there's also some comments in the chat so if you've not had a chance to look at those I'd encourage you to do so too and those who've made comments if you want to voice them further then feel free to raise your hand as well so the first question we had came from from Lillian Joy who's at the University of York and she was asking is there any data on what kinds of assessments are more equitable for disadvantaged students so I'll start with you Tunde but obviously others can come into I mean yeah that's why I said your context is very very important now when we did these data was because we couldn't find any sort of data anywhere on the performance of students with regards to specific assessment times but then we quickly realised that hang on a minute we have this data so we thought okay we will do a statistical analysis to look at the historical performance based on the different assessment times and we're also very careful to also understand also the differences in sort of an instant assessment for example exams or sort of deferred assessment for example when you ask students to maybe write a report and you give the assessment brief to them well in advance so we also looked at the also issues around group assessment and also individual assessment and we also asked the students how do you feel so we asked them about the level of anxiety that they had towards group assessment and also individual assessment and there were very clear differences between the perception of all the various student groups and also you will find and I think this is something that has been going on right now even in our education a conversation that has been going on and that conversation is around how we disaggregate to those data so it's not enough to just say ethnically diverse and group them all so we have different groups within that we have several groups within that and all also have different perceptions and that also needs to be taken into consideration so I would say that your institution is the first place I think you need to go for that sort of data alright and I think there's a need for us to also start to aggregate those sort of data across universities but I suppose that's a project that can be undertaken going forward but I don't know if anybody else has got information on that thank you Thanks Sunday please feel free to raise your hand if you want to come back on any of the questions as well although I'm directing them to Sunday we very much encourage abscess from anybody I'm going to move on to a question now from Mike and actually Mike's put a few comments in the chat during this as well so then Mike if you want to help on the mic feel free to do so but Mike's initial question to Sunday was if chat GPT has been trained on biased data how does it decolonize anything? Yeah absolutely and I think what we need to think about is what we talk about decolonization and I know it's a concept that it's far reaching than just in there the mere sort of looking for textbook from different countries or from different cultures but when we're looking at different texts in that sort of context I think so what I did once was I asked chat GPT to give me one of the leading texts for example from Chile alright and what it did was it returned an abstract of the book of course in I don't know what this is speaking in Chile whether it's Spanish or the Chilean language but it also gave me not interpretation a translation of that abstract and that helped me to then look at that text and say okay does it have the sort of information that I'm looking for so my next step then was to say okay where can I find this text book alright hopefully freely or if you've got library resources then hopefully it should be freely available in your library I suppose that's where we are at the moment with chat GPT and I did because sometimes when you register they give you a form and they say are there any sort of topics you would like us to sort of look into and this was one of the things that I mentioned in their decolonization this is something that we really need to sort of think through and you're right yes at the moment we don't have the right solution but there are little things that we can do using chat GPT in the spirit of decolonization of the text that's okay thanks Tunde, you've got your hand up did you want to jump on the mic if you're able to thanks Tunde I think there's a misunderstanding often by how chat GPT works it doesn't give you sections of anything but in generative AI it's using statistical learning to present stuff from lots of information I don't think with the present version I'm sure with future versions you might be able to do it but you can't say give me an abstract of this book it won't do it it'll generate and it will hallucinate just like it does if you ask it for references so I think it's very dangerous and because it's biased it's more likely to give you biased information you're absolutely right I think we cannot at the moment get away from the bias and we know that yes your right sort of pulls information from the huge database and the question then is how do we even trust the database social media for example is filled with so much on truth for example and we've seen instances of where people put in questions in chat GPT but then it returns something else and people are even clever now whereby they can fool chat GPT to respond in a certain way so it's not a silver bullet solution to give an example of what we see and withdraw my history because what I did was I asked it for a text for a particular country and it did give me the textbook and that textbook is verified and I did ask it for an abstract actually I just said find me one of the leading texts in this particular topic from this country and it gave me a textbook it included the abstract and also included the translation of that abstract and I did my investigation and it did so now but you are right I think we still need to be absolutely careful when we're doing this to verify things ourselves because like I said I don't think chat GPT can help us address that decolonisation in the truest sense at the moment but in terms of looking for text from different countries I think we can use it with caution is what I would say Thanks Sunday just maybe as an aside but one of the other things that's been occurring to me is for obvious reasons a lot of the conversation has been around chat GPT specifically at this stage but there are obviously a lot of other AI tools out there doing more specific things so for example I was looking at one called elicit recently which summarises research papers for you so it's trained on a much different data set so I think it can definitely be an interesting area of tools for that as well okay let me shut up and carry on with the question so the next question came in from and it's phrased as a question I suspect it's more of a comment really but I will read it out comparing sources and using chat GPT to inform understanding suggests that everyone will have equal access to technology but is that the case? Yeah well yeah that's another challenge I'm looking at chat GPT for example from the perspective that it is a tool that everyone can use it's a conversational AI technology so you can have a conversation with it and you can stare that conversation to where you want it to go yes we've got disparity in terms of access to digital technologies of course when you look at it from that sort of ethical point of view and I give you another example when Covid hit it was then that we realised that many of our international students for example didn't have access they didn't have access to laptops so many of them were doing the assessment on their phones and that was a huge problem at the university and it also happened at a time where yes normally provide laptops for students they can use and then return but all those laptops were used up very quickly so we had this group of students who didn't have access to all the digital technologies that we have that is a valid and a big problem that we need to address how do we ensure that students have access to this technology but I'm also looking at it from this perspective whereby if I was in the class and I just said go on for example many of them having a phone they can access it in that way so yes it's still a big issue in terms of the digital divide but I think at the moment using AI as a conversational piece to get information although flawed at the moment and that is an area that we need to look into or continue to look into at least there's still some sort of access in that way, thank you thanks I don't know if anybody else wants to comment on that particular topic but I'll move on now Rob's just put his hand up and I was just coming to Rob's questions in the chat so I'll hand over to you Rob I don't know if you're going to comment but you can also ask your questions from the chat Hi Rob, thanks Matt I thought it would give you a bit of a break actually good to see you Tunday you're so kind Tunday and I presented actually back in February actually on AI and it's amazing actually in the months that followed that how quickly things have actually moved on you've obviously done a lot of work on the attainment gap at your institution do you feel that you've actually made a lot of progress now essentially so have you seen that gap dropping as a result of all of this work that you've actually done so essentially for those of us that have still got this gap very much at the moment you know obviously we can look at chat bots and that's great and obviously we've had that discussion about training data and stuff as well have you made the impact that you wanted essentially or is it still you're on a long road and we're somewhere at least on the road now yeah I mean for us so this project was just started three years ago but it concluded about two one and a half years ago it concluded so we're still in very early stages so where we are at the moment is we've identified some of the gaps problem areas where the moments just introduce you know different solutions to it right so there are different things going on in different parts of the university one of them is this sort of mentorship program that we're rolling out across all faculties for you know peer entering of you know students maybe in their final year or maybe in their second year mentoring you know level one I mean level three level four students first year students so that is currently being rolled out so at the moment we've not implemented the solution and we've not done it long enough to be able to say that okay it's having this impact so I think maybe we need to at our institution at least we need to give it maybe one year more at least before we start to see if there's any sort of shift in the but at the moment noise what I would say yeah so the key message really is to get on the road start doing things because it's going to take a few years to actually start having some impact exactly exactly and then you discussed about assignments as well are there particular assignments that you feel are you know are ones that we maybe should start avoiding you know in the future you've broken it down by different assignment types by different you know ethnic groups and so on so are there some that are really bad you know based on what you've seen? Yes exam is one of them exam is one of them because like I said both student groups white and both expressed a level of anxiety with regards to exams but when we started to look at when we looked at the performance data it was very clear that for all the sorry I can't show you the graph at the moment for all the different assessment types you know the average performance for the exam was around I hope I'm not giving out to you know LGM music right here but was very low compared to the others and when you then look at the two different groups you can also see that there was a huge disparity in there so exam was one area that was that had the biggest gap is what I would say and of course for obvious reasons you can tell because you know in the exam you don't know the questions beforehand you're given a limited period of time and for example non native English speakers you know that's always also very challenging you're trying to interpret what's been required of you and then writing what you think the answers are you always think in your language before you sort of translated but so of course there were anecdotal evidence to suggest that you know exam would be the problem for through that inquiry or inquiry we were able to confirm that this was actually the case with regards to our students I was doing a chat on I was doing a chat on AI yesterday to a big subject group and because of the you know concerns I think from the academics around AI they're saying do we need to move back to exams you know rather than some of these changes because I think they're worried about students overusing some of the AI tools and how do they you know generate or how do we assess authentic you know student grades and so on and there was this discussion do we need to move back to exams but based on what you've said that will also have its own problems and it's not a solution yeah exactly I think we also had this conversation during the meeting we had in February and the reality is you just have to look at your historical performance data for your students for us it was very clear that exam was the biggest disparity induce us if I could use that word within assessment performance and I think the issue of chat GPT also I think it's just laid to bear our reliance on written text and I think that's one of the problems that we have and we really need to start moving away from from that and I think there are two lenses that we can use here the first lens is one there people have been talking about you know how we sort of diversify assessments all right and it's important for us to think about that the second bit is and the older lens that looks at you know what exactly are we measuring so is it just you know really vegetation of knowledge that we're interested in or is it more skill based so there are different streams you know that are you know advocating for less use of exam I'm not saying that we should completely take exam out of the picture what I'm saying is that we cannot over rely on exams so the first question I would ask you or any person in academia is when you look at the life of a student the journey throughout the university what is the percentage of exams that they've written all right as what is the proportion of the credits of the exams as a percentage of the total assessment study is a 50% of the assessment is so how much of this assessment is written requires some form of written work all right and then we can start to build a picture and then see whether are we relying on one kind of assessment compared to the others and I think that's absolutely necessary going forward thank you thanks Tunde and thanks Rob I'm going to got some feedback not sure whether that's coming in from is that me maybe it was you Tunde now I've lost you Tunde actually or have I lost anyway AI is taking over the sound Rob I don't know who can hear me if somebody could just let me know if you can hear me but I thanks Nadia so many thanks to Tunde for taking the time to present to us today many thanks to everybody for attending and for your contributions as well I've just put in the chat the link to both the website in terms of events and the SIG group I encourage you to join the mailing list for the SIG if you haven't already and the recording if you came in late and want to watch the whole thing or revisit will be available on Altsy YouTube channel at some point but let's leave it there for now everybody can stretch their legs because I know everybody will be jumping to other things at 12 o'clock thank you very much and once again thanks Tunde bye everyone