 Okay, so as we meet sort of Thursdays on usually the last Thursday, but as I said last time we have a few sessions, which are back to back every Thursday and this is one of our second ones. As you can see we had a meeting on the 23rd, then this today a second and then there's another one following up on the ninth and they're all around assessment and innovative technologies around assessment and the challenges around assessment. And today with me is Lucinda, but Edith, who is our communications officer for the all south group and they will be in future events Lucinda who is our secretary who looks after membership so if just a quick show of hands of people who are new to this and they have possibly not joined yet our working party on on changes if you if you're new to do those two things and not just new here but also not joined the working party yet. If you have then I thank you for doing that, but if you haven't yet joined our working party I'll remind I'll give you the link if you if you wish to join later on so But it looks like the people mostly are not yet part of the working group so that's that's great because we are looking for people to join from all institutions across the land so well done for for joining us and be even better. Oh, Lucinda is here so there you go. Yeah, I didn't see you join Lucinda sorry. Okay, so we have the full back here. All of us are here. If you want to join drop drop Lucinda the line email and and she'll make sure that you're on on our all south mailing list and you know the group, but let's not go into much into detail of joining because this is easy from the website. Our events are here listed people are here who will be presenting on on the coming Thursdays that we haven't quite fixed after the 9th of March which we will advertise the dates to the following people will be presenting different things around assessment so there's a range of Interesting events for you to come and join and be party to and as I was saying earlier that we launched last time we met the working party whose objectives it is to make Assessment more inclusive across institutions in the UK, if possible beyond as well but let's just keep our heads focused within the country first. It's a big enough task. And we need all all all people who are who are listening to us right now, either on live or on recording you all welcome. What we will do is we will share practice like for today Sarah and myself we're going to share some practice and some lessons that we learned within our institutions. Through the pandemic and basically meet regularly around the area of making changes and supporting people make changes around assessments and making them more inclusive. And work with bodies like the professional bodies, regulatory bodies as well as QA advanced at she takes over, etc. And one thing we will do is we'll be forming a group within this larger group to look at systematic review on specific Inclusion areas and specific outcomes. So if you wish to join us at a later point for a training in systematic review or be part of helping us train other people in systematic review please let us know. So we will definitely be in touch for for that and we aim to produce some evidence from the literature, but also from different organizations that we all work in. So if you haven't already joined I can post this link into the chat and please do sign up. And you will be kept up to date on all our activities and all our events. So what I'd like to then do is to begin by talking about our sort of shared goals that we want to be guided by as a group. I recently did a training session with the higher education with advanced at she and came across this acronym, the Jedi. And so I'm using that as our guiding principles goals. So for example, justice, it's about who loses out in different assessment designs equity is talking about are we creating any systemic barriers to prevent someone from from excelling. Diversity is going to talk talking about do we know who we are assessing is if there is diversity do we know each one of the groups or are we unaware of something inclusion kind of asking the question have we considered everybody's perspectives. And also quite important is the academic integrity as a guiding goal because if we are talking about assessment and this this becomes very important and not only in its own self but also are the Jedi action points that we take on and we implement are they then compatible with the academic integrity that results from such an assessment and likewise vice versa. So if these are our short term or let's say medium term goals. The reason I'm saying that shorter medium is because if it's a long term goal then we are we're not achieving any change what we want is these goals that we are able to make assessment more just more equitable more diverse suited for more diverse populations. And is inclusive and so on. So yeah, so we want those to be our short and medium term goals because we want to achieve them and then move on to something else and hopefully we make that make that possible. Through your help of course and so the way we do it is by taking a snapshot of where we are at the minute. So people who are joining our working party when in their own institutions in their own practice you would like them to start taking a snapshot of the current picture if there is any cell let's say let's just pick justice for instance if there's any awarding gap. And if so is it what is it about is it about social and cultural capital mental health disability whatever it is you take a snapshot of them and then try and affect that through interventions and better design of assessment etc. And then keep on capturing it over a shorter period of time and monitor over a short period of time to see if you if you're making change and then if you have success obviously come back and share it with the rest of the group. And if you haven't got success and you want ideas again come back to the group and discuss and be you know by sharing the problem obviously we might be able to solve it better. Okay so that these are some of the other areas as well I won't go into each one of them but it's important that we take a snapshot in our own practice and around and try and affect the change by doing something about those things and and and yeah collectively if possible otherwise. So the part of the reason why we doing these sessions right now is to is to do that sharing of the practice bit which both myself and Sarah will be talking about today Sarah will be running more of a workshop. And I'll just quickly summarize what we have as a practice that we learn things that we learned over the pandemic about examinations. So that could be an option for other people to to see and pick things from and try and change practice within their own institutions. And likewise Sarah has many ideas and examples to share as well for today. But if likewise you have those ideas please come forward and let us know so we can run more of these sessions where we are doing the sharing of the practice around inclusion in assessments as well. Okay so so that's that. So I'll quickly just run over my own example within my school we have been running exams like any other school in engineering and this is a typical engineering sort of control diagram really so controls not my subject area minus telecommunications but you know I understand this where is it going. There you go. Right so what you see here is in 2020 just before the pandemic hit us well at the time in the pandemic hit us we changed we everybody had to change their exams. We had lots of exams and we had to go open book obviously they take away take it at home that kind of exam. Our exam design was informed by a bit more than what I say here as basic but I in fact released a document which was systematically systematic review of the literature where we came up with some guidance on designing exam papers for this kind of settings. But I'm trying to say that okay you even with the guidance you can't get it right the first time staff are still learning and you notice that the the averages kind of shoot up. And what you do next is you try and react to that situation and I think similar story I was hearing in other engineering departments too. You create you give less time to the students and then you make the questions more hard or according to the guidance that that we all had to our disposal that could result in an undershoot like the exam averages kind of dropping. And eventually you know you learn from from from this experience and when you when you have done it a few times you know how to make it just right how to how much time to give to students what is allowed what's not allowed. Even when even today when the pandemic effects are over we still have open book examinations in our school but we give them no internet access we allow them to bring their handwritten notes specifically we currently we say bring whatever but we are moving towards direction where we're saying do bring handwritten notes only because that enforces that people are are able to summarize their knowledge and themselves rather than just relying on books. But in other institutions what my what I've also noticed is that as coming back into the campus but still still the exam is take away take at home and disruptors like check dot com or in chat gpd in the future could could cause these overshoots and undershoots with reactionary steps like closing the exams to what they were before the pandemic you know making them close. Book again and many people have done that and having done that they may have seen an undershoot on the averages or poor performance in the exam and so on and so forth. But I'm hoping that if we were to keep them the learnings from from the pandemic with us we will see a better sort of stable future. Now these smiley faces are from the point of view of the disabled or sort of inclusion point of view. So we noticed that over the last few years we had increased or sort of closed the gap between the disabled and the and the non disabled in terms of the awarding gap and our assessments tend to be about 60 to 70 percent exams. So we could say that that may be an underlying reason. But equally if you look at the situation in places where they have gone from open book to close book that's sort of certainly closing the doors to some of the people who may have certain disabilities about maybe let's say memory issues and processing and so on and so forth. So so that's where where where we are at and of course for the next week we are going to talk about some other examples from postman with regards to Chai GPD and things. Okay so without taking more time I said I'll stop at 15 past 12 so I'm going to hand over now to our guest speaker Sarah Potbury who's in the School of Animal and Rural Environment Sciences at Nottingham Trent. And she's a principal lecturer along with her colleagues who have joined us Samantha Reed and Melanie Velaratne. Is that correct pronunciation but then yeah so if I hand over now to the three of you who would be taking the rest of the time for the workshop that you that you promised. So over to you Sarah. Thank you I'm just going to share my slides. That's okay. Yes I will stop sharing my. Yeah there we go. If you can you can see my slides can you. Yes indeed. Perfect thank you Manish and apologies for anybody who's not from Nottingham Trent but we like pink and Nottingham Trent so it is quite a bold colour. But to say thank you to Manish and colleagues for giving us the opportunity to share some of the learnings which we've had from the pandemic and some examples of where we've moved more of an authentic work like assessment and the impact that that's had upon upon our students. So I'm joined by Sam who is a learning teaching manager as myself in the business school and also Melanie who's one of our educational developers as well so they will be supporting me as we move through this. Just to I suppose give a little bit of an introduction about the context in which we're we're talking about this when we're talking about authentic assessment we're talking about assessment that calls for students to utilise that same set of knowledge. Competencies and attitudes that they should apply in the criterion situation in real life and our feeling is that in terms of inclusion traditional exams present barriers but also as we start moving towards. We're trying to hit things such as graduate outcome surveys we need to think how can we develop students skills beyond just knowledge recall and recognising that assessment in itself is actually an important form of learning for students as well. So nothing in Trent University is a post 92 where the sixth largest university in the UK and we're one of the biggest if not the biggest campus recruiters of UK students. So 70% of our students come from beyond its region. So we call students from a very diverse range of backgrounds and we're seen as being one of the major skills importers into the East Midlands. We have around 40,000 students and staff and we've spread across six university sites, five of which are throughout Nottingham and we now have confetti London. So we're certainly very very diverse eight different academic schools covering a very broad range of subjects. We three specialist hub campuses as well where we're very much working on the levelling up agenda and again really tried to support that diversity and encourage students to maybe undertake higher education that maybe previously haven't haven't done so. So we think it's really important that we think about how we assess our students we recognise it as I said that an important form of learning. I feel that the pandemic actually was very good for higher education I'd appreciate it was stressful. But we've learned an awful lot from it and our digital skills both of ourselves and our students we saw rapid upskilling and it seems shame to lose what we what we gained from that. One of the approaches within our university was to look at our exams and consider how can we assess those learning outcomes. While moving away from that traditional exam given the constraints which the pandemic introduced and again linking to this graduate outcome survey and this need for our students to have an authentic learning experience we started to explore real world work like authentic experiences. So we work closely with employers we have employer input in terms of a lot of our assessments to make sure that we are providing that authentic opportunity. I will admit from my own discipline which is I'm in the School of Animal Rural Environmental Sciences with the exception of vet me nursing we don't necessarily have students going out into a set profession they need a very very broad range of skills. And that can sometimes make it challenging in terms of how do we prepare them for a workplace which is incredibly diverse. I wondered whether we'd hear the word chat GPT and I think we got in around 16 minutes before Manish mentioned it but we know that chat GPT academic integrity is very much high profile at the moment but I'm personally in the view we need to embrace it. Technology moves forwards we need to think of how we can use it as a learning tool and what we don't want to see is knee jerk reaction back to traditional exams and exam halls. We will lose that richness of the authentic assessment experience but also the the advances we've made in terms of inclusive assessment as well. And what we really want to as well do is reduce pressure on our students we are in a mental health crisis I suppose in terms with our undergrad students. And we need to think why are we adopting these approaches these traditional exams if there may be better more effective ways to determine their learning outcomes and also use those assessments as a form of learning too. So it's given us a really good opportunity for us to think about how can we strengthen our our assessment portfolio, how can we strengthen our teaching and our assessment to enable removal of some of these traditional barriers which we do see around traditional assessment. As I said just to set the context we are a post 92 and as you'll be aware we have the access and participation plans set by the office for students which we need to be looking at reducing these attainment gaps these non continuation gaps to enable us to charge the higher rate fee. So we have a moral and a social obligation to look at this what we know with our students is we have a very high degree of intersectionality. So at the moment we're very much looking at how we assess to reduce attainment gaps for example in black students versus white students. But we know it's far more complex than that that we have this intersectionality that a lot of our students will enter through a non standard entry route. So that will be city and guilds BTEC rather than a level of a level routes within my school we actually deliver FE onsite so therefore we have a much higher proportion than across other areas of the university of students progressing from that further education of those BTEC city and guild approaches. So we have we have a lot of diversity within our curriculum. We know students coming through this non standard entry route have less education opportunities. We see that they also have less cultural capital and therefore we need to provide those opportunities where we can within the curriculum. So again working with employers building that social that social capital to enable them to actually have the same level of attainment and graduate outcomes as students may be coming through that that more traditional route. We also know that around a quarter of our students actually come from households with a combined income of 15,000 or less so we have a lot of first generation students joining us so inclusion to us is embedded throughout all of our curriculum. And what we really want to do is look at approaches that connect students with real world practice. So this work is looked at by four sides and Evans and obviously this is building on this increased need we have to ensure that our students are going out to graduate level roles. And so the better we can support them this with this the greater hopefully we can appreciate those metrics we know that metrics are very highly influential in what is a rapidly changing sector. And as we move towards marketization is even more we need to ensure that we are enabling our students to compete with those which are maybe coming from from different sectors. So we have brand 45 40,000 staff as I said and across a very, very range broad of academic provision. So rationale for wanting to move away from examinations is just because it's been done that way doesn't mean that it's most appropriate and this can sometimes be uncomfortable decisions decisions to have and we need to make sure that the the value to both the students the academics but also those those employers very clearly communicated. So our assessment really needs to consider the diversity of students we are seeing a rapid change in our student body in terms of they are often working additional roles to enable to have the income to enable them to come to university. And we're seeing changes in qualifications etc. So things are only going to move forwards. And if we look at the work of Dolma, for example, you can see that this traditional exam route which we see in some areas, more so than others can be viewed as form of almost ableism in practice that even by their very nature, we're excluding students. We know that this kind of change as I said isn't comfortable. We will face resistance. I first met Manish I presented with Sam at the QAA PSRB forum really trying to overcome some of the challenges which we have so in my area, one of our PSRBs requires an exam for every form for every module. And again, we need to be rigorously challenging this and saying how can we if we are doing this how can we make this more inclusive but with alternatives, can we can we actually go for. So we know that some cultures will see disciplinary cultures of resistance and some traditions is maybe just a lack of understanding of good assessment, but we can't just sit there we can't have this inertia we need to be pushing forwards. We know that authentic assessment will improve students engagement. My own studies have shown that we've increased attendance, which I know is of major concern in the sector at the moment, but we've had significant improvements in attendance in engagement of students outside of the classroom as well. So with our now our virtual learning portal, but reflected in actual satisfaction so potentially will be feeding into NSS, but increases their employability skills too. So if you haven't seen the paper by Sakbana et al from 2021 it really lays out the benefits of authentic assessment that it helps to develop develop students communication skills and we'll give you some examples today. They're critical thinking and problem solving skills, but also that collaboration and I very much rely on team based learning and a very big active collaborative learner learning doctor of that pedagogy and you can see it in the student that we can link this in with both formative and summative assessment, but developing that self awareness and self confidence and as I said if you think back to those student characteristics that we have within Nottington University. We need students to maybe be able to develop that cultural capital to have that self confidence to go out and compete for for graduate roles. And when we look at the World Economic Forum in terms of those 10 skills needed to thrive in 2025 from their future job survey, we can see that traditional exams maybe don't develop and have that same learning opportunity. They want analytical thinking and innovation. They want active learning and learning strategies, complex problem solving, critical thinking analysis and creativity, originality and initiative. These are things that may be more challenging to assess in a traditional exam and therefore what we're going to present today is a range of case studies from across the university where we can use this authentic assessment as an alternative to exams to try to really prepare students better for the workplace. So just as an example in my school. So we're very much a rural campus. So quite a lot of practical underpins a lot of the theory. And one example we've seen here which has worked incredibly well is actually bringing in an employer to collaborate with students on a topical issue so that they are set something by industry. The students work with that employer over almost a patchwork text approach throughout throughout several weeks of that module. Understanding the topic more fully generating outcomes, whether that's that outputs, whether that's videos, whether that's scientific right scientific communication so that they can actually then communicate and actually holding a knowledge exchange event with the local community. So they are working with potential employers, but also that we have that community engagement and the feedback from the students has been overwhelmingly positive they appreciate the chance to work with a leading employer from from the UK. We talk about how that can help to develop that emerging that pre professional identity and how they can also include this on their CV and it gives them that that confidence to go out and to talk to future employers and showcase what they've done within that that curriculum. So assess work like like experiences embedded across all of our courses or increasingly so at Nuttingham Trent and and preparing them for these these graduate outcomes. So moving away from these traditional exams and this allows us through this authentic assessment to align these graduate skills, these qualities, behaviors, values and standards to two professional roles within a very, very diverse sector. At this point I'm going to pass over to Sam and Sam's going to talk through a whole range of other case studies within the university and then we're hopefully going to collect some of your thoughts some of your ideas, which then manage and I can how we can take forward and incorporate into the working party. So Sam are you happy for me to move the slide on. Yes please. Okay, there we go. Thank you. Thank you. Thank you. So as Sarah said, I'm Samant the reason I'm the teacher manager within the business school within Russian Trent University. I'm just going to run through just some typical kind of case studies of how we've included authentic assessment within my own school but then also across the other schools as well in Russian Trent University. So with the pandemic what the decision we chose to make within the business school was to give colleagues the option rather than having an exam to have what we turned on the day time assessments. So this particular form of assessment was brought in so that colleagues can look at their current assessment that would have traditionally been a face to face exam. And see how they could adapt that to be a work like task to be completed within a particular time frame. And what this led to was innovation and how could we look at the skills and the competencies and the particular strengths we wanted students to be able to show as part of that assessment rather than root learning. And we continue to look at on the day time assessments post pandemic and we're now undergoing a review of these to see what kind of forms have been used as part of this assessment strategy and how we can continue to innovate as part of these on the day time assessments. So example of what these on the day time assessments might be for instance could be that students are given a particular set of data and they've got a timeframe to work with that data to analyse that for a company to be able to suggest improvements or make recommendations based on that raw data. And this would have traditionally been on the exam format questions around a set of data but through the on the day time assessment they're now having time to be able to analyse that and come up with their own conclusions around that in order to answer the set of questions and to do their own research within that timeframe, and which they wouldn't be able to do before. And this has been used across our different departments, we have five different departments in the business school, and it could be that we're replicating a task that might be within a particular graduate role. So it could be, for example, for marketing communications that there's a crisis situation and the students got so long to be able to put together a communication response to that for a particular organisation that might be put together a press release for example, or a social media post or a blog post and they've got time to kind of research into what the implications of that crisis might be kind of applying the theory behind that and then providing a justification for why they've put together this piece. And what it's doing is giving autonomy to the students as well to be able to present that in a different way than just writing as they would have done an exam. We did some research into this to see how students kind of responded to this sort of on-the-day assessment approach. And what we found was that students felt a lot more comfortable, a lot less stress at being in this environment because they could kind of control a bit more about how they were putting together the assessment rather than having to remember lots of information as they would have done in the exam. And the response we've been getting from employers as well as we took it to our Illumina Fellows Network and particularly within the business school, what they were saying was that what it's enabling students to do is to be able to come into the graduate roles, come into employment already having looked at some of these typical tasks that they would do within the organisation. So particularly around data analysis is particularly important for our students and by having that data there or having to find that data they're then able to respond to that in a more authentic way rather than just being given a graph that they have to comment on. They're now having to put that together to create something so that they can then provide as their justification. We also found that for students as well that they felt like it had more value because it felt more realistic sort of tasks they would be doing and we're able to kind of level it looking at the different levels and in terms of how we build up those skills throughout their degrees. So again, they could kind of gain confidence through the assessment process and get ready for that assessment rather than just focusing on how they could sort of cram for an exam. And do you want to get to the next slide, Sarah, please? So within the School of Architecture, Design and the Book of Environment, we've particularly been including lots of input from the external stakeholders in order to ensure that the approach is authentic and that it is in line with industry standards. And so for this particular school using the contacts in a way that's allowing the industry to interact with the students throughout the process of learning, not just within putting together the summative. So what we're doing here is using the industry professionals to provide sort of formative feedback forward really to the students to prepare for their assessment so that when they're putting together their reports, they're able to get feedback from not just their tutors but from industry professionals so that they can then adjust their approach before putting together that final assessment for report. What should you would do within work rather than just producing a piece of work to gain that consultation with colleagues? And I found that students have really felt confident more with understanding the standards and that they would need to show not just from a university perspective but going into industry and actually having that continuous involvement with people within the workplace and the roles they're actually looking to go into. It's helping them to understand the knowledge that they're putting in their building and they're putting into their assessment how they would take that into the workplace. Go to the next slide please Sarah. Thank you. So within, so this is an example from the law school, how we use authentic assessments are within a mooting assessment. And yeah, this is an online example that was adapted and how this has been made more authentic is by students having a choice of questions that they would have so that they can then learn from them that a real world questions that they would have to adapt to but having that level of autonomy over what particular areas they're going to focus on. And that really has helped to kind of reduce that anxiety around that assessment particularly in such a kind of open assessment in terms of the muting and they could then put that together in an audio response based on their own analysis of those particular questions. So in that way that they're learning around their own interests but also able to develop their particular skills that's most relevant to them so it's more but it's kind of competency based really preparing them for that professional practice going forward. Rather than just being given a set of questions and that's going across the whole cohort. And the next slide please Sarah. Thank you. So from our science technology school, we're particularly focused on how we replace the exams by looking more at a case study approach it so rather than having the traditional sort of essay style questions, students are actually manipulating the real data using the diagnostic images to actually be solving a challenge. So it's given again that more real life experience and more engagement in terms of how they're having an individual involvement that rather than just commenting on what could be done. And in that way they're able to actually experience how that data might be used how they what they might do based on what they've been learning to be able to solve that particular challenge. And then the research that they're doing in order to benefit them in that assessment again has real world meaning because they're actively looking for how they can solve that particular problem rather than it being presented as a traditional exam that they're preparing for. So it kind of becomes more work life for them and gives them that experience that they can then take into industry. And another example from science technology. So this is a good example of how we take a kind of additional sort of online multiple choice style questions, but it's become again more autonomous for the student to be able to conduct digital investigations. So what they're doing here is looking particularly at the data and this is around a crime forensic investigation. So they're given the raw data that needs to be investigated before they can answer the question. So what it's now assessing is their skills we able to analyze that data and come up with the solutions. And they've given all of those files on a on the virtual memory stick so a kind of virtual area they can use to find the data look at the different files look at all the evidence for this particular investigation before they can then go in and answer the questions and give them that time to kind of prepare for it away as you would do within the work with gathering the evidence, analyzing the evidence before answering the questions rather than just trying to remember all the answers within that particular exam style are actually actively involved in that process of learning. So I think this might be our last example. And this is just another version of how we're using the industry to help help make that more real and valuable by having those industry content tax routes part of the assessment. And I think what this is really showing is that we how authentic assessment can be as authentic as having consultancy projects where you're working with industry throughout the whole completion assessment or it can be more simple in terms of just making one change on an online format where you're using real data or finding that real data. So what we try to show these cases is a bit of a spectrum because we appreciate that not everybody can have industry context with every single module and consultancy based reports. We want that breadth of assessment as well for all the modules. What we're particularly looking here with the consultancy is using external resources and external information external contacts to directly work with the students who are challenged to overcome a real world issue and work with the consultant offering solutions for a real life organization. And in this way, they're not only developing the skills but they're also showing that social responsibility in terms of how they're contributing to society and trying to make a difference through the recommendations that they're putting together as part of this assessment working closely with the external organizations as part of the consultancy to know that it's in line with the kind of standards and expectations they're expected to put together. Thank you, Sam. That's okay. Thank you. Okay, so just just to summarize we've gone through a range of different examples of case studies of where we've used this assessment as a way to try to address inclusivity but also prepare students better for the workplace. And really to us it's become very clear that there are benefits of the students. We're still obviously analyzing data. But certainly we are looking closely at these attainment gaps is obviously therefore benefits of the institution that hopefully our students will go out and they'll have a better experience will get better and assess reduce attainment gaps and do better on things like graduate outcome surveys. We that's the aim, but also for professional bodies as well just we are hopefully through these approaches preparing our students better for that that professional move that they will make once they leave us. So what we really would like to do this point and I am aware of time, but we have put together a padlock. There's a QR code there and in a minute I will post the link into the chat as well. But what we'd like is if you've got some examples that you're currently doing that will be really helpful for myself and manage so we can get a feel of what's out there and find what other institutions are doing. But also putting yourself in the position of as somebody who is employing students employing these graduates. So how do you expect them to actually communicate their knowledge in the workplace so that we can start to share ideas of what is more authentic forms of assessment and obviously within looking at inclusivity. If you do put an example of how professionals may communicate in their workplace. I've identified the discipline or the profession which helps us to keep it a closer track of the information that you provide and really just remembering that this definition of authentic assessment is as an assessment that calls the students to utilize the same set of knowledge, competencies and attitudes that they should apply in the criterion situation in real life. So I'm going to stop sharing my screen at this point and the padlock link into the chat. Manish, I don't know if you're still on the call. Yes, yes, there's a question for I think Sam presenting one of the slides. There was a question by Fiona. I don't know if you want to ask the question yourself. Oh, you replied. Okay. I have just put both together. Sorry, I couldn't see the chat. So I have put the link for the padlock into the chat and Manish, if we could maybe leave that open. Yes, yes. Maybe this is going to recording can also, were there any other questions? I think no, they've already been answered. So I'll just, we'll see if anybody's actually inputting to the padlock. Yeah, Fiona was trying to unmute. So you want to have a follow up question, follow up question I think. So if people do need to leave, I put in a survey basically for about our session, if you would like to give us some feedback and if you have anything that you're going to take as an action point and do it in your own practice, please do complete the link that I'm going to post one more time now. Just take that link with you if you're going. Anyway, sorry, continue. Yep. Thank you. So we've already got one example up here about teacher education. So I appreciate that. But if you, if you do think of any examples of how, how our graduates communicate, then that's a very good starting point of that backward design to actually how we align these assessments actually to that professional that we're hoping to actually produce at the end of that studies with us. There was a question about slides being shared asked. So I did said yes, because I know you won't mind. So we will share the slides and of course the recording is going to be available later on on the also your YouTube channel. Okay. I'm not sure if it's still is it still showing my screen or not. No, it's just a camera at the minute. Okay. So that icon would say is right. There we go. Yeah. There we go. So yeah, that's us. Thank you, Manish. Yes, we can see the padlet now. Yeah. Perfect. Thank you. And that's the main hall on the campus I work on just because it's a pretty background. So we do hope that, you know, these wonderful examples that have been shared have inspired people to do something in their own. Practice as well. They're wonderful because, you know, authentic assessment is this. This is all when I saw them first time I was amazed. I hope the same has happened to other people as well. Yeah, so please ask me questions. I have put my I've put my email address in the chat if anybody wants to contact me directly as well. Okay. And people, there you go. There's another question from Nina in the chat says, does NTU have a repository of these case studies that I can share with colleagues? I believe that I'll talk to the other learning teaching managers just because we have a shared ownership of these. But I think Sam, we'd probably be quite happy to put them on the national teaching repository. Okay. There we go. That's the answer. Yeah. Thank you. I can't stop. There's a question from Collette. This is a hard question, something that I think we all want to be able to do but hard to put into practice. Thanks for sharing examples across different disciplines. Yes, that's how I felt as well when I first saw these wonderful examples. I think Kate did an awful lot to get us doing it. We didn't have much choice, did we? Yes. Those things. Then we have another maybe question. It says, how have you worked with colleagues to redesign, encourage the redesign of their assessments and how have and how have you engaged employers? Or is this devolved to individual academics? That's a very good question. Practical. Yes. Yeah, that's, that is a very good. So I think both in my role and Sam's role as learning teaching manager, we work closely with with academics and professional surfaces within our schools. Sometimes, yes, it's down to an individual who's very motivated. We also have a very strong steer that we do need to have assessed work like experience across our courses. So from the VC downwards into to support us with the graduate outcome survey. So this is very much a top down approach, but I think also we as we're maybe less, less shackled to some of the traditions because we are a post 92. So I think we're often we're encouraged to explore new approaches to be confident to try things and not to be afraid to fail. And I think that that comes from a culture that that's very student focused. And this is ours is that our access and participation plan is within our university is our success for all, but it's across everything that we do. And it's very much a responsibility of every academic as well. So I think that that does motivate the law to be areas where you get better by in the than others, but certainly we have very good links with the local community with local employers. We have Sam mentioned them. We have very good alumni relationships that we maintain and we build. So I think I think all of that helps as well with digital technology, the world's become much smaller. So we can draw on people from a much broader range if we're using online platforms as well. Yeah, they got to say as well. I think we're still very much still at the beginnings of this as well. I mean, within my school, in the business school, we're just undergoing assessment review at the moment and just looking at how people are interpreting, particularly these on the data participants in different ways and offering those workshops to people to redesign them once we've got kind of a shared understanding, particularly post pandemic and where it's moving. And yeah, as Sarah said, our alumni first have been brilliant being involved in helping to collaborate with that. And I certainly found it in the pandemic. I teach one of my topics is the nutrition and I couldn't get students out to to zoos behind the scenes as I normally would do or that link with keepers and working approaching industry across across the world. They were quite happy to invest time making videos of maybe authentic scenarios so that the students could then explore them by feedback and then they actually got feedback directly from the employers as well. So it started to build these networks and this this cultural cultural development of students. So again, I think we are fortunate that we we value our alumni and employers, but also we do have these, you know, things like the teams, etc. It makes us the world far easier to work with. Okay, then there's another question from Fiona in the chat is how flexible are your QA processes for making innovative changes to assessments. We, I think QA wise, we're very rigorous. We obviously have to go through so there are cut off points so that we can't make changes to the following year, etc. But I think we also recognize the need that we do have to be flexible and we do have to respond to our students. So as learning teaching managers, we all have a very good relationship with our quality managers. We work very closely with them and also the Center for Academic Development and Quality, which Mel is part of and right from our design sprints are designing a new curriculum throughout. This is very much considered within course design, as well as within existing ones as well. I hope that answers your questions. You know, there's another question from there's a question from Charlie. She's asking, did you provide support for staff to develop suitable assessment criteria for these new assessments for the new assessments. Yeah, within the business school, we've moved to online grading rubrics. So as part of that work has been putting together guidance and for staff to be putting together that assessment criteria. And yeah, that's been particularly important moving to some of these different forms of assessment. So we have kind of regular workshop and guidance supporting that with that assessment criteria that kind of bank that we put together. As part of moving towards the rubric system. Okay, hope that answers and there's some response to David's question in the chat. David is still around. Then there's another question from Fiona about the app. You mentioned the app. How connected are your digital education teams. And regarding the tech that is provided and working from the WP teams by the participation teams. Yeah, it's. Yeah, we work we work centrally I think we work across we have very good collaborations across professional services as well so we have regular forum to regular meetings. And certainly because it's a very top down approach in terms of that that we all buy into it as well that I think DT are very supportive with this. We also have the flex team who who solely focus on moving things online or supporting blended learning and that supports available for all, all colleagues as well. Okay, okay. Thank you for that response. Is there any more questions. No, I think there is a question again from David, but I think it's Melanie. If you can take that and respond please. Yeah, so I'm seeing that people are leaving. I'm just hoping that you will complete our form which is just to take a feedback for the session. And also if you've taken back something today. And if you want to implement a change into your practice in your or your colleagues practices practice in in your own institution. Do do post that in the form. I can push the link again one more time. Any questions from anyone. We're very close to the end now. Apologies it took me so long to work out to stop sharing the screen so sorry about that. I think mostly the questions been answered so thank you, everyone for your time today. Just on time as we're going to finish the session stop the recording now. That is it.