 Okay, so welcome everyone to this session on JDI, just equitable, diverse and inclusive assessment working party session. We met earlier on as a larger group and many other people joined our working party and these were the objectives we set out to work on, which is making assessment more inclusive across institutions in the UK in different departments, different subjects, everything all inclusive of that. It's a big challenge obviously, which can only be supported by meeting regularly and discussing and sharing practice and sharing lessons learned from both since the from the pandemic and beyond. So we will keep on arranging some of these sessions to allow that to happen, to share practice and learn from each other. In fact, we will reach out to other bodies such as Thaiso, Advanced Achie, QA, PSRVs etc to also join us at different points and share their views and practice with us. And there will be opportunity for us to work on a systematic review of practice around inclusive assessment at some point, which will run a training session first of all, and then we'll recruit teams to work on that collaboratively. Moving on from that point, just to remind ourselves of the goals. So you see a picture of a Nobel laureate from 1913, his name is Rabindranath Tagore, in his own words, he composed a poem which is in Bengali, obviously this doesn't mean anything to anyone. In fact, it doesn't mean fully to me as well. I'm not a Bengali speaker, but I can recognize some words because they have common roots with Sanskrit and I speak Hindi so it's kind of related so I can understand a little bit. So it says, if sorry, yeah, it's this word means if your call is not heard, is not heard by anyone, you must walk alone, walk alone, walk alone, my friend, something like that. And if you were to read the actual translation, it's something like this, the song goes on, it talks about the progressive path, it talks about walking alone, talks about being fearless and not turning back. And others shall follow you is how the song goes. So that's probably what we are all trying to do in our institutions, we are walking that path alone and making the little bit of change that we can to practice around inclusivity, inclusiveness. Another sort of example that I think of is that of sharing light from one diya, diva, as it's called in Hindi, to another or maybe, yeah, so it's even in the darkest night when Diwali is celebrated, spreading light from one diva to another, brightened up the entire home, street, cities, world, one small diva at a time. Taking an example from another culture, Chinese Lantern Festival, there are multiple stories about why this is done, but the contrast here is thousands of lanterns in the sky alongside a full moon night, that's when it's celebrated, to show that even established practices can be challenged if many, many, many small efforts are done. So that's again a good thing to think about. Again, be it other cultures where people are sharing hope and gifts and pleasantries with each other around festivals. So this is just saying, you know, if little by little we can make big changes. And one such light or one such person who's passed, who tread the path, is one of our next presenters and I think all the other presenters are in the same category. People have done different wonderful work and they're here to share with us. So I'm really, really grateful for that. So I'd like to hand over the floor to Dr. Annot Cliff, who's from Kendabri Tratches University, who has created a school based on EDI principle. So hand over to yourself. Please do take your 30 minutes for introducing yourself, a little bit about yourself and your work and followed by some question answers. And that's how you hold the rest of the session. Thank you so much. What do you think? Thank you. Can everybody hear me okay? Yes, I just switched off my microphone and said yes. Thank you. Thank you very much. Some of the challenges of working with Blackboard once you share your slides, I can't see the chat. So welcome. My name is Dr. Annot Cliff. I'm the Founding Head of the Engineering, Technology and Science here at Canterbury Christchurch University and I've been a very fortunate position to create a new school from scratch. That means from nothing being there, creating a physical space and bringing the staff and the remit I was set was to create a completely EDI school. So I've been in a very privileged position to do that and that has involved an awful lot of work. We will be focusing in on assessment, but I think you need to understand the context for that and the work that we have been doing. So we haven't just tackled curriculum on the assessment. Why was it needed? Well, UK has a skills gap. I noticed my spelling error dyslexic, so I apologize. 137,000 need for this year alone in engineering. The profession still recognises it keeps hiring white males and they need to be reflecting on that, that missing out on talent, both females and Black and Asian students. We're in the Southeast and engineering is one of the biggest growth employment areas, hence the need to address the skills gap. And last year there was over 2 million vacancies on prospects related to technology. So as you can see, there is a desperate need for staff in this area and if we're going to promote this out to students, the opportunities are there for employment prospects. Graduate engineering employment reality. So this does impact what we do in our curriculum, but we need to be aware of this with preparing our students to the work environment they're going to. So ethnicity, gender, low social economics, disadvantaged students and their employment prospects. I did a piece of work in my previous institution and they were successfully placing over 50% of our students out on placement. This was a big, I'm talking about big engineering and computing department, so we're talking about hundreds of students. When I analysed down only 11% of my Black and Asian students were getting placements and in the actual fact Black and Asian students which were successfully getting placements have English names. I will leave that for your understanding of that. So this is some of the world work they're going into and some of the challenges we have out there. So back in the curriculum, back in the assessment, some of that is, as we know, there's a Black attainment gap and that is an institutional problem. That is not, it's academics and institution and curriculum problem that is occurring because if you look at the Rust Group students going in, they have the same qualifications. Black and Asian students from London, there is no attainment gap problem in schools. So we have created the problem in HE and then that perpetuates for our students of having that attainment gap awarding gap, particularly for Black students and that impacts their career progression and deployments as well. So we have got to address our curriculum and our assessment as part of that equality. I'm not saying we're perfect, but we're doing a lot of work to do what we're doing. But that places you some in context of why it is important we should be engaging with a more in quality, diverse and inclusive assessment and curriculum because we have a direct impact on our students' future. In engineering, it's even more impactful if we don't have people out into engineering who are female and Black and Asian, we will continue to produce products that only work for white males. Clars are a classic example. I can tell you now the IPD distance between your nose and the centre of your eye and the distance from the edge of your eye to your ear are smaller on women, which means our peripheral vision, our vision around, surround is different to a male, which means the pillars in the car, if they've been signed by white males, actually compound women in the car parking. So actually we know statistically women have more car parking accidents than men, but actually I don't think it's about ability, it's about the fact that we have designed cars with pillars that women can't see because our visual difference around spatial awareness is different because our eyes positioning is different to men. We know that profit increases if we have inclusive teams. Again, you know, the potential of economic growth we can support UK PLC is absolutely imperative if we can address this awarding gap, ensuring our best, you know, the students all have that opportunity as an employment, we can really help UK PLC's economic growth through that equality and diversity. So this is all placed in context for you of why, particularly in engineering, we have to address the EDI and as I say London is 40% black and Asian community and society overall internationally is 51% female, so we shouldn't be, as I say, engineers shouldn't be engineering solutions, it's only addressing 49% of the population. So how do we create an EDI school? So we do phenomenal amounts of STEM outreach which we won't focus here, but some of that work has to be done, those exercises and learning exercises that you do in school, you need to be aware of some of the work that has been going on in STEM outreach hubs about the importance of how you conduct yourself and that we know that the wrong comment can impact whether a female will progress in STEM. So if you're into outreach activities, go work with your STEM hub, but get training in EDI as well because it's important because that will impact your pipeline as students. Now something which also impacts your curriculum is actually thinking about your facilities, are your facilities and your resources and your staff EDI. So it's 38% of my staff are female, we've worked very hard to achieve that, we've written up the HR rule book to do that and I'm more than happy to share another time how we've gone about achieving that and 65% of myself are black and Asian. So we have those role models in that and it's important you have role models of mature black and Asian and female staff in their 50s as well as the young staff because for black and Asian students and female students they need to see the role models that people continue in that career and that's subject to this is so important and that actually can impact students' engagement in their studies and their willingness to achieve because they can see the end goals at 30, 40 years, I can achieve that, I can be who they are. With regard in these states, some of the things we've done is we have round tables or Bruclex triangle tables. The reason for this is to enable particularly with group work that people can't use their body language to exclude people. When you're teaching a class and you've put them into group work, really stand still and look how your students are working. Look where your Asian and black students are set, look where your white students are set, look how your males are sat relative to the female students. I've observed it so many times, particularly you know normally it has been a male who's used their physical presence on an oblong and a square table to physically exclude female students so they can't engage in the conversation and the exercise of learning that's occurring. Now to be fair I've also witnessed as part of our medical school recruitment which I am part of actually a very well educated private school young lady use her same body language to exclude two Asian students from a group activity of assessment. So be aware of that these are microaggressions that people do consciously or unconsciously but as an academic you should be challenging and bring it to their attention in private if that means taking the student out of the classroom and explaining what to them what they've done you might get a very angry response from the student but you need to highlight them that they'll be going into a working environment and employers are more or less tolerant of this behaviour now particularly go to an assessment centre for graduate influence so you know don't be afraid to challenge it. I have done in the past I have had very angry male engineering students in the corridor thinking I'm a waste of space explaining this to them but if I pointed out to them I care about their future and they need to change. So think about your state think about how you're setting up that group work from environment we do engage with employers and again employers it's really important it's a part of the assessment so for us we for the inclusive assessment areas we do big capstone projects and we source those from industry and we particularly identify what relevance about those projects and how we set that up and we're doing group work how you create your groups is really important so we do use a role play of Melbourne so we take it away from personality and about skill sets and that's helping the students to understand that you need it's about understanding those non-technical skills the employability skills that are really important to teamwork and group work and play into each other's strengths now UCL do Clifford's strengths as I say I use role play and Melbourne and my team does and we really explain we spent quite a lot of time with the students to help them facilitate that and that it's not about working with their mates usually they're the first year they may have done that with their mates and then the second year we move more to this Melbourne model and it's interesting how much more the students are reflecting on that more as semester one semester two are on that experience now of seeing actually that person over there they thought they didn't get on they compliment them because they're opposite and that's about inclusive assessments it's helping them to work for a more inclusive digress way to support them and it's also equally the assessment vehicles that we bring in so in the case of last year the students were designing a surgical tool for piles which is to be used in developing countries so it had to have single use and it must fail immediately after use which I know for the students we had to bring that in to make them realise that from a sustainability perspective it's still about sustainability because actually you don't want to reuse it because of the risk of sep but also it's getting them to realise that the majority of surgical instruments are designed for male hands that means women are falling out actually don't continue surgical careers because of the surgical instrumentation so it may help them students understand that surgical instruments have got in order to be British Kite marked British standard Kite marked they've got to consider it's got to work with a female hand as well as a male hand so again it's all you know if that helps with inclusivity if the assessment itself is encouraging the students to think about that inclusivity about that assessment process and playing you know let's say that a mixed group of male and female students actually bring in both their physical characteristics which are going to be important for discussion about solving these engineering problems some of the works I would like to cite you to as well when it comes to group work and inclusive assessment strategies particularly group one two work can I recommend and I will share with Mamish afterwards to share with you is Katie Bodie's work and she's produced a framework toolkit to support academics in facilitating group work she did a massive piece of research in the UK and Europe and America looking at group work in engineering and what she was observing just hold on a second I just need to tell a dog to be quiet who seems to think she can craft quiet settle is the piece of work she did was she observed she observed the observing groups all over the Europe and America in these massive projects that the girls would be sidelined into project management roles rather than using their technical skills in group work the girls are still achieving two ones and first but when you interview them at the end of their degree they don't feel technically competent so they don't progress into an engineering career plus equally on their employment application they talk about project management instead of leadership so it's absolutely imperative her work highlighted and I've you know staff developed staff is about staff picking up on that when the groups are formulated you know who's doing the leadership so who's doing the project management they should be rotating that role and it shouldn't be the girls being sidelined into that project management leadership role so it's absolutely important you as a facilitator and as an academic is picking up on that and encouraging that role swapping and I picked up on it this semester with our first year students when I was giving them some feedback on our current hill and their group final as they prepare for the final submission so we were in semester one and I picked up that some of the girls had ended up in this project management role and they were you know they'd not felt so comfortable in doing some of the technical side and I turned around to the group and I was blatantly honest to the boys and the girls I said to the girls like next time you do group work you are not doing the project management from the chip you're going to get your hands dirty in doing and it stops very dirty but getting on with the technical side contributing on the technical side volunteering for more of that technical side attributes that need to be done in the project next semester and I turned around to the boys I said and you need to take on some of the project leadership management because you're missing that skill so I explained about Katie Bowie's work so it's you know very honest with them and they accept that they recognise oh yes you know this is equally important both way our students today are far more passionate coming through about a fairer society and social justice and sustainability and are far more engaged now about quality and diversity so actually when you challenge them very early on in their first year if they do do something that's not appropriate and the students their peers will call it out and you address it and don't run away from it they will take it on board they recognise that they're in a safe environment in the university space and that we've pulled it out and this is an opportunity to learn from it to move them forward because we know they are aware that we're trying to help them to be a better person for their future careers and supporting them in their future opportunity and I have to be fair the environment we've managed to create the girls confidence is immense so when we did have a misogynistic incident in early on in semester one we were able to deal with it internally as a school and we addressed it and the student was invited into one-to-one and he understood what he needed to do he apologised to the student and you know he's been working on that so we have a real opportunity to shape things so I hope that gives you some thoughts about that assessment some of the other things that and I think Mammish will invite so that my academic colleagues to talk about some of the inclusive assessments that we've done which playing against the strengths and that they'd be best presenting some of the results of that and it has been really positive work with our coming out of Brexit out of Covid was that we now do online it there's an online exams we do in maths and we have pulled together a massive pool of questions applied questions using Blackboard and the students have historically during Covid but we've carried on with a 24-hectare style exam it's probably not 24 hours now I think we're restricting it to more 12 but the students can take two attempts it's fixed time anyway with the moment started they've got to complete it all students are given extra time of part of that fixed time so allowing for students some students having their learning support plans some students not declaring their disability and therefore not having an LSP or students who are still in the process of getting their LSP so we've taken the approach all students get extra time so we've written the exam that would be normally for an hour and everybody gets an hour and a half they can take that test exam twice in the time it's set out over a 12-1-4 hour period so we're saying to the students if you're an early bird doing the morning at six o'clock in the morning if you're a late at night how can't do it in the evening when it's your strength and your brain is functioning and you've got two attempts and we'll take and the system will take whichever is your best attempt we also have a mock one so it also pulls from poor of questions up to a week before the exam they can sit a mock exam so they're prepared for that and we've really noticed the improvement of adopting that approach has reduced that attainment gap awarding gap and the model you know is more evened up so all minorities characteristics we look at disability we look at black attainment gap and we look at low social economics we look at mature students and that's all evening out with providing an opportunity so it works around students who've got caring responsibility in a more inclusive way. I will end there so that gives you an insight to some of that level of curriculum involvement we've done and we're working progress and we're carrying on with that and I would recommend bringing in Shumi and Hani some time to talk about that particular assessment what we've been doing because they can really show you the module results but it's getting you to challenge you to think about your assessment and the impact you can have on your peers or students and working together and particularly the other side to that is about that social capital about building relationships with students because the more you build relationships with students and this is the work I did at my previous institution and the academics did we had no black awarding gaps in the subject area I managed because of working closely with the students building relationships developing their social capital engaging them in their studies helped all students progress in their learning. Okay great thank you so much Anne it's a fantastic talk we have a few minutes for questions if anybody has for Fran. I certainly have but I'll hold back my questions if anybody else would like to take the microphone or type their question in the chat while somebody else is speaking. Was it in-house knowledge? No so some of it yes because of what I did at my previous institution I've been trying to address the black attainment warding gap and the work I did there and that was work I've been working on since 2012-13 I've been involved in. Now because I was developing a school from scratch I went around the world literally I went around Europe visiting places and some of the work that's been doing I read the literature but I also learned from a lot of the work that's been done and research that's been trying to address EDI in the employment sector so going to the West Conference and everybody was moaning how bad it was and saying actually has anybody got any solutions here you know from the employers and started applying a lot of that work into what we did so that's how we've been doing the recruitment processes so I do recommend going to the literature. Delph has done some fantastic work research doesn't mean they've applied it in their institution they've done some fantastic research. Okay so anybody else would like to take the microphone or put their question in the chat while somebody's making of their mind I have a question you've you mentioned about you know taking the test and I see another comment in the in that in chat about that as well like taking the test twice so is it the same test or is it different test? So it's a different test because it's so it's a big pool of questions which you can imagine that's the hard part for the staff is creating this massive pool so they get a random so every set of students every student gets a different random set of questions so when they take the second test they get another random set of questions but they're all meeting the learning outcomes of the module. And students don't come back say oh that test was easier this test was easier you know they don't have any issues like that because they're set in a certain way that they are all at the same level. All students now exams are horrible but they do prefer that exam approach that they have a second bite of the cherry. It helps them with the students for that self-doubt. And I say I would I strongly recommend inviting Shumi and Hanny who have written a paper on the work and the results they've done an analysis of it of where they can demonstrate whether you know the first six was better than the second set but they've got some really good feedback from the students from that so I'd recommend that. And the question about what was the lessons learned that surprised me the most I think it has to go back to my previous institution when we when I reanalyzed the year set of results and suddenly there was no awarding gap or attainment gap for Black and Asian students and I actually went round asking the staff the team what did you do and they just went oh we invited the students to our engineering cafe that we have where we work on problems that aren't to do you know our module assessments but related to modules we made a real effort to make sure the Black and Asian students came along so it's about social capital and that's been the biggest and that's something I've learned where and so at Kingston University recently and for an EDI event for life sciences and that's what we're missing a trick that's the research that's now coming out is our low social economic students our Black and Asian students are being denied that social capital when they're at university it's either happening consciously or unconsciously and we need to make more of an effort so the more we build that kind of relationship academically you know that tutoring relationship and encouraging the students to do the more extra curriculum side of things and will make makes a huge difference it supports the students it encourages them to be part of it and want to engage in their studies and the reading around and that sense of belonging because otherwise if people don't feel they belong they drift and it's the same in the group work and some of the work we did I did at my previous institution was taken due to shore cross work who produced examples for Cambridge engineering students who were going out on placement they were upsetting the placement providers by the way they were speaking so she said she showed them you said this and you got this angry head if you rephrase it you'll get this lovely polite head and I was observing our students were doing exactly the same in group work and so I now at my current institution my previous institution we share due to shore cross work with the students and so the language matters so first past the post competition approach creates a very aggressive atmosphere and that aggressive language actually results in people being switched off and drifting out of group work so I've highlighted those students who want to do really well and get very competitive that actually your language is impacting the other students learning that you are stopping them from learning and that you need to change and think about how you're failing if you're not helping them to be part of the group so it's helping them realize team working is important their skills and their communication skills have a direct impact on student engagement. Excellent thank you so much did any further questions or are we okay to move to the next presenter I know it's my name on the list but I'm I'm going to step backwards and allow or rather invite Fiona who's from UCL to to take that slot for now because and then I'll do my session later on to Fiona thank you so much and maybe you can express our clapping I don't know if it's not possible from but there is an emoticon I'm using that right now but thank you so much for sharing your wonderful work with us and Fiona if you can try sharing your slides next we can move on to yourself for the session and please take your 30 minutes to introduce yourself your work and for Q&A like we did for this first part brilliant thank you Manish okay so hopefully you should all be able to see my slides and everything yes fantastic so I'm a lecturer with the integrated engineering program I promise I will explain all of this terminology for those of you who have not heard of the IEP before or what engineering challenges is but I am not based within any of the departments within the engineering faculty I work across the board and my specialist subject is interdisciplinary teamwork and I'm also the student support lead for the IEP so I'm very interested in how do we make our courses as inclusive as we possibly can for students so a brief introduction for those of you who've not seen the IEP before don't know what it is it's our cross-faculty teaching framework so the vast majority of our undergraduate programs in some way engage with this framework the engineering faculty at UCL covers such stuff as the school of management security and crime science we've got medical physics we've got computer science mechanical biochemical chemical biomedical all of the kind of engineering disciplines that you can kind of come up with and so obviously anything we do has to be somewhat flexible and tailored to each of these different disciplines so the vast majority of our programs engage in some way in this process and this is our framework I have to say it's not to scale as much as I would love the teamwork in the middle so challenges s1 s2 s3 these are all our teamwork opportunities for students it looks like it's like 80 percent of their degree it's really not the discipline technical stuff which is done in their departments it's there you know the the typical engineering teaching is really what's about 80 percent of their degree and you can see we've got maths that's taught across the faculty as well as something called design and professional skills which really does what it says on the tin it teaches design and professional skills and the reason we brought this framework in is because we really wanted to move to include specific space for skills learning for students it's something that came up in discussions with employers was what they they knew we were UCL we could definitely teach the technical that's cool but what they the kind of added extra that they would really like is if we could give students opportunities to display these skills to practice these skills to say come into interviews having already done it and they knew what was going on so this was our approach to build in teamwork opportunities where students could work on projects they could demonstrate not just teamwork but presentation problem solving critical thinking all of that kind of thing also bring that technical knowledge into the kind of project space that engineers tend to work in and then we would explicitly teach stuff you know in the classroom around how do you construct a presentation what are the kind of basics that you might need to think about in terms of teamwork so that's how these two kind of go along what's really interesting for us is the introduction of a lot of projects allows us to actually build in quite a lot of contextual consideration for students so not only are they solving a problem in a project but we can put that problem into a place into a set of users and get them to think about the sustainability or the ethics or the cultural context of what they're working on we can start building that in in a relatively I would say natural way I'm not going to suggest we've gone full authentic problem solving necessarily certainly from the beginning but it's provided us this space and the module that I work on engineering challenges is right at the beginning it's our intro module to project work to teamwork and that kind of stuff and that's the one I lead and we've built started building in how we kind of do some of this teaching so for us the concept how we kind of have tried to bring in the concepts and and think about them within the context of the IEP and our student cohort around justice equality diversity and inclusion is the part of how do we make our teaching and learning accessible but we also are interested how do we teach our students these concepts as Ann talked about our students are going to be creating things that impact the world you know the UCL's engineering faculty slogan has changed the world for a reason right we we are understand what we kind of what our students will be involved with and stuff like AI chat GPT you know all of these things and we are the point where students can start thinking about ethics before they necessarily go out into the wider world but one thing I really need to mention and it is our big challenge at UCL is scale so engineering challenges is our biggest module within the faculty it's the biggest module at UCL as far as we can find we cover seven departments and nine hundred and fifty students take the module so anything we do has to work at an enormous scale it is just kind of massive what we work on and this is one of the big challenges for us in terms of making our teaching and learning accessible is the sheer number of different ways we might do it and what we need who we need to support how we need to support them and a lot of certainly the advice I've received from our student well-being service is where you have a conversation with the student you talk about it which is fantastic except you know I've got nine hundred and fifty students I need to try and make a conversation in a connection with that will enough that they feel comfortable to disclose what's going on in their lives and this is a really tricky thing that we need to talk about and I do have a lot of academics that work with me there's about you have a teaching team of 20 academics and about 50 PGTAs but again this is a really tricky thing to negotiate and the other aspect to UCL's cohort is we're about 50 international students so we are we have a lot of students who come from very different backgrounds and they come from places where attitudes towards stuff like mental health can be very different to what we are attitudes in the UK and that is another thing that we are attempting to negotiate and try and support students through and bring when we bring these students together so I don't think we've we've certainly got stuff that we need to improve but I'm going to talk a little bit about what we've done so far and what on our kind of approach to some of this stuff so just to briefly I'll give you a little bit more detail on engineering challenges as I said it's taken by seven departments it's a team project work module it's first term first year which brings its own challenges it's 10 weeks our team project is actually only seven weeks we spend three weeks introducing students to the overwhelming life change that is coming to university and helping them support them through that process entering their department finding out what is electrical engineering is it what they thought it was who are these people who are teaching them and as I said everything is tailored with day-to-day teaching done by academics within the department and it's really fun because the students build a physical model very as you can see one of them is made from spaghetti not necessarily the world's most complicated sophisticated prototype but we actually get the students doing the kind of act of engineering right from the beginning which really actually helps them engage with this process because to for some quite a few of them that's what they wanted to do that's they wanted hands-on getting dirty in the process and to give you just an idea of how our assessment pattern does we have a very structured assessment pattern I think the the comma we get most often when talking about engineering challenges is how highly structured the module is and that is a consequence of the scale of the module and also being able to put in enough support we quite a lot of our students probably about 50 percent of our students have not had assessed teamwork before they don't necessarily know what they're doing and as I said 50 percent of our students are international students a large percentage of those are working not in their first language and so this is it's it's a very much a quite steep learning curve but we want to support the students in this so the approach we've taken is very much quite highly structured project very typical if you do project long long-term project work there's the kind of midpoint presentation then there's the live demonstration at the end and the final report but the two that are maybe a little bit less obvious are where we've got ipak criteria and we use ipak so ipak is a piece of software we developed within the engineering faculty it's called individual peer assessed contribution to teamwork is what the acronym stands for and it's students peer assessing providing feedback to each other as a way of giving students within the teams a little bit more of an individualized say it's a it's a nice way for students to feedback I've seen some fantastic comments on not it's not just kind of you know you might expect oh they were terrible they were useless but the students are actually very positive you know if they've had a fantastic time they were like you were amazing this was great and that's really useful for building up students confidence as well I think in the this something that's very unusual and new to them and then the other assessment that's maybe a bit unusual is the social impact report so this is where explicitly have started building in getting students to think about things like ethics things like cultural context within and how that relates to their engineering work how what how these might affect their project what's kind of stuff is going on there and we have as I said we have very structured teamwork we put this in for all students but it's I think particularly useful for those students who for example you know we're coming in from different cultural expectations about how to how to work we've also put this in for students who maybe find working with other people anxiety inducing or have find it tricky we've the aim here is to really explicitly say these are the processes these are our expectations this is what you need to do in this process and it as I said it's beneficial for all students so as I said we have the the module design and professional skills so before the students start doing their team project we have they have a two hour session with me where I go through things like conflict happens in teamwork yeah it's you know if you you know if there's some conflict that's not necessarily a bad thing you need to communicate here's how to think about communication here's how to think particularly about cross-cultural communication you know your expertise your ideas your experiences are all going to impact on what you're doing in that and then we also explicitly talk about how to divide up tasks what we our thoughts on leadership are for example we really encourage students not to have a team leader we really encourage them to break down that leadership and for example lead on the demonstration or lead on the project report right and that's leadership there you're not necessarily a kind of dictator telling the rest of the team what to do it's this kind of consensus building is what we're looking for so we really start talking about that we also have a design and professional skills module that happens in year two and I come back to the students in year two when they're doing further project work and say okay let's actually unpack why conflict happens within teamwork now you've got some experience let's really think through these situations and and how other people are working differently and as Anne said we use Gallup Strengths Finder we really picked that because what we were really interested in is for students to have a language that they could talk to each other about how they preferred to work and it really does work very well we find students you know obviously you know there's a broad range of engagement with the process we have students who say that's not me we have students say you know that's not what engineers should be we really try to make the point that you know engineers can be anything and a lot of engineers have a lot of people's skills engineering is surprisingly people-based really but it also allows students to say well actually that's why I'm clashing with that other person is because I want to be highly organized and they want to be leaving every change up to the last minute and this is bringing conflict into our relationship rather than it becoming a personal relationship so that's that's useful I don't as I said it doesn't remove all personal conflict the the scale of the module means that there's always something but it does help our students to kind of have facilitate that discussion and we're particularly aware that we're working with a lot of 18 year olds who maybe have never really had to have that discussion or maybe necessarily that self-reflection process you know I think you'll remember being teenagers and going I am definitely right you know I know what I'm doing I'm an adult now so it does help them with that process and as I said we give them expectations this is what we want a team member to be this is what we expect a team lead to do so we don't expect you to be doing that we do expect you to be doing this conflict does also often come up with lack of communication so we give the students communication tools so we get them to do meeting minutes we say you have to me every week here is a spreadsheet that breaks down what you need to do we also give them a team's channel and say you need to do some collaboration teams is what people collaborate on quite often we use teams because we you know bought the site license for it at UCL so that's where we're coming from so this is really helpful as I said for all students but I think it does help to reduce the anxiety again for all students about what is the expectation of teamwork and in general particularly within the first year model we're as flexible and as generous as we can be if we can give a student some extra time in a demo and be like last awesome leading questions in a live presentation we will if we can obviously with this scale and to try and make it as fair as possible across the board we kind of can't make kind of endless changes to the way we do it we also unfortunately the double test thing sounds fantastic but if you're marking 950 social impact reports it's a lot of work for us so there is a little bit of a limit resource as to a resource limit as to how how much we can kind of go here we also use a range of assessment types we use stuff like recorded presentations team presentations we do written reports we do template reports we do peer assessment with students and we explain to students why we get them to do peer assessment and how and we we do in a variety of different places and and stuff like ipak is used throughout their programs as well so it's not just within it's an introduction within my module but we use it student advisors is a term that we have at UCL it's a fantastic resource that's very new to us and I love so we have a member of student support team so the central student support team is assigned to each department so they're sort of embedded in one or two departments and they are seeing students on a regular basis so I have a meeting with them I say this is what's on the module this year this is how it functions this is what's going on and they're able because they're outside of the kind of marking hierarchy students are much more open with them and because they've already got a relationship with me they can then kind of come to me and say this student is traveling with this as long as this obviously as long as the student is happy for them to get in contact or a kind of generic this issue has come up in this kind of place so that is helpful and we can then also advise students to talk to them if they need to have mitigation or if they're struggling and the student advisors have you know the skill set to be able to really support their students they know how all UCL support systems work they can feed them in so we're in kind of quite not constant contact but we're always available and teams messaging has really helped with that we can kind of message people and just sort of go oh I've got a concern about this student and it may be a student that they already know is in the system and that's really useful and also when Sora is a I've probably you've all got something similar summary of reasonable adjustments so these are students with a diagnosed medical condition and we start lines of communication with all our Sora students to explain that this is a teamwork module it's teamwork is still considered somewhat unusual at UCL even though I mean we in engineering use it extensively so it can be that our central student wellbeing team are not so fair as to what happens in all of the different modules some of our adjustments can be a little bit generic which is something we're in communication them about but how we could get more advice and so we talk to the students and we do try and start those conversations specifically with those students as much as we can and say look you know if there's something that would be helpful please let us know although you know one of the issues we have is we're a first-term first-year module relatively few students have Sora's in place because relatively few students necessarily come in with a diagnosis particularly those international students and relatively few of you know we get certainly students who were coping and then the big life upheaval that is university sort of means that they're in a very different place and and you know it kind of exasperates something that was maybe already there or kind of highlights something that was maybe already there and these things do take time you know the NHS is not very quick at providing a diagnosis for example so we're also looking into how do we make sure you know really actually everything is as accessible as we can and what are the limits to what we can make flexible and we push those boundaries as much as we possibly can and so just to mention because this is the assessment I teach on and run directly so it's my baby is a social impact report it's something that we've recently brought in it was the kind of response to the pandemic because we couldn't kind of we used to run a kind of sort of discussion meetings where students would share progress on on their project and then we'd start kind of bringing in the concepts of cultural context or ethics and we had to put this stuff online and we initially created it as an asynchronous piece of assessment and we've now added a live workshop that's tied to this individual assessment and we provide students with the report so it's basically we give them a tool so if you're looking at cultural context it's stakeholder analysis if you're looking at ethics you've got to identify some harms if we're looking at risk and security you do a pestle analysis so really make it feel a little bit more engineering because we because one of the issues is the students have a very science maths background and so they're not so used to having to kind of work in the gray areas of you know these this judgment call and this kind of kind of work the gray areas of ethics we then get them to pick two things from their tool and go actually can you explain that can you talk about mitigation and then we go okay beyond the tool because we all know that ethics is more than you know what harms that happen cultural context is more than what stakeholders are can you give us some ideas and this is all relatively kind of bullet pointy single paragraphs um three or four sentences because it is very much an introduction and there are subsequent teaching for the students on these areas and they will do it in subsequent project work so it's very much an intro process one of the things I found quite interesting this year is I use Mentimeter I absolutely recommend Mentimeter as an engagement tool for large groups of students it's fantastic I use it when I teach ethics at master's level as well because it's anonymous the students can put their ideas they can put what they're talking about and we know these topics things like ethics cultural context can be a little bit difficult for students to discuss so I really recommend tools like Mentimeter but what I used it in the social impact to do is challenge the students assumptions so here I've asked them a relatively straightforward I guess a science question what is tuberculosis most of them understood there was a bacteria but then when kind of challenging their assumptions about so I should say our project work is based in Uganda it's something we wanted to put it that was really got the students to think about what they were doing so we're building a tuberculosis vaccine production plant in Uganda we've been with the same project for eight years now and it's really interesting how what the students attitudes towards people outside of their own experience are there are some interesting and quite dated stereotypes that our students sometimes come up with so for example I asked them what is the literacy rate in Uganda and I've taught this we teach this with 12 groups of 12 groups of students 80 in each time I've yet to have one of them correctly state that it you know a majority of the students say that it's it's 76% which is what it is in Uganda they tend to have this assumption that you know people aren't going to school they they can't read and write which is actually quite false and and quite as I said quite dated and so we are looking into how we might start challenging those assumptions and really getting the students to really think through you know a lot of this information is online they can find this information it is something you can look up so how we might incorporate that into our teaching so in summary we have made some progress there are obviously big areas that we still need to improve on we're working on things like faculty wide guidelines on teamwork we're looking into how we can improve our support for students and disabled students in general but you know the big issue we always have is that we're at scale it's a huge scale we're also across a lot of departments we have to I don't know how many of you've tried to get 20 academics to agree on something it's very tricky and not even seven academic departments to agree on something very tricky and student expectations are can be quite tricky you know there is certainly a percentage of our students who expect engineering to be some some hard maths and some science and so it they sort of going well why am I talking about social impact or why don't I have to think about teamwork and that how we we kind of we again done some work on bringing alumni in bringing people from you know mechanical engineering team bring in Formula One engineers and say and they talk about teamwork all the time and so you know we're still working on that but it's it's really interesting what our students some of our students are expecting walking into an engineering degree for