 Welcome everybody, this is the first of a two-part conversation that we're going to have with the focus on assessment. I'm delighted to have Dr. Arne Yehey here to chair the session, and I'm delighted that we have a number of speakers for the sectors that are going to be sharing their perspectives. What we're going to do today is going to set a little bit about why we're having the conversation. Our speakers are going to give some lightning talks, they've been instructed that they have about five minutes, and then if they don't stick to the time, well, there's lots of paper and squeaky turkeys that are going to come into play. We're going to have a panel discussion, and most importantly we're interested in your perspectives. So, and I'll tell you about how we're going to collect those in a minute. Next slide. So just in terms of housekeeping, the webinar is being recorded, and that's actually the recording is to support the development of the National Forum. It's insight that we'll publish at the end of the two webinars based on the conversations we've had. We welcome comments and questions through the chat throughout the webinar. Please contribute. And we also want to hear your perspective. So we have set up a Google Doc, and the link will be posted in the chat just now. Please, if you could use the Google Doc to capture perspectives again. They're important, and they'll certainly contribute and be referring to them throughout the conversation today. So a little bit of context. So, just in 20, I think it was in 2015-16, we went and we actually reviewed the assessment profiles of nearly 500 modules across the sector. And we published the report which is available in our resource hub. And our conclusion was that examination is the most common assessment method, although it's popularity and waiting difference between fields, programs and stages of the program. And what we found was quite useful. And if you, is that if this is a profile of the different assessment types across the different disciplines, and the light blue is examinations. And when you look closely at that, you'll see for arts, for example, 60% of the modules that we looked like at looked at contained examination, and their average waiting was 59%. And if you looked at business, 980% of the modules we looked at in business and contained examinations and the average waiting was 67%. And looked at engineering, for example, 67% of the modules we looked at contained examinations and 65. And the average waiting was 65%. And then in health and welfare, 43% of the modules that we looked at contained examinations, and the average waiting was 63%. So it's very clear that examinations are across all of our disciplines, but that there is a difference, they have a different waiting across that. And the other interesting thing is, next slide, is that the examinations and the use of the examinations differs depending on the stage of learning. In education, you can see that when students start, the level of examination is higher in their first years than it is in their final year. In business and administration law, there's a slight increase from first year to final year. And in the natural sciences, it starts at about 60% and it ends up not that far away. So there is a difference in how we're using them across the stage. So what happened was COVID started. COVID hit us all. And I'm grateful to the QQI for this report. And I've taken a quote from it here, that's unseen written examinations, normally conducted in examination 100, restrictive visualization or mainstream summative assessment and higher education, and they were all cancelled in mid-March. And institutions had to find alternative arrangements for assessing students for eligibility for graduation or to progress to the next stages of their program. And we did that. And we looked at alternative assessments. And institutions put in continuous assessments, non-proctored online assessments with a long time window, non-proctored online examinations with a short time window, and proctored online examinations. But it was interesting when you looked at the report, we learned a lot. But what I noticed from the report was that it's what happened has made institutions themselves think about their whole strategy to assessment. So this is included here is one reflection from one institution. Our traditional focus on terminal form written examination is proposed being reexamined. And if you just go to the next one. And it's become clear that there's more than one way to access to assess a module and that reliance on terminus is not always the best way. Initially indications that that's why we are there was some increase in the number of deferrals students performed equally well if not better with met with modified assessments. So that's why we're having this conversation. We want to explore and help with the sector to explore and alter the, the, I suppose the opportunity to relook at assessment strategies. I know it's be done locally, but this these these two webinars, the one today and the one on the 28th of May. It's about to have a conversation to let us tease out some of the issues and that the place for this final written examination. So, to chair the session today, I'm to welcome our chair and also our first speaker, Dr on you, you hate on you is the registrar and vice president of academic affairs at Monster technological University. And she's also a national forum board member on your hand over to you just now. And thank you Terry for the very kind introduction and colleagues I'm just delighted and to be and honored to be asked to chair this session today. And I was reading the blurb for today's session. And I know from it that it says that for some time researching research analyzing assessment and higher education has raised questions about the effectiveness and reliability of traditional exam based assessment. And I thought first of all it might be useful to share with you a few perspectives my own background. Before I became a senior management manager. I grew up in a world of final exams if I'm to be honest, and my two main passions in my academic life have been mathematics and music, and there was so much loaded on the final maths exam, or the final paper, or the final grade exam in practical performance. And then I moved to CIT is was then in 2001 as a maths lecture, having worked for 10 years in primary teaching. And all of a sudden, you know, we were working with on various programs in engineering and science and business. And it was a host of requirements from the professional accreditation bodies who seemed to place value on terminal examinations in terms of mathematics, and you know proof of meeting learning outcomes and so on. And I was always a believer in continuous assessment for forms of purposes I would have introduced a lot in my modules at the time. But there were kind of concerns to in certain in our perspectives about bite size learning and concerns about academic integrity so we were kind of at the time very much focused still on on keeping that terminal exam in that discipline. And it was very interesting at the same time to meet with colleagues of other disciplines, and there's a diverse range of them in empty you cork. And it was also interesting to speak with industry, or indeed to meet with the professional and registry bodies when they came to visit because, you know, they were drawing our attention to program outcomes as well, which are more competency based like, problem solving teamwork, and so on. And all of a sudden I was asking myself hard questions about, I suppose the capability of the terminal exam to address all of those issues so I started to ask my quest myself questions about it maybe a need to provide a better mix. And indeed we ourselves in empty you cork, we rolled out a major modularization semesterization project there in 2007 eight, where we revamped our curriculum. And we really made a very deliberate shift in terms of embracing coursework based approaches, and putting a limit on the number of terminal exams per semester. And now as a senior manager as VP for academic affairs in cork, I'm seeing that, you know, over the years, there's been a demonstrable improvement, I suppose, or increase in the adoption of coursework based assessment strategies and alternative assessment strategies, as opposed to terminal exams in the exam hall context. And, and then I suppose. So, you know, we refine ourselves, ask ourselves very hard questions, you know, students are required to say, for example, to come into a maths exam and produce all the goods in a two hour exam. But yet, I knew I had to look into my heart and soul and admit I suppose that if a graduate working in industry needs to, you know, determine some, you say, partial fractions of an algebraic expression that there is no host of free software on the web that will enable them to do that, so that they still might have the overall concept of the understanding of the whole concept but maybe get the software to accelerate the algebraic computation so I was fine myself asking hard questions of myself, and the degree I suppose to which closed book assessment might not always be appropriate, or that there might be better options available to consider. So we were working away there and encouraging colleagues to, I suppose, consider, you know, alternative forms of assessment, you know, a more I suppose eclectic mix of assessment. And then 12th of March happened, and I will all remember that announcement by the then Tisha from Washington and I may be very naively thought I was heading home to work from home for a couple of weeks. And within weeks, we in CIT had announced the discontinuation of campus based teaching and learning assessment for the remainder of semester two. And all of a sudden, we were working right across the Institute with colleagues from various perspectives. We were looking with our colleagues in teaching and learning and in the academic department simulation to assessment design, our colleagues in technology enhanced learning were I suppose providing the infrastructure in terms of the online piece. We were, you know, I suppose working with our students union and relevant colleagues in relation to academic integrity and promotion of the requirement for same particularly in the online context and work with our colleagues in the area of access and disability and indeed realizing that the alternative assessment format for actually opening doors in that regard. And now here we are. And, you know, we're 15 months later, the vaccination scheme is being rolled out. And I think there is a very good possibility that we'll have significantly increased amount of onsite delivery come September, and certainly moving into the calendar year 2022 I would expect to probably increase. And that's all very welcome we I certainly long for the day to meet students and stuff on face to face again. But I suppose the question is now we're at a very interesting point where we actually have a choice. And the question for me is, is this now going to be a full system reset where we go back and we always do things as we always did. And I suppose there are an opportunity here to move to what colleagues and of mine have called a better normal. And I believe we've come to that critical point where we now have a choice. I certainly don't have all the answers. And so I'm really looking forward to listening to our esteemed speakers who coming from a range of perspectives here today, and to hearing their thoughts on the matter. I know it's still, it's my pleasure to, I suppose, introduce our speakers to you. And just at the outset, we've invited them to consider the final exam as an assessment method. And then to consider the mode through which it's delivered, because each mode will have its own issues, whether face to face or online. So we've asked, as Terry said, these are lightning talks. There's the five minutes indication and Terry and Cole will be issuing very nice reminders. So I'm going to introduce our speakers. And first of all, we'll be joined by Kevin McStavick. And Kevin is the Deputy President and Vice President for Academic Affairs in the USA, and indeed is a board member of National Forum. We then have Dr. Elaine Doyle and Elaine is Interim Head of Department and Senior Lecturer in Taxation in the Department of Accounting and Finance in the Chemie Business School in the University of Limerick. Coming from the languages perspective, we've Dr. Julie Rogers, and Julie is currently Assistant Professor and Head of Department of Fresh Studies in School of Modern Languages in Munoz University. And then we're joined by Peter Cullen, who's the Head of Research and Innovation in QQI. And our final speaker will be Dara Ryder, who's the CEO of AHEAD. So just a reminder that the link for the doc, for that shared Google doc, where we're inviting, we really will welcome the audience perspectives here today. So we're encouraging you to visit the Google doc. And as has been previously flagged, we're also on the hashtag on Twitter, NFSS. Please, those of you who are active on Twitter, definitely post under that. And I'll be keeping an eye then we'll have, once our speakers have spoken, we'll have a Q&A session, so I'll be keeping an eye on the chat. So please post your questions throughout the talks on the chat and I'm sure we'll have a very, very productive and active Q&A session afterwards. So, Ganna Hillemweil, without further delay, I'm going to invite Kevin to come on board and Kevin, I believe you have a small presentation to share with us. And again, Kevin, just to thank you in advance for, you know, just being your work on the board. It's been an absolute pleasure to work with you on the board National Forum. It's a real partnership approach you're bringing to us. Thank you, Kevin. Thanks very much, Anya. And hopefully everybody can see my slides. Okay. Yeah, perfect. Okay, I'll kick off. Thanks very much, Anya. And thanks to National Forum for inviting me along. As Anya's already said, I'm Kevin and I'm the deputy president and vice president for academic affairs for USA. I am being asked to reflect on the assessment of learning with a particular focus on the final exam. So the first question I would always ask when it comes to assessment is what are we assessing? Are we assessing learners understanding of a particular topic or a particular area and their ability to critically apply that in different contexts to analyze complex information and present that in a way that demonstrates their understanding. We obviously want to know their knowledge of a particular subject area and what they understand about what they have learned and how they can apply that within their answers. But particularly when it comes to closed book assessment, the question is whether we're also assessing their memory. And if by including a large portion of closed book terminal assessment, are we actually encouraging students to focus on the breadth of their knowledge as opposed to the depth of their knowledge? So that's a question I would just pose for the time being. How are we assessing and already discussed this today? Do we use mostly terminal assessment? Perhaps we use mostly continuous assessment or perhaps we use a mix. And what are the reasons that we would use particular formats for particular subject areas or particular modules? Why assess using a certain format? So I think it's really important that with setting the assessment for a module or setting the assessment for a particular subject to consider why you use a certain format to measure learning outcomes. Is it to meet their learning goals? Is it because exam, for example, is a good way of demonstrating the knowledge in the learning goals of that particular module? And based on student feedback, do you talk about assessment as part of staff student consultative committee, for example, and take on board feedback from your learners? Or, and it's a bit of a tricky question, but do you use a particular type of assessment because that's the way it's always been done. And there's never really been a conversation around innovating it or doing it in a different way. And then that brings me to another slightly philosophical question. What is it all for? What is assessment for? By using terminal assessment, are we inadvertently causing learners to learn for assessment as opposed to assessing their learning? By that, I suppose I ask, what is success in a particular module? Is it just purely the numerical outcomes of somebody's learning? So is it whether they get 70 an assessment or 15 an assessment? Or is there more to that? Is it about how that particular subject that they're studying or that module actually adds to their overall personal development? Or is it adding to their ability to communicate their teamwork, their ability to analyze complex information? And how are we measuring that? And how does assessment play into that? Looking not just at summative assessment, but also formative assessment. How does low stakes assessment through the module contribute to the overall development of learning? And then looking at the dreaded seaward COVID-19, COVID-19 has really shook things up across higher education for staff and students alike. And Anja already spoke to that about the impact that that had for everybody working within higher education and the need for assessments to be adapted at very short notice. And the question I would pose when we look, for example, at the online terminal exam is whether using proctoring software, for example, whether we adapt an assessment for the environment, the environment that we now find ourselves in, or is the use of this type of software and this type of approach actually us trying to adapt the environment to fit the needs of the assessment? And I don't think there's a right or wrong answer with that, but I think it's a question that we need to ask ourselves. And we need to then also ask not just ourselves, but ask students about that too, and have those facilitated conversations with students. And that brings me to the final slide of my talk. What does the future hold? And I've used a quote here that some of you might have heard before from Grace Hopper, who I think is an admiral, was an admiral and also a computer scientist from America, from America. And she said, the most damaging phrase in the language is we've always done it this way. And I think when we look to the future of assessment, the worst thing we can do is just refuse to entertain the conversation at all. I think there is a need to look at assessment in the whole, to look at it in the pre-COVID world, in the during COVID world, and now to the post-COVID world and what that looks like. I think that involves conversations with all manner of institutional staff that involves academic staff, professional development staff, and of course students. On that note, and before I get a lovely reminder from the Director of the National Forum, that brings me to the end of my presentation. And thanks very much for your time. And I look forward to the discussion as part of the panel discussion there. Anya Jarnout. Thank you very much, Kevin, for that. And thank you for bringing the student perspective and asking some very challenging questions, but in a very meaningful manner. Absolutely. Thank you. So we move now to Elaine. And Elaine, you're loading the presentation, so I'm going to hand over to yourself. Thank you very much. Thanks, Anya. Can you see just the slide there? We can, Elaine. Thank you very much. Thank you so much. Thank you very much for inviting me to be part of this conversation, which I think is really timely and we've spent so much time, I think, probably individually thinking about all of these issues, you know, as we pivoted, I guess, to the online type of delivery. So it's really lovely to have a chance to work with colleagues on this in a broader sense. So I suppose I'm coming at this from the perspective of accounting and finance. I'm a lecturer and tax myself. And we really, I suppose it's very much a computational type of subject matter is what we're engaged in. There is a right answer. And there's only a limited way that you can arrive at that right answer in terms of actually meeting legislative requirements and so on. And so I think the disciplinary context is really important in that sense. And the disciplines that we teach are also very much, they build on each other so that you don't really know how to do a tax computation until the end of the semester because we're building all of the blocks as we go along. And therefore I suppose it is difficult to comprehensively assess something in the early weeks of say a module because the students really don't know how to do very much you're kind of building that that that capacity and that skill up throughout the semester. But I suppose it where we're caught in in our disciplines is this and I suppose link to the professional exemptions that which was mentioned by I think people before before now probably Terri and on you. And we are linked in most cases to the various different accounting professional bodies and the tax. And they again it's a bit like you know they've always done it this way and we in order to maintain the exemptions for our students we actually have to assess and to the the minimum of 70% final exam in hall exam so that we're caught with that. So while we do obviously use continuous assessment and and quizzes and projects and so on all the way along. We have to be confined really to a maximum of typically 30% for the continuous assessment and typically then a 70% in hall and end of semester exam. Which does kind of restrict us and also our students typically go on to study with those professional bodies and that's the way they assess so to some degree you know where we're getting them ready to be able to sit those professional exams by having them be used to I suppose a final final in hall exam. So I suppose that there that that's the context within which we we are operating. Now what's interesting, I suppose, for us in the in the context of the switch then to online and was the whole I suppose integrity is really the key for us. So not only to ensure that our students are meeting the learning outcomes but also to demonstrate to the professional bodies that they've met those learning outcomes with integrity within the assessment process in order that our students would be able to maintain those exemptions. And so it was kind of interesting with the whole the whole pivot because we all immediately began I think individually to think okay how is this going to have a knock on an exemptions if we can't be in hall. And we were I'm lucky to be part of the Irish Accounting and Finance Association which is a group, which brings accounting finance and tax academics together. We immediately in March started having zoom calls and well what are you doing, what are you doing, should we have a sort of united front on this to ensure then we can go to the professional institutes and say look this is the best we can do, we have to, we have to move move to online. And we're going to have to assume, and that we are students are take are taking exams in an open open book kind of way because we're not going to be able to monitor them so we need to switch to switch to that kind of mode, I suppose. So we did switch online and typically I think people did more or less what they would have done they still had their end of semester exam, albeit delivered online made the assumption that it was going to be an open book exam so you're not going to ask anything that could be kind of looked up very quickly. And, but where the university had, I suppose put pressure on all module leaders to look for alternative forms of assessment and, and allowing a long time for students to submit work, because you know a tax computation is that is the same as the next tax computation, there isn't an interpretation as such. So we couldn't do that and stand over the integrity of the assessment we had to use time bound online exams. We had to switch to so the time pressure for us was actually important in order to ensure that the integrity and that students aren't just taking a screenshot of something and passing it on to the next student so that so we have, we have time bound online exams, but we've also in lots of cases asked the students to handwrite their exams and to download an exam handwrite their solution scan the solution and upload it again so it really replicates quite close to the, to the, the in-hall exam. And that also allows them not to be able to copy and paste I suppose if they if they were typing and so on. And so that's what we've been doing they're unproctored, but that's not going to be sustainable for the professional bodies in the long run because they've all moved to proctored usually AI proctored solutions which they're not reversing out of. But we've been trying to get a seat at the table with the professional institutes to say at this point where we're in transition could we please have a voice now when we're really looking at the post COVID world. We don't just continue to say well we have to have a 70% end of semester exam. This is the point of which we'd like to have a bit of blue sky thinking around that. And we've managed to secure a meeting with all the professional bodies at the end of May. So there's a delegation of us and delegation of them. And we're hoping that we'll be able to I suppose continue this conversation in a more productive way going forward. So thank you. Thanks very much Elaine and I suppose very interesting to hear the perspectives, I suppose, in terms of a computational subject, like I would, I would empathize with some of the experience there and indeed in dealings with the professional. I'm going to move to Julie now who's coming from the languages perspective and handle with yourself Julie thank you. Hello everyone. Thanks for the invitation to be here today and I'm enjoying the discussion so far. I was very interested there to hear some of the things that Elaine had to say I hadn't realized that language learning and language assessment could have so much in common with tax, but actually it does. Because in the same way that Elaine explained that her subject is very much a kind of block building process. So is language. There is very little that we can assess after two or three weeks of language learning, particularly when it's from the perspective of beginners in my own discipline, French. So we really do rely on end of semester and even more specifically end of year assessment and examination when it comes to languages because it's really the only real kind of meaningful point when you get a proper glimpse and a proper idea of the students capabilities in the languages and the breadth of the learning that has taken place over the course of the year. So, it might be worth just mentioning that when it when it comes to languages and what we tend to do, well certainly in minutes anyway, the way that our program is is is composed is that there's a certain amount of culture and literature, and then the rest of it tends to be language so written language listening and spoken language. Now things like content. So literature, politics, culture, film that lends itself a lot more easily to continuous assessment to group work and to online activities. But when it comes to the actual bread and butter of the language learning and the language assessment. That's that's a lot more difficult and the languages, I would say have really struggled in the transition to the online environment with with the pandemic. First of all, in the initial months of the pandemic. We'd already had a couple of months of having our students in class. So we had a good sense of their level, and then all of a sudden then we moved online and it became very, very clear to us that our students were no longer doing their own work that the assignments that were being handed up, were so proficient that you know even a francophone would have struggled to put together such a high level of French. So we really struggled with, you know, how do do we make sure that the work that we are now getting online, which in the classroom would have been able to monitor more carefully. How do we, you know, be sure that this is actually the work that is being done by our own students and the same thing occurred then in our end of year terminal exams. We struggled because because cameras hadn't been used a lot during the teaching part of the semester, and this was not due to a reluctance on lecturers parts to use cameras but students who would express discomfort with cameras or say that they went in the right location, or they didn't have access to a camera and we had to accept all of those things. What it meant that was on the day of the oral, we didn't actually even know if the person who was doing the oral was the student who was supposed to be doing the oral. You know, in a couple of cases, it became quite apparent that it wasn't the student. It was, you know, a francophone or someone with more expertise in the language you've been brought in to complete the oral examination. In relation to our language exams, we also struggled with the access to online translation engines. We weren't able to control any of those aspects in relation to the final assessment final exams and yet we felt that, you know, we did have to go with final assessment approaches because like Elaine's subject, we're also bound to the European framework. We're bound to the teaching council, a lot of our students will go on to train to become secondary school teachers and they require a certain amount of final, end of year assessment or, sorry, examination I should say. So we had to fulfill those criteria. I know that I would say all happened the first time around last May. We've now had another go at it and we've been able to come up with a few creative solutions. So in relation to our written language exams to make sure that the exam that's being done online isn't being put into a translation engine. We've come up with things like, you know, we'll ask the students to, in their essay, give us five examples of the use of a particular tense or the use of a particular preposition or try to integrate a certain number of phrases that we will provide in advance into the essay. And that makes it much more difficult to just put the whole thing into a translation engine. It means that the students have to take a certain amount of responsibility for the production of their own French. So that has helped a little bit in terms of the way that we were examining language at the end of the semester. It's still not ideal. In terms of orals, we've just, we're just in the process of completing our orals. And again, we're still struggling with them. We're not 100% sure if all the students who turned up are the students here actually enrolled. We've, this time round, tried to get them to show their ID cards, but a lot of them will hold them so far back from the screen that it's impossible to actually look at the photograph and compare the photograph with the person that you're looking at on the screen. And if you haven't seen them all semester, then, then you don't know what they look like. And yet we do need to assess them in an oral exam because the way that languages work. We need to know that they can react and respond and speak spontaneously in a language. So, you know, you might think that, well, why don't you just get them to do a video or some other kind of, you know, assignment. It doesn't really work because that's prepared language, and we need to be able to see that they can speak the language that we're teaching in a spontaneous situation where the questions aren't predictable and that they can react, you know, in and in the moment. So those are just a brief summary of the challenges that we've encountered in the languages. And I hope I've also kind of given you a bit of insight into why we kind of do need end of end of year examination in the languages. Thank you very much, Julie. And it's again interesting to hear what the challenges and I suppose the innovations that you put in place in relation to assessment design and noting comes with the workload that takes and the time and so on. I'm going to hand over next to Peter Cullen of QQI. And I know I just want to commend again and the report that was done by QQI. Thank you, Karina's in on the call as well. And I suppose to commend you as well on the dialogue you've been having with with the special registry and statutory bodies so over to you and I'll thank you Peter. Thank you very much. It's a pleasure to be here. I'm addressing a terminal exams today like everybody else but I want to take a step back first and say a little bit about assessment in general because I think it's important to be particularly when you're thinking about changing or innovating and to determine how best to assess in any given situation I think you need to consider who you were trying to assess what you were trying to assess and why you were trying to assess it. I'm talking about assessment but I'm thinking of what I mean is the assessment of the influence of a person's knowledge skill or competence compared to the standard or objective. And it's important to have that standard in mind so people often forget about it. There are multiple reasons why we assess and the conduct of each assessment can serve multiple purposes in one career to them. That is the element in programs that students focus on and stress over most for students assessment defines the day factor curriculum. When it comes to innovation and change in assessment. I wonder do teaching staff have sufficient confidence to innovate. If they don't, what are the barriers that they're facing. We've heard some of them today already to be clear about QQ is expectations. I think providers and their staff to assess imaginatively and individually. As far as we're concerned, an assessment need only be fit for purpose. In other words, valid and reliable to be acceptable from a quality of qualifications recognition perspective. Assessment rigor does not come from being conservative assessing your students the way you were assessed is not necessarily a good thing to do. Things are changing far too quickly for that to be a credible approach anymore. Rigger comes from taking a systematic approach based on the sound understanding assessment. Imagination and innovation required effort. But there can be a substantial benefit for both teachers and learners and students in spending that effort. The 2019 crisis stimulated a lot of fresh thinking on teaching learning and assessment to QQ I would invert would merge all involved to keep the momentum. We need to grow the scholarship with science and assessment. We understand, for example, the impact of assessment on students and what they learn. We understand the impact of the systems and procedures on assessment, how they can sometimes unconsciously constrain what can be done. Should PSRBs be prescribing assessment techniques as distinct from criteria relating to the validity and reliability of assessment that is used. If we engage students sufficiently in the design of assessment, how can we maintain academic integrity in the context of new kinds of assessment. The experience we do with some of the traditional. Providers need to support the development of staff competence in imaginative and innovative assessment. They need to be realistic about the staff time involved in quality assessment. It's not just about developing individuals, but also teams, designing and implementing an assessment strategy for a whole program is even more challenging than designing one for a module. And it's important that there are opportunities for program teaching staff to collaborate on this. We are considering our concerns terminal examinations and at the undergraduate levels, term and examinations have traditionally been dominated by mass closed book on scene examinations conducted. In reality there are an infinite number of approaches to terminal examinations. However, I think providers have perhaps become over aligned on the in person pen paper examinations and too complacent about their shortcomings. As a universal solution. There can be challenges with some of the alternatives and the academic issues which are serious. None of these are insurmountable. Traditional written examinations are problematic in another level. The pen paper variety. They are unduly did anything in the context of modern life. Few people spend much of the time using pen paper to the extent that they didn't last century. They've got to have some tablets now, only to write to run applications and process data that contemporary interface for work is digital and examinations need to reflect that. And once you move to digital, a whole world of assessment possibilities opens up. Of course it must change what you assess, but most of the changes I suspect would be better. If you want to know what a person knows at the end of a course, a terminal assessment is a natural place to start, but it is by no means the only only approach that you could take continual assessment can be used to complement this. Some caution is required here as well. Sometimes continuous assessment is used as a tool to keep students engaged. However, overburdening students with continuous assessment can stifle thinking and creativity. Graded continuous assessment can discourage learners from experimenting for fear of penalized for failure while learning. Graded continuous assessment can be done well, but if it's done in a piecemeal and fragmented way it is unlikely to be capable of being valid for assessing program or modular learning outcomes. I'd like to finish up by stating that claiming that it is important not to be nice or sanctify any one form of assessment. I think that technique is a perfect solution to it in at least one scenario. If you want to test the students ability to think of their feet for example there's no point in assigning an essay to rise at home and come back a week. It goes back to my opening sentence, the approach depends on who you're assessing, what you're assessing and what you're assessing. I'm just saying that we think this conversation is really important and we're really interested in continuing to engage with stakeholders in further these discussions. Thank you very much Peter. And definitely the who and the why and the what are very key factors to be looking at. So just that we are looking today at the perspectives kind of and we have an episode two of this series coming up later and we'll be looking then more at the issue of alternative assessment formats to the terminal exam, but just very to round out perspectives today. Our final speaker is Dara Ryder CEO of the head so Dara over to you as many thanks. Thanks very much on it. Yeah, so have a figure on the screen here 86% when we think about that in the context of assessment we think first class honors but there's another 86% that I want to draw your attention to. 86% of students with disabilities receive exam accommodations in order to navigate the exam process. And so these would I suppose have a range of different accommodations which would come from extra time in exams they might be used for reader or a scribe or an alternative venue all sorts of different reasons for some students with disabilities for example thousand the DCD dyspraxia category. That's an accommodation for 19 in every 20 of them. Now for me that's a symbol that something is wrong. Can I just interrupt you there. Sure. If the desktop is sharing my apologies. Oh, sorry. Yeah, sure. No problem. That's okay. I'll stop that sharing. Try it again. Is that okay? Can everyone see that? Yes, perfect. Thank you very much. No problem. So for some students with disabilities and I mentioned that's 19 in every 20 some categories. 19 in 20 a head data shows that that was over 13,000 students who received an exam accommodation. Now what's happening in these instances is that we have every student getting them having one change one time. In the exam process for each module every year and then we're repeating that process again for the next batch of students who come in. So to pose that question really given that the explosion of students with disabilities and a range of other diversities that are coming into our higher education system is a time that we look more focused at the barriers that were right within the assessments or we're going to continue to locate those barriers within the individuals and instead put up ladders for those barriers to navigate the process. So I guess what I'm asking is should we continue to put up ladders or we should continue to look at smashing down the wall in the first place. Next up I'm just going to play some some tarts on assessment from students with disabilities themselves. I'm not a fan of the exam process. When we almost go back to exams or we have some sort of written exam we all kind of forget. I find having a one-off exam that really gives you the final grade of your module doesn't really show you as a complete student. It only shows you in this little two-hour slot where you've been pressurized and you've narrowed down to these little topics. It doesn't really show you the skills that you've learned throughout the module. I think from an academic point of view it makes much more sense to write two good essays with the sources and they're getting somewhat of a peer review rather than how much can you write in the time we've allotted to you in a cold draft gym. Like for me being dyspraxic I can't write that fast let alone think that fast you know. So I prefer having continuous assessments in general. Throughout the year we get two assessments so they add up to 60% of course so you can pass the course without actually going to the end of your exam which I love because it means you don't have to wait for one day where do you pass or fail one test. I think if students were to be offered an alternative to maybe a choice between two different types of assessments I think it would work for them on a personal level like it would suit them better than being forced into an assessment that they're not comfortable with. Everyone does not learn the same, everyone doesn't come across the same even in an essay. You know if you're not particularly good, if you don't have a writing style you're not going to come across the same in an essay as you would in some other forms. Okay I'm actually going to kill that because it's time but there's more lovely stuff in there I'll post it in the chat maybe later. So what I want you to quickly do is imagine your greatest learning challenge, maybe your greatest learning weakness even the thing that you find most difficult is a learning skill. You can post that into the chat there so it could be your writing exams, your planning, your organization. And I'm going to give you the flip side of academic integrity, you've talked a lot about it today. I want to give you this example of Billy who has dyslexia, he has very high IQ but he has the jagged learning profile and that means his reading and writing skills are not as strong as others. He's actually in the 8% according to his reports which mean that 82% of the population read and write more proficiently than him. And the course downing outcomes he's taken have nothing to do with writing or writing speed within there. So what I'm asking is, is it a fair deal for Billy that we're asking him to demonstrate what he knows to a tree hour written assessment, even with some of the small accommodations that we might be able to add on as piecemeal. So, the academic integrity question is, are we holding up our end of the bargain as institutions so we often talk about put that burden of academic integrity always on the student, but are we holding up our end of the bargain as institutions as well. And the final question is, does your final exam have disabilities. And some of the takeaways I'd like from the video and everything else that's been said today is the importance of choice in assessment and I think, for me, if we want to maintain our end of the bargain in terms of integrity and recognize the variability that now exists in the higher education population, we need to provide different pathways for people to demonstrate their learning outcomes. And obviously that provides many administrative challenges, but I think we need to rise and meet that challenge. There's a much more student partnership in design, if we're going to meet the needs of those diverse learners in our cohort. So we need to understand the different challenges much more clearly that people are facing and not always try to just add ladders on but actually address the issues, true design of the assessment process itself. And the need for more assessment for and as learning as we go on. So if there is that, and that final terminal exam and that isn't a necessity that we're making sure that that's not an autopsy for some students that is more of a health check, you know, there is health checks along the way. So they understand where they're going wrong and how can they, they can actually address them. So yeah, that's me. I think I managed to make my five minutes. Thank you very, very much. And we've got a little bit of time for some questions and comments that are coming up in the chat. And so I suppose one question that's coming up in the chat I noticed and I'm going to throw it open to the speakers is coming up and I'm going to be on the whole registry piece, but this whole idea about how to discipline and actually, then the need to provide maybe a mix, and you're going as far as choice Darah so I think that brings in maybe the role of the program team. And that would include student representatives and same so I would just like to throw it open to our speakers. Any other thoughts on maybe what a program team can do going forward we're seeing lots of nice ideas come up like cross module assessment and so on in looking at the assessment across a program and what the opportunities there are and indeed any challenges. Thank you so over to yourselves. Thank you, speakers. Any thoughts. Maybe. Maybe just just, first of all, the. It's important that there's a strategic approach to a system. In other words, there's an overall overarching plan. Once you have that ability to make trade offs to to assess learning outcomes. In the context of the program other than necessarily in the context of the module where they were achieved and that can benefits to to integrate learning and more efficient use of people's time. In order to do that. You need a program with with elements in it. To avoid other programs and so there is a certain tension between taking a strategic approach to assessment and having extreme implementations of maturation where people can make up their program based on building blocks as a broader units. That's something that simply has to be worked through. I think it's also very important in to create space for the teachers involved in a program to talk to you to talk to one another. I think it's been time. Not only developing that program assessment strategy I was talking about earlier on, but implementing it, because, you know, strategies always change when you're confronted with realities, and after the operation. Really, I mean, I completely understand the disciplinary approach I think it's, it's absolutely essential to recognize the different disciplines and completely different needs. Well, on the other hand, there can be conservatism there can be. We do it this way because this is the way we were assessed from students, and it is important to question whether it's really your discipline that needs this approach to assessment or you just assume that you just need to be surprised. I think people would be surprised how different you can do things if you really want to. Finish off by saying that I think choice is a great idea. It comes with a whole lot of challenges in terms of, you know, if you try to assess, you have to figure out how to fail your rank students if you need to rank them. But, you know, these things are prices worth paying. Thank you very much. I'll quickly say something there on your paper. There is so many challenges with the provision of choice and, you know, but there is actually a lot of great work and it's actually a lot of it's located in Ireland. I think Gerald in on it was actually in the realm here is done a fantastic amount of work. Also recently published a paper and follow people at least upon the bill, you know, and it's sort of evidenced way on how you can provide equity within those choices for students and ensure integrity. So I think there is great work and it's actually a lot of it's coming out of Ireland, which is lovely to see as well. And in terms of the administrative challenges, there is huge administrative challenges here will provide in choice. But there's also massive administrative burden that's coming on with the institutions right now. I mentioned those 13,000 plus students. Well, that's those 13,000 of them that have to be accommodated every time in every module. So it's absolutely massive and the amount of different implications that has. There is a challenges either way we go about things and it's just about where we want our focus as institutions to provide quality integrity and maintenance of the variability that we have in our institutions. Thank you very much. And if Geraldine is still in the room, Geraldine might invite you just to kind of speak you flagged that study that was done so but maybe just first, Kevin, any thoughts there and how students might contribute to that dialogue further fleshing up from your talk earlier at a program level. Yeah, at the program level I think there's a key role for staff, student committees and program boards to play in this and for the student representation system. And I think going back to some of the points that both Dara and Peter Breeze, there is a need to consider the program in the round and looking at where there is that crossover of learning outcomes are being assessed twice or maybe three times. So I think another key one is assessment deadlines because often you will find that if students study in three modules they may end up with three major pieces of continuous assessment due within a couple of days of each other. And that obviously then puts a huge pressure on the students in terms of their ability to perform effectively and to carry out all of those to the highest standard. So I think that overall holistic view of program and I get that that can be particularly challenging where students have a lot of options around modules, but there definitely is a need to consider that in the round. And as you've already alluded to on the ensuring that students are part of that conversation too. Thank you very much, Geraldine sorry to put you on the spot there but you had flagged there the work done so maybe just a quick. Sure. Yeah, can you hear me okay. I don't think I share my video but as long as I can hear me. Yeah, yeah, there was a couple of things there's and Dara has mentioned it there that the choice of assessment is really one great way forward because that allows people to to take small steps in a way that they can slowly move away from the exam. I think there is one thing about, you know, moving away from the exam but it's not throwing the baby out with the bathwater because you don't want to disadvantage all those students who do well in the exam. So I think there is something here about having a more balanced approach to the exam and certainly it's not about throwing it away completely but the choice of it because as I say some students have some do well in it so I think it is about coming the other piece and I know Lisa's in the room here as well that Lisa Padden and myself did in UCD when we did a piece of work quite recently around diversifying assessment. One of the things that we found was that you don't have to diversify a lot you can just do small diversifications and have big impact. And I think there is something about equally not throwing loads and loads of new stuff at students and they become overwhelmed by the newness so I think there is this is where the program approach that's been mentioned by many of the speakers is really key that it's not coming at it and throwing everything at the students all this newness because there's a lot of work to say that too much diversification, particularly for students who are already struggling or maybe international students, you know, there's enough diversification. So I think it's about having a balanced small incremental step to this change. I'd say that would be my message and the program approach and the choices are two key parts in this. Thanks Caroline and I thought Julie's example with the French spoke very well to that and you know and I see Elaine you're not in your head so Elaine, Julie any final observations on that incremental approach at discipline level in the context of the program. You're on. Yeah, yeah, I was just thinking you know because when you try when you think about, you know, all the changes that you could make. When it comes overwhelming, and you think how would I ever start to make all my modules and all my assessment and all my programs, you know, inclusive, and the best that they can be. So it is, I think, more useful to think in the way that Geraldine has just mentioned about the small steps and you know what you can do just bit by bit and then it's less overwhelming for for both lecturer and and also students as well. And even, even though you're operating on a small kind of scale, those changes will be beneficial and add up and lead to, to something better. Further down the line it's, it's when you just look at the big picture and how much work you need to do. Sometimes that can be, think that can be that can be a bit overwhelming particularly given the conditions of the last year where it's just felt like we've been in a permanent fire fighting. Thank you Julie Lane any final thoughts on that from your own perspective. I think the program approach and somebody I think mentioned in the chat also, and trying to maybe assess across modules which I know we've done a little bit in tax so some sometimes we're in danger of the students compartmentalize the learning and they go that's that module I you know I figured that out and you know that you know I can't possibly mention VAT in the same, you know, pumping as income tax so we've done that a little bit in tax where we might give a case study and it'll try and bring all the bits together for one client. And it does require a bit of coordination who writes what who grades what but I think those kind of things are important because I definitely think we're over assessing students and not maybe coordinating and we do try to program level to make sure using our class reps to say look tell us if there's pressure points throughout the system and we can just we can, you know, you don't want to have all your midterm exams in week six or whatever. So those things again are small but I think it's always useful I think to to make good friends with the class rep because you know you get that kind of communication going and you're not just listening to one student who has a problem on one thing and keeps pestering it you're getting a class view so I think it's useful on course boards to have the student voice in there and because I think working together I think what's really the highlight I think in the in the course of the pandemic as well as students do appreciate all the efforts that we're making you know so I think you know involving them in that dialogue around assessment at a program level is really useful as well yeah. Thank you very much Elaine yes and there's a wealth of info coming up in the chat colleagues so it's been suggested that we will draw that together after the meeting and make it available lots of useful resources and links. But just again, thank you to our fantastic speakers and thank you all for your inputs coming from your various perspectives. Thanks to everyone for their contributions in the chat and in the Google Doc, and I'm going to hand back now to Terry, who will just give you a few further announcements about further events before we bring this session to a close thanks Terry. Thank you really and thank you to all our speakers and most importantly thanks to all our participants I've never seen such a busy chat during your webinar, and we will pull all that together. And I suppose from my point of view the words that I remember I think are we putting out ladders are we breaking down walls I think it's important that we decide what we're going to do. I think the word partnership is was a very strong theme. I think that balanced approach and that caution to take small incremental steps don't try and do it all together. And they're, I think, very good starting points. And as I said at the beginning this is our first of a two part series or our next webinars on May the 28th at the same time I hope you can all join us. And Oya has agreed to chair the session as well we'll have some other good speakers but most importantly hopefully we have a lot more conversation around this what's a really important topic. So thank you all very much. And, and I hopefully look forward to seeing you on the 28th at the same time. Thank you again. Mary and thank you to you and all of the teams who are excellent working this slowness. Thank you. Bye. Thanks everyone.