 Well hello and welcome to this the next I suppose the next webinar in our series as you know we ran a number of webinars and over the last academic year on work based assessment it's a topic that I don't think we can actually talk about too much because it's a tricky topic it's something that we need to to keep talking about to be able to find ways forward with some of the issues that are are facing us just to let you know that the webinar is being recorded and we will make it available and after once we've got it downloaded and that would make it available in the national form website. And on behalf of the national form and our partners in this in running this webinar QQI we really thank you for your support and thank you for your interest in the topic. There's going to be opportunities to talk. So don't worry during the webinar, just to remind you the webinar is going to extend from 12 until one party. And so that you have that kind of timeframe if there's, if you have commitments after one o'clock and you need to go we completely understand. I'm going, I am delighted actually that I don't have much to do in this webinar except to hand over to the person who's going to facilitate it. And I'm also delighted that Geraldine O'Neill, who is going to facilitate it is going to be talking about some initial findings from the research she's doing as one of the national forums inaugural teaching and learning research fellows. So I'm going to hand over to you. Great. That's great. Thank you very much, Terry. I'm delighted to be here. And I already see people in the room that have been part of the research so great to see them. And so I'm going to share my screen. And I will get started on the presentation. For those who, for those who know me, I'm, I'm, or don't know me I suppose I'm Jordan O'Neill and I work in UCD teaching and learning but as Terry says I'm one of the national forum teaching and learning research fellows. And this research is coming from from that from that work. So I'm delighted to be able to present some interim the results the the data is still being analyzed further and will be presented later on this year. So I'm very excited about that and also we'll be publishing it but we thought we give you a bit of a taster, particularly as those some of those who've actually been part of it it's great that they can see some some of the stuff coming out. And so some of the stuff that we saw coming out of the work was things around competencies, expectations, and learning contracts and work based assessment and we really thought it's useful to see how these might be related. And we felt it was an useful opportunity to look at these kind of some of these key themes that are coming through. And I suppose some of the challenges in assessing work integrated learning are the first one of first is the key challenge is expectations of what is to be assessed. We're going out in very different placements. So what are the different expectations can be a challenge, the diverse learning opportunities by the placement or experience for for these competencies can be a challenge because they go to different places and it's not all the same, and that's the strength of it, I suppose. On top of that you've got the unique learning goals of the students, so students differ their experience differs, and they're going to these diverse places so there's a lot of diversity. And yet we're trying to assess in this context so it's quite a challenge in relation to these three things. In particular is presenting as I say my National Forum Fellowship Research and assigning some of the initial findings and really what we want to try and do in this session is to understand and discuss. That's why we do want you to be part of the discussion and these three challenges, and we're trying to do it in the context of the higher education landscape, which is a very challenging landscape in itself. So what we thought was very useful is Sue Huckett from QQI, you're just going to give a quick overview of some of the assessment challenges in the higher education landscape. So Sue I'll hand over to you for looking at some of the assessment landscape issues. Thanks Geraldine and hi everybody. It's great to be here. And it's, I know it's going to be so interesting to hear about Geraldine's fantastic work that she's been doing, and some of the findings. I'm just here, I'm here actually on behalf of Peter who could Peter Cullen and my colleague couldn't be here this morning, just to give you just to give us an opportunity over the next few minutes to step back and look at the perhaps the assessment landscape in in all its complexities reflecting QQIs remit in relation to assessment. So if you may recall since the publication of the assessment green paper, the publication of the report on the stakeholder responses to this to this screen paper. This year, earlier on we held a number of one to one interviews with key stakeholders focusing on rethinking assessment that was the kind of overarching topic asking them what they would like to see, ideally addressed assessment, and what were the current issues as they saw them. So this graphic really summarizes the issues that were raised and the related considerations under each heading, which came up and of course we could have gone on and on and on, putting more and more issues underneath each one but you know, I hope that at least captures the main, the main points. As you can see, it's extremely wide ranging and of course complex from that high level of distilling confidence in our qualifications and the many reasons why this is so critically important to identifying all the stakeholders who need to be need to have an active and collaborative role in rethinking assessment developing it and the practice using it as well. The need for students as partners in the co-creation of assessment approaches, academic integrity and all that involves and I'm sure you've heard a lot more about that in other webinars, etc. Thank you so much for your time and indeed the critical role of change leadership. Now, those are the ones I've picked out there are other things there as well, but obviously you can have a look at these in your, at your leisure later. Look at this slide and indeed if you have any questions, please let, let us know. So, what do we know and what do we need to rethink. And these are some really obvious headings. I know you probably can look at them and go for goodness sake, we all know that. But of course it's what is under the iceberg, what is under the water, and what do they really mean. So many people brought up, you know, assessment has been so critically important. It provides a purpose and a focus for teaching and learning. Of course we know that. But, but equally, that assessment is not some ivory tower sort of experience or activity, which takes place discreetly. It must be planned, designed, developed, fought through in the same way that teaching and learning is fought through as well, and in an integrated way. We all know that probably in theory, but the real challenge is how much we have managed to realize this to date in practice and what further do we need to do. I've already mentioned confidence in our qualifications this was a very common comment from from stakeholders. And indeed came from our interviews with the department as well. So confidence and the quality of our provision on our qualification system, obviously rest on assessment which is valid consistent, aligned to intended learning outcomes that was a key issue that was coming up, reflects the target community needs, reflect as Geraldine referred to their student goals, and upholds our national reputation, not least. Trust in the education system is sort of related to two, so I won't say too much more about that that was again talked about again and again, valid assessment is a core part of that needs to be consistent there, equitable, reliable, all those all those things we know about we talk about, but have we actually managed to incorporate it into our practice, and then concerns around the impact of assessment. When we see, for example, particularly during the COVID period the rise in online cheating of various types from perhaps minor to really serious the more contract cheating end of things. We understand the pressure students are under to to manage their assessment and the law of misconduct. So, how are we addressing that how are we really helping students to avoid that sort of thing, and how aware are we of the impacts of assessment in the long term, and in its sort of life changing potential for those students. That's really a heavy responsibility. So some key check key challenges staff expertise the need for staff training was raised again and again, also underlining that it's not some kind of ivory tower science. We need also for learner training, including learner orientation around infrastructure institutional infrastructures which need to enable assessment which is responsive to its context context and maintains validity and again this was a very common issue that was raised, which was, well, it's very difficult to our infrastructure is very plunky. It takes a long time to make any changes in terms of assessment and we really have to convince so many people, etc, etc. So the programs and learning outcomes, particularly as I just said the alignment of learning outcomes to assessment and, interestingly, this issue around modules and programs that was a certain sense from the interviewees that the program has disappeared, or is disappearing when we focus so much on the modular system. In terms of academic integrity I've already talked about that I'll, I'll skim over that and indeed that we have a lot of material on that if you'd like to learn more digital assessment, and how we are articulating that, rather than just transposing face face assessment into the linear, again, that's a whole issue, and QQI has some separate work going on around e-proctoring and we will have a report being published in late October on that. And finally the assessment of competencies and I know Geraldine's talk is going to talk more on that. I suppose just as a final word to say that a central component of QQI's new strategy and the focus in this strategy is to continue to take a lead on assessment in relation to QQI's remit around quality assurance and qualifications. And we fully recognise that there's need for a close scrutiny of this key area and support that we need to provide for all our stakeholders. And of course we will continue to liaise and collaborate with key partners such as the, such as the National Forum and it has really been and is such a pleasure to be able to work with them on this work-based assessment initiative. Thanks Geraldine. Okay, thank you. Thank you very much Sue for that. That's great to get the kind of the broader background about what's happening in assessment, both from the, I suppose the enhancements and the quality of the QQI kind of their remit in the assessment system. So thank you for that Sue. I suppose moving specifically then more on to assessment of work-integrated learning. Back last year, it seems like three years ago, back at the beginning of last year, we ran a series of webinars and we asked people what was the challenges that you had, what did they value in work-based assessment. Different themes came through, but what really key theme that came through was the thought that assessment should be real-life meaningful assessment in the work-based. So really important that it makes sense, it's very linked with sort of what is seen as authentic, where it was used, but it also needs to be consistent and Sue alluded to that, that you need to be able to stand over. That means to be trustworthy and it needs to do these two things. So the research then that I decided to do was trying to look at both of these, how can we do these both well and how do we optimize these. So that was the focus of my research project from the National Forum, funded by the National Forum. So to tell you a little bit about the research. And again, if people are in the room are happy to share, I won't say that they were involved, please do. I won't name them because I said there was been honest, but you can name yourselves if you'd like to. But there was two big methodologies in this. The first was I interviewed some key experts in the field about, you know, to interrogate this is quite a complicated thing. How do we be consistent and authentic? How do we get both of these? So I had some really interesting conversations with people from around the globe. And there's a lot of work going on in Australia. Those who know this field well and Canada is a lot of work going on, but different stuff has gone on in Europe, so the Netherlands, Germany and South Africa. And so a lot of the research is coming from these areas. So this is why I went with these because they were key authors, even though there's lots going on in many different countries. So today I interviewed seven authors, so that's part of the research. And then I did nine, what I called solution focus workshops with different disciplines across Ireland. And they were in three kind of broad areas. The first group were on campus and this is work integrated learning also includes that type of work that goes on in campus, like project work and problem based learning that links very much with industry. So I included that in my research, although I'm not going to talk to that as much today possibly, but I certainly included that group. The other groups that I included were those that are in placement out on practice, but are primarily assessed by the educator. So these disciplines they may go out and they may do sort of work experience, but they come up back to the institution they write up a report or they do a presentation. So that's that category context be so educator assessed primarily. And then there's another group that are primarily assessed by practitioners. So these are people like in the healthcare they might be physios nurses and healthcare workers social social work. But they are very much assessed by practitioners could be industry assessing them could be teachers assessing them so they are the sort of the sort of two broad groups of placement. I had about 120 participants, but what was really nice was there was an equal mixture of students practitioners and higher education staff in these in these groups across across the different disciplines. So the different disciplines that were involved the blue beings that are more on campus was civil engineering and diagnostic radiography. And because more project based public based learning types ones, and then the light green and the dark green were more on placement so there was hospitality Institute of Technology, survey and construction management a technological university, teaching was university business information systems a technological university occupational therapy was university and veterinary nursing was university and they were assessed by practitioners those those those two there. So they were the three that sort of the nine types of disciplines. I'd like to get a feel if I can in the room for in your context, if you are currently doing it. Are you primary is your placements and these are placements out of practice, primarily assessed by the educator, or primarily assessed by the practitioner. I know there's often a mixture but where's the waiting. And if none of these maybe you could just put other just to get a bit of a feel for the people in the room. So Colin here has shared the poll and give you a minute or two to have a go at the poll, just get a feel for those in the room. If you get a chance in the chat. Is there any any rational for either including or excluding the practitioner is there a reason why you may for example I'm looking here slowly but I'll share the results now in a second, whatever is the chance. It's leaning more to being assessed in the institutions and education is there a reason why you're not sticking in the chat actually just to get a feel for is there any reasons is it the way it's always set up. There's no rationale why you may not include practitioners. Okay, so 5050 okay Catherine you're in the middle so 5050 okay mixture. Yes a lot of people mixture. I'll share the poll I think now Colin actually why people are putting stuff in the chat. So, looking at this that the mainly by the educator and institution but about 30% are in practice and others are known. Practitioners to the whole assessment that's Maggie. Yeah. Okay, thank you for that for an. Thanks, Leslie, hi, Leslie academic guidelines with academic online modules that run concurrently for placing with the perceptor determines whether competent or not okay so that's. And again, there's different languages. perceptor is a practitioner, some people use perceptors sometimes it's industry sometimes employer different words for the practitioner practitioner may not at the time, or not wish to make assessment decisions Jonathan thank you for that that but the lot of literature as well that the practitioners don't particularly want to assess sometimes. And they don't feel it rather not assess it's coming through quite a bit literature. Practitioners email are directly observing competence and become provided with training in advance yes training coming through. Yeah, regular expects clinical assessment competence yeah Caroline yeah so sometimes professional bodies regulators. So when that that's not there that sometimes, maybe people don't practice or want to assess. And then clear placement assess by practitioner to meet competency based standards yet the regulatory bodies yeah. Okay, great. You can stop that sharing column or just close the poll but that's that's an interesting. Okay, thank you for that and keep them coming through. Mary Rose practitioners pass the students we assess the written report according to our standards that's a mixture. Yeah. And practice is not always trained. Thank you for that Trish Mary Rose thank you for those. Okay, so just moving on to tell you a little bit about what we did and actually the methodology itself, people might be interested in, because it was a it's called participatory learning and action methodology very much about discussion. With different stakeholders. So I say those students practitioners, higher education staff in these workshops, and they were very much mixed in all the breakout groups as well. And just the methodology we did was all online in code times. So all done on zoom for three and a half hour the workshops were polls and jam boards breakout groups for discussion, coming back to a planary, voting on the key challenges, coming up with some solutions and coming up with some potential actions. So hopefully all that were involved actually got it up from actually being part of the session, but it's methodology that's really does push interest stakeholder dialogue so something you might be interested in yourselves. So we have qualitative analysis done, which I won't bore you with here but it was done and it's been further done and some key actions that are coming through some key themes, and these were actions that people were thinking about. The first is the importance of clarification of the expectations for all the stakeholders was a really key thing coming through the many many mentioned lack of clarity and expectations we don't know what to expect. All the stakeholders, including the students, the appropriate sensitivity of the assessment form, not having too many competencies this was coming through quite a bit in the literature and from the attendees at this workshop that the assessment form too many competencies on it and come back to that valuing and recognizing the role of the practitioner that coming through, certainly in some particular disciplines this was coming through quite strongly some of the newer disciplines. Exploring the appropriate grading sales that came through a lot and we look at that. Supporting the interstakeholder dialogue. So this, although the workshops are interstakeholder dialogue. They were also saying we need to do more of this. This is the first time I was ended up in a room with a practitioner and a student and myself so the higher education staff so and actually having an honest conversation. They weren't being assessed at this particular moment and the students found it got a lot of positive feedback from the students and that they felt that their voice was heard, and that they that they weren't being assessed at this moment it was an opportunity to have a dialogue around the issues around assessment. Developing student empowerment came through a lot and peer support groups students supporting students came through as an action that many of them thought about doing. Developing students ability to self monitor, or something that came through the development of learning contracts and supporting the use of authentic assessment and feedback approaches. I've got a few things coming through there. But what I thought was quite interesting and what I thought we would do today was looking at some of these things so it's a subsection of some of the findings. And these three things came through quite strongly and are quite related. So competencies was coming through quite a bit, but clarity of expectations was coming through a bit. Interstakeholder dialogue was coming through we need to do more of this and from that maybe the use of learning contracts. So I'm going to work my way through these as teams for the next little while. So starting with, I suppose competencies. This is a challenging one and I know we've talked about this before in some of our previous webinars, but competencies was coming through and I suppose companies are often assessed on the generic and or disciplinary competencies. And some people are using very sophisticated models, and this is just a model, and it's out there, but some are using very sophisticated models, others weren't using particularly sophisticated models often, often the professional bodies, the ones in that context see that have professional accreditation regulations, often happen quite sophisticated models of competencies. So there's often generic or sort of more discipline specific. And the language around this is quite challenging sometimes and people are using different language around it. Standards is a term that's used to describe which students should know and be able to do in relation to establish criteria. They're just in statements and provide the building blocks for competencies. So standards are often building blocks competencies. The word competencies is one way of describing it is how students apply and transfer the learning in new contexts and situations. So standards can be a bit more narrow, whereas competencies are allowing in new context and new situations and they encompass often multiple standards are assessed at multiple times and allow for the transfer of knowledge and skills across content areas. So that's one kind of definition of it. Another definition from the European qualification framework is competence means the ability. So it's slightly different from competencies, which are more like statements and how they're written competence is more kind of a thing you own. I suppose it's the ability to apply knowledge and personal social and methodological sense in the workplace or during learning, as well as in personal and professional development. So when you start those that are looking at this language of this can be quite confusing. I mean competencies is even used in a way that describes a kind of a journey from competency to expertise. So it's used in that way I think quite a bit nursing. So there's different ways of how it actually is used. I'd love to feed into your work. Yeah, well, happy to talk to people and just looking at some of the things in the chat. Happy to meet up with anybody afterwards to chat. Yeah. And John from your industry are involved in getting feedback for our work placements. Yeah. Great. Yeah, so yeah, thanks for that in the chat in the line the chat. What were people saying about competencies. So in my research in this national forum research, some of the challenges that came out in this were from the first one here from hospitality to wider ranges standards. This was people saying that the spectrum across hospitality where they might be in a in a big hotel versus a small coffee shop. There's so many different wide range of standards. How do we deal with these standards. There's too many of them. And there's two, they're too wide. That was one of one particular challenge. Lack of consistent opportunity to demonstrate skills, veterinary nursing mentioned that that they don't always have the same opportunity students in physiotherapy is they talked about students struggling to achieve competencies and high pressure and specialized sites. And some by the nature of the work experience struggled because it was kind of more high pressure and diagnostic radiology talked about a lot of, there's a lot to learn in a short time that was sort of the length of the placement have trying to pack things in and business information systems talk about the wrong balance between technical and soft skills that the technical skills can take a while to get the soft ones you can see earlier. But, you know, trying to get the balance right between those two is challenge. Occupational therapy mentioned the appropriate sensitivity of the assessment form too many competencies. Again, a little bit like hospitality. There's a lot there. And sometimes you can't see them all in every context. And then PE teaching pitching the task at a level that's meaningful for students. So getting a meaningful competency was something that came out of PE teacher. Alice just competence feels like it's one end of a scale, come you come you're leading a master year Alice that is something that was talked about in the literature a lot that competencies on competencies on the spectrum. And it's one way of that it is described yet. So one of the challenges that was coming out from these groups was how you, what way do you grade these, you know, in the kind of a spectrum of grading the competency, you know how do we grade them, what's the challenges around grading. And interestingly, and there was a real spectrum between the researchers that I talked to, and the discipline groups that was that it had the spectrum of that red to green on that scale there. For example, some people were pushing narrative feedback. So really just pushing feedback and the narration and description as one way of assessing. Okay. And particularly in some of the European groups were looking at this focus on what what they said she said was focus on the narrative feedback that is more meaningful and please let's get rid of these numbers in our assessment system. I think this ideal world but this is what they were talking about. These are experts writing about it. So this is one end of the spectrum. The next one is pass fail and I will be getting you to actually vote on this in a second so have a little think about these are you using which ones you're using. If you are using them and I'll get you to vote in a second pass fail. Which is similar to competent net not yet competent to this kind of binary of, you know, pass or fail. This student spoke very highly of this because it released the pressure of actual grading and that they could relax and sort of concentrate on the learning. This was what feedback on that was. And then there's kind of these ones in the middle here that are sort of bands or scale some people call it bands of proficiency. The next interesting one I think that was used in medicine is above at or below expectations from three grades. There was five point scales and seven point scales, but they aggregate interesting one about that one is they aggregated them across a number of placements so it wasn't just one number. It was aggregated across and some of the language around that was architect modules are graded so professional practice should also be valued by grades this was an argument for that. If we did our modules why aren't we grading our placements if it's all if we value placement and aggregating scores has a meaning for the outliers. So it actually even though some people didn't like the grading where there was outliers it was useful to pick students who were either, you know, maybe struggling. And then finally, there's percentages and some of the language around that is what I want to do a masters and grades matter. So that is the spectrum of thought on this. So what I thought I would do is, and again, Colin, do you mind putting up the poll and what grading approach do you use, and I'm just kind of collapsed them a little bit there to narrative binary which is pass fail competent not your competent bands or grades percentages, none of these. And again, and maybe stick again in the chat, why you use, or don't use them. Once you voted and I'll share the vote back in a minute when everybody's had a chance. Narrative feedback is coming through as a small number but it's coming through somebody's using it that's great. And requirement of the module descriptor. Thanks for that Sarah. Binary is coming through at the moment I will share the results in a second is coming through as the top at the moment. Placement is also important. Reconstruction performance and assessment. Yes, it's census but with the detailed rubric. Yeah, thanks for that Trish. Order use bands above meets blow expectations. Great. Thanks order placement is 15 credit modules. We mark the rubric news percentages pass failed due to variety. That's a strong argument for the past fail because the variety like to pick narrative. Yeah, but yeah, a lot of people would like to pick that you see that yeah. Bands and grades to forensic students and the regulatory boards are quite pass. Yeah. Great. Thank you to seeing all those coming through. Could you end the poll their economy and we'll see the results. Yeah. Yeah. So, you can see that pass failed is coming through as the most common with bands the next percentages the next and narrative is being used by by six people. Great. And some of the arguments coming through with some of the things I was saying so. I might stop that. Thank you for that but you can see it's not there isn't a right and wrong answer, but interesting the past fail is probably the most common in this group here. So, some of the solutions are supposed to some of these challenges. The, and there's some really nice stuff coming through the in the chance there people might like to look at as well. So some of the research participants suggested some solutions to these things so reduce the number of companies on the assessment form is one thing just let's collapse them a lot of people are doing that internationally as well and they should just dropping them and reducing the number because of this wide range allows students some choice of competence competencies, do they all have to do them all. And again that goes back to the hospitality I think an occupational therapy word we're saying there's so many. Could this is an opportunity for choice, develop a more elaborate competency framework for those that don't have it along the competency stronger with industry or placement needs. Allow formative assessment only on some competencies, could it be feedback for some of them, or earlier on only need time to develop don't assess too early that was coming through. I think that was coming through in the in the survey and construction management one flexibility for different competencies in different settings and levels so flexibility coming through a lot as a suggestion so flexibility choice. There's some things coming through. So the first task we wanted to do was actually get you to chatting about this for a few minutes. So we're going to get you into breakout groups. Okay. I would. I think this is about 150 in the room I think at least so it will be quite a few breakout groups. So can you take a note of the number when you're going into it. Okay, when you're actually getting into breakout group. And also you'll see it in either the top right hand corner at the top left hand corner, and when you get in but if you can notice when you're going in better still. But what I want you to talk about is what are your experience and ideas for assessing competency. Do you have a view on this like in relation to the number, the level the type. You know, do you assess them all, you know, do you have large numbers is that a problem, are they pitched at the right level do you have ones for different levels. And are there any implications for practice or policies. Okay, do you have any implications for practice and policies. So that's the question I want to put to you. I think in a few minutes column would you mind clicking instructions thank you for that column. We've put the instructions in the chat here. And when you get put into the breakout group, what I would like is if someone would make some notes, and just to take notes for the group we will have it all and on a Google share documents so we can share it back afterwards. You can avoid using people's names, but just some of your ideas around experiences implications for practice and policy, the link is there, but I would say to you, don't share the document to talk to each other's faces, but whoever is described can can actually be typing away on the link. So when you get in to skype on who might take a few notes, and they, they can click on that link. You'll have to write it in your group number so you'll have to that's why the group number is important. So, with that, I'm going to get Colin to put you into breakout groups, and you have a bad week we're going to give you 15 minutes because it does take a good time. So, what are your experience and ideas for assessing competencies and any implications for practice or your institutional policies, particularly those grading scales should we be looking at institutional grading scales part of this. So what are some implications. So, off you go Colin. Okay, thanks for that I think some people are coming in. The link there is to people still people who are typing still want to type the camera document is there. I'm just very quickly going to show some of the points that are coming through but we will tidy this up and send it back to you. Can you see that Terry can you see that screen or right can they see the document just I can see you. Yeah. Yeah. Okay, so you can see that there's some some nice conversations coming through and as I say people can type it up afterwards different ideas, former placement competencies group one there. I just got to go down to group interesting to to to group eight because I thought just this two points they brought up that actually are quite interesting from generally. The point is that employers are often generous with marking. This comes through a lot, the literature, and it's, it's a really challenging one because is that a good thing or a bad thing. And I think this does, I did see a nursing article described once as the, the, the, nearly the scourge of great inflation and placement or something like that so. When you think of it employers practitioners are working very closely with people, and they are often more generous they don't work off a normal curve I've said this before in previous webinars. Don't work off like a normal curve that we often do in higher education further education where we're trying to map people and differentiate. And so they do are often more generous and it can come down very much to sort of personality as well. You know what you think about a's and B's are particularly nice. And this is why I think the graded system is actually quite challenging because of differences in standards and different views of grading, but generally practitioners are often more generous and higher education often see that as a problem, but maybe so this really needs to be interrogated a little bit more how do we do is around more training with rubrics. Is it around going with different grading scales, is it getting them to give different types of feedback so it really is something that comes through a lot. And the other one I think that that came through in a lot of your things is too many, too many competencies. So I can't go through them all the reason why we put this in the things that you can actually have a look yourselves and what people have done. And we'll also tidy it up and share it back with you as a, as a group. So, I'm going to now go back to my slides. Okay. And go back to where I was. Okay, just a little bit more on the literature and this and what some of the researchers that I talked to what they said about this. They talked about things about having broad competency having broader competencies this gets to the idea of having not having too many but maybe broader, but equivalent projects and activities that people could do. Okay, so maybe having less loaded ones, many of you mentioned too many. So the British researchers saying maybe broader competencies but having equivalence of how they could show them some choice in what they learn. Is there some choice that came through from the researchers as well. And one a nice quote that they said, if you want everything to be the same or equivalent across students then by that, then by its nature, it can't. There's a tension between authenticity or getting sort of meaningful relevance of assessment, because it won't necessarily bend and flex to the situation and to the individual's goals and desires and expectations, and the needs of the workplace and that's the tension, the tension between standardization and authenticity. So trying to get standards and competencies that are the same and yet trying to get it relevant is a real challenge. So some of the stuff that people were talking about having broad and choice. And there's an interesting literature which I just allude to you because some of the professional bodies and regulators are looking at this but there's interesting literature around principles based regulation and rules based regulation So let's have a look at this article. And it talks about maybe we should be looking sort of certainly at the regulatory side and many of you have professional bodies and, and I know there's there's different awarding bodies that maybe looking at the regulatory you laugh at me for the principles and know I'm always a person that loved her principles as a way of actually getting some sense of, of what expectation is, but actually not having it so pinned down so have a look at that article for those that are particularly interested in that. I want to move on to expectations. This came through a lot and it is related to competencies. Expectations that are shattered. Hence the image came through by is probably the most common theme that came through when I talked to the disciplines. And here's a couple of examples I suppose other things that they were saying. Physiotherapy talked about managing expectations of all participants, the students, the staff and those on sites, the different sites different context. Hospitality talked about clear communication of academic to expectations to the workplace so the workplace knowing what the academic expectations are that that was a challenge. Guidance for employers on the requirements for student learning that came through its survey and construction management similar point to hospitality and students expectations of what they need to learn. So the students themselves being aware of what they need to learn. Communicating these expectations came through in civil engineering, what was expected the activities and then business information systems talked about students expectations of workload at the start at the placement. So these were particularly expectations so really common. So the competencies and expectations are quite related. They can be enhanced I think by two key things that also came up. That's more dialogue between the different stakeholders and possibly learning contracts as one idea that came through. And what we meant by dialogue was a little bit like of what was meant for the research that is really getting these three key broad stakeholders into the same room. This is sort of the practitioners, the students, and the institutional staff, and there's other stakeholders, policymakers and things but certainly these three key ones are key to have more dialogue what do you mean by that standard what do you mean by that competency what does that mean in this place and the students saying I don't understand what you mean by that competency I don't understand what you expect to me about competency because I haven't done it before I have done it before. So really having this kind of dialogue around what is expected is we really, this came through a lot we really need to enhance this and some of the solutions that the disciplines talked about hospitality talk to very practically about maybe this discussion with the academic and workplace mentor in advance the work placements maybe there's a piece there we really need to enhance development possibly replacement charger clearly articulating what is required from all stakeholders, and all will be involved in developing that maybe that's a potential solution. More university partnerships between education and practitioner occupational therapy we need to strengthen this partnership. I mean nursery talking as Sue was talking earlier, and some of you have mentioned, you know, this training, we need to enhance training maybe through the professional body. Is it something to enhance this. The other survey and construction management talked a bit about body systems for the students so these were maybe stakeholders between the student groups so maybe more senior students or more graduates that could be bodies to students on placements and that kind of a dialogue enhancing and business information systems talked about opportunities to hear from students employers. So a lot coming through around increasing that dialogue. And one way forward is not the only way forward, but one way forward that was coming through a lot was the idea of a negotiated learning plan, or sometimes called a learning contract. Some people were using this others weren't. And what it is is really is some sort of a plan that is sat down that that the stakeholders sit down with at the beginning of a placement and negotiate is why negotiate it's not that the students goes in and says I want to learn the student might go in and say I want to learn and say well we don't do that here, or we do it partly or maybe we could do this instead so it is a negotiation and that's a key term in it. And what the literature says about it and this is coming from Australia but not written on this is that then these types of contracts or plans whatever you want to call them, they help students be clear about what they want from the placement. They have confidence to get them thinking about their role that empowerment piece and response to students irritation that what much of it assessed and learned what much of what is learned on placement is not assessed so they're learning things and doing things that don't relate at all to the form that they're being assessed on and students get irritated. They're doing loads of this but it's not on the form you know so again it gives an opportunity to actually put that down in this sort of a learning contractor or plan. It recognized the immediacy of the work placement that was going on in the work placement and enables a more convergent set of expectations between the university employers and students. So this diversity of work placements that many people talked about, and many students who often hear say well I don't want to go in this placement because they don't do a's, whereas here they do and then again it's related to the grading but it allows for diversity of placement that came up up I think again in many of the different kind of disciplines. My colleagues those that know me know that I was an occupational therapist in my past life and my previous colleagues have kindly let me show from Trinity Occupational Therapy have kindly let me show an example of their good learning contract form that they use in their placement with the link there at the bottom for the template and the guidance so you know for those again they're happy for you to have a look at it's publicly available on their website. So this is an example of one that they use, and it does link on the left here with the competency so there might be this competency so it might link to something on that competency, but you can. You can have your learning objectives what you intend to learn from students what you what strategies and resources that might be there so well this I'd love to do teamwork but you know something. And then you have an opportunity to work in pairs so you know in this context, this is what you could because the resources are only this. What's the evidence to show that they've met it, the validation that it's happened timescale it's happened in, and did they achieve it and any feedback. And this kind of thing. That's why these things are related they work between those competencies that can be broad and sometimes too many and even if they're too broad. What do you expect on the placement expectations. So these kind of things I suppose help with that tension between expectations and professional bodies and institutionals with competencies and diversity of practice so it needs more work in this space and learning contracts is one one particular approach that that can help it. Some of the suggestions from the discipline groups business information systems talked about maybe doing a learning contract between the employer students at university would be would be useful something that they're thinking of doing need to define what the skills are will vary between disciplines. Again this is, you know, you have a competency called, you know, ethical, ethical or professionalism what does that mean in this context you know, increase responsibility for students to increase their ownership with something physiotherapy felt it would do. Again physiotherapy said each site identifies more specific needs for the student coming to them and what they can offer for the teaching and role of the students to educate. So there's a role for the student to educate the educator and what some competencies mean and give students evidence of what is achieved like self appraisal and that came through an occupational therapy so students have a role in this to sort of empowerment. There's a nice little resource that's coming through in in deacon in Australia around assessing work integrated programs, and they also mentioned the learning across contract they told us a learning agreement. And I noted that it's easier if it's not graded so if it's the past fail or not graded so but not necessarily always used in that context. So what we wanted to do again is maybe another breakout group just for 10 minutes and a short recession. Just, have you used learning contracts. So Colin if you didn't mind in the chat putting in the instructions. And again that would be helpful. He might be ahead of me and don't already but if you would put these instructions again like likewise into the chat Colin. And the question is, have you used learning contracts or something similar. And how can they or if you haven't used and other other approaches, maybe to support the stakeholder dialogue and clarification of expectations. Again, what we've done same thing you probably be in the same groups again. Maybe or maybe not we're hoping to get most in the same group again because we know each other, and we'll give you 10 minutes this time. And there's the link again so if somebody somebody again will take a few notes that will be great. And there's the link. I'd say to you again to a little look at the number you're going to go into might be exactly the same number as last time so just a little check and it certainly is on the top left hand or right hand of your screen if you don't see it. Okay, great. Thanks for that. Sorry I couldn't give you more time to chat I'm sure you probably liked even more time but just to keep on time and that I hope you had a good good conversation. Interesting stuff coming through I won't share the screen but the link is there if you want to have a look at it yourselves. You can pick on the link some really nice stuff coming through. Quite a few people are using learning contracts, or something similar the couple people have asked is this a learning contract, because there's lots of kind of maybe contracts or more formalized contracts that are negotiated. I think one question was would I call that a learning contract. I think the key thing about learning contract is there are all sorts of different sets of agreements and things that are set up a more formalized and. But I think to call it more a learning contract the student individual needs needs to be sort of addressed in it, but whether they can do it or not do it at least they're asked what what they want to learn and I think that makes it a learning contract. This is sort of the way I'm using it here. But any sort of agreements that help clarify expectations are very valuable. I say some people have said things that don't let her like letters of expectations very good idea, you know roles and responsibilities. In advance that's a really good idea. There was a question about maybe we couldn't use learning contracts because we have diverse placements but actually to be honest I would think that's the very time they're actually quite useful because of the diversity. Certainly, you know, happy to talk to people a little bit more about that if they wanted to again, but it's the very time that actually learning contracts are useful because of that diversity. Where you have these broad companies say well you know what does that mean and this this place versus this which is so diverse so. And you know maybe we could have a little look at that further if people wanted to chat to me again another time about that but that's the very time I think it is useful. Some people have had some good experience of it internationally as well and others have have kind of experiences where they do it and sometimes don't do it or they have kind of student tutor agreements if maybe different language on it but and that's fine there's no one language for this but but these kind of tools and these kind of dialogues are really, really, really key. I'm just going to move on because we're sort of coming towards the end. Just sort of a kind of a final thought, just to wrap it all together. So we've got these standards we have these competencies we have these diversity of placements, and we're trying to work in this in this challenging space that's why I think lots of opportunities for dialogue is good. And I must this this quote quite a recent quote from medical education literature and I thought it actually is quite a good one maybe to try and summarize with. And that's the that it says that the standardization of outcomes, although attractive in the accountability and Sue talked a lot about QQI and accountability and it's really important each roster system. So this accountability that it appears to offer nonetheless risks over simplifying the complex nature of competence, potentially creating a false sense of security around the capability of graduates. So this is because competence to some extent is inseparable from the context in which it is developed. It is not an immutable attribute of an individual, but rather a socially constructed notion that may reveal its fragility when context shifts. So I thought this was a really nice way of putting it back together from some of the medical aid literature. We need to do it. It's important to do it. Sue said in the beginning, we have to stand over our system. We have to, and many people have to stand to regulate everybody's and not everybody does, but you know, many do. But we do need to be very careful to not oversimplify it and have a sense that the standardization is actually making it less meaningful and less less learning experience. So I thought this was a really nice way of kind of pulling it together just for the quote from that literature. So I'll finish on that because we just a few minutes left and my time back to Terry, actually just to talk to this slide and I'd like to thank you in particular for your engagement in it and your chat and the discussion groups and I'll hand back to Terry if that's okay just to put it together. And thank you very much, Jer, and also thank you to Sue. This is a hugely interesting, complex space that we're working in. And I think our shared wisdom will actually help us try and find whatever solutions or pathways they are through this and we already saw from what happened in the breakout rooms and also what's been shared in the chat, a number of different approaches that people are taking and I think, well, I always say it anyway, we're stronger and smarter when we work together and we try and find solutions together. And so, you know, we're saying to you at the moment say, what, how else can the forum and QQI help you to try and I suppose facilitate conversations around this and what I'd like you to do perhaps is in the chat a few ideas about what our next steps might be. I can give you one that I definitely have is that the National Forum are running a week from the 8th to the 12th of November called valuing Ireland's teaching and learning. So that's part of that week. Each of our fellows is going to present in part of our scholarship hour. And Jordan will be giving more insights and more detailed insights from her research on the Wednesday session so that's from 1230 to 130 on the 12th, if anybody would actually like to join us with that and we'll, we'll, we'll publicize it and send registration links and that out. So Jordan has just showed you all the different the references that she's been making into and that's so that they'll actually be on the recording. But what I'd actually like you to do is there anything is there a big next step is there is more conversations like this are useful are these are these kind of webinars sharing what we're doing having opportunities to talk to each other. Well, if you any ideas at all, we'd appreciate them if you could put them into the chat now in terms of how we can keep these conversations going and how we can learn from each other. So with that, I just like to thank everybody and for their support today. You can, the documents that recording will be available on the National Forum channel and also to our website. We'll share with you to all the summaries of the what you put the what you put and shared into the Google Docs during the breakout rooms. And so we'll make that available as well. And if you have any thoughts at all, please send them to myself to juror or to QQI and we'll be delighted to try and and to respond to your needs. Thanks everybody.