 Fy enw i gael i gael i gael i ddefnyddio'r ddau'r deiladau cilio ac oeddwn i gaelio'r ddau'r ddau, Pwylleth Cymwyth. Fy angen, rym ni yn methu yn ffysig iawn, yn dod o cyflwyno yw ddych chi'n gwneud i'r ffordd mwrdd. Mae cwm yn ymellanodau cyffredinol, ac, os nid, rwy'n ddau'n i'ch gael ei cafodol ar y dyfodol i'w gael, amserau i gael i'r ddau'r ddau cilio a ddweithio. Mae'r first agenda item i dda i ddweud amgoeddu gwaith oherwydd y byddwyd nid o'r agenda item 4 ar gymf fort, rwy'n bryw'n mynd? Ard y cyfaint, ddweud. Felly, o'r ymarfer y panel hwnnw, Overfyn i'n gweld i'n mynd i ddysgu'r gyffin i ddweud cael ei ddweud i ddweud i ddweud yr yrhaf a nu o'w gwaith o'r lleiw sydd ymddillau datblygu Cyfaint. Mae'n ddweud â'r cyffin amddilliedig yn ddweud i ddegwyd o'r ddanion what it was like for them, and how they thought assessment could be best managed in the future. They really were a credit to themselves. I hope, but I also hope that they found it as helpful as we did. We certainly found it, provided us with a great deal of insight into what the issues were from their perspective, and anonymised summary of their views is provided in the committee's public papers this week. Our first panel of witnesses this morning on the alternative certification model in 2021 are from the Association of Directors of Education in Scotland. Joining us from Addis are Tony McDade, Executive Director of Education Resources, South Lanarkshire Council and Audrey May, head of service, children and families, Dundee City Council. I would like to welcome you both to the meeting. Do you have any initial comments that you would like to make? Maybe you have something that you want to say before we get into the questions. If you do, Tony, it looks like you do. Good morning. It was just to say that we are more than happy to take some questions this morning straight into some questions. Okay, good. And Audrey, you feel the same way. Yeah, and just thank you for the opportunity to be here, and it's lovely to hear you talking about young people first and foremost as well, because that's why we are here too. Thank you for that. Yeah, it's one of the determinations of us collectively as a committee is that we want to hear from young people, we want their voices to be heard in an unfiltered way, and last week we had a really good experience with the young people who we're speaking to. So we'll go straight into the questions. I'm going to take the convener's prerogative as usual and ask you directly some questions about your experience over the last period. I want to ask you how well supported by the Scottish Government and the SQA did you feel during the last 18 months. Tony? Thanks for that. I think that there was a genuine attempt for people to come together and collaborate. Certainly around the model of last year, the 2021 model, I was part of the national qualifications 2021 steering group and there was a working group as well. I think that that was a genuine effort to try to understand what was a pretty complex situation and a willingness to try to engage with local authorities, with teacher professional associations, school leaders Scotland and a variety of others. Young people were on the group as well. I think that that was a genuine effort to collaborate. I suppose that ultimately you have a qualifications body that are delivering a national qualifications so that there are some individual decisions that need to be made by that organising body in terms of taking those decisions and that will never be a perfect match. There will always be a scenario in which some of the things that you suggest might not actually be possible from the qualifications body perspective because they are delivering the national qualifications versus what you're looking at for your schools or your children and young people or via the local authority. I think that it's finding that balance around what an appropriate national qualification looks like that's got credibility for those young people, at the same time making sure that it's manageable for us, for teachers, for young people on the ground. I do believe that there was a genuine willingness to collaborate and there was a genuine opportunity through different engagement exercises with SQA and our teachers or central officers, but at the same time you then have to reconcile that with some of the practical things that then go on and that might sometimes work in theory but then you try to put something in place and practice and that becomes a bit more challenging. You're going to get a number of questions about the communications dimension of all of this which we really want to understand more about Audrey. Yeah, obviously support what Tony said. I think working through a pandemic there was no script. We were all finding our way and having to be adaptable and flexible and agile as things changed and it changed a lot over the period as we all know, so people really rally drowned. I felt that people were working in collaboration maybe more than even in the past to come together because we were all so concerned about what the big impact for all of our communities, our children, our families or young people, but in particular around this agenda concerned for the young people and we were hearing from our young people because I hope to be able to share with you that we were trying to listen to their voices throughout as well and so we were hearing their anxieties, their worries, their concerns and so it was really important that we worked together to make sure that we delivered the best job we could for our young people in the toughest of times and I think yeah we will come on to communication and there was regular communication I think you know in terms of Scotch Government we certainly had you know regular updates and although things might change which then kicked us into the practicalities of the operational work again maybe having to change at least we were getting very regular communication and of course we were getting updates from SQA as well at the same time and again that changed throughout the piece and so this brought stress to the system but that's a part of our job like people like Tony and I in the local authority to try and manage that and help our schools and establishments to deliver on what's coming along next during a pandemic which was new territory for all of us. It must have been quite difficult to maintain any kind of meaningful communication with the young people during this time. Teachers were struggling as well, how did you cope? Absolutely, well will we have a citywide youth voice group that we meet regularly and discuss the big issues for children, young people, growing up in Dundee, growing up in Scotland and so we have a lot of these regular communications obviously we went online we had to then do this in teams and Zoom calls to keep that communication open absolutely and we did a lot of electronic surveys and forums you know that we brought together young people in smaller groups to just hear from them and they were trying to do that back at their own schools as well. There's been a lot of criticism of the SQA, the evidence that we've received for today even, lots of criticism I mean regarding communications, regarding decision making, time when it's a decision making given the circumstances that we the country found itself in during the last 18 months do you think these criticisms are fair? I think they're people's response to how they felt at the time when things were constantly changing you know so from the perspective of someone in the system I'm sure it felt fair to say the frustration and the you know the worry that and the concerns we would have for our young people going through the system but the SQA were in the same boat as everyone else they were trying to manage this through a pandemic you know so people had to had to do the best they could with the information they had at the time and that was another changing picture and to be fair the SQA did keep in touch with our we have coordinators for all our local authorities who keep in touch with our officers so there was regular communication to and fro yeah people would feel frustrated because there was often change to what we thought was happening and then it changed but that was again part of the the world we were in at the time and I think to be fair you know we need to take that into account for all of us yeah we could all be wise after the event can't we absolutely that's a reality Tony do you have anything you want to say on that I think that that's exactly the case and also remembering maybe the sequence of it as well you know the sequence of of us moving forward with a series of national five exams that were changed in October we then have a period in December in which we changed the higher we're still anticipating coming back to school even in December we find that we're not coming back to school in January time you then have a period between January and March and I would describe that both Audrey as you've rightly pointed out as a former headteacher as am I a former secondary headteacher I would describe that January to March period as the workhorse term around senior phase particularly for young people in higher and in advanced higher and you missed that that term is about you know the kinds of assessments that you do the on-going activity for those young people and then that's taken away from you so ultimately you're trying to find a balance and you're trying and inevitably that those forms of communication there were some confusing forms of communication I think the general communication was about trying to take assessment as an example of that was genuinely trying to say you don't need to have a big hit at the end however when you get into some of the subject guidance by the nature of our qualifications you leave lots of our activity and our assessment to the end not in every subject area but a good number of our subjects the the balance of our qualifications will be that exam at the end on May the 14th you're sitting at the two hour exam inevitably trying to get a quality activity that's accurate assessment you need to be able to do the learning and teaching that comes with that if that learning and teaching if the advice says you can do general assessment and then some of the subject specific advice says it needs to replicate you know for it to be valid almost what we've done then that I feel that that can sometimes appear confusing to people if you're trying to navigate that on the ground as a class teacher or as a faculty head or indeed as a school leader we're going to come back to that okay but generally I think what Audrey's saying is she you think you think some of the criticisms a little bit after the event it was a very difficult situation that people were managing their way through it it was it was imperfect I don't think we could ever look back and say that all of that went perfectly I think that if you're genuinely in the most challenging scenario around young people and our schools and obviously from a school perspective they're looking for clarity and young people are looking for clarity at that point and I felt that lots of the decisions around young people you weren't able to give them the clarity that they needed whether that be even to study yeah because that allows them to be in control of some other decisions if that was always changing then I think that that's legitimate for young people to say when thinking back to that period were you looking for more guidance than you were getting at the time no I think sometimes within the it's the nature of the guidance isn't it we have spoke to some young people and indeed members of the Scottish Youth Parliament were on the NQ 21 group I think it felt if you were asking those young people it felt very technical guidance it felt as if the language wasn't necessarily always designed although there was a learners panel I think for young people they needed a consistency of guidance I'm not sure if you were able to do that you know I think you can apply a retrofit but I don't think that was always possible in the scenario that we were given but at the same time I think consistent guidance is what young people were asking for I don't always think that was possible and indeed our teachers were asking for that level of guidance as well so I think in terms of the complexity of that some of those messages if you're on the receiving end as Audrey said might very well look mixed to you because they were. Audrey did you want to say anything about more or less guidance I think yeah no just agree with what Tony said you know that there was a lack of clarity sometimes that's how it felt to the people receiving it it wasn't that there wasn't guidance there but it was changing and and there was lack of clarity and it wasn't reaching young people in the way that if you were in dare I say normal times you know you would be preparing young people for for what was coming next you know absolutely we would be doing all the work that we would do in our schools to help young people to prepare for exams and study support and putting in additional tuition if that's what's needed or you know giving them feedback on the work to date and then helping them to plan their study timetable all of the practical things that you help young people to do to be prepared we weren't able to do that because it changed frequently thank you deputy convener um thanks Stephen for that um the I'll try and drill down um a little bit further into some of the uh themes that were raised by Stephen there um I'm interested in the communications a little bit more so um if he could tell us a little bit more about the quality and the quantity of the communications that you received um and um cover the timescales that that sort of cut you know sort of like did you get the right information um from the right people and over what periods of time that thanks um as you look back to it I think that there um there was regular communication um I think there is no doubt that that was the case I suppose it's about timing of communication isn't it when do you know that information and how do you know that information and again there was a genuine attempt I believe through the national qualifications group although there were SQA communications to look at how what kind of communication that was into whom um there were communications that would have been to central officers from a local authority perspective to our school leaders um to teachers and to young people and indeed there were general messages out into the public as well to try to reassure people around trying to navigate that ACM and what that model might look like I think sometimes then people can feel overwhelmed by communication if it's then saying some of the same things as you've heard before well what is the clarity of that message that comes along with it and what's the importance of that message is that does that change from the last time or is that a build up from the last time or does that build on in the period before so I do believe there was regular communication I think you could always look back and look at the timing of that um I think that as a teacher you would want it as early as possible if I'm thinking about course construction the course construction the learning and teaching and assessment and what that assessment might look like you're going to try to have that as early as possible I think you're also trying to say to young people and I think schools up where we're very good at this the individual school with the individual teacher in front of their class are trying to communicate that message to those young people I think you then get layers on top of that that can add complexity to young people if it's not the same message and also what they're then hearing starts to they start to become a bit anxious understandably and I think you picked that up certainly towards that April, May, June part of the term when we came back so I think that that part was was confusing for young people and challenging for young people simply because of the scale of what was being asked of them and how they were trying to navigate what what am I doing what counts as my assessment especially when you're trying to have a if you think about it normally a consistent assessment die at the end well this can be based on your teacher judgment it doesn't need to be that at the end it can be based on some activity so that's very different and that can vary not just from school to school that can vary from subject to subject and therefore how do you get that consistent message for the young person to say don't worry it's about understanding standards for your teacher it's about being able to moderate that so that we come to a collective view of your that's your grade with your teacher but you know that that's been checked and there's a consistency about it okay thank you thank you thank you very much turning to Michael Marr who has an interesting declaration of personal interest to me yes sorry thanks convener I yet declare an interest as a councillor seven councillor on Dundee City Council who's an Audremae's employer and Audre is a former teacher of mine so there you go the more interesting end of it absolutely a very young teacher as she was certainly yes yes I'm interested in the first and foremost I think we're going to talk quite a bit about qualifications and the outcomes but the other side of that outcome I'm interested in I suppose is knowledge and what did young people learn so you know we'll talk about whether they got grades but you know it was your point 20 I think about missing the workhorse term missing an awful lot of time in school for an awful lot of people you know as we look forward for these young people what concerns do you have about what they might not know in terms of what they have missed and what they have learned it's just an excellent question I think as a teacher you know that notion is it knowledge is it skills is it ability I think we would I think you need to separate the grading and qualifications by the knowledge that young people learn because between January and March I think there was a tremendous effort I think young people have shown on the whole incredible resilience I think that young people should be commended for their engagement with their schools and how seriously when I spoke to young people in schools as Audrey has described as well and I'm not glossing over the fact that that was really difficult for them but you were able to get involved I think that this time around in terms of the engagement from teachers around the learning and teaching was much better than the first time in which you were just reacting so you were able to set whether it be via google classroom there was the national offer through eSchool as well which was really that live lesson activity there was engagement with your class teacher as I'm the part of the west partnership regional improvement collaborative I'll lead it there was the west online schools pre-recorded lessons as well I think that young people have shown us that we can learn in a different way as well you know so for me I'm not as worried about the knowledge gap because from a qualifications perspective there was adaptations made there was practical adaptations around some of the content I think the question will come maybe as you're rolling forward into this year's qualification I think you still need to make sure that those adaptations are there so that those young people I think of those 50 years just now if you think about that experience of a fifth year at the moment they are they've had two periods of lockdown they've come through a period in which it was an alternative certification model they're going into their hires this year I think they have all have a responsibility all of those young people count but I have an eye I think to those young people themselves so I think that we need to make sure that that knowledge if you like her that sense of understanding is built upon I also think that you make sure that the alignment of the qualifications meets where they're at just now because I think that would be my worry that you've got a qualification system that if it has a full expectation that there's been nothing missing then that leap for them will be considerable so I think that we need to make sure that that continues to be tailored to their needs if that makes sense it makes sense to me but it does worry me a little and in terms of you know hearing from these young people in terms of going from higher and to advanced higher for instance speaking to university principles and lectures about people coming out of school with perhaps a lack of knowledge in comparison to what they may have been otherwise and whether we are adapting as a country to address that I don't want to bury too deep into this convener because I have another question if that's okay but I don't know if Mr May has any. I would just add to that yeah there's a gap that we're aware of but the adjustments that have been made I think have been put in place to address that but that doesn't take away your worry about the future and I think that's important for the immediate future that we address that because one of the things that I think is really important is when our young people did return to school is that we didn't use the language of catching up and putting extra pressure on them we were very much concerned about their welfare and we wanted to care for them you know because of the impact of their experience through a pandemic as well and the worry about qualifications so it was very much a health and wellbeing and welfare approach that we took to the return so we haven't focused on well you've got a huge gap in your knowledge and I think we need to you know say that out loud and admit that that's actually the case so we do need to think about what does that mean going forward and it's especially for the young people that Tony's described this year and yourself Michael so it's a really important thing for us to think about but I would just reiterate the support that people really rallied round in terms of that online and digital offer and certainly in the Tayside Ric we worked with the west partnership to grow that across the country not just even in our own regional improvement collaborative we worked nationally and the great work we built on that great work of eSchool in the first place so there's there was a lot put in there and I think it's important to to do what Tony said and recognise it some young people actually engaged very well online and some people returned to school hitting the ground running not all of them but some of them did because that medium for them was actually really successful you know and it suited their style of learning and and they totally engaged so so there's that part of it that we need to take forward in terms of learning as well you know but what we do in the future for any disrupted learning so I think that's important. One last one so data released yesterday showed that the scale of the attainment gap produced by the ACM so the alternative certification model trying to avoid acronyms as best I can so they show that around 75 per cent of private school people's got A grades compared to less than half in the state sector so do you feel that the system benefited the most affluent at the expense of the poorest? No not necessarily I think it's a if we think of the poverty relief attainment gap that existed pre the pandemic didn't it and actually that is a mission for all of us every single day to try to make sure that you close that gap and to have as much aspiration around all of our children and young people I think from our own perspective we probably have seen more ease I think we could speak from the local authority we've seen more ease than we've ever had before but nonetheless that's I think it probably replicates a scenario I think you're seeing a scenario around the poverty relief attainment gap and disadvantage rather than necessarily I don't think that ACM accentuated that I think that what you're looking at is the kinds of assessment that we have for young people I don't think it necessarily accentuated that I would probably just support that but add in you know that we we're very aware that again feedback from young people and from from our schools that there are sometimes young people and often associated with disadvantage or deprivation or poverty however we wish to describe that who actually benefited from not sitting exams because they didn't they didn't have the support if you like in their wider community to be able to turn up to an exam ready and prepared and so for some of those children who would probably be in that gap if you like this system actually suited them better quite a number of our colleagues have reported that back to us okay so if I understand you're saying that you talked at Audrey about how some came back to school hit the ground running as if nothing had really interrupted their stride but for those that didn't have the same connection the gap didn't widen I'm asking it that way because I can't I find that quite hard to believe that if you weren't able to take advantage because of home environment or material basis you weren't able to plug into all of the that you didn't fall further behind I'm using a pejorative term there I apologize but that gap didn't widen no I think for some young people it definitely did I think that we look at the data and we can see that the gap widened but it's not all of the children and all of the young people I think it's about you know dylan down to what suits some better than others and making sure they did have access to digital for example which was part of what we tried to do but it's also remember when and most one of the things our schools did during lockdown was to bring some children and young people into our hubs to you know where we were so concerned about them not being and not being seen and not being you know and being vulnerable for a number of reasons we would want to then bring them in and so a lot of these young people did have actually want to teach in our small group teaching in the hubs when they were there as well so it was a mixed picture but despite your best efforts I've no doubt yeah there were some young people that absolutely that were that were left behind yes all of them down I think what starts the supplementary on this no thank you I can be not I'm just a bit confused because I mean I wonder about the difference between 2020 and 2021 because my understanding was in 2020 the the grades of state school pupils improved faster than those at private school and then in 2021 the opposite seems to have happened and we seem to have seen a reverse of the kind of progress in narrowing the gap we saw the year before do you have an explanation for that it's fine talking about A grades but you know actually for a lot of young people you know they're looking to pass qualifications you know and leave school with you know something meaningful and it just seems a bit odd and it's trying to understand what changed between 2020 and 2021 I think it's a good question I think also it's maybe reminding ourselves that the system wasn't the same so you were going from an internal estimate system that wasn't necessarily the same model and if you think about it schools were giving the grade on the base of an internal estimate I think that the grades in 2021 were broadly aligned if I think about one three and five five and I actually do take your point around whether it be A's or simple passes I think that that if you compare that to 2019 I think there was a considerable improvement if I take this year's grades I would accept it yes I would accept that so we've gone in a year of having more time and more time to get the ACM organised in that time it's actually disadvantaged those you know from the most challenging communities is that or has the system just adjusted back to what we'd normally see no I don't think it has I think that what you've tried to do is you've tried to look and try to give young people the bent you will you will work with individual young people you'll try to put the model in place you will try to quality assure that model as well I think that the system was slightly different I think that the internal estimate system and the ACM were slightly different I think that the moderation of that activity also think as well that some of the progression rates if you look at a young person moving from fourth year into fifth and sixth year some of that progression activity the technical part of that so if young people were doing well in fourth year through an internal estimate process actually the jump into fifth year might have been considerable for them so I think that might have been a contributory factor in that as well if that makes sense so you're looking at something in which those young people might have got an A or a B but in actual fact as they go into fifth year that alternative certification model that required to look at the demonstrated evidence of it allowed I think that that jump could have been difficult for some young people believe it there I come back in okay of course um bob bob Doris thank you thank you you know I was very interested in Mr Mundell's line of questioning because my understanding is that both 2020 and 2021 sorry bob can I just stop you for a moment is there something wrong with bob's sound he sounds very far away it's over on the broadcast sorry bob it's just in in the committee room we were just you're a little faint but please carry on is it any better at all now to know it's a little it's fine on the broadcast it's just in the room itself I hope that our can you can you hear you can hear you can hear right okay carry on please sorry to interrupt you bob okay thank you can you hear I was really interested with Mr Mundell's line of questioning because I think that this gets to the heart of some of this and it's going to clarity as to what happened rather than looking at snapshots of attainment so my understanding for clarity is that both 2020 and 2021 in historically challenging circumstances technically the attainment gap closed compared to 2019 and previous years I wonder if that's the understanding of witnesses but more importantly in understanding the difference between 2020 and 2021 which is absolutely important I wonder if some of that has got to do with the role of intel with moderation procedures whether it's departmental, whole school, local authority, across local authority so we'd like to better understand the moderation procedures that existed in the local authorities in 2021 as compared to 2020 because one of the issues of course is without an exit exam is as unsuitable as those are for many young people you don't have that benchmark and in 2020 we didn't have that benchmark to to refer to so I'm really interested to know about 2020-21 compared to 2019 and previous years that I'm also interested in on the role of moderation in schools in a local authority level in 2021 compared to 2020 because that might flush out some of the issues as well of course as that lockdown January to March which I think clearly had a massive impact and it might not be ACM that led to that differential that may have been that January to March lockdown so I'm really interested to know the witnesses' views on that thank you Bob who's going to go first okay I'm happy to start yeah and I'll probably start with you know we know it was a completely different system in in terms of lockdown came in 2020 in March you know so there was that that gallop towards we need to to help our young people get the best qualification we can it was a completely different system as has already been said we submitted estimates at that point and then the sqa awarded the awards if you like so that's the fundamental difference in in the ACM alternative model we had a bit more notice although it was still staggered notice from that october to december then a lockdown in january so we had more time in that term when we would normally be you know really really going for given every young person the best opportunity to get their internal work done and and to be prepared for the exam so so what we had to do was set up a system of moderation and quality assurance at every level so we had moderation and we we produced and that was part of the work of the local authority officers working with sqa around understanding standards offers that were out there for supporting our teachers in the system but we also then had to look at our quality assurance so that we were really confident that we were submitting provisional awards that was the difference in in 21 it was provisional awards and they were based on robust and reliable moderation across every level so we were moderating at department level within subjects and curriculum areas we were moderating across the school we were moderating across the curriculum improvement networks where principal teachers of same subject or curriculum leaders were coming together so we were moderating there we also used our sqa colleagues to support us that with that work around understanding standards we then took that to the rick level so across our regional improvement collaboratives where we had subjects where there were just maybe one or two members of staff in the department we would go across the three local authorities to moderate and then of course across the rick so we partnered with other ricks to do that even wider and we even got to the point of getting that online as part of the offer around the digital offer for our colleagues and build up hubs of resources in fact in our rick we even started to look at national 123 which is also now a national development that we're leading on so to make sure that all of our young people were given accreditation for the qualifications they deserved so i would be confident that we did everything we could in that time to make sure our young people were getting the best results that they deserved because people were working really hard the commitment of teachers you know at that point to do the best job they could for the young people i think was second to none and so there's learning there and we would want to take that learning forward around how we better benchmark our standards how we better moderate and quality assure and so as local authority and colleagues we have to think about that maybe there's stuff we need to stop doing to do more of that going forward because we felt very confident that we presented the provisional awards and you know out of thousands of children of around 3000 in Dundee we had 59 appeals now you might want to come on and talk about appeals but that process also allowed us to absolutely deliver on a no surprises agenda so so young people were being tracked and monitored because of that continuous assessment to know what was what was likely to be their award right to the wire and our schools all agreed to release that information on the same day so that we were all on the same page sorry maybe that's more than you needed to hear oh no that's excellent so important to say that this has been about making sure from a teacher's perspective there's been confidence in their judgments as well because it's based around the young people and based around what they have done in that class at that time and and ultimately so if you think about it as a young person you want to be assured that if you're in one school that you're treated as fairly as possible that's why demonstrated evidence based on teachers professional judgment was really important teachers professional judgment is based on something and it's trying to make sure that what that is based on has a consistency and that young person is able to look at that and to be able to know even though the assessment tool might be different even though you may look at a collection of evidence slightly differently if we think about it last year we even in South Lanarkshire and some of our schools some of our urban schools had young people who were self-isolating two or three times versus some of our more rural schools that weren't so you couldn't possibly have the same assessment tool that would be completely unfair for those young people so you needed to have a balance in there therefore the moderation became vital that was really just an understanding that the teacher when it was a when they were awarding the grade that teacher had been through that understanding standards process that they were confident and also I think it helps the teacher know for the young person that when the grade that they're awarding gives the teacher a bit of protection it's not just me although it's me awarding you that provisional grade because it's based on your work I've been involved in checking that work across my department across my school with other schools within the local authority and beyond the local authority so I think that that notion of demonstrated attainment becomes quite important based on the professional judgment of teachers bob can we have time for one brief follow-up question because I know time is precious and other members still have to get in firstly I took a lot of confidence there in the robust processes underpinning the alternative certification model for 2021 but of course there will always be room to improve that and we have to wish that that is consistent across the country but I suppose the question I wanted to ask was that contradiction we had in some of the SQA guidance in relation to relying on exam style evidence but encouraging schools and departments not to give those traditional exit style exams and we heard from young people that in some schools they got that large exam but the difference was if they didn't meet the standard then they got a second exam then a third exam so that they got multiple opportunities to prove they reached a standard unfortunately it was top heavy within the last few weeks so what guidance did your local authorities give out in terms of what best practice looks like in unavoidable assessments in those last few weeks and is there need for greater consistency because there seems to be a bit of a patchwork experience across schools actually within the schools a bit of a patchwork experience as well and across local authorities. Thanks I do think there was an element of that but maybe putting that into context as well I think our guidance was trying to say base around the learning teaching that's happened if you think about it teachers maybe diagnostically need to come in when young people have come back into school you need to do it although they were engaging with young people having the class in front of them does play an important function at the start I also think schools were caught in a dilemma with individual teachers as well you do by the nature of our qualification system some of that drawing together of concepts and knowledge comes at the end we need young people to practice we need them to develop those concepts and gain an understanding yeah if you assess them too early and in fact you might do the young person a disservice at that point because they do grow in their confidence they do gain an understanding so I think there's that element I also think that teachers and schools were trying to give young people certainty around when you might get that assessment the the one thing that young people would want is that notion of what to when what do I need to get in order to get that a in order to when I'm being assessed and trying to do that you're trying to leave that as long as you can but there was a nervousness for us if a young person disappears for 10 days because that was happening all of the time as well so the practicality of that is that teachers were saying we need to put something in the diary don't worry if you go out for 10 days we can catch that up with you that happened on a regular basis I was part of a number of conversations well that happened but the reality of that is you're then coming down to the real brass tax of how you manage that situation and the practicalities and I think the nature of that comes for young people I'm at the end of 15 assessments I think that that's what young people would present what I think is worth pointing out I think that schools managed young people's anxiety on the whole tried to take that into consideration and if a young person was saying to them I'm struggling a wee bit what they were able to say to them and I have again numerous examples of that they were saying don't worry we'll do that next week and just trying to build that in from a practical perspective I think Bob was going back in Bob no community I'll come back in later if there's time I know that Covid's got questions that don't ask. I want to bring in Fergus then and then Ross Fergus yes thank you thank you very much convener and just pursuing this this line of questioning I was very heartened by both witnesses statements at the outset about the genuine co-operation between all players involved and their positive comments today and I think that gels with what we heard from young people from Inverness in the session convener last week who gave very positive report as to their experiences in lockdown despite all the difficulties some of which we've heard about this morning what I wanted to ask was to what was this really that looking to the future and the use of assessment the use of alternative certification alongside examinations can I ask this very basic question how do you believe that we can provide confidence that any system of assessment as opposed to objective examination where the testing is done by independent third parties how can we provide assurance most ecologies universities and to employers that if you like the the qualifications have been earned and how can we avoid any criticism that in a sense it's unfair to expect teachers to do anything other than have an optimistic and favourable response to children who after all they have taught in a sense they're marking their own jotters and that's not a criticism with the great work teachers do but it is a sort of fundamental question and I feel we've been skirting around this a wee bit because quite rightly we've been looking at the difficulties of Covid but if we are to move away from examinations and you know in my day when I was at school such a very long time ago it's probably irrelevant but examinations were the be all and end all there was no assessment of any sort and I think that's wrong but I am concerned to hear from our experts how can we demonstrate the bona fideis the robustness of an assessment and see how can we get it right for Scotland moving forward? I suppose it's a question there isn't there for trust in the system and trust in our colleagues in our schools and I would absolutely support that ask you know because of the teachers that I see in my schools every day and the leaders in our schools you know just wanting to do the very very best for the young people that's why they came into profession and so can we have a big debate around this I think this is part of what we need to take forward from the OECD review and the recommendations there to maybe shift the balance you know there may still be an exam who knows we've not had that wider discussion yet and I think we need to have it but can we shift that balance for a more continuous assessment so that young people get the opportunity to to see the progress in their learning throughout their learning not just at the end as we've had in what might be described as a high stakes exam system which doesn't suit all of our learners and certainly doesn't according to the OECD which is a really good fundamental starting place around as a foundation we have got a great curriculum for excellence that's renowned throughout the world but maybe it doesn't articulate well into the qualifications part of that into the senior phase so there's a work to be done to look at all of that I think together and I think when we when we start that dialogue the most important things it was mentioned right at the start we've not come back to this communication you know we really need to communicate with our stakeholders we're young people they need to be part of that story they need to be part of that debate and and our parents are carers in the system to give them confidence as well as well as you know our further education our higher education and our employers you know so that everyone has confidence in the system but also we need to look at what we've learned which I think I tried to touch on and I know our international council of education advisers also touched on how fabulous we actually did do during that pandemic so can we take that learning around moderation and quality assurance and make it open and transparent so that there is confidence in the system I do think the question does strike at the heart of what we want for our national qualification system I think it does it does lend itself I don't think anyone is saying that exams don't have a place to play we need to clarify what important assessment is though if you look internationally and I heard doctor point say that last week I was I chaired the practitioners forum so I spent a lot of time with the OECD with this other countries as professor Stobart's second part of the review says other countries do this differently it doesn't mean that you don't do exams within class tests it doesn't mean that but if you can have the baccalaureate in France that will do an oral assessment if you can have a report writing part of your assessment I think you need to do you need to strike a balance the fundamental question is what do we want out of our qualification system and I think that's a bigger part of the debate I don't think you can tinker around the assessment tools and just say let's just change the assessment tools you need to look at the course you need to look at the skills that we're trying to get for young people we need to look at the fundamental form of our ambition for our children and young people when it comes to that I also think there's a couple of unintended consequences that can come sometimes by the notion of just continuous assessment it can become we have had internal assessment and we do have internal assessment sometimes it can seem you spend more time doing the assessment than the learning and teaching it has to be proportionate it has to assess the things that matter and you have to make sure that it's not over bureaucratic that the teacher is spending time filling in but it's a paper in order that rather than doing the learning and teaching part so we must strike a balance I'm P is my background so I'm used to familiar with a practical context of learning the reality is that you then need to strike that balance that's meaningful and then there's a debate that I think we need to have with our families and our communities to say don't worry we'll put a moderation system in place and those skills can be robust so when we get the A and I do feel strongly the young people have earned our grades this year fully earned our grades this year and that can stand up against any other year and I think that we need to continue to give that message but I do think we need to tailor our assessment around the things that we're looking for and the ambitions that we have for children and young people it's interesting that in the OECD report they did they did call on us to restate and reevaluate our vision of curriculum for excellence which in essence is what I think you've just both said before I come back to Fergus Ross wanted to have a supplementary on this so it was on bobsline of questions so conversations moved on a touch I'm happy to bring it in my line of questioning okay that's fine in that case I go back to Fergus so I'll pass to other colleagues appreciate tiny short and I thank those witnesses for their full answers I was I was interested in the ssta's survey that they did and I want to ask you because it's connected to what Fergus has just been we've been talking about we were talking about demonstrated attainment and in their survey they said that 36 percent of teachers believed that the evidence that they'd collected truly demonstrated their pupils attainment but that means that 64 percent don't I mean is that just what's your reaction to that data I mean is that um should I be questioning the data no I think you have to listen to what what data tells you again that was at a fixed point in time I think you need to be careful about you need to get to the end of a process I think in order to try to understand it fully and also my understanding at that point in time we're right in the mix of the the assessment processes so I think that demonstrated evidence we need to be there was lots of talk about what was high predictive value versus low predictive value ie if you mirror some of the assessment activity that the exams would do that has a higher predicted value than potentially a class test that you were doing in october time I think you need to be confident that there's a there's a range of assessment tools that young people that teachers can have at their disposal I think the danger is that you then assume that teachers professional knowledge is not based on anything I think it's really important that teachers professional knowledge is based on the professional experience of the assessment tool and the balance of that assessment looking holistic like that young person that it wasn't sitting at two or three exams in may time and that was the exclusive giving of a grade that they were able to take into account some of that work in august, november and december that was at a class level that might be a contributory factor around a grade boundary so if a young person sitting on an A and B that you're then looking at it and saying well actually I have some evidence that does suggest that they're at the A they're on a boundary here just now with a couple of assessments that I have I can now go and look at something else and I think that's where teachers professional judgment understanding standards is crucial at that stage okay moving on I'm going to turn to willy renny thanks campaigner thanks for the evidence so far I want to focus on the stress for pupils and also for teachers it's often claimed by pupils that an assessment model is less stressful than a big exam but we also heard from pupils that the model this time meant repeated assessments which for some of them were just the stressful especially those who didn't get the marks or the results in the first assessment that they did and they had to do over and over again. So I'm just wondering whether it's claimed that assessments are generally less stressful than big exams is true or was it peculiar to this year? Audrey I'll start I'm sure Tony will want to come in I thank you for the question again something that we're all grappling with because we we want to look after our young people and care for them as well as get them qualification so their stress levels and their anxiety really important that we understand and so I think if I give a very short answer it would be I think it was because of this year you know I think there's there's wider research out there that we want to look at we've already referred to some international research around that you know the balance between internal assessment and and I won't stop exam at the end of a course and so I think we need to still you know learn from that and look dig deeper into that research but I think for this year there was so much uncertainty and so many changes along the way you know that I think Tony's partly covered some of that and an earlier response around you know young people being isolated for 10 days and I mean we had in one of our schools in Dundee you know young people isolating three or four times during their fifth or their sixth year you know in school and these were huge pressures and so yeah there were repeated assessments but probably because of the other part of your question there you know because teachers were stressed teachers were worried that they weren't doing enough you know in order to get the young people every every bit of evidence because there was much more emphasis on you know this provisional award will be based on evidence of assessment so that was the you know the overarching message that we needed to get good evidence to be confident that we were given the right award so I think this year has really just been you know well we know what it's been we've been through a pandemic and and so nothing was was as it should or could be in terms of bringing in a new system that was an emergency setup that we put in place to do the best for our children and our young people and and of course there was lots of anxiety around it because it kept changing as we've heard earlier. So just as a follow-up to that then there was criticism from the EIS but also from some people and there was particular criticism from EIS towards the SQA that there were kind of they were saying there were assessments but they were reverting towards a more traditional model and I'm just wondering what your view on that is and then just a final follow-up is there's been some claims that this year and the previous year can teach us a lot about what we're going to do for future years in terms of assessments and exams and I'm just wondering whether that is actually true or not. You've kind of given an indication for that already but that any model wouldn't be like this in the future so I'm just wondering really what we can really learn from this last two years for the future. Did you get the whole thing? I've missed you. Right at the start of your question there was a broadband blip. Can you just go back to the first part and I think we got most of it but just the first maybe the first 15 seconds or so? Yeah so the EIS and pupils were criticising the fact that these were effectively examined by any other name and the EIS in particular were critical of the advice that came from the SQA that they were effectively reverting back to a model that was effectively examined but they were calling them assessments. I'm just wondering what your view on that is. Right that's excellent. I managed to catch up, thank you very much. I think there were some of that mismatch activity around some of the very technical advice for some subject areas. I think versus the general advice that said don't look at it as a broad assessment piece. I think ultimately you can understand a qualifications body talking about high predictive value around an assessment that therefore that looks like a small exam at that point rather than being able to look at the more broader based activity. In terms of what that means for our qualifications going forward I think we were repeated in the OECD report about balance. We do need to be careful of unintended consequences for young people that in fact that if you then go into a continuous assessment model that you don't overburden young people that every single assessment feels like you're doing an exam in May time you need to need to find a balance for young people during the course of the year so perspective becomes important in managing young people's wellbeing becomes central and indeed I think staff's workload because the over assessment activity can then take away from the quality of the dialogue and the learning and teaching and for us I think that if anything trying to run a dual system I think you need to run an integrated system not the thing doing two you know trying to achieve an external exam for the bulk of it and at the same time having evidence that sits separately to that it needs to align completely and that if it aligns completely you can take some of that assessment evidence out and only focus on the things that really matter that allow you to give a grade so I think I would say that there were some aspects that were confusing around some of that activity I understand why though and I think if you're in the middle of something you can you can understand why that was given out I think when that translates into reality that can cause some confusion but I think we need to be cautious about what that does and tells us about how we move forward and what the new any new qualifications model looks like. Audrey do you want to comment? No I think Tony's probably covered yeah I tried to pick up some of these points I think that that's where we are just now just very conscious I mean we're getting we're gonna have evidence from the NASUWT got to get these acronyms right later and they talk a lot about this compression that you've been describing in terms of assessment and how difficult really impossible they describe it as to complete all of the work that we're asked to do including the quality assurance and certification and the effect that had on the teachers I'm sure you're conscious of that. Completely conscious of that and you know that for all of us I think that we need to make sure that teachers workload is manageable because that is integral to the experience that young people have and ultimately the qualifications that they have I think it also proves that our teachers are schools not just our teachers we have school support assistants heavily involved in this type of activity our school leaders I think we can often ask we've shown that our teachers can do pretty much anything I think it's asking them to do everything all of the time it's too much I think that we need to be really clear about about what we're trying to do I think in those circumstances I think if you find yourself in extraordinary circumstances that changes national 5 in october higher in december and goes into a lockdown between january and march inevitably april may and june was no matter how you manage that I think teachers were trying to search for it as manageable as possible but inevitably there was a workload associated with that. Just a comment you made about the teachers experience reflecting in the experience of the young people one of the young people one of the schools we spoke to last week highlighted the need for fun to be put back into learning and I think that's an expression of what you've just been talking about I'm going to bring in Michael and then straight across to Oliver. Thanks, convener. This move to demonstrated attainment from teacher judgment in the second year obviously created many of the kind of pressures that you're talking about in terms of the assessment model and having to go through that. Also as part of that process it removed the circumstances that many of the young people you're describing were facing you know they had to get the exams done the exams are not the other name they had to be a demonstrated attainment rather than judgment I mean does that then not talk to and listen to you carefully the absence of exceptional circumstances and appeals does that not go against the kind of lived experience of young people? Absolutely I'll start sorry we're falling over each other yet yeah absolutely would agree with that Michael it's got to be part of a discussion going forward because you know the exceptional circumstances in the previous model if you like before Covid was about on the day of the exam and of course we had huge disruption for some people's learning throughout which of course has led to some of the other stresses in the system that we've been discussing you know just even under the last point so I think that that's an area while we look to the future that we really need to get that bit right. But there is a cohort of kids now who now and actually from the previous year in essence who are just feel that the system hasn't served them well and those exceptional circumstances have not been taken into account. Maybe if I come back on that maybe just a couple of things in further attainment as we spoke about in 2020 how I think we again need to look at the timing of that and further attainment happened after march the 20th in lockdown what you've had up to that point was dent so teachers professional judgment was based around some demonstrated evidence and the demonstrated evidence they would have had would have been some of prelims activity as normal they would have had in the first lockdown scenario so you would have a teacher judgment in prelims you would and indeed some schools had second prelims you had internal assessment evidence that wasn't taken away so your practical elements in drama and music and PE they were already there as well it was the last part that you were inferring so it was the drawing together you know they're on a great boundary to have enough evidence to go back and look at how they might have developed what kind of test that we have so I think that what this year tried to do was to say we need to look at the demonstrated evidence there was a second part of this year and you can see that from the EIS's presentation to hear there was an opportunity for candidates to further present any evidence now during this term if they had missed any assessment evidence what we were able to say to our schools is you need to be relaxed it's not about the quantity of evidence that you have you need to be confident and speak to those young people about not feeling you need to cover every single aspect of course coursework I accept though that that's not how that might come in reality in reality for young people I would say though that in terms of the evidence of our schools I do believe that the vast majority of young people were given those opportunities were given the opportunity to be assessed were given those opportunities to come back and do it again and there was a further opportunity that was put into the system to try to make sure it's easy for us to say you know get if you're in the heat of the moment it's it's really easy for us to distinguish that I do think that that was an opportunity I don't think many candidates I don't know the numbers of candidates will have taken that up but that opportunity was there and I think that that was well argued both by SLS and the EIS as part of the national qualifications group that that activity post this into this new academic session was there for those candidates so that was there I mean but that was additional evidence and not exceptional circumstances well I take the point at that I take the point at that all of that thank you I kind of want to return to the earlier line of questioning and I wondered I'm HMI did a did a review of what local authorities were doing in terms of the ACM and they said that most local authorities have developed bespoke data analysis to analysis tools to support school level quality assurance and that that was used to check against three in five year data trends and that those were used to identify and address any unexpected provisional grades is that your understanding of what what happened across the country? It was a sense checking exercise just to be clear that candidates this was based on candidates work that absolutely this was the teacher and what those candidates demonstrated what for us is a quality assurance system and if you if Professor Priestley put that in if you look at the review of last year although it wasn't ideally followed Professor Priestley makes a reference to sense checking activity it wasn't for us to give you an example if you had a to use that data if I think it's legitimate as a local authority if you're in a there's a classroom or a school environment in which candidates are doing particularly well on a train basis and suddenly that dips I think it's legitimate that we speak to our school leaders and say what is there something specific about that group it was about trying to understand that picture and trying to have a confidence in the moderation process it wasn't under any circumstances about trying to have an algorithmic approach and putting a ceiling on candidates numbers of ways this was about making sure actually that candidates had a sense of and teachers were reassured at that process and so for me there was a sense checking activity what was good about that is our school leaders came to those activities in those meetings and were completely clear that they were had a robustness around understanding standards that they were able to talk about when when it was a slight difference how would they were managed they were it was the class teacher that had come up with that grade that there was a professional discussion and dialogue about that activity and and there was a confidence in the grade that was given that's what that was about I mean I guess the reason for asking is we mean is looking to try and identify what changed between 2020 and 2021 and you know I think that you know the heavy kind of moderation process or the reintroduction of that because I mean I note you know that Addis were in discussion with the SQA as early as October 2020 regarding statistical analysis quality assurance and moderation and I think there's a feeling that that basically the normal SQA processes rather than taking place at the SQA end or front-ended in this process is that what you think that is a fair assessment I think that that's fair absolutely I mean we had been and I'm sure lots of local authorities were because we share a lot of our work we're absolutely looking at the best kind of way to track and monitor progress across the piece and it's part of our quality assurance as a central team to open up a dialogue with our head teachers and provide them with a toolkit certainly the toolkit that we used in Dundee was was commended because we looked at five years of data of evidence and also outcomes versus estimates so it was again it was just a sense check in other departments other areas where that's not been a good pattern if you like in terms of correlation so we were able to open up that discussion when we could see some sort of discrepancies and so we would go back to the school so that the head teacher would then have the opportunity to use that data again just to open up a dialogue to dig a bit deeper and I think that that was the point of having developed these systems to give a wee bit of confidence as well that we were we were not we were the tracking and monitoring that we want to happen in our schools is about individual learners and where their progress is in their learning but this gives us an overview as a local authority and it gives us an opportunity to have a conversation and ask the right questions if there are discrepancies and that's how it was used. I listened to that and I agree with your earlier comments that trust and transparency are important and I personally feel and I'm not asking you to comment on that bit of it but I personally feel there was a lack of transparency in the run-up to this year's grades being awarded from a pupil's point of view and from a public point of view because we had the cabinet secretary here in the parliament 16 something quite different appearing on the news 8 June saying that the assessment process is judged by your teacher and they will submit the grade no one's coming in to overrule it or to second guess it or to look at any other material around that your teacher will decide your grade and I think people then hear about what is a normal moderation process and I'm not trying to suggest that it isn't what happened in a normal year but I think there was a suggestion that somehow the ACM the ACM was different from what happened at the SQA but in reality it was very similar to what would normally happen I'm not sure if that's fully I take the point actually around it the moderation activity I think that the grade was given by the teacher you know I think to be to be reassured but of course sometimes no one's coming to second guess it besides to me like that this process was trying to second guess what you would expect not all this is about it is about just making sure there's a consistent professional judgment so if you think about it from a moderation perspective teachers will come at it from a variety of experiences some will be very well versed in higher some will not necessarily be around the marking schemes around the application of that simply for us as a local authority I think it's right and proper that on behalf of the young person that we get to be able to say I'm not quite sure whether that makes sense to us what that then happens is it's a legitimate conversation that might have then involved a more experienced marker from another school so ultimately that's not about us changing any grades at all that's just about saying is that accurate does that sit with the understanding standards does that allow a degree of consistency so there was no this was a teacher's teachers awarding those grades but we need to make sure that that understanding standards allows a consistency across from one school to the other it's not about applying a you know a threshold or a benchmark it's just about making sure that those standards are achieved and and us being confident in it and I think that's about firms for young people and actually I think that's about firms for the teachers as well in those circumstances you know and I can understand that logic of the argument I just don't think that's a message you know that people were getting at a political level and the explanation we got of how much work had actually gone in because to me the idea you know that I guess when meeting with the SQA you know in October you know and discussing quality assurance seems perfectly logical it's just then the sort of suggestion you know that somehow those normal SQA processes weren't happening was the message that was was being delivered here politically and that classroom teachers would be making the decisions by themselves and I don't you know I don't think that that is actually what was happening but obviously I'm not comment on the political part of that but the but in terms of the the discussions that would have been part of the national qualifications 2021 activity as opposed to a separate conversation between ADES and education Scotland that's simply about the formation of an alternative certification model rather than anything very specific as a separate conversation all of those conversations were part of the trying to develop alternative certification model I mean I just say on that there's a very last thing Stephen I was the you know on the Thursday the 22nd of October ADES did meet separately as part of a joint CEAQ network meeting with the SQA and from the minutes of that meeting there appears to have been quite a lengthy discussion around the need for quality assurance statistical analysis around the appeals process so there were separate conversations to that body ahead of it. Thanks I take the point of that I think what I'm trying to say is that that then fed into the alternative certification model as someone who was I don't know if I was at that particular meeting it might have been Stephen Quinn as director from Renfrewshire who chairs that group but Stephen is also on the NQ21 group this was to try to feed in to the NQ21 group was the purpose of that thank you. It was just in the spirit of sharing I can only talk about my own experience obviously and and we do share across our ADES network but one of the things that I think brought confidence to our system was actually we shared that timeline of all of that quality assurance with our parent council chairs so that we were meeting regularly throughout the process with our young people but also with our parents and carers and we met the chairs of our parent councils throughout and had additional meetings for our secondary heads sorry our secondary school chairs and we actually the timeline that we had for our processes of quality assurance and moderation we actually just shared it with them so there was there was certainly local transparency from our point of view and I think that brought confidence to the system as well and so maybe there's learning in there for all of us to do more of that. Would you share that with us just so we could see that as an example? Absolutely, yeah, no problem to do that. Good, thanks a lot. I think Bob has a question on this if you can be quite brief Bob and the response as well because we've got to go to Stephanie. I'll be very brief because again I'm concerned that we confused the purpose of moderation and Mr Mundell mentioned about the quote from the former cabinet secretary about second guessing. It would be helpful to hear from the witnesses that moderation is about professional support and assistance for teachers. It's about checks and balances that teachers also want to see within the system and it's been that way in relation to continuous assessment parts on processes within departments and some departments have only have maybe one teacher in it some of five or six teachers and there's the need for that. Did you have a question or did you have a question? The question is do you recognise that moderation is not of its second guessing but rather providing that professional support and assistance to assure the professional in the classroom? Yes, I would agree with that. Absolutely. Yeah, okay, right, thank you. Stephanie. Thank you, chair. I just want to thank you for coming along today. I think our teachers, our pupils and our parents have all been absolutely incredible throughout this situation, so a huge thank you to all of you. We've spoken a little bit today already about supporting pupils with mental health and I'm quite interested in what worked and what didn't work and what kind of recommendations that are going forward there. I think pupils really did step up and demonstrate their resilience and they also seemed to, at the evidence sessions, it was really clear that they had a huge amount of empathy and respect for their teachers. It felt very much like teachers and pupils but a team, which I thought was really, really nice. So that's my first question and my second question is about how will your organisations make sure that young people's views are listened to and acted upon going forward? We got some of the evidence that came through there. There were some suggestions that were really quite different, things like putting stuff up in Google Classroom that young people could comment anonymously throughout some of the young people that struggle with exams and have additional needs saying that if they had some music on in the background, that could make a really big difference to them. Just how we can incorporate those views and then I have a very quick final question as well. I think I touched on it right at the beginning that we try and I know across the local authority as we all try very hard to value young people's voice in all of the big decisions that we make as a local authority and an education authority. So I could talk about my own experience in Dundee where we have a youth voice forum and the young people, maybe about four to six from every secondary school, come together and meet with me on a regular basis. I hear from them and I share with them and through this you can imagine the top of their list of things they wanted to talk about was SQA and their qualifications. So lots of discussion there and I think there's something about just that on-going and regular communication and the way I described a timeline of this is trying to make sense of what was coming nationally and helping young people to see there's a pattern here and this is what you should expect but also listening very carefully to what they were saying and listening to their concerns and asking them to be ambassadors and to go back to their own schools and to ask the questions. They were running surveys, they were having people focus groups, they were having online discussions but there has to be a feedback loop in that. So the young people have to have a forum to bring that back so obviously in their own local school they're getting to speak to their senior leadership teams, you know, good practice in their schools. We have many young people who sit on leadership team meetings and so on so there's youth voice happening across the piece. So something you know whatever that might look different in different authorities but something around the principles of having a forum where young people could come and share their views and absolutely feed in and we need to be able to feed back as well so and to feed that into the system about what's being raised and what are their concerns and worries. I suppose the other thing, again I touched on it at the beginning so I don't want to repeat myself too much but you know just keeping the message from our leadership team to our head teachers and their teams that you know that health and wellbeing was right at the centre of what they needed to be doing you know before we get to thinking about everything else that has to happen in learning but we need to make sure when our young people especially after the lockdowns returned and even when we were engaging online or bringing them into our hubs that we were addressing the most vulnerable issues and concerns that our young people had. We've also put in mentoring programmes, we've put counselling into our schools so that where there's individuals who need a bit more help, you know targeted to help, we've targeted mentoring programmes around our care experience young people, our young carers, you know other young people who would have additional challenges in their lives you know that might again be a barrier to them you know maximising their own potential so what else can we do? Our counselling programmes in school we started a few years ago in Dundee, it's part of our attainment challenge work to really address the mental health and wellbeing of our young people so we've managed to do that and as I said act upon that whole piece around you're telling us this, this is what we can offer, this is what we can do, that feedback. I think one of the things from last year and no one would want to repeat any of the last couple of years but there's learning in there, there's absolute learning we need to not lose and one of the things for me is that that really no surprises agenda that we stuck to right through the process so that when young people got to the provisional award they knew exactly what that was and yeah there was an appeals process but there was very very small numbers across the country who actually appealed compared to the number of presentations so I think we need to think about what does that mean in terms of how we engage with our learners about their progress all the time and be really clear about that kind of learning conversation if you like so they know that they're on track for whatever they're on track for and they also know what else they can do or get help and support with if they want to raise that bar so these are the kind of things I think that we want to do and I think by you know I certainly know in our local authority and I'm sure my colleagues would say the same that we spent a lot more time listening to our young people and a lot more time discussing with them and I think that's the message as well and certainly in terms of how frequently we met our chairs of our parent councils that's also increased significantly and we've had nothing but positive feedback and support to the local authority because we've kept people informed throughout the process and I think that's key in terms of communication so we're very aware of the stress in the system and I think we want to take forward the learning into whatever happens next thank you thanks me with just a couple of quick points to build on that as well I do think and I'm heartened by what you've said about the young people about that I do think that teachers have a particularly special relationship with young people I think that we often think about the system but in actual fact schools are built on that positive relationship that teacher that you have that's been with you since you've taught them in first year and you've seen them develop and grow through the course of your schooling so ultimately that attachment informed activity is is fundamental around wellbeing for all of our children and young people and having somebody to turn to I think there are structures as Audrey has said you know around for young people that are struggling particularly I also think from a very practical point of view about communication with young people I think it's about clarity for them isn't it I think for them to feel that they are in control of some of their decisions and that they're able to make some of those decisions and are part of that decision making process and they can see the change their influence has made I think the second part of that is that information to them needs to be as consistent as possible I was struck in the evidence by the young people and in the notes that they spoke warmly about the communication that they developed about the appeals process I think that they need to be involved in developing communications for young people because what that does is that means it's more it's appropriately tailored to meet their needs and that they're more sophisticated I think in terms of how how they deal with their peers than necessarily us telling those young people that's great thanks and just finally which I suppose is quite a big question really what are the key lessons then would you say and what priorities would you like to see the Scottish Government looking at for this year specifically I think in terms of this year obviously the model is out in the in the sense that there's an adaptation you know it's an assessment activity there's an external series of exams with with modification and indeed there's a potential for further modification depending on what happens I think we've all I don't think any of us can predict I think that for the past two years taught us anything and also then the third part and that does become important around estimates is to make sure that there is no need for a dual system that is overburdened for our staff and for our young people I think that's the clear message that for us that we're making sure that that we're confident in the kinds of assessment when we talk about teachers professional judgment that we understand what that means that we all have a collective understanding that might mean sometimes an exam at the end of an assessment activity or it might mean a piece of work a report or it might mean a practical element of the performance that we have a clarity around what that means if when we're giving an estimate for young people themselves so I think that would be really important for me that we're making sure that we're not saying we're running exams and then but also at the background we're running the ACM that's not happening I think that we need to make sure that the evidence is tailored and as the natural assessment points are used this year for generating an assessment for young people and an estimate for young people. That's good. And the last question is welcome from Ross Greer. Thanks, convener. I'd like to return first to the questions around moderation and make sure that Oliver Mundell raised about the use of historical data. I completely understand the need for a level moderation to ensure that an A grade in one school is equal to an A grade in another but moderation that includes the use of historical data, school level performance data, seems to me to do the opposite because we've had an attainment gap in Scotland for a long time. That attainment gap is obviously socioeconomic one as well as between those with them without additional support needs etc. Surely any moderation system that uses historical data is automatically putting more of a question mark over higher levels of achievement for young people from a deprived background compared to those from a more affluent background. It would have been viewed with more suspicion for a class full of higher pupils in Drumchappel to have got straight A's than a class full of higher pupils in Newton-Merns or Clarkston. How did you deploy a moderation system that included that use of historical data without simply having far more conversations with teachers at your schools in those areas of higher levels of deprivation? That is an absolutely important question to reassure people on that as well. You're quite right. Historical data is backward looking. You're looking at the past and not necessarily the future. What you hope it does is it gives you a sense of what you're doing. For us, it is about continuing to make sure that teachers are confident in the grade that they give. This was about a conversation with their school leaders. That wasn't us saying, well, you need to go back and make a change. It was about a genuine professional dialogue. There was a dialogue that said, here's what the grades, the provisional grades are looking at, here's where we have been in the past. I think that it is a legitimate question to say that, in particular, subject areas, here's what it's looking like. I think that that's about engaging in the qualifications process and a moderation process. That, for me, was about understanding standards. It wasn't about saying, well, you've just made a grade boundary or you're looking at a percentage and it's just gone up or gone down. That was just about making sure that candidates and young people were given the benefit of the doubt. That part was about understanding people on the back of last year worried that you're having an algorithm applied to that. I think that, for us, the whole way along in all of our conversations with our school leaders and our teachers, we were saying, this is about the candidate. This year, I think that question about if there's 20 A's in that class in the Navy and that they get it. That's really it. Nothing more or nothing less. I think that, for us, it's legitimate. If it's never happened before, there's a legitimate question to say, you're confident with the standards of that, that there's been a process that's gone through. I think at that point, and that's about the teacher delivering that grade and there's a confidence level for across different schools and across different subject departments. I completely understand that there's a legitimacy to that question if you've suddenly got 20 straight A's in a class that's never had it before, but that's disproportionately likely to have happened in a school in a more deprived area. Have you done any assessments since then to check how many of these quality assurance conversations we're having with our school leaders and our most deprived communities versus those in the least deprived? Have you checked whether there was a disproportionate amount of that kind of quality assurance going up? No, it was the other way around. The schools would come to us, so the agenda was set by the schools, so the schools would come to us and talk through their story and talk through their assessment story. Here's what we've done, here's the understanding standards activity, here's the grades that young people will be getting, and we are reassured that that's the case. It wasn't us saying, oh, by the way, this looks out of kilter. It was our schools because I think that schools have been doing this for a long time around understanding their grades. I also think that it's about making sure that schools have an awareness and school leaders, faculty heads and right to the individual class teachers have a sophistication understanding that they were applying a grade. That, for me, was the important part of it. It wasn't about a statistical analysis. Sorry, Audra, do you want to come in? I realise I keep trying more questions at Tony, but I'm not giving you a chance. No, we just be, I'm sure, much of the same answer. Just to emphasise in terms of our responsibility as a local authority for the quality assurance, it's that overarching data that we have to look at that opens up that professional dialogue. This is about us being part of the story. It's not about individual moderation of a standard at a child's level of qualification. That happens in a classroom, it happens with a faculty lead or a principal teacher, then it happens with an SLT lead, and then it happens at a teacher level. We've got the overview of that data so that we could just give that data to our schools. As I say, this is for us, the third year we've used that approach for quality assurance. It's not something that we brought in because of the situation here. No, we haven't analysed what's the difference. What we tend to want to ask questions around is where there's significant improvements. Tell us the story because we want to share the good practice as well. It's about listening to, I could give you examples, where some significant improvement in results, and we're able to go back to the teacher and say, yeah, we pointed to a new principal teacher last year. There's a whole story there about how that shifted, so it's about that dialogue that's important and it's part of our accountability as a local authority to have an overview. Most local authorities who used historical data as part of the process used something in the region of the 2015-19 data, the excluded 2020. Can I just ask what did your local authorities and RICS do? Did you include the 2020 data in that historical average? Yeah, we did as well. Does it told us another part of the story? Exactly, I'm keen to hear the rationale for that, because obviously the local authorities who excluded it were the ones that I was certainly hearing, and those in my region, I was hearing much more from teachers in those areas and pupils who were coming forward with concerns because the one year in which the gap closed quite considerably was excluded for moderation purposes. Could you just explain a little bit why you felt it was appropriate to include 2020 data for moderation? For me it was just about it was there, why would we not look at it? We didn't overthink it, I think we were gathering data, we had at that point because we'd done some longitudinal study around our data, we had five years worth of data, we added it in, but we were still, as part of that professional dialogue, very aware of what that data was telling us and it was a very exceptional year. For me there was no point in just ignoring it, it was there and we had to ask questions around that too, it informed our discussion. And there were similarities, although it wasn't the same model, there were similarities around the 2021 model and 2020 model, I think it's legitimate to look at that and also those young people in 2020 earned that, those grades, so I think you need to take that into consideration as well. And just, I've got time, communer. Just, just. Right, I'll be very brief then. Going back to, I think I can't remember who raised this before, it might have been Willie Rennie around the volume of assessments that young people had to set in that kind of three, four week sprint in April and May in particular. I spoke to a young person yesterday who had 30 assessments in a fortnight and that was, they were taking two hires into advanced hires, so that was on top of dissertation deadlines etc. Did you receive any guidance from the SQA as to how those final assessments should be timetabled to avoid that kind of compression? So a lot of that was due to the perfectly valid motivation of teachers to let pupils set the same assessment over again a couple of times to maximise their chance of getting a good grade. The cumulative impact, though, was quite negative for the mental health of some young people. I think that cumulative impact was really challenging. As you just described, would you ever set out a system that would want to do that within two or three weeks? I think, without repeating what I've said, you are taken into an exceptional circumstance in which some of that evidence that you would have naturally gathered between January, February and March and actually some of that concepts around learning, you're not really able to get into that at that stage in time. In terms of the SQA, the general data was about not trying to repeat. However, if you get into the subject guidance that said that needs to be quite closely aligned, then that's what's then going to happen. I think that we need to, that's the again a point I would make about the kinds of professional judgment and what evidence can you use, not just talking about in absolute terms about high-predicted value and low-predicted value. Actually, it's higher and lower. Some of it can all be used and it's to the extent in which that can be used. If, understandably, schools try to keep it as high predictive value as possible, then that's what you end up getting. I think that for this year, if we're looking forward, we need to try to make sure that something like that doesn't happen. We are now out of time. I'm sorry to do that to you. Cocab is just going to have a brief word. Thanks very much. I just wanted to come in very quickly to actually thank you, because I think that the implementation bit at local authority level is a huge challenge. And my sort of like thanks is I worked as a teacher throughout all that time myself. And I have to say that I would like to just on record pay tribute to the local authorities that had to do such a rapid response under unprecedented times. And I think today you've explained, I think, to a wider audience the complexities that are involved around that. So I just wanted to thank you. So I'd like to thank Audrey May and Tony McDade for joining us this morning and giving us the evidence they have. It's been very insightful and very useful. We will now suspend briefly to allow us to change panels. Apologies to Willie for not getting him in. I'll repent, get you in again. Are we ready to go? Welcome back. We will now hear further evidence on the alternative certification model from our second panel of witnesses. And I'm pleased to welcome to the committee Larry Flanagan, general secretary of the EIS, Searson, general secretary of the Scottish Secondary Teachers Association, the SSTA, and Tara Lillis, who is a national official Scotland of the NSUWT. And let me start off by thanking you all for providing us with your written submissions, which were very interesting and useful. And I'm conscious we have a lot of ground to cover today and not as much time as we ever want to cover it in. So what I propose to do is to move straight to questions, but I'm conscious I want to make sure that you, if you have anything short and specific, you'd like to say beforehand that we allow you to do that. Larry, are you happy to go straight to questions? Yes, that's fine community. And Seamus. Seamus is happy to go straight, yes. And Tara, you're happy to go straight to questions, are you? Right, good. So once again, thank you to all of our panellists. And let me begin by turning to Larry, who's in the room with us. Great people have you here. If I can turn to page 3 of the submission that you gave us, you said that the Scottish Government and the SQA were determined to push ahead with national sampling of all courses. Did you ever ask for a rationale or a justification at the time for their determination to push on with that? If you did, what was it they said to you? Well, I don't want to put words in the mouth of the Scottish Government, but the discussion was largely around whether S4 pupils could bypass qualifications last year, last school session. On the basis that somewhere around 94 per cent of them would be moving into S5, they could progress from S4 into S5 based on teacher assessment, and that would allow them more time in terms of teaching and learning to consolidate their understanding ahead of an S5 diet. It would also have created or lifted a significant level of workload because S4 is the biggest year group in terms of qualifications, so clearly that would have been beneficial in terms of teacher workload around the S5 and S6. We also said that if there were S4 pupils who were exiting school, they would have to be catered for. The vast majority of those pupils go on to college courses, and most of the articulation between school and college courses at S4 at S16 revolves around unit assessments, so arrangements could have been made. We didn't get into a discussion with the Scottish Government around its rationale or reasoning on that. I would be assuming that it thought that it would be politically challenged and they were keen to be seen to be delivering qualifications for S4, S5 and S6 pupils because it would have been a big political decision. They clearly had a view that they wanted to certificate that all pupils were far as possible and in a normal manner. In your submission, you referenced a number of times very clearly that you believed that there was a hugely political dimension. On paragraph 1, page 3, you referenced the awareness of the forthcoming Scottish parliamentary elections. How much of the decision-making that was being made in the process was driven in your view by a political agenda from the Scottish Government? If I am trying to be even handed, I would suggest that all the political parties viewed what was happening in schools through a political prism with a view to the elections. In fact, I would be fairly critical of some of the noises off because the politics around what was happening in schools was unhelpful to schools. I think that we would have preferred to have much strong consensus from the parliamentarians about supporting teachers to a credit young people. In your submission, you said that the noises off were unhelpful. By and large, what happened once we got to the point of the ACM was that teachers focused on delivering for their students. I reference in the paper a herculean effort on the part of teachers to make sure that young people were not disadvantaged any further than the obvious from the pandemic. Ultimately, although there are lots of issues around the ACM and aspects of it, it was not our model, but ultimately it delivered the qualifications for young people. That is partly due to the resilience of the young people themselves, but they are also larger to the work of schools. There was a lot of noises off, as you call them, in political terms around this issue. Clearly, it is a huge issue, particularly in Scotland. Education is a priority in the lives of the people of Scotland, and it is a totally good thing that that is the case. However, the Government clearly was in the seat. It was making the decisions. I want to ask you one more question. I am sure that there are many questions that I could ask, but I want to bring my colleagues in as well. I want to go back to the sampling issue, because there is a rather incendiary comment in the submission that you have made, which is that the EIS's trust in teacher judgment was not matched by that of the Scottish Government. On what basis would you say that? That is quite incendiary comment. The Scottish Government and the SQA were keen on what it referred to as a robust quality assurance process. We were not opposed to a quality assurance process, but we were keen to limit the role that the SQA would have in that, beyond promoting the understanding of standards. What we did not want to see and what we were absolutely insistent upon, particularly on the back of the previous years' experience with algorithms, was the SQA, in any sense, being able to overrule the professional judgment of teachers. Quality assurance process, as far as we were concerned, was supportive of the professional judgments in the process to arrive at those, but we were not prepared to concede any kind of veto to the SQA. If we have included the Scottish Government in relation to the issue of trust, our challenge was to both the Scottish Government and the SQA that it had to agree a system that was based on trust in the professionalism of teachers to arrive at the estimates and grades on behalf of the pupils. The point that I am trying to make is that what you are saying leads you to say something that is fairly damning of the Scottish Government about their trust and belief in teachers. Before I bring co-cabin, the Deputy Convener, to take the questions forward, I want to turn to Tara, because we touched on that in the last session. In your submission, you talked about over-assessment and the compression that went on this year in terms of the assessment and the concept of over-assessment. Have you or any of your members sought to define this and what conclusions have you reached over assessment? I think that it really does get to the nub of the issue about manageability and workload within the system of the alternative certification model. When we talk about over-assessment, that question was posed directly to Education Scotland by one of our executives. When they were coming in to review local authority process and procedure, did Education Scotland themselves have a concept of what over-assessment would be within the alternative certification model? That answer was not forthcoming. Education Scotland did not have an idea of what over-assessment would be. In terms of our membership, the experience on the ground was that the alternative certification model was unmanagable. When we look at, for example, in the Stobart review, any model of assessment has to encompass a balance between manageability, validity and reliability. Within the system that operated in 2021, manageability fell off a cliff in that there was this compression, this last-minute dash and an over-assessment that was unmanagable for teachers but also for pupils who were undertaking those assessments. We could talk longer about the costs that go with that compression and over-assessment, and I have no doubt that my colleagues will get to them. I want to bring in Fergus, who wishes to ask a question at this point. Thank you very much, convener, and good morning to the witnesses. I just wanted to pursue something that Mr Flanagan said in evidence, which was that although the Scottish Government did not accept the approach to ACM alternative certification model that the EIS had recommended, nonetheless the outcome was satisfactory, and I very much welcome that as a positive comment. With that in mind and looking forward, given that the national ffives higher and the advanced hires will be held in spring 2022, if public health advice allows, and the decision will be informed that public health advice and course content will be reduced compared to a normal year with further contingencies if there are further difficulties of disruption because of Covid, could I just ask Mr Flanagan, is he broadly happy with the approach that has been set out? I very briefly describe, convener, for looking forward, and does he have any particular views and suggestions about how that should operate in 2022 looking forward? The EIS supports a system of senior phase assessment, which is predicated on exit qualifications. Across the senior phase, we should be looking at the qualification that young people leave school with as their passport to the next step. Currently, what we have is a year-on-year step-ladder approach, starting with S4 qualifications, then S5, then S6, and that robs time from teaching and learning. In our view, we have a poorer assessment system now than we had when we had standard grade and national qualifications and intermediates because of the challenge of time around how the senior phase is structured over three years. In terms of this year, I think that the Stobart review and the OECD report create a context for us looking afresh at the issue of exit qualifications so that we can achieve the ambitions of breadth of learning, depth of learning and parity of esteem between vocational and inverted commas academic. We argued for this year that, if the Government chose to revert to the exam diet, again, we thought that this year's S4, which was last year's S3, and in many ways you will hear young people saying that S3 was forgotten last year because it was outside the qualification focus. We would have targeted this year's S4 on a two-year pathway to qualifications in S5, and that would have allowed more time for consolidation of learning, particularly given the challenges that those young people had faced across last year. That was a step too far in relation to the political decision. We are now going for the qualifications in terms of the diet. I can understand why the Government has decided that. There is now a review under way. That review will have to avoid the mistake of the last set of changes, which is to push ahead with them without getting by in from the profession. The review will have to take about a time so that any changes can be bedded in. Reverting to the existing system is probably the most straightforward. We could not move to a continuous assessment system this year. That would have been a challenge too far. People are already exhausted, although it is only in September. I think that the diet was almost inevitable, but we would like to have seen a bit more courage around the S4 cohort with a focus on their learning and their wellbeing, because it has been a challenging period for them. I understand the line of argument that Mr Flanning is pursuing. Briefly, that is the line of argument that you are pursuing, which you pursued last year not with a successful outcome, but you had the good grace to acknowledge that the overall results in terms of the highest achieved was a good outcome. For the record, I should say that my understanding is that the Scottish Government placed immense trust and values the work of teachers enormously. Are you still in discussions with the Scottish Government about your tweak or amendment to the approach that it has with regard to S4? Is this an on-going matter of discussion between the Scottish Government? We are still involved in what is now called the NQ22 group, which is not meeting as frequently as the NQ21, because there is not an immediate pressure on contingency, so there are discussions in that group. I do not think that there is no discussion around S4 that that decision has been made, but there is discussion around in the event of the diet being cancelled what the contingency would be. We do not think that it should be the ACM model as ran last year, nor do we think that it should be the expediency of the previous year. We are quite keen that schools are clear what the contingency would be and that any contingency minimises additional workload or additional assessment for students. The SQA has put out guidance to say that any contingency would be based on naturally produced evidence, so tick to that is okay. What does that mean in terms of quality assurance? The issue last year was not so much around the evidence, but it was around the quality assurance process to validate that evidence. We would be comfortable with the notion of naturally produced evidence, but there are issues around that. Cocab Stuart is the deputy convener. Thanks very much, Stephen. It's nice to see you again, Larry. I will sort of declare an interest at the beginning. I am a previous member of the EIS and currently I am an associate member of the NASUWT, and I am registered with the general teaching council currently as well. Having got that over and done with. We took lots of evidence from young people over the last few weeks, so I have been speaking to children and I am trying to put that together with your presentation as well. I am interested further about schools defaulted to running an exam-type assessment, and that takes us back to Tava's comment about the compression that happened as well. We heard a lot from the young people about the additional stresses that they were at great pains to say that their staff and teachers helped really well by the way. Can you go into a little bit more about how that happened? The message was that it was not meant to be exams, so how did that happen at school level that it turned into multiple assessments that were regarded as exams? I think that there are a few aspects to that, so there was quite a contested dialogue within the NQ21 group around the validity of evidence. If you look at the ACM guidance, it is very clear that it is for teachers to decide what is appropriate evidence to arrive at their estimates. That evidence can be a class test, a preline paper from another year or class assignments. There was unilaterally issued to the system SQA subject guidance, which in some areas, particularly maths and physics for example, was lent very strongly towards valid evidence based on exam practice, because it tests the full range of skills and a number of schools immediately latched on to the SQA advice. It was hotly contested within the NQ21 group as to what was appropriate. Ultimately, it was left for schools to make decisions on what they regarded as the best assessment approach for their students. As you will know as a practitioner, the arrangements will vary from subject to subject. I am an English teacher, so it is pretty straightforward for me to have pupils write a critical essay, write an essay to do a close reading passage, and I have the evidence that I need. In maths, where the topics are much more disaggregated, they will tend to go for an integrated paper. Without doing a disservice to maths teachers, maths teachers would probably quite like a past paper because it covers a broad range of skills. That was left for schools to make decisions. It was not required that a school had one approach. The other big factor that then came into play was the three-month lockdown, which was not anticipated. Pre-Christmas, we were arguing with the Scottish Government that there should be an extended break over Christmas as a fire break. That was being resisted, and then we came back to three-month lockdown. The advice to the system was that evidence gathering should not happen during lockdown remotely, because of the challenges around validity, equity and access and so forth. When schools came back, there was a very truncated window in which that evidence could be generated, and bearing in mind that hires had only switched to an ACM model in December, so some schools had not carried out pre-lamps. What then happened in a lot of subject areas and I would think that the majority of schools have not actually factually assessed that yet, was that people started going for many exams in the classroom. If you are in fourth year in particular and you are doing seven subjects, that is just a lot. The reason that we did away with unit assessments was because the unit assessments were ending up as just a treadmill of assessments, particularly around February and March. Some schools organised timetables and put people off timetables for study leave. I can understand why a majority of students thought that this was just an exam diet. What schools did in a lot of instances was that if a young person did not achieve what was perceived as their potential first-time round to try to give them a second chance. In one sense, that adds to the assessment, but it also gives the student a second chance and you can go with the best result. All of that period, and that is when a lot of the media coverage around issues of equity were happening. All of that was not the ACM model per se. That was the impact of three months lockdown and the fact that there was a very tight guideline. To finish, you will see in the documentation that we pushed within the NQ20 group one group to get extensions to deadlines to try to maximise the time that schools had to overtake the assessment challenge. I think that that was very full yet. I am happy to say that Willie Rennie is going to come in on this subject. Willie Rennie, I am keen to hear from the other panellists who are online as well on this. Lari, you were quite critical there of, effectively, that there were exams of sorts, although we were trying to do assessments. Some have claimed that the assessment process is much less stressful than an exam process, the big-hit exams. I am just wondering whether this year really tells us anything about whether that is right or not, because, as you say, many teachers and pupils were incredibly stressed by the repeated assessments throughout the period. I am just wondering whether it does really tell us anything. Secondly, on your point about changes to the fourth year this year, does that not really bring up a much more significant question about the leaving age from education? If you want pupils to leave with something, does that not mean that we actually have to change the age that they leave school or education? Lari, you come in first, and if you would briefly answer that, then let's go to Seamus, because he hasn't said anything yet, and I want to bring him in. On the second point, part of the curriculum for excellence senior phase arrangements is that, at the Brinwick Christmas of S3, schools are supposed to map out a three-year programme for young people to the age of 18. That might involve leaving school, college or work, but there is meant to be a three-year plan. Schools have a responsibility for planning 16 to 18 in terms of the onward learning journey of young people. There are real difficulties around that because of the way that our qualifications work at present, and because of the challenges around S3 and the lack of connection between BGE and the senior phase. On your point about whether we need to raise the school leaving age, de facto is being raised because students are staying on at school. When standard grade was introduced, 90 per cent of boys left at the end of S4. We now have 94 per cent of pupils staying on to S5, and around 85 per cent I think it is staying on to S6. You don't need to raise a leaving age. Young people are staying at school. We just need to cater for that. On the first point, do we learn anything from this year? Absolutely. We need a mixed economy around assessment because what works in English does not necessarily work in science, and what we need to get away from is high stakes do or die assessments, because that is unfair. Some people relish the challenge and do well, and teachers, by and large, are winners of that because they have come through the education system. However, it is not the only way to assess, and that is what Stolberts tells us as well. It does not mean that you do away with exams. It is just that you put them in a context whereby that is not the be-all and end-all, and we know that the high stakes exams disadvantage people from poorer backgrounds. A mixed economy in which progression can be made by young people and they can bank. That is one of the things that Scottish education likes—to get something in the bank. That is why we have S4 qualifications. People like to get something in the bank as a fallback, so we need to build that into the system. The really big thing that we will learn from this year is that young people and their parents trust teachers. Our qualification system should be built upon that premise to put trust in the professionalism of teachers to validate young people's learning experience through qualifications. In terms of some of the background, I want to focus on that. What has been said this morning is what many of our members reflected on what was their experience. Partly before we got into the ACM that was applied after Easter this year, our members were very worried about the young people that they were teaching and the impact on not only lockdown but Covid and the disruption that it caused to their families. A major thing for us was that, when we got to after Easter and the ACM was implemented, the teachers were confined only producing evidence that was demonstrated during that period. That did not take into account that children and teachers had been off during that period. In some instances, some youngsters were missing a vital period of time in terms of the assessments. However, we can really focus on that. What about all the knowledge and experience that those teachers had of those young people that has been gathered over many years? Every school, as soon as a youngster walks into a secondary school, is trapped and monitored on a regular basis in all their subjects. That helps in their development. When it came to the new model that was introduced as of April 2001, it was purely ignored. In many cases, that hurt our members. That is why many of them were frustrated that they were under a very strict timeline to produce evidence. Again, it is not quite clear what was the evidence that was required. In many cases, they over-collected evidence in the short periods that tried to get something that would be approved by the SQA. Our view was that we should have been able to use previous knowledge and experience. It is all about trusting teachers. Trusting their professional judgment means that the teachers would have to, in some instances—we are not talking in every instance—overwrite the evidence that was available. They could justify that because of other things that had gone on in the past or the difficulties that youngsters faced at that time. Again, that is an issue for us—the communication. Communication appeared to be only one way. I raise that because SSTA represents six and a half thousand secondary school teachers and people-ahead teachers as well. When it came to the dialogue with the NQ21, SSTA refused commission to join that group. It has not been invited to join the NQ22 group either. The communication from those who represent teachers in the classroom is not getting back to this particular group. If you look at the membership of the NQ21, there is only one representative that represents those who deliver the exams in the schools. Therefore, there is a deficiency there. When an argument is taken place at the NQ21, it is one person against the rest. I feel that we need to make sure that we have balanced the communication of what teachers are trying to move forward from. In terms of where we go in the future, the ACM has got some real good things that we need to look at. It was the application of it this year that actually caused a greater problem. The last bit that I want to focus on is the moderation. If you think back to 2020, the teachers gave their professional judgment and that was accepted. There may be some people who had concerns, but when it came to the 2021 exercise, it involved a whole series of different moderations, as if they did not trust the teacher. There was moderation at the department level, then at the faculty level in the school, school level, moderation with other local schools, the local authority, the RICS and then at the end of the SQA. Therefore, we got to ask ourselves how much of that was necessary, because that is one lesson to be learnt. We need all those different levels to deliver that. Unfortunately, the impact on the pupils is the downside of that. If the youngsters experienced between April and June this year is what they think exams are going to be like in the future, many of them were locked out. At that period, we should have been building on teaching and learning, encouraged them and, as you mentioned earlier, put in the fun back into education, but we did not do that this year. We drove them into the ground to get the assessments. If that is one lesson to be learnt, we have to make sure that we assess youngsters properly and use teachers' professional judgment as an overriding concern issue in the process. Willie, do you want to come back in? If you do not, I will go to Oliver Mundell. Oliver Mundell, I just wondered off the back of those comments from Larry Funnigan earlier, whether there is a feeling that the SQA was given too much say in the ACM model and that they were trying to reintroduce their own normal methods earlier in the process. It is something that came up in the earlier session, but it really feels that the voice of the SQA was stronger in developing the process than of teachers in the classroom. Do you feel that balance was right? One of the points that Shaymie said was that there was one voice in the Q21 group. To be fair, School Leader Scotland and ADES were often on the same page as the EIS. There was quite a collective voice around the realities of schools. In terms of the quality assurance process, there is a level of quality assurance, even around pre-alarms. You get cross-marking, you get principal teachers sampling and so forth. That clearly was expanded. That was why the quality assurance was one of the biggest drivers of workload, because people were marking other schools as a double check. Local authorities were supposed to provide supportive quality assurance processes using their QIOs, although not all have QIOs. I think that all that local authority side of quality assurance sits as a cohesive approach about supporting understanding standards and making sure that schools are getting it as accurate as possible. The SQA quality assurance is a different agenda. That was where it sampled scripts. It probably wanted to sample more scripts than it did. The SQA was impacted by three months lockdown as well. The key issue for us, and we were supported by ADES and SLS, was that that sampling was about a professional dialogue with schools and was not about SQA overturning any school results. SQA had its fingers burnt from the previous year with its algorithms and its quota systems, which, if you want me to talk about that, I am delighted to because that is where the real fault lies in high-stakes exams when you have quotas for A's and quotas for B's and the kids who are season D's that end up losing out. They were quite clear that they were not looking to overturn school-based judgments. That was one of our red lines in the NQ21 group. The SQA could have a quality assurance input, but it could not veto what a school was deciding around qualifications. There was, even during the lockdown period, a lot of engagement from staff around understanding standards, which were webinars run by the SQA. The whole thing was under huge pressure because of the truncated period when pupils and staff were back in schools. To reiterate, I think that it is a testament to the work that was done, that when the results were officially announced and pupils knew the results before the summer anyway, there has not been a huge outcry the way there was the previous year. By and large, the student cohort is content that other learnings have been appropriately accredited with the qualification outcomes. Do they feel that the SQA had too strong a voice in developing the ACM given the clear failings in 2020, or that we are trying to retain influence over the process? I think that after 2020, the SQA wanted to put control back on the system. What we got was an SQA model that was implemented by different groups on the way, but it was the same thing. I think that we have to realise that it is about the young people that we are teaching and that they want them to have a positive experience in schools. However, the national qualifications and the way that it was implemented this year were just re-voting back to time. I will go as far as to say that, as for 2022, the intention is that we go back to where we were before. The worry for us, even though the SQA has sent out messages, is that schools will already start to collect information for the version of the ACM for this year, should the exam not be able to go ahead. Therefore, we are already increasing the amount of workload that the people are doing. I think that there was an opportunity with not knowing what the future was going to hold to work with teachers. That is the point that I wanted to try to make earlier, is that you need to listen to the teachers who are delivering in the schools what their experiences are and try to work a system that works to everybody's benefit and does not put horrendous pressure on everybody during a very difficult time. My concern is that the SQA decided to go back to an exam. We were not consulted on that basis. It was made by a decision by the AQ21 or AQ22 group, whichever one it is, but there is real concern that we should have planned for this and put something else in place as a make-weight that we could be managed early for teachers to plan and guide their youngsters through this academic year. Tara, I think that it is only fair that we should bring you in. Thank you. Just to add that since 2020, NASWT has been calling for the SQA to bring teachers into their confidence and to make them a core part of that discussion and decision making because of the wealth of expertise that they can bring to those discussions. As has already been outlined, there has been a lack of collaboration and engagement. NASWT is not on the national qualifications group either. It was hoped that, given the controversies within 2020, that there would be recognition on the ground that a more inclusive approach which put the views of teachers at the heart of those decisions would be taken, but that has not been the case. As it turned out, the approach adopted within 2021 very much felt to teachers as a done-to, not a done-with. Thank you for that. I am going to move us on so that all colleagues can contribute. I am going to turn to Stephanie. We have somebody else waiting there. Thank you for coming along today. I am quite interested in how you guys will incorporate young people's views to how you feel we move that forward and how that influences you. Clearly, the three of us here are primarily concerned with representing the views of our members collectively. Sheamus and Tarr have outlined that they are not being involved in the NQ21 or the NQ22 group. We have a dialogue within the teachers panel of the SNCT, but that is not a direct line in that. That is a decision that was made by, presumably, the SQA, who invited us to join the group. I think that it is important that the experience of young people is captured. That has been representation on the NQ21 group from the Scottish Youth Parliament. The SQA has, I think, got a learner's panel. One of the issues is that, because of the relationship between teachers and pupils, we often get feedback from our members about how young people are feeling about things. We have some insight into some of the challenges. If you look back at our EISs' communications around the pandemic, the health and wellbeing of young people has been one of the key issues, because it is difficult to teach young people if they are in a stressed place and facing challenges on a personal level. There is a balance to be struck around looking at qualifications, because every S4 cohort is new to qualifications and they do not know what they do in the sense of what are the alternatives. Young people who do well in exams like exams, because they do well and young people who do not do well in exams wish there was something else. It has been interesting last week to hear some of the comments on continuous assessment. You need to think about how young people learn and make sure that that has been factored in. I think that part of the approach that was taken at a school level, and the convener referenced it earlier, is about the fact that lots of young people felt that the teachers were there on their site. That is a critical dialogue. Even in terms of young people having given information about their progress last year and being kept in touch, there were those in the NQ21 group who wanted that to be a summative thing at the end of the year, basically the school telling the kids that it is the last-week term that they managed to get a C. We were clear not that that has to be an iterative process across the whole year, so that teachers are talking to their students and keeping them informed about progress, what the next step is and what they have to do to improve. It is that kind of on-going dialogue that is critical. I am just slightly wary about setting up panels, and we think that that is capturing how young people are feeling, because the key issue is that, as a panel itself, it is represented above most young people. We need to be alive to listen to young people at every level, but particularly at a classroom level. I do not know whether you want to put that to anyone else as well. I have been interested in hearing from both of you. It is very important that we learn from the experiences over the last few years and talk to youngsters that have been through the process. In addition, when you start into the process, you do not really know what the options are, but teachers do. We probably need to think about different ways for pupils to go through the system, but teachers should also be listened to, because they are engaging with young people all the time. I am not saying that they can speak on behalf of their pupils, but the teachers have a view of how that has impacted on the young people. That needs to be taken into account as well. Just because we are teachers does not mean that we cannot speak totally on their behalf, but we can put a view of how we see our young people and how they fit. During this period, during and able to do, our members are very worried about the stress and the pressures on young people, which they felt was totally unfair. That is something that we need to take on board and make sure that we do not repeat. I reiterate the comments that were made by both Seamus and Larry that it does not preclude engaging with young people, but our experience of feedback from our members over this period has been that they have strongly been advocating on behalf of their pupils, in addition to making representations about the impact of the alternative certification model on themselves, in their own wellbeing and in their own workload. The frustrations voiced from practical subjects about pupils, in essence, being made to set a driving test without ever having sat behind the wheel of a car, were representations made with the best interests of their pupils in mind. While, under the UNCRC, there absolutely is an onus on the system to consider and incorporate the views of pupils and children and young people, we should not think that those of us who are advocating on behalf of teachers are not also incorporating those considerations within our own representations. I just wanted to ask as well where you feel the focus of this committee should be, going forward? Where can we offer the best value? Larry, would you like to ask that question? The big issue going forward is the OECD report and Ken Newe's consultation. That will require systemic change. As I said earlier, I feel a bit as if I am just looking at the age profile here. I used to go to pictures and I used to go in, sometimes you went halfway through the movie and you waited and saw the second half first, and then you got up and you said, this is where I came in. This is a wee bit of how I feel, because I came in here with the last big changes around a senior phase. It now looks with the OECD report as if we were getting back to where we wanted to be 10 years ago, which is to look at how you have an assessment system that captures the ambitions of the senior phase and CFE. I think that this committee might want to focus on how we progress that, because it would be absolutely brilliant if we had all-party buy-in to changes that were beneficial to our senior secondary schools, not our senior secondary schools, as we come back in time again to our secondary schools. That is really helpful. That is what the OECD encourages us to do, is not it? Go back to the vision. Ross and Ross will be followed by Willie. I am afraid that we are doing our own bit of compressing in terms of the time that we have left this session. Larry has distinguished a couple of times now the conversation between the problems that were inherent to the ACM and those that were compounded by the lockdown period and school closures of January to March. When our predecessor committee was scrutinising the SQA last autumn and spring of this year, it was very hard to get an understanding of what scenario planning had been done for a period of prolonged school closure during the year. What was your understanding of the scenario planning that had been done by the SQA and by the Scottish Government from last summer? The answers that we were often getting were essentially just repeated affirmations that schools were not going to close. Were you aware of any scenario planning being done of the impact of prolonged closure on the certification model? Before the national qualifications 21 group, there was a SQA contingency group that was called. There was an on-going dialogue around the 2020 diet and the whole algorithm agenda. There was discussion about what would happen in the subsequent year and some of that was about the S4, for example, and what might happen. To my knowledge, there was no serious discussion around the potential for a three-month lockdown. I do not think that that was anticipated. As I said earlier, pre-Christmas, we were trying to get a firebreak, and that was being resisted. The Kent variant, which hit Scotland in December, was a game changer in terms of that. The contingency that was being developed around national 5 in October, when the national 5 exams were cancelled, was mainly to do with disruption rather than lockdown. That became the template for the ACM model for S5 and S6, and that became challenged by the full lockdown period. If your question is asking, do I think that there was sufficient contingency planning, then probably not, but hindsight is a wonderful thing. If Seamus or Tara want to provide any further thoughts on that, please do, but I will move on if you could wrap up the answer to the next question. That would be great. Seamus, do you mention the... I have just been advised. You do not have to be so compressed. Would Seamus or Tara like to make any comment on scenario planning and whether it did indeed take place? As far as we were concerned, we raised real concerns in the summer of 2020 about what was happening in 2021. We talked about discussions that took place in other places, but we were not party to that, not only the AQ21 but the SIR group as well. As far as we were concerned, we were going ahead as planned, so your answer to the question is contingency plan. We were unaware of that, and I would imagine that most of the public would be those of the signed view. Tara, were you in a similar position representing a union that was not on any of the relevant Government groups? Yes, we were in a similar position, but obviously we do engage in informal dialogue with the SQA, with the national qualifications group. I probably would stress to the committee that it is a concern of any SWT that the lessons about that lack of contingency planning have not been learned looking forward to potential disruption for this year. For example, we met informally with the SQA on 28 May, and at that point we were asking that it was really important that there were sufficient contingencies in place for the year coming so that we could avoid the mistakes that had happened with the ACM. There was clarity in the system as to what the potential options were moving forward and more than the options that we were clear on what the triggers were for moving from one option to another. Further in direct communication with Government, we were stressing the need for a decision on exams to be made before the summer to again provide that clarity and consistency of messaging to the system. As it turns out, we did not receive any clarity on that about exams going ahead until 18 August, over a week and a bit after some schools had returned. Indeed, the SQA scenarios were only published on 15 this month, again over a month since schools returned. That scenario planning is attached with a caveat that says that here is a broad outline but more information will be provided in October. There is a consistent jam tomorrow approach being adopted by the SQA in terms of information provision to the system. Thank you. That sounds weirdly like exactly the same conversations that we were having this time last year. I move on to a question around the moderation system, so shame is you listed the various levels of moderation that provisional grades had to go through before they were approved and the workload issue that that created. I am interested in the feedback that you all had from your members about how much that moderation changed potential grades from what a teacher might have initially been minded to give compared to what was eventually assigned. Did that moderation process result in much in the way of grades changing? Was there a particular level at which that was most common? Did grades typically change based on the conversations that were taking place at faculty level within a school? Or was it at local authority level? Did the RIC level moderation influence grade changes? Was there much change as a result of that process? Feedback from our members, there were some alterations in that going on. There were pressures in some places to downgrade grades because they were out of keeping the previous years. These examples do exist. It was not too many asking to be pushed up. It was more questioning what level the grades were. It did happen at different points. Sometimes it happened in a department, sometimes it happened across schools, and local authorities, whether they like it or not, did do that exercise of looking back at the history of the schools and using that as a guide. What the schools did not want to do was to put their heads up above the parapet and be identified as some concern. Many schools were conservative in their grades and tried to keep them in keeping with previous years. That is only what we have been told from a range of different members. I can imagine that happening in many cases. I accept that this is anecdotal feedback from your members. In the areas where there was pressure to downgrade, was that disproportionately in schools and areas that historically had lower performance that is typically linked with socioeconomic status and deprivation? If I have said that we had very few from the high-performing areas, that might be a way of answering it. Tara, what was your member's experience? Was feedback similar or was it a different experience? Feedback is variable. It is one of the challenges within the Scottish education system that consistency is not necessarily seen across the picture and that within individual schools and local authorities and ricks, there are different approaches that were taken to moderation. The feedback that we had came from members who felt confident in themselves to push back where they disagreed and where they felt that their judgments were sound. The feedback that we received was from members who were informing us that those kinds of discussions and interactions had taken place. The concern may be that we may not have heard from members who had those discussions and who felt that their judgment had been overturned, perhaps wrongly, but did not let us know. I do not think that we have that oversight across the system in order to make evidence-based decisions on what did or did not happen in practice. Partly that is down to the flexibility and variability inherent within the system. If there is time, I have one final question for Larry. A key part of the quarter assurance to stress was the understanding standards initially. There were a lot of school teachers engaged in professional learning about understanding standards. A lot of the cross-marking is a second marker. In small departments, it is useful to get another school involved in it. We do not have any difficulty with the quality assurance other than the time that it requires, which is always a challenge. Similar to Seamus and Tarot, where we were contacted by very few members who said that they had been told to raise the grade boundary or lower the grade boundary, we simply said that that is not what the ACM model says. It refused to do it and it was few and far between. There is a big discussion around grade boundaries because, in the absence of national data in the SQA setting a grade boundary, some schools varied it, some schools stuck with 50, 60 or 70 as their defaults. There was no easy solution to that other than setting a national standard. If we send cases back to the point around, we will just get national exams if we set a national standard. Over a piece, I would say that our members are assured that their judgments held sway in relation to the final qualifications. In the few exceptions where that was not the case, we certainly challenged that. The final question is just for yourself. Given that your union also represents college lecturers, the experience of college students and lecturers is probably a significantly underexplored aspect of the past two years. I have had a college student in touch with me recently to point out what they felt was the inequity of a system in which they were in the end graded on the same terms as any school pupil, but they had spent the entire year learning remotely whereas there were at least periods of time where school students were actually in school in a classroom, that was not the experience for colleges. How would you reflect on the communication that the SQA was issuing to both college lecturers and college students and what their experience was relative to teachers and pupils in schools? We had a very specific dialogue. College of Scotland represented on the NQ21 group, but we at EIS Feela had a very specific dialogue with the SQA around college national qualifications and other qualifications in college. It was a robust dialogue in terms of the discussions. There was clearly a different dynamic because you were talking about adults rather than pupils, so as you say, lockdown measures and remote learning had a different role to play. I think around national qualifications, the same exams that pupils would be sitting, ultimately we were comfortable with the arrangements, which was the same professional judgment based on the evidence. Your point is that some students might feel that they did not have the same opportunity to produce the evidence, and I would accept that that would certainly be the case. As part of our dialogue, we provided them with SQA with quite detailed a student survey that we carried out in relation to the impact of Covid on students from disadvantaged backgrounds. It was quite dramatic in terms of the impact on their potential to learn. There were other qualifications that SQA oversees in the college sector, some of which had to be deferred because they were based around a competency model, so you cannot accredit somebody with safety standards if they have not completed a course. However, our message around colleges would be similar to the one around schools, which was that, ultimately, college lecturers pulled out the stops to try and make sure that young students were accredited appropriately. However, I take your point that the learning experience for many students was disadvantaged because of the fact that it is a different cohort. Young parents with families and childcare responsibilities have to miss lessons because their kids are off school, so it might come back to the point around pupil student voice that we should be listening to students about what their experience was. Thank you for asking that question, because I have representation from students in colleges who have reported similar experiences to the ones that you just described. I want to turn a little bit to slightly beyond the qualifications process in terms of the learning. Maybe less about the immediate assessments. I mean that I was struck by Seamus' comment about pupils being driven into the ground to get assessments and the convener's remarks about compression being the kind of the theme. We understand why there was, in essence, less teaching across the year for a variety of reasons, particularly for some cohorts, so less time in school, difficult access. Taking evidence from young people in the last couple of weeks is very clear to me that many of them feel that they haven't learned as much as others have learned, understandably. I wonder if reflections from the three of you in terms of the challenges that that is going to present as young people progress into the next stages of their qualifications or indeed their lives. I think that there is a challenge there, because there is an element whereby qualifications are a passport to your next step. However, there is a certain point where, if the qualifications are based upon reduced content to the extent that you don't have the skills that the qualification should represent, that creates a challenge for you. I heard Tony MacDade talking in the last session about young people who have got a qualification and moved on to the next step, but they haven't consolidated their skills and are now struggling at the next step. One of the issues that goes back to why we have said that S4 should not be doing exams this year, should be focusing on teaching and learning. One of the issues that we have raised within the qualifications group is that universities had to be flexible in the way that they accommodated school leavers on to courses. The whole system is focused on the relevant grades so that young people can get to university, as if the university can do what it does. I was saying, why doesn't university look at the fact that there has been a disrupted learning here? It has to provide in the first two or three months some consolidation work in the way that it would do for people who go to university without necessarily having formal qualifications. Do we think that that's happening? No, it's not happening. The response to that was that we need to concentrate on getting young people the qualifications. I was saying that there was a case in the papers that a student who had not got into medicine because she had got four A's and a B rather than five A's, and I was saying, well, that's just ridiculous. If that young person has been denied that course because she has got a B instead of an A, what responsibility has the university shown in terms of whether there is a good candidate? We might have to bring some additional support initially, but why shouldn't it be doing that? If we have an integrated education system, the university shouldn't be doing their own thing separate from everybody else. I'm sure that the universities would talk about the cap on student numbers as well that the Scottish Government put in place. Siamys and Tara, when did you have comments on those issues? In terms of following on from Larry's said, the system needs to adjust, but I think that it's assessing what the youngsters have missed is important and teachers will be doing that, but what they need is the support to actually make that difference. Obviously, we have additional teachers to try and break down the challenges that teachers have, but one of the obvious ones is the multi-course teaching that takes place in the senior part of secondary schools, where we have national hires and advanced hires in the same class. That means that they only get, if lucky, a third of the lesson. That's a thing that we should have been striving for this year to make up the work that needs to be caught up with, focusing that they had a dedicated teacher on doing national five or higher and not on advanced hires separately. At the moment, we've still got that situation where you've got three different courses, and they are different courses that take place in many subjects, particularly the softer subjects. Therefore, the youngsters aren't getting the dedicated time with the teacher that they deserve, they're getting a bit of time. I think that that's something that we should have been trying to address this term. I think that, just to add on to that, it's a question about what the purpose of education is for in the first place. We would want to move away from putting children and young people, putting pupils into lanes and shoving them through exams. The purpose of education is broader than that. It's to instill in individuals a love of learning that they can take forward with them throughout their life and their career. I think that some of the feedback that the committee has heard today shows the stresses and strains that have been placed on fostering and embedding that love of learning within the system. In addition to looking at moderations at further and higher education, which we would absolutely agree with, one of the aspects of education that has somewhat been lost has been the discussion on the impact on children and young people with additional support needs. Some of the feedback that we had regarding the 2021 system is that a lot of the supports that would otherwise have been in place were diverted, whether that was PSAs or absences due to Covid-related absences. That had a knock-on impact. I think that, as a system, we need to ensure that when we're looking at learning in the broadest sense that we're doing it in an inclusive way and that we're ensuring that we're getting it right and that the challenges of the system are not only talked about in the broadest sense, but are also talked about specifically for pupils who have additional support needs. All of that is very useful to me. When we're considering the design of the 2022 system, absolutely the broader impacts have to be part of it. It can't just be the assessment model, it has to be the reality of what teachers are facing in the classroom and the circumstances that young people have faced. My last question, if I can, relates to appeals. We had a situation where we've had a low number of appeals. Mr Flanagan has already said that most pupils seem to be satisfied. I've had representations from a significant number of pupils across both cohorts in the previous years under the algorithm this year, under the alternative certification model, who are greatly concerned that exceptional circumstances have not been accepted within those appeals. Many people have faced exceptional circumstances. Should they have been included in your views and should they be in the future? For one point of clarification, I'm not too interested in issues about additional information. I know that the provision was made for additional information to come through in September, and that's in some of the submissions that are written. I'm talking about exceptional circumstances that were faced. From an EIS point of view, we have supported the idea that the awards should be evidence-based. That was partly a protection for the teacher, because if you don't have an evidence base and a student supported by their parents appeals it, you're in a difficult situation as a professional if you just say, well, it's my opinion. That has to be an evidence base. Even in 2020, when we had inferred progression, we did have an evidence base because we were at the, in March, we had, by and large, prelims, we had classwork. What teachers then did was infer the progression from the prelims. If the prelim was December, you would normally expect 10 per cent improvement to the exams. Teachers did, and people went in during the Easter break and the schools during the lockdown to get the evidence. There was an evidence base for that. The key issue is that the evidence base wasn't only exams. In the ACM model, not the SQA guidance, the evidence could be a class charter. It could be an observation that a teacher had made, somebody delivering a talk. An evidence base is something that we support. On the question of appeals, Michael, I think that there is a challenge here because the previous appeals system, which SQA operated, was evidence-based. So, if on the day a student couldn't sit their exam because of family circumstances or whatever, the schools could then submit alternative evidence as a basis for an appeal. But the appeals this year are based on a reappraisal of the existing evidence. So, if a student is given somebody a C rather than a B and the appeal, the school will submit the evidence that is used for that judgment, and the SQA will decide whether to overrule that. That doesn't take account of why the evidence is as it is, which is the exceptional circumstances that you are alluding to. However, I think that there is a difficulty then, because somebody has had a very traumatic experience and you would be sympathetic, you would necessarily give them a grade that they cannot justify. I think that there is an ethical dilemma in that. In an exam situation where they have missed an event, you could say that you can understand why we are looking at alternative evidence. However, if it is simply based on, let's say that it is based on somebody who got knocked down and had to spend three months in hospital and therefore was ill-prepared for the exam, could you still award what you think they might have got if they actually have not had the teaching and learning? I think that there is an issue there. I think that our view would be to be favoured evidence-based approaches. Seamus, Tara, is that any comments from them, convener? Is that okay? Very briefly. Again, the appeal system, as Larry has outlined, was very much based on the evidence that was provided. Therefore, any chance of getting an alteration was limited because of the moderation process, and that would be geared against it. However, the problem was—as I mentioned earlier—teachers did have a worry that the youngsters were not getting the evidence that they needed to the attainment levels that they should have received. That is the problem for us, is that we want the teachers to be able to override the evidence. I understand where Larry is coming from. If you have the evidence, you cannot argue about it, but you have to, at some point, accept that teachers want to give the youngsters what they should be getting, not just what is in front of them. I think that the ACM has taught us that we need to have that debate on what point does teacher professional judgment take over the evidence, and we are not there yet. It does happen with standardised assessments that teachers can override the assessments that I have done at that level in S3. However, we need to have that conversation, because teachers' professional judgment is something that has gained through years of experience, and we are not using it at the moment. I feel that what we are doing is relying on the evidence because we want something to forward. It is a built-in braces approach, and I think that we need to go beyond that, but we are not there yet, and that might be something that will come in the future. I quoted Seamus, your statistic earlier, from your survey. 36 per cent of teachers believed that evidence that he had collected truly demonstrated the people's attainment, which means that 64 per cent did not feel they had, which is not a startling statistic from my point of view. I think that for the remaining 10 minutes that we have is over to Bob. He has the privilege of the last round of questions. Thanks, convener. I will try not to use that 10 minutes. I should put on the record that I am a former secondary school teacher and a former member of the IS. Indeed, they may still be taking my subscriptions. I am not sure, Mr Flanagan. I want to check that out, but I have now put that on the record. All witnesses have said that they agree with a moderation process, and that is important. I think that that is what everyone has said. It is about the extent of moderation, the burden that that might be put on teaching professionals and the proportionality of it. I think that everyone is wedged to the idea that moderation is important. Mr Flanagan also put in his evidence that moderation provides teachers, teaching professionals with additional protections. I remind him that Seamus spoke about teachers who think that a pupil is working beyond what evidence they have, but the contrary can also be true. That might be a teacher who thinks that a young person is working at a sea level, but they want a sea level unless you can evidence-based how you would use your professional judgment. I can see that teaching professionals get into all kinds of difficulties. I might ask Mr Flanagan more about the protections that moderation provides to teaching professionals. In my constituency experience, I found that there were difficult conversations to be had between teachers and young people when they were informed about the grades that they think that they were working at and that might not have been the grades that they were aspiring to. It needed a lot more direct, good-quality communication between teachers and pupils about that kind of thing. The key point is that teachers want to get it right for their students. There has been a huge commitment from staff investing in the quality assurance process and the estimates so that they were getting the outcomes. Every teacher knows about unconscious bias and so forth, so we put in measures routinely to make sure that we are being as objective as possible. You touched on it and said, by the way, Bob, that there is no refund policy in the EIS, so just in case you send me an email. You touched on it there yourself because one of the key things is that over the course of the year, teachers are actually feeding back to students about how they are doing. I know that you want to get a B, but at the moment, your words indicate a C, so here is what you have to do to step up if you want to improve on that. It is not difficult—it is part and parcel of the job—but it is critical so that young people know where they are. I might say to a student that I think that you are capable of an A, but it does not mean that I am going to give them an A unless they demonstrate the knowledge and understanding and the skills that apply to that. It is a constant dialogue. The key point around the protections is so that parents and students have confidence that their teacher's final iteration of the grade is based on that dialogue across the term and the evidence that the student knows she or he has produced. That is part of the teaching and learning process. I would not want to see—there have been very few appeals, around 4,000, which is relatively small in terms of the number of processes. I would not want us to be going into an appeal system with one of our members and the evidence to support the judgment is that I thought that the candidate would do better or that I thought that the candidate would deserve A because it would be hard to defend that. You could have parents going, I think that my son or daughter deserves five A's and you have given them five C's. The only way you can resolve that is by having evidence. I do not think that Seamus and I are too far apart because I think that there is a much greater range of evidence that is available than just past SQA papers. That is the key point. It is not just past papers. There is a whole range of evidence that is available. I can see that Seamus wants to come in and I would be keen to hear from him. I have one final question. Can we roll that together and perhaps Seamus could respond to both if that is okay? I suppose that my final question is focusing on the year ahead. We know that there has been a reduction in course content to support teachers and learners in the coming year. We have heard today that there will be more details of what alternative scenarios might look like if there is disruption to schools. That is likely to emerge in October. It is really to ask for witnesses today about what they think should happen. God willing, that does not happen, but there is additional disruption to schools once more. In terms of does it mean that more course content should be taken out, we should refer to a revised form of the alternative certification model. What would that look like? I get that teachers and learners and parents want to have that information as soon as possible. Ross Greer mentioned that the previous committee was keen to see a bit of assurance about what those contingencies look like at the earliest opportunity. For this individual academic year, there will be larger systemic changes to the system, but for the current year that we have just commenced, what would their witnesses like to see happen if, unfortunately, there was additional disruption? In terms of the first question, there is a lot of information on pupils, and there is constant dialogue with pupils and parents about how they are progressing. From the day that they walk into a secondary school, teachers are tracking and monitoring those youngsters. An age student that runs through the first three years of a secondary school, the likelihood is that they can carry on in that direction. The same with a youngster at the university level, they will be moving along with that. It is the joined-up assessment that I would refer to, because, at the moment, it is like that it did not happen. Everything did not happen in the school. That is something that we can be doing, because it ties into the next part of the question. If we have already got this information and we have already got and understood where the youngsters are at, and understood that as they have gone through the system, then it should not be a hard problem at the end of that, to say, in the past six months. That is where we think that you are, and that is where you should be. That is probably the answer to the second part. We should be using that ready to move into a situation should the exam system not be able to continue. I do not want to revert to an ACM model just for the sake of it, but the evidence that the youngsters are doing at the moment and continuing in teaching and learning should be continued right up until the finish, if you like, because we break that to then go into gathering assessment materials. We stop teaching and learning, and I think that we do not need that. We cannot afford that, so we should be looking at what they are doing at the moment and keeping that evidence. That is the evidence itself. We do not need to create new evidence, which the ACM did at the end of the last session, although that might not be what was intended, but that is what actually happened. Schools, as I said, are always starting to do additional exams in any event that exams do not take place in the next summer. I would argue that we need to build on the evidence that we have already got and do not disrupt teaching and learning for this year, because we have lost enough already. Cutting back on the course is not of any benefit. We need to make sure that they have covered the full area of the course, especially in what might be a disrupted experience over the next number of months. If we are looking forward to the year ahead, we would be looking for reassurances that we are not going to have a repeat of the confusion and chaos that we had last year. We would be looking for some protections in the system to ensure that the workload burdens that were associated with the alternative certification model are not replicated, and that is whether those burdens were associated with national guidance or within local implementation models. If we look backwards into the 2021 model, our position was that there should have been some adoption of some form of arrangements that took place in 2020 as the only practical solution to addressing the inherent difficulties within the 2021 ACM model. That is not in any way to say that that is a best practice model. There was no off the shelf. 2020 had its own difficulties, no matter which option was undertaken. They were all suboptimal. In the discussions that we have today, it is really important that we distinguish between moderation and assessment under a normal circumstance and moderation and assessment while we are simultaneously contending with Covid and school closures and all the disruption therein. As part of that process, the key consideration is putting teachers at the heart of those discussions. That is the critical message that we would want the committee to remember, is that in order to ensure that we do not have a repeat of that disruption and that chaos, the views and the voice of teachers has to be considered at the core. The course adjustments are marginal. I am wondering if you want any final comments on that final question. Yes, thanks, Bob. The course adjustments are marginal. They are largely around removing some assessment instruments areas, not necessarily course content. There is a limit to how much you can remove before you invalidate the study. What we absolutely cannot have, and we have already had this discussion with the SQA, is a dual assessment approach whereby schools start banking materials just in case, but at the same time prepare for pre-lims and an exam diet. That creates a workload burden for staff and it creates an assessment burden for students. All that combines to detract from teaching and learning. What SQA has finally put out after some discussion is that the contingency will be based on naturally produced evidence. We are content with that general phraseology, but it comes back to the point about what is valid evidence. In our view, naturally produced evidence at this stage in a year is not going to be particularly useful for summative purposes because students have only started the teaching and learning process. The pre-lim exams this year are potentially quite significant in terms of any fall back that might be required post Christmas. In 2020, a lot of students said that if we knew that the pre-lims were going to be so important, we would have worked harder. Of course, they said that. They should know that the pre-lims will be important. Pre-lims do not test a full year, they only test what has been taught up to Christmas. All of that has to combine. Another thing that we have said very clearly is that just because you cannot have all the students in the big hall does not mean that you cannot have an exam system. Even last year, we said to SQA that if you are pushing for the use of past papers and effective exams, then schools can administer that in class and you can mark it all. That would have been an exam diet, but not necessarily the way that we have staged them previously. All of those things have to be kept under consideration, but I echo Tara's point. One of the big concerns in school is that we do not create this workload burden that will just drain energy from teaching and learning. We do not end up—obviously, lockdown was precipitated by the pandemic, but we could have had decisions around the ACM much earlier, so we do not end up with a last-minute decision. I do not know why we had to wait until August to decide whether there is a diet this year or not, because it was clear that that is what the thinking was mid-summer, and schools could come back to a clear decision around that. Early decision-making is critical. Brilliant. Since we are coming to the end, I think that I might do a quick summary. One thing that I just wanted to get a bit of perspective on was that 36 per cent of teachers the statistics that was referred to by Stephen earlier, I was just looking up the membership figures, so I am still correct in thinking that the EIS represents about 55,000 members, that would be about right. I am not sure about the figures for the NASUWT and the SSTA is that around 7,000, so with the SSTA would that be 36 per cent of your membership? Is that correct, Seamus? It was just to get a bit of perspective on that, about the numbers of the representation. I think that what I am hearing actually marries quite a lot with what we have heard from the other agencies, from universities, as well as from the young people, from the broad spectrum. I am thinking that we have pupils, practitioners and policy, and right at the bottom of that is politics. The OECD said that as well. Moving forward, I am feeling quite optimistic that a lot of your evidence is triangulating with other evidence that we are hearing. I am hearing consistency here about how to move forward, about timescales, clear consultation and communication. Is there anything that I have missed that you want to take an opportunity just to put in, of how we can move forward as a committee? I take Tara and Seamus's points around Serg and the NQ group, so we would certainly welcome all trade unions being involved in those bodies. There is a collaborative process in place in Scotland, which is significant. Even within the NQ21 group, I spend most of my time arguing with the SQA, but collaboration does not mean constant agreement. The fact that we are having a dialogue is important, because I know from colleagues in the EU, for example, and Tara knows this from the NES in England, that the teacher unions are outside of the discussion full stop. The collaboration that we have in Scotland works to the benefit of Scottish education, even when we are having difficult discussions. I think that building on that would be a good way forward. On the Stobart and OECD review, it is important that all the professional voices are part of the discussion. Ultimately, you need the professional voices, the professional associations, to carry their membership around the changes. If you do not carry the teachers around the changes, we will end up with another review in 10 years' time looking at why we got it wrong again. That was a teacher in you that required you to sum up to the lesson. I would like to thank Larry Flan again, Seamus Sears and Tara Lillis, who have been our panellists for the last hour and a half or more than that now. We really appreciate you giving us your time like this. I thank you for the oral evidence that you have given this morning and the written evidence that you submitted in advance of the hearing. We are very grateful for that. We will now conclude our evidence-taking on the alternative certification model next week when we will take evidence from the Scottish Qualifications Authority. The public part of today's meeting is now at an end. I will now suspend the meeting and ask members to reconvene on Microsoft Teams, which will allow us to consider our final two agenda items in private. Thank you.