 A i ni gwybod i eich ddechrau i recognitioni gwaith yn 2021, rwy'n gweithio fewn i gywch blaenauol iaith yn Ffarmatt y Llywodraeth Cymraeg, neu i ni'n gweld i'r iaith i ffarmatt a'u sefydliath yn cyd-gorffygiad spr move yma yn cael ei cyflosigol. Mae'r lleidol yn cael ei weld i'r lleidol a'u lleidol i'r lleidol a'u lleidol i'r lleidol i'r lleidol i'r lleidol i'r lleidol i'r lleidol i'r lleidol i'r lleidol i'r lleidol i'r lleidol Fyonah Robertson, chief executive, Dr Gill Stewart, director of qualifications development and Beth Black, director of policy, analysis and standards. I would very much like to welcome you all to Parliament today and to our committee. I understand that Fyonah intends to make a brief opening statement on behalf of the SQA, and then we'll get into the questions. Thank you very much, convener, and good morning. So thank you for the opportunity to appear before the committee this morning to reflect on the experience of delivering the alternative certification model, the ACM, in 2021, as well as looking ahead. It goes without saying, but bears repeating, that the ACM was developed and delivered in the context of a global pandemic for which there was no script. The whole education system came together through the national qualifications 2021 group to design and implement the ACM with clear roles and responsibilities and worked incredibly hard to deliver for Scotland's learners and to ensure credibility and fairness remained at the heart of our qualification system. This was not straightforward and nor should it have been given what was at stake, but I am proud of the way that learners, parents, schools, colleges, professional associations and local authorities collaborated in the most exceptional of circumstances. That pride extends to my colleagues across SQA who are passionate about doing their work and doing their very best for learners and who over the past year have worked very hard to deliver. Chuffed choices had to be made at times, but the NQ group were clear that the ACM was the best possible approach when we were faced with a situation that was rapidly evolving before us in real time. On 9 August, almost 137,000 learners across Scotland received SQA qualifications and they can be proud of their achievements and have full confidence in their results. The experience of the ACM in 2021 has helped us to develop our approach to this coming year. Following the announcement by ministers in August of a return to exams, should it be safe to do so, we have announced contingency plans that have been developed in collaboration with partners from across the education system and have been clearly communicated with messages and materials shaped by advice from those partners. An NQ 2022 group will continue to be a feature of this year. We will continue to keep learners, parents, carers, schools and colleges updated as further information becomes available, but we have been clear that learners will not be expected to undertake dual assessment, thus avoiding any increase in workload and stress. Ministers remain very supportive of the work that we do, but they have considered that the time is right to look at the national organisations supporting and serving the education system in line with the OECD's findings on the alignment between a curriculum and assessment. We will play a full and positive role in the review process that is under way, reflecting the skills, knowledge and experience of colleagues across SQA, while continuing to deliver for thousands of learners during the transition period. In terms of the future, I welcome the most recent OECD report by Gordon Stobart on assessment and qualifications. The pandemic and the alternative assessment models that have been used in the absence of exams have shone a light on those issues, and it is imperative that we learn from this experience and from the lessons of recent reforms. I sincerely hope that everyone, including members of this Parliament and this committee, positively engage in this debate and get behind Scotland's future approach to assessment and qualifications and the organisation that will be tasked with delivery. With my colleagues, I am very happy to answer questions this morning. Thank you very much, Fiona, for your opening statement. You already alluded to this, so let me ask you this first question. In June, the cabinet secretary announced effectively the abolition of the SQA. Can I ask you why you understand that the SQA is to be abolished? On June 21, the OECD report on curriculum for excellence was published. At the same time, the cabinet secretary announced that it was an intention to accept the recommendations of the OECD report, including the suggestion that an assessment agency was established. It was on that basis that the cabinet secretary intimated that the SQA should be replaced. Of course, there is a review process to consider those issues in more detail, now under way, with the appointment of Ken Muir, former chief executive of the General Teaching Council for Scotland, acting as professional adviser to a review that is being undertaken by the Government. That review is now under way, and we are discussing those issues both with Ken and with Government colleagues on an on-going basis. Where has the SQA failed to live up to its purpose, such that it needs the recommendation that the decision of the Government is to abolish the SQA? Where has it failed to live up to its purpose? I do not think that what the cabinet secretary has said that the SQA has failed to live up to its purpose. What the cabinet secretary has said is that she wishes to consider a new organisation with a new purpose. As part of that, I sincerely hope that there is some continuity of function around assessment and qualifications and the role that the SQA plays. In particular, the role that colleagues with the skills, expertise and experience play within the SQA. I think that what the cabinet secretary has intimated is the potential establishment of a new organisation with a new function and a new purpose. She did announce that the SQA would be replaced. So the reasons for that decision, are they a failure of confidence in the SQA? Ministers have not expressed a lack of confidence in the SQA. The cabinet secretary on the 21st of June set out her reasoning behind the announcement that was made, aligned with the publication of the OECD review. You may wish to ask the cabinet secretary further questions on that matter when she appears before the committee next week. As far as the conversations that I have had with the cabinet secretary and the announcements that the cabinet secretary has made, those are the reasons that I have outlined to you in my answer. Just before I move on to co-cab Stewart, the deputy convener of the committee, how would you describe the role of the SQA in 2021 in relation to what the SQA did? Effectively, teachers' assessments were taken as read. There was some quality assurance work that went on and we are going to come back to that. What, in functional terms, did the SQA do after the announcements of October and December last year? In terms of the responsibilities that we have as an awarding organisation and a set out in the 1996 Education Act, our responsibilities were in principle unchanged. We were responsible for devising and awarding qualifications. Obviously, we were in a set of extraordinary circumstances for a second year. We made a commitment—I spoke to the predecessor committee about this in March this year—to work with the whole system this year to develop an alternative certification model that would fill our statutory functions but, most importantly, deliver for learners to ensure that learners could be certificated and learners could continue with their learning or progressed with their next steps in learning or, indeed, their careers. Our focus was ensuring that we did that and working with the system to do that. Of course, the committee is understandably interested in national qualifications and that is a very important part of what we do, but we are also responsible for delivering many thousands of other qualifications in relation to the college sector to training providers for apprenticeships, for foundation apprenticeships. The reach of the work that we do in delivering qualifications to learners across Scotland, including young people at school, but not exclusively young people at school, was important that we took that work forward. We are also responsible for regulating qualifications provided by other awarding bodies as well. I hope that you do not mind me spending a moment just highlighting the breadth of our responsibilities. I know that the committee is keen to establish on the alternative certification model. At the start of the year, we consulted on modifications to assessment because we knew that there was likely to be some disruption to learning in the coming year. We undertook a public consultation around those modifications. That was across the 150 courses at national 5, higher and advanced higher. My colleague Jill Stewart led that work, and we would be very happy to talk further about those modifications. Those modifications were announced early in the academic year. We then formed the national qualifications group to help to advise us and to develop the approach to our work. Bearing in mind this time last year, plan A was exams. We were looking at a set of contingencies, not an alternative certification model as plan A. The Deputy First Minister cancelled national 5 on 8 October and higher and advanced higher in December. There was a lot of work during that time to develop an alternative certification model, first for national 5 and then subsequently for higher and advanced higher. In tandem, we provided advice on the assessment approach for each of the courses that were available to provide understanding standards events and, as you have highlighted, undertake quality assurance. Setting the framework for assessment for the year ahead. I know that other colleagues are going to come in on the issue, so I would like to move on now. Thank you for your answer so far. I know, as a previous educator myself, that it was extremely challenging times for all those in education, and I literally had to turn on a sixpence, so I really do get that, and I want to put that on record. I am looking back on as a reflective mode. I would be interested to hear from all of you. What are your main reflections on the past couple of years, and what lessons do you think we could learn from that, or what lessons have you learned in your professional capacities? That is a very important question. We have all been in the midst of a global pandemic, which has affected every part of our lives and has been felt very acutely in education, and the impacts have been felt very acutely by young people in particular in education. All public organisations have faced very challenging circumstances. The tradition of a spring diet of exams, which was in place for, I think, uninterrupted for 130 years, was interrupted in 2020 with the cancellation of exams a matter of weeks before they were due to take place and with schools closing, obviously, at very short notice. As I highlighted in my opening statement, there was no script. There was not a model sitting on the shelf waiting to be activated, if you like. I think that we have been very conscious of the impact that the last 18 months have had on young people in particular on their learning and teaching, and how that impacts on their assessment and, from the perspective of the SQA, how that has impacted on their ability to undertake assessment and achieve qualifications. We have been very focused on ensuring that young people can continue to progress in their learning, working with partners and, from our perspective, to achieve their qualifications. We are looking at, in particular, around lessons and, in particular, in relation to 2021, which is the focus of this morning. Obviously, the move to remote learning after Christmas and that becoming an extended period put a lot of pressure on young people and on schools and colleges. The assessment window and the national qualifications group absolutely acknowledged that. I acknowledge that. The assessment window was quite constrained. That was a very challenging, busy period with perhaps more pressure on young people and on the system that we would have liked. I think that there is a lot of learning to take on board, but, in particular, this year, in relation to the ACM, I think that that particular compressed period post Easter. What we have sought to do in the arrangements that we have for this year is to learn from that and to put in place very clear contingencies around disruption to learning and communicate that to the system as quickly as possible. Dr Gill, would you like to add anything, Beth? Just to add to what Fiona said, I think that there were some key communication things that emerged from the 2021 experience, particularly from speaking to young people, that they wanted to know what the plan A and the contingencies were right up front. We have done that as a system and we will continue to do that. It was very important for them to know what is ahead of me. I think that there are other more far-reaching lessons for the education system as a whole around how we develop our remote learning and teaching, not as a substitute for face-to-face learning but as an additional strengthening of learning and teaching. Similarly, around assessment, we need to invest centrally in the SQA and its successor body, but also locally in technology to enable assessment so that we do future proof against things such as a pandemic or another type of scenario. Another reflection would be about co-creation. That has been a journey, not an easy one, because everybody comes to that table with different perspectives quite rightly, and that is what co-creation is about. Inevitably, it requires compromise on everybody's part. You learn to listen and to try to understand and find a way through that. That was very challenging. We had working groups on a weekly basis and the steering group meeting weekly helping us to develop the ACM model, and we have adopted the same approach for 2022. We also had a similar group for HN vocational qualifications with all the key stakeholders from colleges, training providers, sector skills councils, the Government, SDS, Skills Development Scotland and so on, to help us to make modifications to vocational qualifications. We would like to see some of those modifications retained as we move forward, because it has perhaps highlighted some areas of over-assessment in our vocational qualifications. Just to add a couple of points to colleagues. I take your point, Mr Stewart, about turning on a sixpence. That is something that we have learned. We have got to build into our contingency planning for 2022. The ACM-21 model is quite a big model. It involves a large part of the system to have to change course. What we have set out and that is featured into our thinking for 2022, what we have set out through a series of contingences, which really means that learners and teachers can focus on the business of teaching and learning, with a clear course set ahead, knowing that, if exams are cancelled, there is a clear plan in mind. If there is further significant disruption, there is also a clear plan in mind. That should not get in the way of teaching and learning. There should not be a dual assessment model and there should not be additional workload for teachers so that qualifications do not get in the way of teaching and learning, but they very much support teaching and learning. Thank you. We learn lessons, but the most important thing is how we then apply that reflection. Fiona, would you be able to give me a couple of examples of how those lessons and how you are going to apply them in the immediate future for the year coming 2021-22 and then in the medium term? I will endeavour to be brief. I will pick up a couple of things, including a couple of things that my colleagues have highlighted. I think that we have already reflected a little on how the experience of 2021 has informed our approach to the coming year. I think that that is an important part of the development of that approach. In particular, as Beth has highlighted, there is clarity around contingencies at the start of the year, as far as that is possible, and ensuring that the assessment burden is appropriate. I say appropriate because it is important that qualifications remain valid and credible. It is a serious business, but it is also very cognisant of the fact that there has been disruption to learning and we are still in an extraordinary set of circumstances with some disruption to learning evidence in the system still. Jill's point about communication and engagement with young people is important. We heard through our learner panel and other discussions with young people that they wanted to hear more directly from us. A lot of the communications is absolutely appropriate through schools and colleges and teachers. That is very important, but they also wanted to hear directly from us. We endeavour to do more of that in 2021 and will continue to do that in 2022. Ensuring that we strengthen our engagement with young people through this process is going to be very important to us and any successor body. You said this morning that your pupil should be proud of their achievement and have full confidence in the results. I want to know how you know that. You presented in your document in advance of the committee that the results this year cannot be compared with previous years but then go on to give confidence in the system by using those exact results. I wonder whether there is a bit of inconsistency there and why you have published those results if people should not draw definitive conclusions from them. What I said in my chief examining officer's report, which is published on results day, I will repeat here because I think that it is quite important. I understand what you are making because we have had a couple of years in which the assessment approach and the approach that we have needed to take in those circumstances has been quite different. However, we have all pulled together to ensure that young people get qualifications and that their hard work is reflected in those qualifications. It is an important collective message for Scotland that young people have worked hard and that the 137,000 young people who got the results on 9 August worked hard and deserved those results. We see variation in attainment and the composition of attainment to an extent every year between courses and indeed over time. Every year there will be some differences in the attainment pattern for different reasons. Over the last couple of years, we have seen more movements in attainment than we would see in a normal year when exams are held. There are a number of reasons for that. Disruption to learning would be one and periods of remote learning. There were also modifications to assessment last year and the absence of external assessment. It was a different assessment approach. The flexibility in how courses were considered and assessed by teachers and lecturers required due to the levels of disruption may have also impacted on attainment. It is important that we highlight the credibility of the qualifications, because we put in place an approach that we felt was the best approach that we could in those circumstances. However, we acknowledge that there have been differences in the way in which young people have been assessed over the past couple of years, and that is reflected in the results. In terms of our run of data, there is a discontinuity, but those results are credible. It is important that I highlight the quality assurance process that we undertook. Quality assurance is an important part of any qualifications that are internally assessed. If you look at a college sector, quality assurance is absolutely central to the work that is done year-in, year-out to assure students getting an HND. One institution in Scotland can have parity with an HND in another institution in Scotland. The quality assurance gives additional assurance, as I suggested, that standards have been maintained across Scotland. We provided advice to schools on samples of evidence on whether the standard was being adhered to. A range of approaches were taken, including by schools and colleges themselves. On that consistency of evidence—we have had reports from pupils and teachers about the inconsistency of the evidence that was provided between schools and subjects—how do you know whether there was a consistency of evidence across the piece? There is considerable evidence that contradicts that. There were some flexibilities in the way in which young people could be assessed. That was part of the reason why exams were cancelled, because it was not safe to assess young people in the same way and at the same time in every school and college across Scotland in the way that we have done for the past 130 years. That was obviously a decision taken by ministers on the basis of public health advice. Therefore, the alternative certification model that needed to be put in place needed to have sufficient flexibility to recognise that disruption was being experienced in a different way in different schools and colleges across Scotland. There needed to be flexibility in the approach. However, the NQ group and the SQA also agreed that the centrality of demonstrated attainment—the evidence—was very important in determining what young people achieved. We provided guidance, and Jill Stewart, my colleague, can say a little more about that, about what that would constitute. So, was that the only reason for the inconsistency? Was the impact of the pandemic—there wasn't a different application by different teachers in different schools—for other reasons? It was all to do with the pandemic that there was inconsistency. Are you sure about that? I would like you to explain what you mean by inconsistency. You were talking a moment ago about the difference between the 2019 results and the 2020 results. Are you now talking about issues between institutions? You asked teachers and schools to provide evidence of their pupils' performance, but different teachers applied that in different ways in different schools and between different subjects in the past year's set of assessments and qualifications. You said that that was to allow flexibility to cope with the impact of the pandemic, particularly outbreaks in schools, which I imagine was one of them. Are you sure that that is the only reason that the evidence guidance was applied in different ways in different schools? We provided guidance that we expected schools to follow. Within that guidance, there was some flexibility. I wouldn't describe it as inconsistency, but I would describe it as flexibility for the right reasons. Schools and colleges and local authorities—you heard from local authority representatives last week—provided advice and guidance around local quality assurance to ensure that there was credibility to the evidence that was produced. We also undertook quality assurance across every school and college across Scotland. In addition, we undertook lots of understanding standards events so that we could provide as much assurance as possible that schools and colleges were following our guidance to ensure that the qualifications were credible. It is to do with the moderation process. You were still looking at historical results to question individual performance or class performance. No, we weren't. Well, there was pressure put back on. There was feedback provided to schools using historical information. We had that from— No, there wasn't. Well, the directors who were before us last week acknowledged, and so did the EIS acknowledge, the historical performance in a school was used to provide feedback to your deny that component. I think that you are asking me what SQA did. What SQA did for its quality assurance process was looked at evidence that was provided to—that there was request from schools and that there was no information sought on historical attainment. We looked at the evidence in front of us and we looked at whether that was to standard. If it was to standard, we provided that feedback. If it was not to standard, we provided that feedback. In addition, for every course across Scotland, we also provided course reports, which would help in ensuring that other schools were understood that guidance and were taking account of the feedback. The actions that local authorities and schools took. There was some work done by Education Scotland, by inspectors, and local authorities were very clear with schools that historical attainment could inform a conversation but would not define results. That is your system that you devised, together in partnership with others, where historical information was used to provide feedback, but surely that provided a cost for those schools that previously were disadvantaged and provided poorer performance. SQA did not— Hold on a second, let me just conclude this bit. That pressure was not there on previously better performing schools, so why was that appropriate, even just in an inviter or a feedback loop? Why was that ever allowed to happen? We surely learned the lessons from last week, last year. Why was that allowed? SQA did not look at historical attainment as part of its quality assurance process. Ministers asked Her Majesty's inspectors to look at the quality assurance process that was in place at a local level, and inspectors did that. Local authority representatives last week highlighted the approach that they took locally to quality assurance. What I am seeking to explain to you here is the guidance that we provided to schools and the work that we did. I assure you that historical attainment was not part of that. I thank you, convener. I was listening carefully to the first answer that you gave to Willie Rennie when you said that these years could not be compared to any other, but that you expect some change in grade outcomes. Do you therefore expect grades to fall this year? Do you expect grades and passes to fall if you go back to the old system? I think that we are very mindful of the need to be fair to learners taking exams in 2022. Very few learners will have taken a spring diet of exams, given the disruption that young people have faced. With modified course assessment to take account of disruption to learning, I think that that is really important. We will work through the detail of awarding in 2022, and that includes some discussions that are under way around the nature of that awarding. What is your expectation? Do you expect grades to return to normal? I think that it would be wrong for me to speculate about grades. Do you not think that unfairness learners deserve to know whether or not their grades are likely to reflect past years or the two exceptional years that you have talked about that can't be compared to? We do not grade on that basis. I think that it would be very wrong for me to speculate on the precise outcomes in 2022. I think that what I am very clear about is being mindful of the disruption to learning that young people have faced, and that we are considering further the issues with respect to awarding in 2022. The modifications to assessment are not trivial, and they are helping and will help learners during their learning and teaching experience in this coming year. James Dornan, we have a sound problem with James. You are loud and clear. I have just heard from my last two colleagues. I wonder if you could just clarify that. Obviously, it was a unique year, but how did the SQA assure that that year's results were consistent across the country? You have talked about it, but not really in any detail about how you can make sure that the results were consistent. We have heard from Willie Rennie and others that local authorities in schools are doing it in slightly different ways, so how could you make sure that those results were consistent? I appreciate that I may be repeating myself a little, but I will do so in order to endeavour to answer the question the best way I can. In developing the alternative certification model, we set out clear roles and responsibilities across the system, including SQA, schools, colleges, local authorities and others. That was an important part. We acknowledged that everyone needed to play their part in ensuring that young people got their awards in August. In broad terms, we set the framework for assessment in 2021. That involved some modifications that we hoped would help young people and ensure that teachers could focus on learning and teaching. Those were undertaken across all subjects. There were particular challenges for some of the practical subjects, and we can talk about that if you wish in more detail. There was also a big focus on understanding standards. With a system of teacher-assessed grades, I acknowledge that teacher-assessed grades is effectively moving from a system in which every young person is taking the same exam on the same day. It has all been marked by SQA with lots of processes and procedures around that. We are moving from a very centralised system to a system in which individual teachers, schools and colleges are making those determinations. With some guidance from us, all the support that we could put in place was really important. Lots of understanding standards, events and materials are building on a significant catalogue of guidance and support already in place. We provided guidance on providing estimates, so we provided guidance to schools on how that could be done and on which we received good feedback. We also worked with and discussed with schools and local authorities about how some of their local quality assurance was being undertaken. I mentioned the Education Scotland report that was published that was commissioned by ministers. Importantly, we undertook national quality assurance as well. I will maybe get Gill Stewart to say a little bit more about that, because I think that that is quite an important part of that assurance process. Every school in Scotland involved a sample of evidence from a number of different courses to see whether the evidence was appropriate and was to standard. We would provide feedback to individual institutions and individual courses, and we would provide some national reports to schools on that quality assurance process. I acknowledge that there are challenges around some of that, given the extraordinary circumstances that we were facing, but I want to provide assurance here that everyone—you heard from local authorities last week—and all the teachers that we spoke to and all the work that we did during the course of the year, everyone was focused on making sure that young people got the qualifications that they deserved. All parts of the system were working together in the best way that they could, albeit in very challenging circumstances and, albeit, as Larry said, in the context of sometimes quite challenging conversations, to ensure that we got that absolutely over the line. Gill, would it be helpful for my colleague to say a little bit more about quality assurance? That is a big part of the assurance process. I add to what Fiona Stewart said that we have a very strong programme of understanding standards. We have lots of materials for each individual course where we exemplify what a performance at a C grade is or an A grade. We have done lots of CPD, obviously online, with teachers around about that material and exploring it with them. We also have lots of online materials that teachers accessed in their own time to clarify the standards. That is a big plank of how we maintain standards. We also have a local level very positive feedback from local authorities about one of the positive spin-offs of the ECM model was about how it helped to strengthen their subject networks at a local authority level and the strong role that those subject networks had played at a local authority level. That is particularly challenging. If you are in a one-person department, you need to have a colleague to discuss the standard and get some of the assurance truly about what the standard is. Local assurance at a local level. SQA carried out over a kind of short window working with teachers national quality assurance. What we were doing there was we were not assessing individual pupils. We were looking at the centre's application of national standards from a sample of evidence that they sent us. We sampled every centre and the number of courses that we sampled from varied according to the number of courses that that centre offered. If there were lots of centres, they might have had up to six selections and a lower number might have been two or three selections. What we did was we were looking to see had that centre for higher biology applied the correct standard and making their judgments and then we provided feedback to each centre on the evidence that they had submitted. It was supportive feedback. We were not saying to them that we have to amend your grades. In the main, the headline figures were that centres were applying national standards. There were not a lot of centres that were not applying national standards. Some were maybe a little bit off in some particular aspects and we provided supportive feedback that they could use locally within their centre to have a look at that and to have some conversations within their centres. In an ACM model, it is a distributed set of roles and responsibilities for quality assurance. SQA's role is to provide very clear guidance about the nature of assessment for each subject. In modern languages, when you assess speaking, listening and writing in maths, it is about how you can integrate all your skills and apply operational mathematical processes and carry out reasoning across the course. In English, it is about looking at candidates writing through a folio of writing that is generated within the centre and assessing lots of different things in P. It is about performance as well as an understanding of how you improve performance. There is a wide range of things that you look at. That is very helpful. I am not sure whether that is your feeling. What role has SQA played in the overall quality assurance process? We supported the centres through the provision of guidance for each subject, through our understanding standards materials for each subject, to exemplify the standard and CPD. We provided assessment resources as well that centres could use if they wished. We answered centre and local authority queries relating to individual subjects and so on. We were supporting the local processes of assessment and local and regional quality assurance. SQA's key role at the end of that process was to carry out national quality assurance and to look at samples of evidence across a small number of courses and to provide feedback to the centre about the application of the national standard. It was that feedback. It is teachers that do that. It is teachers from schools and colleges that were carrying out that quality assurance on behalf of SQA. We were very mindful about the need to provide crisp and clear feedback to centres that are very supportive if there were any issues. There were not a lot of issues that were picked up during that process. SQA's primary role was to provide up-front guidance, support on-going through the process and then the national quality assurance exercise at the end to provide feedback to centres on their local application of national standards. What role would SQA have in feeding into what the quality assurance programme was going to look like? If I have understood your question correctly, it was for SQA to determine what the national quality assurance programme looked like. This year, we were mindful of the circumstances. You have choices about the extent of quality assurance and how you do it. We felt that it was important to cover every school and college, so every provider. We had good coverage across all courses, but we were also mindful of the fact that schools had been in remote learning until Easter, so we had a small window and it was proportionate. It reflected the circumstances that we were facing. We made choices that the circumstances of last year were important. We get feedback sometimes that quality assurance can create workload issues for teachers. We did not want to create undue stress in schools, but we also had to do our job, which provided the assurance that members of the committee have been seeking. The quality assurance piece was important. I have been a member of the committee before, and I have not always been kind to the SQA, but I am not really sure that there was enough law board that it could do in the circumstances that was thrust on them on this occasion. I just wanted to quick follow-up, because I was a bit confused with what the conversation here was with Willie Rennie earlier, Fiona. Has it ever been part of the guidance of the SQA in relation to the use of historical data? I think that it is quite important that we do not get—there is not confusion about the role that historical attainment data might provide. Was it part of the guidance in the past, part of the process, part of the system in the past for the use of historical data? There is nothing to prevent schools or colleges from considering data in informing the evidence that they look at, but it is important that I get this right, because I do not want there to be confusion on this issue. What Mr Rennie was asking was, did historic attainment in 2021 inform any moderation or any changes to grades? The answer is no from the perspective of the SQA. We were not looking at historic attainment when we were deciding what quality assurance we undertook. That was not part of the approach. You said that, from the perspective of the SQA, it did not have a role. Clearly in the system, historical data did have a role if a question was asked that a pupil or a class was out of line with previous performance, a question was asked. Even if it did not lead from you to a change in a result, it put pressure on the teacher or the class or the school to question whether the result was right. That never applied to better-performing schools, where, let us say, a poorly performing pupil was never questioned because they were sticking in with historical performance. Even though you never asked it and you never moderated, the system did and you allowed that to happen. Your system should take responsibility for that. I do not have evidence that that happened. The director said it, the EIS said it, the trade unions were very clear, the questions were asked and they did not like it being asked. I think that the discussions that we had on a number of occasions in the NQ group, which included the EIS and local authorities, was that young people, and it is very important that learners should receive grades on the basis of the evidence in front of teachers. That evidence might be subject to quality assurance, but that evidence might be subject to questions from the local authority or, indeed, part of cross-marking. A whole range of approaches were put in place around providing assurance of the very consistency issue that you have been very keen to stress, but that results absolutely should be based on the evidence of demonstrated attainment, and that was at the centre of the approach this year. Just around this issue of historical data, I think that we put it to bed a wee bit there. I know that, certainly, there were some comments that there were small numbers of teachers that felt that they were under pressure to lower grades. Certainly, that was something that was said, although it was not about big numbers. My understanding of that was that, when we were talking about it last week, it was really that historical data was something that they would look at, but it was really more about if there was a massive difference, then that would be maybe a bit of a red flag that they should reconsider and look at that again and make sure that they got it right. Is that correct? I think that the Education Scotland report highlighted that, in some cases, historical data was used in that way to ask a question or to have another look at evidence. You asked local authority representatives some of those questions last week. I can only say again that what was very important this year and what we said time and time was that results were based on evidence of demonstrated attainment. It was not a model in which historic attainment played a part. I think that that is key. There is nothing wrong with using data to ask questions, if I may say so. Data can be very helpful to ask questions, but the decisions around grading were very clear throughout the year and all partners were very clear that decisions around grading were around evidence of demonstrated attainment. A lot of the work that we did was to try and ensure broad consistency of the evidence as far as possible in what was obviously an extraordinary set of circumstances. You said that you would read the Education Scotland report, which I would have expected you to do, but what did you do in practice when you read the line that local authority officers expect staff to use those tools to review concordance data, including young people's prior attainment, identify and address any unexpected professional grades? That does not fit with what you are saying today. Did you read that report and think that something was going wrong? I think that that extract needs to be looked at in the context of the wider report. My reading of the report, and I will acknowledge that, has obviously been out for a number of months, was that it was highlighting the range of strategies that local authorities worked— That says most local authorities, and it specifies within the report that that means 70 to 90 per cent of local authorities were using three to five-year historical data, and that local authority officers expected those staff to use that to identify and address unexpected grades. That does not fit with the picture that you are giving. As a person responsible for the qualifications that are handed out, did you not have a problem with that at the time? We were very clear, and we did have some discussions about that as part of the NQ group, including the report. Indeed, address representatives were also clear that data was being used to ask questions, to look at the emerging picture, but evidence of demonstrated attainment for individual young people was the basis on which awarding was made. You said nothing publicly. You did not raise any concerns, despite the concerns that were being voiced in the Parliament. You were happy to let this report from HMIE, from the Government's other main education agency, sit there in the public domain and give the impression to young people that previous data from their school might be used to identify unexpected grades and address them. You were happy to just let that sit there and say nothing about it? I think that in the discussions that we had in the NQ group and the communications that we issued, we were very clear about the awarding approach in 2021 and consistent in our position and a position that I have been very clear about this morning. Thank you, Dr Robinson. The consequences of that to me seem pretty clear. The attainment gap between the richest and poorest increased, the performance gap between state and private schools increased, and the gap between disabled and non-disabled students increased. Do you not feel that that is the consequence of the issue that we have just been discussing? The attainment gap in Scottish education is long standing— It has increased this year under this model. Are you saying that compared to 2020 or compared to 2019, is there a different picture depending on which year you choose? Compared to the previous year, because the changes that you made as the leader of the organisation over that year to put in place this new model, we have been discussing this issue about using the evidence in terms of previous attainment, putting it into the model that was used in local authorities, as Mr Mundell has just pointed out. Does that share the consequence of that suppression of those grades? No, I would not accept that characterisation. I think that I have made it clear the position with respect to historical attainment, first of all, and I have made it clear the position explaining what the ACM was and what the ACM was not. The issue with respect to the attainment gap is that we have a responsibility to report on the gaps in attainment. We have done that this year alongside other equalities information that we have undertaken during the process. There was an equalities impact assessment that was undertaken for the alternative certification model and for the modifications to assessment that took place. We have a responsibility to ensure that the assessment approach does not exacerbate gaps that exist in Scottish education, but attainment gaps have existed in Scottish education for a long time for lots of different reasons. The attainment gap this year did widen slightly across different groups, but the picture that I have highlighted in 2020 and 2021 is quite different to how it looked in 2019, where the gaps were much wider. We are interested in the model and how it was applied. That is what colleagues are getting at. It is pretty clear from the data that those gaps increased. How do you account for that between the two models, 2020 and 2021? I have highlighted that there have been differences in the assessment approach in 2020 and 2021. There have been differences in the experience of learning and teaching in both those years. That is crucial in all of us. The Scottish Government's own equality audit highlighted issues in particular with respect to deprived young people and their experience of learning and teaching in 2020 and 2021. That is an avenue that the committee should explore, because the learning and teaching experience is the most important element in determining what young people can achieve in school. That is fair, Dr Orson. If it is okay on this, you are saying that the data applied had no role in that. Can you increase that gap? I am saying that every part of the system this year works together to ensure that the alternative certification model was based on evidence of demonstrated attainment for each and every learner that was the basis on which they received their awards. Every part of the system sought to ensure— To be fair, you have said that several times. I think that we move on to all of us to take the evidence forward. I just want to return to the convener's original line of question and to quickly ask the question in a slightly different way. Do you think that the OECD's recommendations in relation to assessment are right? Do you think that the recommendations have made are the right ones? From your professional experience over the years and from your experience at the Sgrade, do you think that the changes that they identify are the ones that we should follow as a Parliament? There are two OECD reports. The first OECD report did not have a specific focus on assessment, although it highlighted issues around the alignment of the curriculum and assessment issues. We published our submission to the OECD, which documented the journey of reforms to qualifications, in particular through the development of curriculum for excellence. The Scottish Government commissioned Professor Gordon Stobart to undertake an additional report, which was published a few weeks ago with respect to assessment and qualifications. That was a comparative study that was looking at different countries and looking at what he called the British tradition of exams, highlighting the fact that Scotland and other parts of the UK have a tradition and a culture of exams. I think that an important part of what Gordon Stobart said and what I agree with is that assessment and qualification approaches are effectively a kind of cultural phenomenon. They reflect the culture and practice and capacity of the system. In thinking about any further changes to assessment and qualifications, it is important that we highlight those wider issues. Do you think that the initial recommendation to move the exam or assessment part of our system in with the curriculum is the right decision? Are you talking about the organisational change or changes to assessment and qualifications? I am first talking about the first OECD report that suggested that the SQA or an equivalent body should be merged in with the curriculum body. Do you think that, in your experience, that would be a good move? I think that ministers have set out their position with respect. I am not asking about ministers, but I was asking about your professional experience having headed up Scotland's exam body. If anyone is going to stand up for exams, if anyone is going to make the case that what we are doing at the moment is right, it would probably be you. Is there another side to the story that the Parliament should think about, or do you think that those recommendations are right? There are issues to consider around function and form. The organisational structures that exist in Scottish education are important across Scottish education, including the national bodies. It is also important to consider what the constitution and structure of those bodies is that it is also important that we are clear about what it is that we want those organisations to do. I realise that that is quite an obvious point to make, but it is a really important one. My observation is that we have reformed quite significantly Scotland's qualifications alongside the development of curriculum for excellence in the 2014-16 period, in which there was a balance of continuous assessment through a unitised structure, which was continuously assessed in schools, alongside exams and coursework. Following the removal of units in 2016 by Scottish ministers, we have seen a slight move back to a greater emphasis on exams. A lot of good systems of education have a balanced approach across continuous assessment, external assessment in the form of exams or some form of externalised assessment and other forms of assessment, for example coursework. Even at the moment, I think that sometimes the debate feels quite polarised around assessment and qualifications, because at the moment many of our courses do not rely fully on the final exam in the spring, but there are a relatively small number of courses that do that. We also need to, as I highlighted in my opening statement, I think that it is important to reflect on the experience of the past couple of years and indeed some of the questions that the committee has asked today about some of your concerns about the alternative certification approach and, effectively, the kind of federated system of assessment in considering those issues. That is the final question, convener, to reflect on one of those issues. It is not one that I am enthusiastic about asking about, and I absolutely believe that all young people in the past two years have got the grades that they deserve, but it is just to ask—obviously, people are positive to see young people from more challenging backgrounds doing better than they have in the past. I would like to see that continue, but it is just whether there are unintended consequences of grade inflation and whether that is something that we should be mindful of. It is not something that is popular to talk about, but it is whether that brings with it other challenges in terms of what a qualifications body should be doing. I guess that it comes back to the previous question that I asked as well. Whether there is an expected number of A grades that you would want to see in any qualification system to get that differentiation? I have highlighted in answer to previous questions that the pattern of attainment has looked different over the past couple of years. There are a number of moving parts that might have contributed to that. The awarding of qualifications is a serious business. We have a responsibility and, indeed, a statutory duty as things stand to determine the level of competence that is required to gain a qualification and the means of assessing learners in order to determine whether they have it. Sometimes that means that we need to be both clear about our expectations of the system and sometimes we have to say that the level of competence has not been met and that is hard. I welcome a debate about how important qualifications are and what they do, what the function they fulfil, both here and elsewhere, and the way in which we do that. However, I think that that debate is to be had. Some of those issues are policy issues and are issues quite rightly for the Scottish Government to consider. I will play a full part as chief examiner in those discussions. I am sure that the cabinet secretary will want to talk more about that next week. In 2020, you pushed to have the algorithmic element to keep grades where they would be expected to be. Is that something that you think is important in the system? Standards are important as part of the responsibilities that we have because the qualifications that young people get at the time and over time remain important. All of us will have been through that. At different periods in our life, our qualifications remain important in terms of what happens next. That is why, when we were asked to maintain standards and develop an alternative certification model in 2020, that is why we did what we did. We were commissioned to do that, and we did that to the best of our ability. We have a responsibility to maintain standards over time. In addition, we have been through a pandemic. We have had to put in place very different arrangements. We have worked very hard to do that in the best way that we can. I am glad that you have acknowledged that young people have the qualifications that they deserve. It is important that we all get behind young people to do that. On our united support for our young people, there is no division at all. On the OECD report, I think that it might be fair guess that you want to come in on that. Yes, thank you, convener, and welcome to our witnesses. I just say that no country in the world suffering the global pandemic had a syllabus ready and waiting to instruct us about how to proceed when schools were disrupted and shut down. Hindsight is a marvelous thing, is it not? I would like to look forward, not back, and really ask two questions, the first arising from the OECD report. I am sure that Fiona Robertson and our colleagues have read the evidence. I was struck by the very positive comments made, for example by Beatrice Potts, who said that Scotland was amongst the leading countries in global competency proficiency. Scotland is above average across OECD countries in terms of equity, and they saw the CFE, which I want to question Fiona Robertson, as a pioneer among education systems internationally. There were lots of positives. The take-away criticism, for me, is the central conceptual thrust of the criticism from the OECD, seem to me to be that the competencies set under four competencies of successful learners, confident individuals, responsible citizens and effective contributors were competencies that, if you like, needed to be better worked and assessed in the overall system. In other words, it is not clear at the moment how we assess whether individual children pupils' learners have or have not attained those competencies. That seems to me, Fiona Robertson, to be the central tenet of the criticism. If that is correct, then how do we address that in future? What needs to be done to, if you like, take Scotland forward and make sure that, in driving the CFE forward, we can do so in a way in which we are able to say with confidence that, yes, our children are successful learners, they are confident individuals, responsible citizens and effective contributors. I am happy to answer those points. I would agree with your sentiments and the sentiments of the OECD. The previous OECD report in 2015 highlighted the pioneering of an innovative nature of curriculum for excellence and its endurance over time. Many other countries have followed the broad approach of curriculum for excellence. It is a very strong basis on which to build. The issue with respect to the four capacities of CFE, one of the emerging issues in the OECD report, was that, somehow, the qualifications and assessment system focused on one of the four capacities, which was on successful learners at the attainment that, therefore, was in one sense seen as part of quite a traditional model. If you go back to the building, the curriculum documents of curriculum for excellence—I should probably highlight at the time that I was the chair of the curriculum for excellence management board, so I was not in the role of the SQA, but I had a wider role with respect to curriculum for excellence. There was a lot of ambition in those documents about how curriculum for excellence would develop, including in the senior phase, including in the curriculum models that may emerge over time. If you look at the suite of qualifications that SQA offers, I am pretty confident that the four capacities of curriculum for excellence are covered. We provide leadership awards, mental health awards, and there is a huge range of awards that we offer. I would accept that curriculum for excellence should not simply be valued in terms of the qualifications that you lead school with. That is important, too. You do not always value lots of things that you do not measure, as well as some of the things that you do. However, we offer a large suite of awards, but I think that there are also issues that we need to consider as a system about the curriculum models that are followed right across the system and that the choices that young people have, or, in some cases, the choices that young people do not have. There has been some really good work. In fact, there was a report just over the past couple of weeks that was published, which I would commend to the committee on school-college partnerships. The work that has gone on there to integrate the vocational offer within schools across Scotland and some really important lessons there about what works well and some of the perceived barriers there. There is more to do in ensuring that the offer is there, but there are also choices available to young people in schools and colleges. The curriculum offer is such that it can take advantage of those opportunities. You have given some very good examples. Indeed, I noticed that the OECD witnesses said that Scotland is viewed internationally as an example of high performance. When we compare data with that from other countries, we see that Scotland is above average in a number of indicators, especially the new OECD indicator on global competencies. It is easy to forget all the positives in the OECD report. If I might ask one more question, please, which is of a more practical nature, it arose from the comments earlier that Fiona Robertson made about the importance of the need to help children to prepare for examinations next spring. In the background where they have not really been used to examinations over the past 18 months or a couple of years, I guess that it is a long, long time. Five decades since I sat my last examination at school, such a long time ago, the dinosaurs were prowling around the classroom cave outside and maybe my experience is therefore a little bit dated. The essentials of exams from a children's point of view have not really changed. There is the need to put work to prepare and the inducement to work and prepare. There is the anxiety and there is a fear factor about the unknown and the consequences, and all those are constants irrespective of the passage of time. How, in practice, do you think that we can best equip and help to prepare our learners—if that is the word that we use these days—for examinations where they have not been used to doing examinations in the recent past? I put forward a particular suggestion, which may or may not be relevant these days, but is the use of practice specimen papers so that children have the chance to rehearse, to practice, to try out the process of an examination before the real thing? Is that a structured part of the system these days so that, if you like, children are not going in cold to an experience of which they are completely unfamiliar? That did seem to me from my long-forgotten examination preparation to be the fundamental benefit of preparing yourself for an experience that you had tried out in peacetime before the real thing. I think that schools and colleges are very experienced in ensuring that young people are able to prepare in the very best way that they can. Many schools, including over the past couple of years, have maintained prelims or other assessments. Young people are assessed on a range of things across their school life. Assessment as a thing is not new and will not be new to many young people. I accept that, in the spring of next year, with an exam diet, that may well be new to young people, but schools will be very aware of that and will be developing approaches to ensure that that is as stress-free as possible. I would like to explore the functioning of the NQ21 group, and specifically the level of participation of those involved. You will be aware that, on the day that the appeals process was confirmed, Cameron Garrett, who was the member of the Scottish Youth Parliament on the group, said that, as the only young person who sits on the SQA's NQ21 group and the only member representing young people, I have not had an equal input into discussions around the appeals process this year at NQ group meetings, young people have been let down and ignored by this process. You went on to say that organisations such as the Children and Young People's Commissioners Office and SQA, as well as the Scottish Youth Parliament, have been calling for no detriment policy and exceptional circumstances to be taken into consideration as substantive points have neither been considered in the process. Subsequent to those comments, did you reach out to Mr Garrett to better understand why he felt that that had been his experience? Could you talk a little bit about what you believe you have learned from those discussions and how subsequent processes to this one can more effectively involve the voices of young people? It is a very important point. A cabinet has been a very helpful and very good member of the NQ group, and it has been really good to see him on a weekly basis alongside colleagues from the Scottish Youth Parliament. Indeed, we commissioned the Youth Parliament to deliver our learner panel this year, so there has been a variety of involvement. Of course, I have had a number of discussions with Cameron and others in the NSYP, and I am keen to understand and ensure that we take all appropriate steps to ensure that our engagement with young people is as good as it can be. With respect to the appeals process, there are probably two issues. One is the operation of the NQ group more generally, and one with respect to that particular issue, which Cameron and his colleagues felt quite strongly about. In relation to the NQ group, I think that Larry Flanagan highlighted some of the issues. It is quite a large group, with a lot of loud voices and a lot of strongly held views. Gil highlighted the issue of having to work through some of those complexities and come to agreement. Sometimes agreement has been quite difficult during the course of the year, and I think that Larry alluded to that. We have agreed with the Scottish Youth Parliament that we will increase the number of young people on the NQ group this year, and that is in train. The number and variety of voices is important, so we will do that. That was good feedback, and we will take that on board. In relation to the appeals issue specifically, the new appeals process, with the enactment of the UNCRC, we thought that there was an expectation through discussions with the Government and, of course, the priestly review that we would introduce a learner right of appeal this year. As you will know, previously appeals have been, absolutely, involved young people, but have been initiated by the school or college. Indeed, the final decision about whether to institute an appeal has been through the school or college in previous years. This year, we were met with a set of different circumstances in relation to the alternative certification model, but the expectation, perfectly reasonably, that we would have a learner right of appeal. We thought that it was important that we consulted on that, so we undertook a public consultation on the appeals process this year. There was quite a lot of discussion, and that included a learner panel discussion. In fact, I attended that particular panel in relation to appeals. In particular, we have highlighted, in relation to the symmetry or otherwise of the appeals process, i.e. whether appeals could result in an upgrade or a downgrade, as well as a grade remaining the same or whether there was an asymmetry, i.e. only up or the same. In addition, there were issues with respect to exceptional circumstances, which I am happy to go into, if you wish, in relation to that. We had lots of discussions about that, and that obviously went through our governance as part of the SQA to our board, through our qualifications committee. There were lots of discussions, and there were lots of discussions with the Scottish Government around those issues. We decided that appeals should be symmetric, i.e. could, in principle, go up or down, or indeed remain the same. Obviously, that was not a position that was supported by SYP and Cameron on the panel, but what I was keen to highlight to Cameron was that we took the feedback from young people very seriously. There were lots of discussions about that, both with them and more widely, but we took the decision on the basis of fairness and that that was an evidence-based process and that we saw evidence that an award had been given incorrectly as a result of an appeal that we needed to take action to do that, and it was on that basis that we took that decision. It was a very considered decision. Sorry to jump in at that point, but the specifics of the appeals process is not particularly what I would like to pursue with a sign of questioning. Given that you raised it, and we have had exchanges about that in the past, there was not a particularly large number of appeals this year. Did any of those appeals result in a downgrade? The appeals process is not yet concluded, so we will not produce official statistics at the end of the year on appeals outcomes. I cannot share the outcomes of the appeals process, because we have not concluded the outcomes of that process, but I am happy to share that with the committee when the information is available. At the time, we have always had—there has always been—a symmetric appeals process in Scotland and, indeed, in other parts of the UK there is as well, including this year. Downgrades are very rare. They do not happen very often, for all the right reasons, because most of the time the system gets it right first time. Our approach was to ensure that we got it right first time, but there needed to be an appeals process in place because that was an important final part of the process. I am happy to provide further information on the outcomes of the appeals process when I am able to do so. On a similar vein, the Children's Commissioner's Office, in fact, the commissioner himself had some words to say about the concept of co-production. Dr Stewart mentioned that earlier on as there has been an area of learning from the process over the last year. The commissioner's comments were pretty scathing. It was at the same point that Cameron made his. Bruce Adamson, the Children's Commissioner, said some very strange examples of co-production are being discussed at the moment. If you have very limited involvement of young people and then ignore their views, you cannot call it co-production just because they were in the room. Similar to last in regards to Cameron, have you met the commissioner subsequent to those comments to discuss the concerns that he had around the concept of co-production? No, I haven't personally, no. Can I explore that in a little bit more detail? When you mentioned learning about co-production now, Dr Stewart, what did you mean by that? Is that taking into account the concerns that not just the Children's Commissioner but a range of other experts in the field of children's rights have raised about the concept of co-production and participation? It is an area that we are all learning about how best to do. To support the work of the NQ 2022 group, and we have a working group as well, we have supplemented, as Fiona said, additional members on the steering group but also on the working group. However, we are also looking to expand the learners panel and include a wider range of young people on it. It is only by engaging throughout the process that we will improve and learn about the whole process of co-production and how best to do it. Do I have any kind of magic answers, Mr Greer? No, I do not, but what we are trying to do is to increase the level of learner engagement and to be in very active listening mode as to how best we might do that. That is a reflection from a different piece of work, but we did some work with young people about the future of assessment prior to the current Covid situation. It was very interesting because it was work where we involved young people along with some teachers, and young people had very different ideas about assessment from their teachers. What you saw there was a bit of a… It was co-production in that sense in that young people made suggestions and then teachers did not poo poo those, but they pointed out some of the practical difficulties and they reached different conclusions about how you should best assess in particular subjects. The one that struck in my head was about history. Teachers often want to assess the way they have always assessed because they know how to help young people to do that, whereas young people come at it from the perspective of, well, I would like more of my work to be taken into account on a kind of on-going basis that I am producing throughout the year. To me co-production is about the future of assessments, but it is about how you get those two sets of people together in the co-construction of what the future of assessment should be in different subjects. I use that as an example, but it is challenging because the young person does not necessarily have the precise answer, and the teacher does not necessarily have the precise answer, but it is by bringing people together and getting them to actively listen to each other that you can come up with ways forward. I think that these are the sort of learning experiences that we need to use to inform, for example, the future of assessment. I understand entirely that there is no easy answer to that. Given the comments of the Children's Commissioner, I would expect him to engage with his office to understand those concerns. I know that Dr Tracy Kirk has engaged with you on those issues. There has been engagement and correspondence with the Children's Commissioner, and I am very happy to continue to do that. I think that what Jill's highlight is really important is to seek to understand the different perspectives and the different responsibilities that we have. The awarding qualifications is a serious set of responsibilities. I have statutory responsibilities to fulfil, and statutory functions to fulfil. It is important that we engage in those conversations to seek to understand, to reflect, but it is also our job to ensure that we are fulfilling our responsibilities as an awarding body, and sometimes that means, unfortunately. I say that with some regret, and unfortunately that sometimes means that we are not able to do all the things that everyone wants. I accept that. I am conscious of time, and I am probably intruding on other members' time at this point. If I can come back in at the end, that would be appreciated. I am interested in some of the questions about form and function, as you put it. We all know that the very important job that the SQA has to do over the coming year and the pandemic challenges remain vast for the education system. On 21 June, Dr Orbson issued a statement welcoming the announcement that your organisation was going to be scrapped. Did you consult the SQA staff before issuing that statement? I did not use the word scrapped. It is quite important. I highlighted the announcement that the cabinet secretary had made. The announcement of a new specialist agency with responsibility for both curriculum assessment is an opportunity for significant change. Yes, I did say that. I think that it is important to be accurate in what I said. I appreciate you repeating that. I had a meeting with all staff on the morning of the announcement, but the statement was in my name. I obviously had discussions with the board and with all staff on the morning of the announcement. It was very important that I did that. There are some significant upsets among staff. I have spoken with trade unions who have said that they are both upset in terms of the announcement and the way that it was welcomed by the leadership of the organisation. Is that fair to say? I have good on-going engagement with colleagues across the SQA and our two recognised trade unions. It is important that we engage positively with the review process. For every organisation, you have highlighted in your questioning today, more broadly, the learning, reflection and the need for some change. It is important that we, as an organisation, along with other organisations in Scottish education, reflect on that need for change. However, I wish to highlight strongly in answer to your question the commitment, professionalism and the integrity of every member of staff in the SQA in the face of a challenging period and quite a lot of political comment. That has, in my view, been at times unacceptable, because we have a set of responsibilities to fulfil. We have had a clear commission from the Government to undertake tasks, and we have done those to the best of our ability. When people like yourself say scrapped, abolished and other adjectives, yes, you will get a reaction from staff, and you will get a reaction from me as well, because there is a lot of skill and expertise in the SQA that Scotland will need in the future. I think that it is really important that I highlight that to the committee this morning. I think that it is really important. I welcome you putting that on record. I know that having spoken to the trade unions and the SQA, there was real concern in terms of the way that that happened. I am interested in part of the relationship between leadership and expertise as we talk about that. For the model for 2020, did staff make representations to you telling you that that would be, in their words, the disaster that it turned out to be? During the 2020 process, within the SQA, we had a number of conversations across a range of issues in which we were considering some of the challenges of awarding in 2020. There is no doubt about that. It was a very challenging period. My colleague Gill Stewart led the process of the approach in 2020. She confirmed that it was a challenging period in which we were very concerned to deliver on the commission from ministers and to get it right. That was our concern. However, there were a range of conversations that we had about that, and we are concerned to get it right. I appreciate that. Given some of the previous question, I am a little bit worried about the distance from the Government and how independent some of that advice and the expertise that you rightly reflect is so important to the education system that can be drawn on. I want to look forward a little bit to next year, if we can, to pull this together, and the plans that you have set out. The guidance that was issued on 15 September might contain the lack of detail and clarity, and I recognise the context that everyone is operating within. That clarity is needed for teachers and for young people. We have heard that very strongly through the evidence that we have had from young people—a complete lack of clarity last year, even less the year before—looking for more clarity this year. In particular, guidance sets out decisions that, if mitigations to exams are needed, that will be left until March of next year. It does not set out whether any detail of what criteria those decisions are going to be made on. We comment on that. That would be useful. Whether that will be pupil by pupil, school by school, council by council or Scotland-wide, what level will that approach be decided on? Will disruption be experienced differently by all pupils? We really need more clarity on that. Can you give us some clarity for those young people? First of all, I totally understand the need for clarity and I understand that. Looking back on 2021 and my pedants before the committee this morning made me look at all the communications that were issued in 2021. I was very struck around the fact that we were dealing with a lot of change in real time. We were, as an NQ group, mindful of the fact that we did not want to bombard the system with changing guidance. Things were changing every five minutes, but at the same time we wanted to provide as much clarity as possible. There is quite a tricky balance around that. I do not think that any of us wanted to be seen to be chopping and changing in seeking to provide clarity. There is something quite important in there about what is the right balance around all that. As things stand, Scotland is the only part of the UK that has been clear on arrangements for 2022. We have come out quite early and I think that there is a context there that is quite important that we have set out the contingency arrangements that will be in place for 2022. The contingencies relate—my colleague Beth Black would be happy to talk about that in a bit more detail—but the contingencies relate largely to national contingencies. We would expect that, if exams are to be held, as appropriate exceptional circumstances would apply for in circumstances where there were very significant issues that would impact on performance for young people at the time of an exam. That happens every year. In some cases, that is highly individualised. It is a bereavement or illness. In some cases, it is a little broader than that. For example, if schools have suffered from—we have had in recent years school closures for different reasons or there has been fire and other things that have happened to schools and exceptional circumstances have kicked in there. We have discussed with schools changes to deadlines and other arrangements that can take place. We have the flexibility to ensure that, if circumstances apply locally or individually, measures can be put in place to mitigate the impacts around all of that. In relation to the contingencies, in broad terms, what we are saying is that if there is very significant disruption to learning or further disruption to learning, that would be beyond that experience in the 2020-21 academic year. We need to take advice from the education recovery group and public health on that. There is a context there that is important and, indeed, that the Scottish Government could consider further modifications. Exams would only be cancelled, and that is the further contingency if public health advice in the spring was such that they could not be held for public eye gatherings of young people. I am going to ask a colleague to tell me just to talk through that. On one point, just within that, if I can, convener. Michael, because there are some other questions on this land. Yes, just within that. We are now, in recent weeks, at a level of absences in schools that were equivalent to when we cancelled exams previously last year. Do you have concerns or reflections as to lost learning and where we might be at the moment? I do not mean to be alarmist on that at all, but I think that in terms of the decision making process, it is appropriate that we consider that. Do you think that that? I think that the focus at the moment is not on assessment, but on learning and teaching at this point in the year. That is really important. Schools and colleges will have responsibility for ensuring continuity of learning and teaching, even if there are some issues around absence levels. We are very aware of that, but the focus at this point in the year is absolutely on learning and teaching rather than assessment. Beth Cymru can highlight some of the measures that we could put in place, should that be necessary. However, keep in mind that we have made modifications to assessment so that learning and teaching time can be maximised. Those modifications are in anticipation that there will be disruption to learning this year. We have taken that step up front and reduced the assessment burden so that it should be more straightforward for young people and should allow for a degree of disruption, should that be in place. Beth Cymru, do you mind if I bring in Bob? I think that he had a question in the series. If we hear Bob's question, then he can give us a fuller answer. Thank you. I welcome that, convener, and it is a really helpful question that a colleague Michael Marra was exploring. I know that initial guidance has been issued, and we are all holding our breath for what that more detailed guidance will look like, perhaps that is what we are going to get from Beth. I am just looking at the guidance that we have out there, so there are three scenarios at present. We run with exams as planned with modifications that Phil Robertson has highlighted that will exist in terms of assessment. We go to an additional modified process with exams or we return to some form of alternative certification model. That is that, convener, that I wanted to ask a question about. I quote from the guidance, the type, quality and volume of evidence that would be needed to support quality, assured estimates and abnormal year. That would be used to support provisional results. It goes on to say that provisional results would be based on in-year assessments that normally take place during the school year, such as prelims, practical activities, performances and class tests. Although, convener, we have before us, in theory, three different models, two of which exams take place but modified, and one of which exams do not take place but within another formable term of certification, but, according to the guidance, those are the types of assessments and observations and evidence that the teachers will be doing anyway. Can I ask Phil Robertson or Beth what the difference actually is? I will cover that in the answer to Michael Marra's question. I hope that our answer was slightly confused at the end about the difference. Modifications are already in place to acknowledge and anticipate some disruption. As Fiona Henneth said, those are significant, they are not trivial and they take account of the assessment burden. That means that teachers and educators can concentrate on teaching and learning. However, we are keeping a watching brief on the disruption. We are very aware of the disruption and the trouble that this is causing in the system, so we are keeping a watching brief on that. If there is disruption beyond a certain point, somewhere in the region, the kind of disruption that we saw last year, then there will be some further interventions that will help learners in the support in that final period, in that run-up to exams, so that it will help take the heat out of that revision period and help support them through that. We will be keeping a watching brief. Obviously, it is very difficult for us to put numbers on these things and to announce numbers because numbers can become targets. It is incredibly complicated, so it is very difficult to be entirely reductive about that, but it is definitely a watching brief. In teaching and learning, assessment takes place naturally. That is naturally occurring, part of teaching and learning. Rather than trying to bring in a very late notice, possibly a large ACM that would then involve all sorts of semi-formal assessments, the instruction would be that if exams had to be cancelled for public health reasons in the spring, then the normal assessment that teachers rely upon in their normal working lives will form part of the support for the provisional grade that they would submit. In any year, teachers always submit estimates to SQA just before exams. This is an ordinary practice in any year. What will happen is that, in any ordinary year, the estimate is being prepared, it is being ready. Teachers are mindful about how to collect that. They know how to make judgments about the evidence in front of them and we are further supporting that with the understanding standards work, which we have heard was very popular. Teachers welcomed it and it increased their capacity and capability in understanding standards. That also feeds into the normal practice of estimates. Should we find a worst-case scenario in March or April and we hope that we do not that public gatherings cannot go ahead or large gatherings cannot go ahead, then that normal activity becomes repurposed to become the provisional grade. The advantage of that is that this is quite important. It means that teachers should not worry about taking additional assessments to keep in their back pocket and learners should not worry about being dual-assessed. It is not everybody's first choice. It is not the first choice to cancel exams. It is very much a plan C but we have to have a contingency in place. We have listened to what the system has told us about the workload involved in the ACM. That is part of the thinking in that plan. That is very helpful. Beth had mentioned what the difference was. That is what I was trying to tease out. What I think is true is that you will embed the normal day-to-day practice of teaching professionals into any alternative certification model if that is what we have to end up. That is very helpful. I wonder what role moderation would play in that process, because we are still in quality assurance. We are still in the same situation about one person departments and different approaches within local authorities, so that would be helpful as well. I am not sure if I will get back in a second time or add this second question to who I would like to explore some of this further. Teachers will feel very empowered now, as they should do, that when they put in an estimate for a young person that that estimate will be a true reflection of the competencies that a young person will be operating at, because, after all, that is what they have been asked to do with moderation and quality assurance, and we then go to the exams. As we know with any exit exam, not every young person will perform as well as anticipating those exams, and that is what the appeal process comes in. Has the SQA considered that should the exams go ahead—I hope to do in next year—that we can anticipate many, many, many more appeals than before, because teachers and young people will feel empowered to have high-quality estimates showing young people operating at a very high level if they do not perform that way in an exam, then a significant amount of appeals are likely to come forward? Has the SQA also given consideration of that? I'm happy to start. I'll be brief if I can. On quality assurance, if we were in a position that exams could not go ahead, for all the reasons that we've discussed during the committee session this morning, we think that some quality assurance would be an important part of the approach that we would take. Keeping in mind that evidence is key, so what we would be asking teachers to do would be to ensure that there would be some further guidance around estimation over the next couple of weeks just to crystallise some of that, but also not to create unnecessary workload, because we are very conscious of that this year. That will be important. Obviously, all the understanding standards work that Jill has highlighted will continue to ensure that that programme is developed and delivered. Obviously, the appeals process that we had this year was with respect to the approach that we took. We will be looking at the appeals process for 2022. I anticipate that it will be a learner right of appeal, for all the reasons that I have highlighted, and we will be looking at the detail of that. There is an existing appeals process that has been in place for a number of years up to the point of 2019, which we can use, but we will be doing some further work to establish in more precise terms what that appeals process looks like. However, your point about high quality estimates is important, because the communication between the school, the college, the teacher and the young person in setting expectations and also setting aspirations about what young people might be able to achieve is really important. That is an important part of the discussion. The good high-quality discussions that go on day in, day out across Scotland in relation to how people are doing, how young people are progressing in their learning and what they are likely to achieve either through an alternative certification model or through an example process. Those are very important conversations to be had. I will ask Michael Marra to come back in and be brief, and if I can have brief responses as well, because Stephanie has been very patient and I must get to Stephanie. I am not asking for numbers for a number of days of disruption on those bases. I am interested in, in particular, if there is one school in one part of the country or one local authority that has been very significantly disrupted and another part hasn't. Could we possibly see a different approach for those different areas? Could we see exams cancelled in Glasgow but not in Edinburgh? We are talking about a national approach, because you and I see a lot of shaking heads on that basis. You talked about exceptional circumstances and taking the individual approach into account, Dr Robertson. How do you square those two things? I think that we would be looking at a national exam process, but the exceptional circumstances that I have highlighted to you, which has existed in previous years, and Jill might want to say a little bit more about that, but that is important because it can apply individually. It is the individuals that are important here. It applies individually for particular reasons, but it can apply more broadly if a school or a cohort of young people has been affected in a particular way. That needs to be important. In 2021, the whole year was about exceptional circumstances. The cancellation of exams by ministers was reflecting the exceptional circumstances across the country and the flexibility that schools were able to deploy was effectively the way in which they could use the exceptional circumstances to ensure that young people got their qualifications. I think that I have a class there. Can I go back to Bob? Bob, do you want to have any follow-up? It has to be very brief, though. I would be really interested to know whether the SQA anticipates that the appeals process in the coming year should be more likely to be successful because teaching professionals are always doing a good job in estimating grades, but through the last couple of years, we have had a really detailed, specific approach to evidence gathering and submission under the further certification model. Can we anticipate a really robust system of estimates if those are high-quality? Should we expect much more successful appeals in the year ahead? Should young people know those estimates before they walk into their exams? I appreciate that, Bob. I think that you asked that question before, as well. Can I get a little bit more specifically on the volume of appeals in any year? I think that your point about the experience of, in particular, this past year and the feedback that we have around a real focus on understanding standards across schools and colleges, lots of discussion within departments and across subject teams and across schools within local authorities and regional improvement collaboratives has enhanced the understanding of standards in Scotland's schools. That has been a good thing and there has been positive feedback in relation to that. I do not doubt in any year that teachers are absolutely doing their best to ensure that young people are being kept very closely involved in discussions about their progress, but the feedback that we have had this year is that the focus and the responsibility that teachers had in determining grades has involved quite a lot of investment and a lot of hard work and commitment in understanding standards and applying those standards appropriately. For the final round of questions, given the time constraints that we have, I thank Stephanie for her patience. I do appreciate that the past couple of years there have been lots of huge decisions that you have had to make, a huge amount of work in a very short period of time and not necessarily the work that you have been used to doing in the past. I am really interested in what work it is that you are doing just now. It has come back a little bit to what Ross was talking about earlier, but what work are you doing just now around planning our strategies for models of co-production and giving young people's views parity, taking on board teachers' views, going forward and getting that real collaboration where you are able to arrive at decisions together? That is a good question. I have alluded to some of that work during the questioning. I think that there are a number of strands here around broader communications and engagement more generally. We have increased our capacity internally around that, with the appointment of a new director of communications and enhanced our internal capacity around particularly young people and parental engagement. Communication and engagement is important, but there will always be a variety of views about how effective or otherwise it is. I could go through in some detail all the work that we did this year to try and do that. We did a lot of communication and engagement, and, indeed, bespoke communication with young people and parents that were informed by feedback from those groups. It is fair to say that there is an never-ending appetite for that, and we need to make sure that we are doing that in the best way that we can and that we are mixing up the approach through social media and other things. In relation to engagement itself, I have highlighted that we are enhancing young people's representation on some of our groups, including the NQ group itself, and expanding the learner panel, which is important. Having a forward work programme was in real time last year, and I felt that sometimes there was not the leading time to things that we would have liked. The review that Ken Muir is undertaking in looking at some of those issues will have a big focus in relation to any successor arrangements, both to the SQA and the reforms to Education Scotland. We will look very closely at young people's engagement. Jill highlighted the work that we did before the pandemic. It was before my time, but it was just before the pandemic around assessment futures. There are some really good foundations around that, and some of that gets into the policy space. The ongoing discussion and engagement with the Scottish Government on some of those issues is really important. I hope that that is helpful as a fairly high-level summary of some of the work that we are doing. We are keen to ensure that young people's views are integrated into the work that we do as far as possible. We are here to serve learners. A lot of our work is supporting the profession to serve learners, but our job is to serve learners. I trust that answer will be yes, but I take it that way, including care experience to people, additional support needs, etc. What we very often find is that all the things that work for those pupils actually work really well for the other pupils as well. We have an annual event with Who Care Scotland on results day. Of course, in the past couple of years, we have had to do that virtually, but my first year in SQA in 2019 was attended by the First Minister herself, giving certificates to care experience young people and reflecting on the very impressive achievements of those young people, given some of the challenges that they have faced. The short answer is absolutely important. I also highlight that we have in place a substantial piece around assessment arrangements for young people who, for a large number of different reasons, cannot take exams in the standard fashion. There is a lot of work that we do to support young people to gain qualifications each year, to engage very closely with centres to make sure that any arrangements are in place, whether they are scribes or font type. There is a very impressive array of assessment arrangements that we can put in place to ensure that young people are treated fairly. We are taking account of personal circumstances in undertaking our awards. You mentioned earlier on about some of the bigger lessons that you have been talking about. We have talked about remote learning assessment quite a bit. You talked about E-enabled assessment, as well as the creation part of it. I was wondering if you could expand a little bit on things such as E-enabled assessment and the remote learning stuff that you were speaking about there, if you have a bit more information on that, if that is at the very early stages. I appreciate that, too. E-assessment is very much part of our toolbox of assessment approaches. We have started to explore even with the existing courses, some of the practical courses, how we could turn what are effectively short question papers now into E-assessments. That is just one example of the sorts of things that we could do. However, I think to how SQA receives evidence from schools and colleges, if we could do that digitally and build an infrastructure to do that digitally and to be able to carry out all of our marking electronically in the quality assurance processes. There is a lot of coursework that features in national courses. If we can find digital means, for example, for a music course, people have to perform on a number of different instruments. In the future, could we do that remotely? Similarly, art and design, there is a big portfolio of artwork and design work that young people currently have to have to be physically sent to SQA, but how could we use technology to do that digitally in a much more streamlined way? That would also open up opportunities to the sorts of things that could be assessed in the future. It might widen out the types of things that could be assessed. One of the things that comes through very strongly from young people is that they would like to see more skills development, so technology could be really helpful in that sort of area, in the assessment arena. Those are just some of the things that we could explore as a system. Some of them might be shorter-term and some of them would require a medium to long-term kind of running to do some of those, but I think that that is definitely the way that I would certainly like to see things develop. There is a big system piece in all of this. This is a broader conversation for the wider system in terms of the investment that is required and the need for all parts of the system to have the appropriate digital tools in place in order for us to do that. This has been picked up in the SFC tertiary review, particularly in the context of colleges and universities. There is also a programme for government commitment around the national digital academy, which is focused in particular on learning and teaching, but there are opportunities there around assessment as well. There are potentially some really good developments in thinking about some of those issues, which could lead to the possibility of remote assessment. Many of the challenges that we have discussed today—some of them, at least—would fall away, but there is a broader issue there. That might be something that Ken Muir will wish to pick up on in terms of the review that he is undertaking, particularly in relation to how curriculum and assessment both integrates and is delivered. E-School has been a very important development and has been used in the pandemic for broader purposes. There are opportunities there, but it is a systems piece. That is great. I think that we did not get everything right and we did not quite get the balance right in some of the areas, but it is good to know that we are in a better place this year in that, looking to the future, there is a lot of positive stuff there as well. Thank you to all of you for your commitment to working over the next couple of years as those changes come through. Indeed, we are all united, as I said earlier, in the respect of caring, passionately about the future of our young people, because they represent the future of our country and our planet. Those are big issues. I do not think that anyone would deny that. That is the big stuff of politics. I thank our witnesses today, Fiona Robertson, Dr Jill Stewart and Beth Black from the SQA. I am very grateful for the time that you have given us. We have tested your stamina. You have not been found wanting. We have kept you going for two hours and a quarter. That is quite a lengthy session of cross-examination. At this point, I would like to bring our meeting to an end. The next week's meeting on Wednesday, 6 October, will take evidence from the Cabinet Secretary for Education and Skills on the Cabinet Secretary's priorities for session 6 of the Scottish Parliament. I will now suspend the meeting and ask members to reconvene on Microsoft Teams, which will allow us to consider our final agenda item in private.