 Mae'n fyddechrau i fy mod i ymlaen i gwybodaeth y cyfnod rhai o gwybod y afael gan unrhyw yng ngyfyniadau yn danai, sy'n nhw'n hynny'n i wneud ein oeithi arbu addysgwys wahanol a'r ystafell yw arfermwag yma, a'i weithio bod sefiaid y cyfrifol yw'r cyfrifol yn sylwg, ond hefyd, dywedadol y cyfrifol bwysigol mewn pan ar spokenu i dod y sefydliad a'r bwysigol trys hynny ond rwy'n i gweithio arweithio gallu llunio i'r rhain i'r llunio'r llunio. Ac mae'n gweithio, Steve, yn ymwneud arddangos yllanod y Rhym Aelodau Rhym Cwlau. Mae'n gwneud bod ddweud yr hynny'n cael ei ddyfodol sydd wedi bod dyma'n ddweud o gweithio, gwneud cyfnogi, gwneud yn cael ei ddeimlo, i ddweud o'r ddweud. Mae'n cael ei ddweud o'r deimlo, mae'n ddweud o'r ddweud, Finland, Iceland, Sweden, Wales, Ireland, Scotland, and Ontario. I want to talk a little bit about what we've been saying in these discussions about accountability and also give you my own views about some of the issues. I think there are five basic questions you have to ask yourself about any accountability system. Pretty basic ones and you'll know what they are but it's worth just going through them quickly. First of all, is your system reliable? In other words, if it's an assessment system, is the marking consistent? If it's an inspection, would you get the same judgement with a different set of inspectors? If it's teacher assessment, is it moderated thoroughly and quality assured? So is it reliable? Is it consistent? That's the first question. But it's not the only question because you also have to make sure that it's valid. That the assessment is appropriate for what you're trying to assess. If you want to assess well-being, children's well-being, using an academic examination performance is not going to give you the right approach, it's not going to be valid. If you want to assess whether schools are developing a broad education, just measuring them on examination results is not going to be valid. So it has to be valid as well as reliable. But thirdly, and this is crucial especially for school leaders, it has to be fair. You have to really understand that whatever type of school you're in, it'll be fair. Treat your school fairly. If you have a privileged intake or you have a disadvantaged intake, it will be fair. If you have a lot of children with special educational needs in your school, it will be a fair accountability system. That you can be judged on progress not just on outcomes. This notion of ipsitive or improving on previous best or compared to what you were doing before is crucial to make a fair accountability system. To use a football analogy, Manchester City are a very, very rich club. Clubs like Burnley or Swansea or Cardiff are not so rich. Making a judgment about which is the best run club, who is the best manager, is not straightforward. You've got to compare like with like. In the same, you have to have a fair system comparing like with like in schools. Fourthly, different purposes. Now, you see this mistake being made that some systems try and use the same information that you use in the classroom to give children feedback. I've created it up to a school and I've created it up to a system and think that system would necessarily work. It doesn't necessarily work because there are different purposes, as Steve said earlier. So being clear about the purposes of each set of data is really important. Finally, the one I think is obviously close to your heart here is about consequences. Whenever you have an assessment accountability system and you decide that some things are going to be part of it and some things are not, you will inevitably, especially if it's high stakes, get schools that will focus and prioritise on those things for which they're going to be able to account. It is inevitable. It's going to happen. At the more extreme level, you get gaming going on. And at the very extreme level, you get cheating going on, as we had in United States and some states when they had high stakes testing. Because schools and education are an ecosystem. You can't just interfere with one aspect of it by changing the accountability system and think that nothing else will change. It's an ecosystem. If you start meddling with and impacting on the accountability system, it will have a knock-on effect on the rest of the system. In England, for example, which is a very high stakes accountability system, the number of children who have been off-road, the number of children who start off in year 11 in a school, on a school's register, but don't end up by the end of year 11 on the school's register, has gone up significantly over the last few years. As schools are aware that if you have difficult children who aren't going to perform well, it'll do badly in their overall examination results. And the number of children in PRUs has gone right up in England over the past few years. Now, some of the consequences are harder to spot because it's a complex ecosystem. So you could decide to take the pressure off, reduce the accountability. The profession takes a collective side of relief and a few years down the line academic standards start to decline. That's an option. You could go the other way. We're going to really push the pressure up. You have stronger and stronger high stakes accountability and you find you're putting more and more pressure on students to perform well exams. You get mental health issues, an increasing problem with children's mental health. Or you find it increasingly difficult to recruit and retain teachers. Or you find it increasingly difficult to get head teachers to work in challenging schools. It's an ecosystem. It has unintended consequences. Given a set of criteria, a policy or programme is said to be systemically valid if it results in improvements in one or more valid criteria but without causing significant deterioration in other valid criteria. That's the holy grail. So you get improvements without knock-on negative impact. So back to the Atlantic Rim Collaboratory, we discussed the accountability issues across our nine systems when we met in our summit in Ireland last September. These were the conclusions that we came to. We said that we're clear about different purposes. Don't try and use the same set of data to perform different purposes. We said both big data and small data have their place. So of course it's good for you to know if your school, based on big data, is performing better or worse than other schools. It's good for you to know that one department in your school is performing better or worse than departments in other schools. That's important information for you to know. But big data does not answer these two questions. First question is why is that happening? It doesn't tell you that. It doesn't tell you why one department is doing worse or better than another. It doesn't answer this question either, which is what am I going to do about it? So unless you get into the small data, unless you get into actually what's going on in classrooms, you're not going to have an approving system, you're just going to have a system that's comparing one to the other. So big data and small data have their place. And where small data really comes into its own is in things like peer review, in lesson study, in joint action research, in school self-evaluation, when you're getting under the skin of what's going on. You're actually looking at what's going on in classrooms, you're looking at what's working and what's not working based on evidence that you're observing. You're seeing it happening and you're trying to make judgments based on how to improve what's going on in your school. And that's the power of small data. We also, at the arc summit, we said that consider how to use sampling as a way of working out what's happening over the standards overall across the system. You don't have to gather information from every school and publish it to find out how overall the standards are in your system or your country. Israel is a good example how it uses sampling very well to monitor its overall standard in its system without publishing school-by-school results. The next one we said is that when designing an accountability system involve teachers and school leaders from the outset to help to think through the negative impacts and get it right. It's crucial if you're going to have a good system. And by the way, it's great to see the Welsh Government doing this today with you as secondary heads to help to shape that and be involved in the design of it, not just beyond the recipients of some system that's been developed by officials. Two more points. Consider use of a balanced scorecard which gives a more rounded view of how a school is performing. Ontario are currently looking at this, how they can move from just publishing exam results to publishing a balanced scorecard, a report card for each school. Equally, they're looking at this in California. And then finally, systems need to ensure there's both credible accountability and investment in capacity building. Let me just say something about this. Colleagues, we need to have accountability. We all know this. Huge amounts of money are invested in public sector and in schools. Parents entrust their most precious thing, which is their children, into our care as school leaders. Of course, we should be held to account for how we use that money and how we support those children in our schools. And if you remove accountability, and I've seen this happen in other countries, especially in Africa, you get some of the worst aspects of an education system. You get complacency and you end up even getting corruption into the children that lose out. So if you have an accountability system without capacity building, without investing in support and help and training, we'll have a serious problem on our hand in terms of implementation. So at the Atlantic Rim Collaborative, we're saying have credible accountability and have support and capacity building to make it work. And the implementation of this is so crucial. I've been working with Victoria State in Australia around some of these issues. And in Victoria, they've got some great policymakers, really bright people who are great at developing the thinking about what would work. The trouble is it's not having an impact in the schools. It's great in the officials office, but it's not having an impact on the schools. So I shared with them some of the things we know about how to manage change across a school or across a system. There has to be a perceived need to change. So the first question for you in Wales is, do you have an accountability system which needs to change? The second question is, there has to be a compelling narrative. Not just we need to change, but is there something exciting we want to change too? Have you got that compelling narrative that this is what we can all believe in and come in behind? Thirdly, not just done too, but are you invited to participate to shape that? So it's not just top-down. Are you involved and invited to help to make it work? Fourthly, is there peer support and pressure once it's going? So your colleagues are challenging you and helping you to make it work across schools. And next, are you putting in the capacity building, the resource, the support, the time to make this work, otherwise you could have a spectacular failure on your hands? And finally, are the consequenters of not doing anything, of ignoring it? That works at school level and it works, of course, across the whole system. So I want to finish with five key aspects of an effective assessment accountability system. And the fifth one is on the next slide in case you're wondering if I can't count. First one then is involvement. It's crucial that you as school leaders are involved in these discussions from the outset. Not just talking about purposes, though that's important, talking about workload, talking about bureaucracy, talking about consequences and unforeseen consequences at the outset. No-one will know better how to gain the system than you, so you should be involved in designing a system that will minimise the negative impacts. But secondly, incentivise school-to-school collaboration. Now I don't know any system in the world that's improving or doing well, though it doesn't really push school-to-school collaboration. But there are some lessons we're learning about this. And one is be wary of making it compulsory. Be wary of making it compulsory. Be wary of making peer review something you have to do. Be wary of being told which groups you have to collaborate with, whether you want to or not. New Zealand tried this in the early days of trying to get collaboration going and found big pushback from the schools. But when they incentivised it rather than required it, they got much stronger buy-in and really some quite important development. So here I think the issue is making it voluntary but inevitable, voluntary but inevitable rather than compulsory. And the second point is equally important. You will never get really strong and deep collaboration if your accountability system is always about each individual school's performance only. You'll never get strong and deep collaboration if your accountability system measures, schools and compares them against each other. Unless you get some accountability, it's about a group of schools, you'll never get deep collaboration. So when you're developing an accountability system, I'm not saying you shouldn't hold schools to account, of course you should. But some of that should be about collective accountability across a group of schools and that's where you'll lead to deeper collaboration. The next one is about sustainability and be wary about quick wind. You probably followed the work of the Centre for High Performance in England which looked at 160 academies in secondary schools in England over five years. And they came up with different types of leaders for these schools and the surgeons were the ones who got the quick winds, almost instant results, focusing entirely on the accountability system. Unfortunately when they left, which they often did quickly, the results went backwards and the school ended up in a worse state than it had been before the surgeon arrived. The schools that had the best leadership and the most sustained improvement were led by people called architects. And interestingly, architects nearly got sacked in the first two years for fearing to make enough impact on the bottom line of results. But long term they got sustainable and long term improvement. So if you have an accountability system that measures the declared success or failure too quickly, that measures it on an annual basis and says your success and your failure, you could be making a big bad decision for how your system is going to run. My own view, a three year rolling average is much more effective and appropriate than making judgments on one year's results. Fourthly, something about capacity building. Michael Fuller and I is a great Canadian writer. You must have heard of him, seen his stuff. He and I wrote a think piece together a couple of years ago on what we're seeing around the world in different systems. And this is what we said. Too often the end result of high stakes external accountability is exhausted, discouraged teachers and leaders stressed on the rack of contract accountability but not given the capacity, the time, the resources or support to make any of this really work. Policy makers are left scratching their heads wondering why change is so resistant to their will. Colleagues, if you go down this journey of accountability, even though it's well thought through, unless you address capacity building, if you're requiring teachers to behave differently, to get into that small data, to do peer review, self evaluation, ongoing research, you have to invest in capacity building and training and support in time to make this work. Otherwise you'll have what we described there. And then finally, colleagues, it's all about leadership because no matter what happens, no matter how clever you are in designing an accountability system, it will not be perfect. And there are bound to be school leaders who say this doesn't quite work for me. It's not exactly a perfect fit for my context. And again, the quote that we got from Michael Fullen was saying in the end, what Michael and I said, in the end, let's not be a victim of external accountability. Let's show leadership at school level and at local level. Here's the quote. The idea is to change the game from compliance as a victim to purposeful focus. It's the responsibility of leaders to shape the culture until that although they take account of external national accountability requirements, which you have to do, of course, you develop an internal collective accountability system that leads to the right outcomes. In other words, no matter what the national accountability system is, you take into account that, but you as leaders make the weather in your own school, in your own context, and you provide strong, clear leadership and an internal accountability system that gets to the right outcomes for your context, for your community, for your children. In the end, I think it's about asking one key question. I want to introduce this question by saying I did a lot of work in Africa over the last few years. Now, when the Masai tribe greet each other, sometimes they don't say hello, sometimes they say, Qasirianingera. Qasirianingera means how goes it with our children. Because the Masai believe, unless it's going well for our children, then the culture is not in a good place. And they don't say how goes it for my children or your children, they say how goes it for our children. When you develop an accountability system, that's the most important question you can ask. How goes it for our children?