 Best excellent to the 15th meeting in 2022 of the Finance and Public Administration Committee. The first item on our agenda is to take evidence on the national performance framework. Before we start our evidence session, I would like to put on record my thanks to all those that took part in the workshops in relation to our national performance framework. Ambitions into action in quiet last week. I hope that everyone found the events in Dundin Glasgow useful and interesting. I know that I certainly did, and colleagues have spoken to it, but I believe that we have wirionedd hyn o'r dupos y cyfnodau. Felly, os ydych yn gweithio dwylo'r cyfrifiadau i gyfnodd gwerthu'r cwyrdd i ddweud, a'r gweithio ar y gyferwyr Ian Elliott, lleidiau sefydliadau yn Llyfrgellol yn Llyfrgell ym gyfrifol yw Llyfrgell i unrhyw uneddiad ac yn ôl i, mae anodol o'r Ysgolfaen Unifedig yng Nghymru, mae'r Argeddot! Argymai, addysg ac i'n ddych yn fwy i'w athgwrs oswn i'r 90 yma yng Nghaer claywch gael eu gweld. Dr Max Frenge Llywodraeth Cymru yn Llyfrgell yng Nghymru, 64 Speydydd i ddechrau i Llyfrgell yng Nghymru yn Gwelch Cymru. O'r fawr, rwy'n ddull i chi ddwyny yn ei ddokol i niw. Rwy'n fawr o gwerth o'r ddodol. The Auditor General said that, in terms that I have published on a blog on September 7, Scotland is suffering from a major implementation gap between policy ambitions and delivery on the ground. It goes on to say, I am not convinced that public sector leaders really feel accountable for delivery change. What is the panel that is feeling about that? Rwy'n distracting ministers to think more strategically to move away from a mindset of thinking in granular detail about inputs and outputs to thinking more about outcomes. That proved to be very successful at the time. One of the aspects of that was freeing up local government in particular through the concordat that was developed at the time to free up local government o'r bernawed cyfyrdd o bo cyffredinolaeth, a i nefydd gwnaer daddiw 현 yn wirionedd, a Filln mayaf ar Wordig i'r hunain. Fy gyd, predictorfa g подтurer diwrnoddiad, hem excesson llun 문hach mwy oim, rydyn ni'n cuanto ochr i unrhyw gwin ditur isiorenau iddo tytofï mae'n a how they can better implement these things. There are a lot of challenges that we have got to recognise in that. At the same time, I do worry that some of the strategic mindset within Government has been lost and there is slowly creeping back again. I focus on things like how many police officers there are on the ground, rather than thinking more strategically about the outcomes that we are trying to deliver on. rwy'n iddynt i bobl gynllenio meddwl Caerdydd amser. Rwy'n meddwl i bobl yn rwy'n meddwl i bobl gynllenio meddwl i bobl gynllenio meddwl i bobl gynnalol. Rwy'n meddwl i bobl gynnalol. Rwy'n meddwl i bobl gynnalol. Mae'n meddwl i bobl erbyn mewn i bobl. Pryddo i'r fawr i confiol y gwir yw darllen nifer oes dynamiteil. Mae'n fawr i bobl gyda ni'n gwirio'i cyfacebydd Cymriol i bobl gyda ni'n wneud yn incidence. Rydw i ei fyeiciant i'r pam eff celeryg i ofyniol. Maes hiaw Yong Playing, i arniw ym designed i gynhyrch, i ddinch theiar a grapesaf—olaeth y gwybod늘 mi. Maelineis i dдаe uddig mor rhai i Gunan agon wch gyda i Abergiol neu i g tacker Fenesia eu ysbyfryd gener. Rydyn ni'n tref boundsief i ddweud ad While a gwnaeth localwys i'r fitting i pa defnyllu ysbyryd i'n symud eiибwys i'r Herape Port— ac weithio yn ymикиw bod sgaflding yw'r Busan hyn chweithu that that scaffolding is missing in Scotland and part of the problem with implementation? Okay, Dr French. Thank you. Just to agree also with the other general's comments and I think what we're beginning to see now is quite clear evidence of an implementation gap even within the devolved nations and Scotland has achieved less implementation depth within Government dweud i fynd yn y weilol, ac ydw i'w cymdeithasol yng Nghymru arall ar 2018 gan gy точu amser oes o boblion yn cyflwymonau tynnol i'r digw sydd benedd yn y weilol, fel i gael arferfynol yn énodol yn amser o'r cy malesyn i fynd yn rhan o'r holl o'r cyhoeddiadau a'r adnewyddol yn ddefnyddol. Diolch yn ffwrdd i fynd yn têpat i ddim yn ddod ar gyflwymon, rhai i fynd i'r holl o'r cymdeithasol y yn fan y rhen i'r bigger I hope that Scotland has been more in measurement than what you do with the measures. I think that the lack of a strategy is really becoming evident in the patterns of implementation outcomes that have been made to paper which systematically compares Scotland, Wales and Northern Ireland on their implementation of the national level outcomes indicators that support that view. Rydyn yn fwy o ran o'r cyflogwadau newydd yn ymgyrch yn ddwyngiadau'r lleol a'r cyflogwadau'r llwyb. Felly, rydyn ni'n rhoi ddod o'r ffocws gwirionedd ychydigol i ymgyrchu'r cyflogwadau. Felly, dr. Frynes, yn gyflaen i'w ddim yn y gynnwys, mae'r NFF yn anghwm i'n fwy o'r ffordd yn fwy o'r lleol a'r cyflogwadau i'r cyflogwadau. The Scottish Leaders Action group says that the current status of accountability against the NPF is patchy, and typically the NPF is not actively used to shape scrutiny, provide sponsorship, undertake commissioning work or shape the allocation of funding. Is there improvements and where are those improvements taking place and where is the area where we need to focus most on in terms of ensuring the NPF delivers on what it is supposed to do? I would say with in terms of the improvement, since 2018 there has been dedicated leadership within the Scottish Government within the performance and outcomes directorate. There has been a small team doing external communications and engagement in that and working internally with the Government very slowly to change some of the processes that are involved in reporting and policy making and so forth. I would say probably too slow, but it is a very small team. In terms of the implementation, there has been a very slow gradual improvement and focus on that. Also, the externalisation of this to an external whole of society approach rather than a whole of government approach has really galvanised some external interest. Also, as you are seeing through the accountability and incentives group within the Scottish Leaders Forum, when you get external people involved in some of the scrutiny, which is something that Scotland has lacked and has not had an external scrutiny organisation for its NPF, there is more energy and more criticality brought to it, as you can see through their contribution to the inquiry. We found when we were out last week a huge amount of enthusiasm and energy for the NPF, but it is about how widespread that is. I think that your research is very important in identifying where there are issues and indeed the Scottish Leaders' Action Group is, too. Ms Wallace, in your response, you say that, while there are some sectors and directorates within national outcomes and more visibly embedded, there are many places where other statutory duties and non-legislative frameworks are seen to take precedence. Do you want to give us some examples of that? Of course. There are two recent examples where we have drawn attention to a very surface-level alignment. The first one is the national strategy for economic transformation, which includes only two references to the national performance framework and no explicit references to the statutory national outcomes. It is difficult to view that as a way to say that it has had regard to it. I think that it has put in some sentences and added a reference to the national performance framework, but it does not feel that the national strategy for economic transformation is aligned to the national outcomes in a way that we would have hoped from an external organisation. The other one that we have been actively involved in—we are members of the Children and Scotland Policy and Strategy Forum and have been for a number of years—has been very actively interested in improving children's outcomes through that cross-sectoral group. We had engaged with the Scottish Government on the Children's Well-Beings outcomes, hoping that that would be a children's version of the national performance framework. That was what we understood the intention of it to be. The process that was gone through was to identify indicators that are currently in use by local government that relate largely to children's services. What has come out of that process is a children's services outcome framework. It is a framework that looks at Shinarie, which is the personal outcomes for children and young people, and Girfech, which is the service outcomes for children's services. Why does that matter? It matters because there are a huge number of things that are important to children's wellbeing that are not included in that draft framework. That includes, for example, access to place spaces, which we know are really important to children and young people. They say themselves how important that is to their own wellbeing. There is no information in there on air pollution, yet we know a huge amount about the relationship between air pollution and asthma and children's long-term health outcomes. By taking that narrow prism instead of a holistic wellbeing approach, it does not, to us, conform to the ambition of the national performance framework. Those are just two quite recent examples that I hope help you to understand what we are trying to say in our evidence. What you have also said in your submission is that we need a strong advocate with powers and duties to ensure their prominence in policy development and delivery of national outcomes. What sort of powers and duties would this individual have? We are attracted to the model in Wales, as many people in Scotland are at the moment, because, as we have seen, the Welsh legislation and the Scottish legislation have come into force at a very similar time, but have had very different success rates in terms of implementation. One of the factors that that has been put down to is the existence of a wellbeing commissioner and office for that. What is underneath that is the commissioner's ability to request information and a very strong memorandum of understanding between the commissioner for future generations and Audit Wales to pull together the powers of both organisations for maximum scrutiny impact. It is not merely that they produce an annual report, as is expected in Scotland, or that they produce research. They also use powers to request information to hold publicly to account Welsh public services and bodies for delivering on their national outcomes. We are missing that key accountability route. We know in the Scottish legislation that public bodies have to have regard to the national outcomes, but there is no mechanism in the Scottish legislation to hold them accountable for having regard to the national outcomes. The other interesting and important point in that is that, in the Welsh model, accountability is not for delivering improvements to the national outcomes. It is for having processes in place that show how they are trying to have regard to the national outcomes and the goals and the work that they are doing. The accountability is a process-level accountability to show how they are trying to change their ways of working, not an accountability for a set of indicators that quite rightly many organisations say they cannot be held solely accountable for. There is a complexity there, but we think it would be quite helpful in a Scottish context to learn from the Welsh experience. Thank you for that. Dr Elliot, you are nodding vigorously there. I am just wondering what your thoughts are on that. Also, if you could touch on the issue of the strategic state, which you have mentioned in response to the first question, you have obviously talked about the strategic state in your submission, so I am just wondering if you can expand on that. Just to pick up on something that Max had said, I completely agree with what he had said. I think that there is a risk in looking at the NPF again that the decision is taken to change the indicators or change the outcomes or do something like that. I think that it is really important for this committee and for others who are involved to consider that implementation gap and to consider how to implement the framework that we already have rather than necessarily making some sort of radical reform of that and spending years of time coming up with a new framework rather than considering how do we make this happen in practice. What Jen has said around the focus on process is really important as well, because this was never in its original iteration. This was not intended to be a kind of performance measurement framework. It was not intended to be counting specific indicators in that way. It was meant to be a decision making tool to get people to think more strategically and more collaboratively across different directors within the Government but also beyond the Government as well. That is where the idea of the strategic state comes about. That is something that has come about from the OECD originally. My research is really looking at how the Scottish Government's approach to the NPF is quite closely aligned with what the OECD considers to be the idea of a strategic state. That is where that concept comes about. I think that it is important that we do not spend too much focus on measuring indicators but on thinking about that process. If we think about things such as the UN Sustainable Development Goals, in a similar way, it is a similar kind of philosophy. They are not intended to be tick-box exercises. The UN is not expecting that all 193 countries are going to measure against every single one of the goals, but they are meant to be stretching ambitions. The expectation is that people will be making progress towards those goals. It is how you demonstrate how you are making that progress and what steps you are taking to do that in the way that Jen has described thinking about the processes, rather than that granular detail of specific measurement of things. I would say that that is an important factor to take into account as well. Dr French, you have said in your submission that there is a lot of evidence that the NPF has been meaningfully incorporated in the organisation routines within the organisations on changing decisions, promoting learning or altering policies. How can we ensure that that takes place? There are probably a few things that we need to do in order to achieve that. I think that a lot of learning can be taken from Wales. It is the combination of statutory duties, scrutiny, accountability, and a lot of the submissions to this inquiry have reinforced. It is also building on some of the strengths that surrounded the NPF and some of the Scottish Government's worker on that, which is positioning that as a holistic societal approach with broad ownership that galvanises people and excites people and makes them want to engage. From Wales, the learning is that if you strap on additional duties in the new bill that would force public bodies to plan for, promote, set objectives and account for those objectives, they can choose to do that in a very passive, superficial way if they are so inclined. If they are encouraged to do that through a supportive package of supportive challenge through a commissioner or an auditor and an inspection body and to see the value in that duty in promoting their own objectives, a community planning partnership setting a loyp that sees the NPF as promoting its own agenda and being important in that regard, you need the will as well as the duties in order to get effectiveness. Otherwise, I think that you get lip service to those indicators and to those duties. I think that you see that in Northern Ireland, the Government departments that have really gone for their outcome-based accountability approach are the ones that wanted to and the ones that saw the value in it and the ones that saw that their own ownership of that reflected in their contribution. If they wanted to, they could do it as a very passive reporting mechanism and not really do that, strategising that Ian's talked about. I think that both of those things are important. You have talked about galvanising stories that capture the public interest and communicate its values rather than merely list statistics in your section on soft power and hard power strategies. I wonder whether you can talk us through a wee bit of that. What I mean by that is that the NPF in some way needs to connect with a public interest, but it also connects with media interests and parliamentary interests. It needs to connect with the stories that people want to tell. Statistics showing indicators going up and down and staying the same. Even in aggregate, they do not really capture the public interest. Scotland performs has had historically low engagement rates from the public and very little uptake in the media. Again, if we compare it to Wales, the commissioner has produced guidance for parliamentary members and also for media organisations so that if they want to tell a story about Wales's act and its national outcomes and indicators, they can do that and they can be provided with the guidance and the structures and the scaffolding that enables them to do that. Pure performance measures have historically low interest in the public except if they are accompanied by a narrative that explains why those are important. Just in the last final question review for the session, it is to yourself, Ms Walts, which is about—we went into the last suite of those, a lot of energy enthusiasm for the NPF. One individual said to me that the problem that we have with the NPF is that it is yet another series of things that the Government expects us to do. We spend a lot of time having meetings thinking about when we will get this set of objectives, we will get that set of objectives, and they can be overlapping, they can be kind of not quite contradictory, but there's a huge amount of energy and resource that seems to be taking up, saying, well, what do we prioritise? How do we cut through that, you know, and through the Guardian, not of all these different kind of objectives that are being put on local authorities and other organisations, for example, so that people can see more clearly and effectively help to progress the aims of the NPF? Thank you. I would recognise that experience that the person in Dundee was describing, and certainly in the conversations and what we have reviewed, there are many instances of the duties to provide statistics landing on officers within local government who have to furnish the information for these plans to be produced, I think, in one local authority, and because there will be better place to do that, but one local authority reported to us that there were nine separate plans related to the same thing, in that case about children and young people. My favourite quote on this again is from Wales, the Auditor General in Wales said, we need a radical decluttering, and I think that after a generation of legislation in Scotland, we also need a radical decluttering, we're very good as a nation in adding on additional duties, and particularly additional duties to create a plan, but we need a process to assess all of those now and to assess which is the most important. I think that one of the, I hesitate to say failures, but one of the disappointments in the Community Empowerment Act when it created the national outcomes was that they did not, in any sense, sit above that sense of the plans and the frameworks and the other activities. It's quite often when you see them referenced, do you actually see them referenced at the bottom rather than at the top? In the flow, it feels very much like they are an afterthought, rather than being the thing which departments and agencies can hook their activity onto. So it's a retrofitting rather than an analysis from the outset. A radical decluttering is probably required in Scotland in a process to do that effectively in consultation with the many people who work for the benefit of Scotland, and then an exploration of how do you create a system that shows very clearly that the national outcomes are meant to sit at the top of that run of plans and frameworks and not as an afterthought? I'm a big fan of decluttering, I must say. Anyway, we'll open up to the colleagues around the table and the first person to ask questions will be Daniel Thorford by Ross. Yeah, I like that phrase radical decluttering. It sounds like what I'm constantly being told to do at home. I'm very interested by the conversations so far, and I'd like to just pick up on a couple of different points. I think there's a structure seems to be the thread that runs through this and a number of different levels, but I'd like to start with Dr Ian Elliott. You were talking about national forms framework originally being a decision making tool, but when I actually look at it and you step through the outcomes and indicators, the outcomes are relatively straightforward to understand. The indicators, you suddenly just get landed with a sea of bullet points that don't immediately, difficult to sort of intuitively see what they are trying to tell you, or even whether that's one thing or a number of different things. Actually, if you step through so, I was just looking at the children's happiness, and actually it turns out that that is just one thing, that's one survey that measures four quite narrow metrics, which are valuable, but not necessarily entirely encompassing what I think we'd all understand to be children's happiness. So is one of the problems here, the structure of the NPF itself, in that we have very good high level outcomes, and then actually a kind of quite asymmetrical set of indicators that sit below it, none of which are kind of intuitively easy to understand what they're telling you. In other words, is that NPF too difficult to use in its current structure? There's a big issue with focusing on metrics and measuring metrics within what is essentially a complex system that many different agencies and government bodies and third sector bodies are all working towards, and that's where I think we get into these issues that I was describing in terms of thinking about it as a performance measurement tool, that it becomes actually quite problematic, and I know that Max has done quite a lot of research on that as well in terms of some of the issues with having a very metrics-based approach, and that was what I was meaning in terms of the reforms that were brought in way back now in 2008. We're really about trying to get the Scottish Government to think on a more strategic basis, and it was very much for the internal government at that time, and there was quite a lot of leadership around that and quite a lot of leadership development. Just picking up on some of the things that have been mentioned in terms of Wales, I do think that that is a really interesting example to look at, which I've referred to numerous times, particularly that they have Academy Wales, which does a lot of leadership development activity, and they've got really strong links between the Government there and academia, the Welsh Centre for Public Policy, being an example of a really strong research unit that helps to underpin a lot of that work and provides some of that scrutiny. I'm not sure about the idea of the commissioner, I'm not as familiar with that, I must admit, but I think that there's certainly a role for Audit Scotland in some of this work, and I can see how they're starting to enter into this space more, which is really encouraging, but also arguably for committees as well, for all committees, not just this committee, but for all committees to really consider how the work of the Scottish Government is being scrutinised in relation to the performance framework and in relation to those outcomes, rather than, as you're pointing to, the granular detail of specific indicators. Maybe Dr French is picking up on that, but is there a sense in which the metrics need to be split apart from the capturing of the outcomes? On why I say that, I accept what Dr Elliott is saying. However, if you just had qualitative outcomes with absolutely no measurement of them, again I think that we'd have a problem. I'll just relay something that was said at the focus group that I was at in Glasgow, we had Parallel Ones in Dundee in Glasgow, which was that a sense in which we are not using data properly, we've gone very narrow metrics. In the world we're in, in the 21st century, it's about big data sets and much more richer data analysis. Do we need to split apart the capturing of the outcomes from the measurement and do we need to overhaul how we're conceiving of what that measurement looks like to capture that 21st century big data approach that's available to us? I think that one of the big problems with the NPF is that it's a robust framework, internationally comparatively speaking. It has a lot of documentation, statistical backing and some sort of agreement that whether an indicator is going up or down and there's been a lot of work done within the Scottish Government on getting the statistics right. I think it's a robust framework but I don't think it's an accessible framework. I think it's hard for people to see what role they can play within this framework. If this is presented to a community planning manager, when they begin to think about the construction of a loyp or a locality plan, it's very difficult to see how that can relate to local manners much easier to consult with the improvement service who have more localised measures and who have better working experience with councils and with local organisations. The accessibility for me is less to do with the indicators and outcomes. I think that indicators have to feed into outcomes. There has to be some way of making sense of movement in the outcomes and that will always come about with trade-offs and with uncertainties around the measurement. It's never going to be perfectly objective and that's fine. One thing that the NPF doesn't have is that it has values and they are good in a multilateral cross-sectoral setting. It doesn't have ways of working. One of the things that's been really effective in Wales in the scrutiny context is that it has a set of ways of working that are going to allude me, but they include things like collaboration, participation and long-term focus. Things that we know that if you're taking an outcomes focus and a focus on well-being, on collective well-being, those are essential principles that you adopt. That has proved much more effective in galvanising some of the practical actions that are taken, both in the planning and the strategising that Dr Elliott was talking about in government and out with government, but also in the scrutiny functions. It's much easier to look back and see whether the quality of long-term planning has been there and whether the actions have lived up to the ambitions over a long-term or five-year period, for instance. That layer is missing. It seemed to me a perfect opportunity to revise and reintroduce the Christie principles, the pillars, which align very well to Wales. Some of the practical guidance that organisations can access and can relate to their own context. High-level outcomes frameworks need a process of localisation to make them relevant locally. There needs to be a process of really taking stock of the indicators and the outcomes and coming up with a valid, stretching interpretation of that, what that means to a local context. How we've done aligned frameworks in government, there's a justice outcomes framework, there's a children's wellbeing outcomes framework, there's what's it, health and social care. The alignment is done after the fact, so we come up with our outcomes and say, well, that one's related to that NPT, that national outcome, that one's related to that other national outcome. It's really done after the fact rather than at the beginning and ensuring that the alignment runs through it. I would say that there's a few things that could be done to make the NPF more accessible and more legible, particularly as it moves to a whole society context. The other thing I would say just very quickly is that the focus on measurement can't be done at the expense of a focus on implementation. It would be, I think, a missed opportunity if this review and its national review, the statutory review, focuses just on the measures and the technical elements of this. It's a difficult problem, but it's difficult in a straightforward way. Can I ask you for a clarification, when you're saying about making it more accessible, and also in terms of what you were saying there in terms of, it seemed to almost be a point in the right interpretation. I wonder if what you're saying is actually, there needs to be a both a government-wide level, but then an individual sort of, whether it's a directorate or agency level, an almost interpretation of how outcomes are going to be influenced, because it does strike me that you bring two people together and they'll have completely different ideas actually, what will impact those outcomes, and unless you actually have a stated view as to how those outcomes are going to be implemented, you're not actually going to get any consensus at all. Is that a fair interpretation of what you're I would use instead of interpretation and negotiation, so a meeting of parties that incorporates both interests and comes to a formalised agreement. That might mean that some local measures are important, but it might mean that some national indicators also take precedence, and that is an agreement that set out the outset. Finally, Ms Wallace, I'd just like to come to you. I was struck by your written submissions where you were saying that the processes for implementing the national forum for us are weak. Ultimately, I wonder if we're sort of dancing around the issue here, and I think it's good to talk about processes, good to talk about agencies. Does it not actually come down to individuals? Do we not need to actually put people on the spot and make them accountable for delivering those things? In terms of our conversations, there's a sense in which it's almost voluntary for both individuals and agencies as to how they're electing to play their part in the national framework. Do we need to have cabinet secretaries and ministers reporting against this, directorates and agencies and make that much more explicit? If we were looking at overhauling the community empowerment act legislation and putting in place a robust piece of legislation, that is precisely the type of thing that you would do. You would follow through the process to say that we are going to require those agencies to have regard to that, and then we are going to nominate who they have to be accountable to for having regard, and that is the piece that is missing. I think that we are saying that you can do that in a number of ways. That could be to the whole of Parliament, it could be to specific committees, it could be to commissioners' office or to Audit Scotland. At the moment, in the absence of that, while it's not voluntary because it is a statutory obligation on them, it feels voluntary because nobody is asking them, as you say, to report on it. I think that that is the gap, which is quite often expressed in terms of an accountability gap because people are then interested in the relationships. The perception of that then becomes that you can't make us accountable for outcomes, and that's where we lose some of the conversation about what's actually possible here, because we all understand in a complex society that a linear accountability for outcomes is neither possible nor desirable. So we're trying to find a way through this to hold people accountable for the bit that they actually can be responsible for, which is in terms of how they are having regard to explaining how they are changing their behaviours in light of the information coming through the national outcomes and the national indicators, so that we can see that golden thread. Is that where an agreement might come into play, where rather than just assigning an outcome or area to an individual, you've got agreements that explain the contribution to it? Yes, absolutely. You would say within an agency it may not be possible within an agency or a local government to have action across all of the outcomes. It may not be desirable to be doing that, but an agreement to say actually we're going to focus on these three outcome areas and this is how they're related and this is who we will work with to make that happen would be a conscious step forward from where we are now. We're so much of the change process is put on individual culture change rather than externalising that and trying to change the structures that create the cultural behaviours in the first place. Thank you very much. My understanding of this is considerably decluttered from when I started from it. Ross, to be followed by Joan. Thanks. The group of individuals that Daniel and I met last week in governing the organisations that they represented began to call us around the word implicit when we were asking them about the alignment between their organisation strategic plan and the NPF. If I could start with a relatively general question to Jennifer in the first instance, is it fair to say that at the moment the NPF is operating more like a set of general principles that shape public sector culture in Scotland than a specific set of measurable outcomes? I'm not sure I would use the word principle, but I think the principle of working towards outcomes is in place. The idea that there is an agreement within Scotland that we need to collaborate more, that we need more joined up working and that we are doing that in order to improve the lives of the people that we serve is well understood in that. The process underneath that for how you transform that set of ambitions into something that an organisation can do, there is not a sense of shared understanding or agreement on that or guidance or strong case studies or the types of things that people would actually be able to assess whether or not that was important for their organisation. On the culture of Scottish civic society and public services, as I see it, working in that direction, yes, that is what they are trying to do, but they are doing it without much of a shared sense or a clear sense of how to do it. You mentioned in your submission that administrative leadership around the NPF has diminished over time, and you have alluded to that a little bit already this morning, but could you go into a little bit more detail on who you are referring to there and why that has been the case? Yes, and that picks up as well on something that Daniel was saying and Jennifer earlier about accountability, where does accountability lie. Certainly when it was first established, there was that clear sense of leadership and ownership within the Scottish Government, but what I have found more recently in speaking to people is that there is a degree of confusion as to who owns the national performance framework. It has been curious speaking to people and some people will say that it is the permanent secretary, some people will identify a particular director general, some people will identify a particular cabinet secretary, other people will say that there are Scottish leaders for them and some people will just say that they have no idea. There seems to be a lack of really clear ownership and leadership, which was there previously. For me, that is one of the key fundamental lessons that needs to be taken away from this process of review, is that there needs to be sustained leadership in order to really make this effective. We need the sustained leadership and the focus on implementation. We need to close off that implementation gap, but we need the sustained leadership in order to have that identifiable accountability and in order for people to see that this is something that is going to last, it is going to be around a long time, it is something that people need to take into account and they need to recognise it explicitly in their plans and so on. That has diminished over time. That is certainly what I found through the research that I have done. When it was there previously, where was that leadership coming from? Was it the permanent secretary or was it director general's? Again, going way back, I apologise for the slight history lesson here, but going way back to 2008, it was ultimately the permanent secretary who developed the performance framework. There were ministers who were particularly keen champions of it at the time, John Swinney, someone who has been mentioned numerous times in terms of providing leadership in the political side for development of this, but certainly it was the permanent secretary as well, John Elvidge, at the time. Going through to 2015 in the community empowerment act, it is fair to say that quite a few people who I have interviewed have spoken about Peter Housden's approach to the performance framework, and he certainly had more of a delivery focus in terms of how he provided leadership to that. Derek Mackay, who was the finance secretary at the time, had a key role to play in that and the Scottish leaders for him as well, collectively. Over time, arguably, it is events that have taken focus away but there has been a sense of a lack of focus, particularly within the Scottish Government. That may be, as it has shifted from being a Government approach to being much more of a whole of society approach, but who owns it if it is a whole of society approach? I think that is one of the unanswered questions. Where does the ownership lie if it is suddenly everybody's responsibility and everybody's accountable for it? That may be something that a commissioner could help to unravel or other bodies such as Audit Scotland, for example, but there does need to be central focus in order to provide that place where people can see that it is being led from a particular area. I think that that is important in terms of putting it in people's minds. I think that Jim McKay can add to that. If it is okay to add another piece of information, I again apologise for the history here. Ian and I have been involved in this for some time. There was, for a number of years, a cabinet secretary round table on the national performance framework, which was cross-party and was convened in this building with members of all the parties, and it had oversight. That was actually the body, the parliamentary piece that had oversight of the development of the first round of national outcomes and the indicators, and it met with a number of external stakeholders representing different interests. It has not met since 2018. It had no statutory reason for meeting. It had no real place within the structures of Parliament. It was something that was done because, at that time, Mr Swinney and then Mr McKay were committed to it and wanted it to meet. In the absence of that, there has not been a space where the parliamentarians have been able to explore that in the way that they were able to under that round table. At that point, there is a very clear change in 2018, where you can see that lack of impetus in external engagement happening. Thanks. Sticking with the question of leadership and ownership, Max, you made an interesting comparison in your submission between the Scottish model of appointing champions for various outcomes within civil services opposed to Northern Ireland, where there were more defined owners. You also noted that there are quite significant levels of internal opposition in Northern Ireland within the public sector. Are those two things related? Which two things, sorry? The model of appointing owners, rather than champions in Northern Ireland, so you put that very direct accountability. Has that contributed to that internal opposition or are they unrelated issues? Yes. This comes back to the accountability point. There was an expectation in Northern Ireland's civil service that permanent secretaries and senior civil servants were going to be held to account at some stage through likely a parliamentary process to account for movement in national indicators. There was an expectation that there was some sort of accountability on the horizon, so that surfaced fears of being held to account for things that you cannot control, the perennial problem with outcomes-based accountability. There was some sort of relational collaboration amongst the senior civil servants around this approach, but, broadly, it was a top-down imposition. At that time, as there is now, there was either uncertainty or discomfort around these uncertain accountability relations, or there was a dislike of the approach being forced to take account of different measures. There was buy-in in certain sections of civil service leadership and not in others. I had a very conducive head of the civil service at that point, and that subsequently changed. Really, that did not broaden into a collective endeavour. There were struggles to make the technical elements of that work, so the decision was made to assign national indicators that were the closest to the domains that civil servants were working in. It is only fair if you are going to be held to account that it is in your area. Consequently, if anything entrenched the silo mentality, here is my indicator and I do not take account of other ones. I would say that linear, attributional approach worked against that endeavour to work across departments in Northern Ireland, which is difficult for a whole host of cultural and statutory reasons. That was where some of the opposition emerged from. When the executive collapsed in 2017, many elements of that outcomes-based approach fell by the wayside, and there was the opportunity to disengage and then to establish a resistance to it being reintroduced. We do not want to go back to that sort of system. Then there was leadership change over the head of civil service and such forth. It lost its champions. Now that is a difficult challenge for the Northern Irish Government, the executive office and the civil service to be working with Thank you very much. I have just got one final brief question. You mentioned again in your written submission that parliamentary scrutiny in Scotland using the outcomes was an example of good practice. I will be honest, most, if not all of us, did not think that we were doing a good job of this. Could you tell us what we were doing right? To be honest, we were not aware of that. Well comparatively speaking, national outcomes and indicators are used more in the Scottish Parliament than they are in the Welsh Senate or in the Northern Irish Assembly. Statistically speaking, mentions of the national outcomes, for instance, are used more often. There are a few caveats to that. One significant thing is that, in Scotland, one of the real benefits here is that we have an NPF that has a degree of solidity and recognition that the national outcomes and indicators in Wales or Northern Ireland do not have. In Wales they talk about their act, they do not talk about their wellbeing goals or their wellbeing indicators or their milestones. The NPF has a kind of solidity, albeit one that is rightly challenged on a number of accounts, but it has that that maybe gets in the minds of parliamentarians but also clerks, research staff in the Parliament. When members reconvened for this parliamentary session, my understanding is that you were briefed on what the national outcomes are, here are the indicators, and here you might be able to use them. They have been used in different parliamentary committees, which is my understanding, than they were in the prior session, so there is maybe some movement, some indicator of greater usage. However, I would say also that you might be feeling how you use the framework as a parliamentarian, just as a community planning manager, as a department head in the Government department, and how you use it can feel very difficult to access. Thanks, Ross. John, to be followed by Liz. I hope that, following on a bit from some of that, the question of language and how important language is. Various people, when we spoke to some of the Government officials, they talked about language as intangible for outsiders, and I think that it was the system's advice that talked about the language differing, but they felt that they were broadly lined up, they said that their own language is different, differs from the language of national outcomes that two are broadly aligned. I am looking at one of your points, Dr French, which is that we should rebrand the national performance framework as Scotland's national wellbeing framework. I am just wondering how important it is that we have the wording right, and should we be changing some of the wording? Two reasons for that. One is technical. You have already got an NPF, and it might be given more cognisance in the Scottish Government than our current one, you have a national planning framework. There is a technical point, but also the larger or more significant point is that the branding of this is, I would say, a crucial part to getting it right. A national performance framework does not excite people as much as a national wellbeing framework. A performance framework is something that an organisation uses to regulate its processes. I would say that a national wellbeing framework would link with the galvanising interests nationally and internationally around collective wellbeing as a kind of organising principle for society. I think that that is something that has attracted cross-party support and links with a developing international agenda, which has a real relevance for people locally and for working in public bodies. I think that Jen can maybe give a bit more fleshing out on that. I think that the point that I would make is that the branding and positioning is crucial to the life or death of a performance framework. It has to excite people, a wellbeing framework excites people more than a performance framework. I feel that we do not talk about the national performance framework very much, and although MSPs have been briefed about it, I do not see it coming up in the chamber or on committees specifically being mentioned. Is that the experience in Wales and elsewhere that people will use them more if the words are better? In Wales, wellbeing for future generations has established a real cultural significance. Policymaking takes stock and takes note of that as a matter of course, and that has galvanised a vision that is long-term, that is societal, that is cross-party and that has really influenced the discourse and has exerted some of that cultural impact through the language that it has used. It could not have got to that point if it had used different words or if it stuck with a performance focus. It really articulates in that collective aspiration that people can get involved with, that they want to engage with, and eventually that they cannot not engage with should be one of the aims of an implementation strategy for the NPF, or the NWF. This is something that we are really interested in at Carnegie UK. How do you take those concepts of social progress and translate them into things that people can understand and act around and convene around? Generally, we find that people can convene and bring a huge wealth of lived experience and professional experience to a conversation around wellbeing, so they are able to engage with that in a way that other more technical language are less likely to be able to engage with. As Maxis said, we do not really believe that the national performance framework is a performance framework in the classic approach of that. It is not a linear process to identify an A to B of how you improve your performance. It is antiquated. It is historic about where it came from. It could be due to a refresh, for example. We have been arguing that for a number of years. The Scottish Government seems very committed to using the word performance. We are not entirely sure why that is. We have said that you could call it the national progress framework. That would work. You could still call it an NPF, despite the problems with the national planning framework. Again, that is not something that they have been willing to take forward. However, we believe that it is a significant barrier to the wider societal use of the framework, because people outside of the Scottish Government do not see themselves as having a role to play within a performance framework. They do not feel that they have evidence to give to the development of a performance framework. Whereas you ask them whether they have evidence about how we can all live well together and how we can all work together for a better Scotland, they have plenty of evidence to bring into that conversation. It is part of that opening up to try to brand it in a way that really accentuates the value that we all have to bring to it. In terms of the extent to which you can engage to the public in statistics, there is a difficulty with the number of statistics and the way that that is presented on the website. Again, we have experimented with summary statistics of what we call growth domestic wellbeing. That is for English data rather than Scottish data, as it was an exploratory programme. However, the message is that when you process the information in a way that is geared to communicate with the public, it will engage with it. You need to separate out which bit of this is about communicating with the public about how we are doing as a nation and which bit of it is about communicating with managers in terms of how they make decisions, or indeed with politicians, about how they make decisions about public finances and how to help people to account. At the moment, the framework looks and feels too much in one direction rather than the other. I am interested in the use of the word performance. You are quite critical of it, and others seem to want to keep it. Is it because performance suggests that we can measure things such as the performance of a car or 99, 100 or 98, whatever the figure happens to be? Performance suggests that we can measure it and we can hold government to account or hold someone to account, whereas wellbeing is a vaguer word? First of all, the science of wellbeing has grown massively over the past 10 years, so in terms of how we can identify actions that would improve wellbeing, there is a significant body of research and evidence on that now. In terms of the outcome element of it, a performance framework gives the impression of there being a linear relationship that you can attribute the change to the action that was carried out. You can do that in some public services, in some interventions. When the outcome is, for example, improving children and young people's wellbeing or improving their lives, that is not an A to B thing that you can create an attribution chain. What you can do is say that we are making a contribution to that outcome. By delivering high-quality education, we know that there is enough evidence that says that high-quality education improves children's feelings of wellbeing and their external objective indicators of wellbeing, so we can make that chain. We are not saying that an education department is specifically responsible to create the entire benefit of wellbeing through that one line of accountability and through that direct attribution. That is where it differs from a traditional performance approach, where you would create a much smaller link between your activity and your indicator, and you would be able to say that, yes, we moved from A to B on this, but the chain is much, much broader. The Parliament tends to question how many people have got hires, how many people have got degrees, how many are at college, those very fixed things, rather than how is the wellbeing going. The point of intervention would be what do those indicators map on to what we know gives people a good life, or is something changing within that? Is there a change in the evidence that says that having a degree is no longer a guarantee to having a good income, and are we therefore using those indicators, so those are output indicators, as proxy measures of a good life when they no longer hold true? That is where you want to come in and put the scrutiny role in saying that does this indicator act as an outcome indicator, is it a proxy indicator, and is it a relevant indicator to the current times that we are living in? Do you want to come in on any of that? The performance word is a bit of a problem. It can be perceived as being a punitive thing, as soon as you say that it is about performance, and it can also be very much perceived as top down as well. An individual charity or an individual local authority is being held to account by the Scottish Government. Again, that is not really how it was originally intended. The idea of a wellbeing framework, I think, has a lot of merit to that. I suppose that perhaps one of the challenges to that is around the value for money duty, and how we demonstrate value for money if there is not that clear link between the spending going to this particular initiative with this particular outcome. I can see why the Scottish Government is keen on the idea of performance. I can see where that comes from. I am just not sure that the national performance framework is the right tool to do some of that activity. I think that the idea of a wellbeing framework has a lot of merit in trying to develop a different mindset and a more collaborative approach. That ties in quite well with what I was just going to ask next, which was about the budget that Ms Wallace mentioned. There has been a comment, too, from Government officials that they do not see the national performance framework being used in the budget process. Are you saying that that is not necessarily a bad thing or should the national performance framework in the budget be a bit more closely tied together? What I would say about that is that, over time, there has been more ring-fincing of budgets, particularly to local government, and that in itself has proven to be problematic. I would caution against seeing the review as an opportunity to do more directive approaches to service delivery and to ring-fince budgets around that. That has not proven to be particularly successful in delivering on the outcomes. I think that there needs to be some caution around how it is used in that way. Would you then end up with a situation where, instead of the Government saying that we have got to be 1140 hours for childcare, some councils would say that we will use the national performance framework, so that we will have 1,000 hours or 1,200 hours or 1,900 hours, and that would be a very varied picture around the country? Would that be a bad thing or would that be okay? I suppose that we have local government so that local government is accountable to local communities, do not we? It is really up to communities. We are supposed to be embedded within the community empowerment act. Where are communities in all of this? How do communities decide what is important to them? Where do they feed into this process? Again, the more top-down you make it, the less you are empowering communities, the less you are doing that work. I would caution against any directive approach, and I do not think that that would be in line with either the Christie commission principles or the principles of the community empowerment act. In relation to budgeting for children's wellbeing, you mentioned about how the budget and the NPF tie together. The first thing that is worth noting on wellbeing budgets is that this is a very new area of international activity for Governments. There are very few Governments who are experimenting with this. Everything is being done in real time, learning from practice in other countries and learning from colleagues at the OECD who are applying their resources to understanding it. There is not a core set of principles for a wellbeing budget that you can apply, so what I am going to state needs to be within that context. That is difficult. The analysis that Dr Travec did for us on children's budgeting was again that, where the national outcomes come in, they are very much an afterthought. They are a process at the end of a budgetary process, rather than at the start of it. What she meant by that, and what we have explored with others since her work, is that there is not an initial assessment of how people in Scotland are doing against the national outcomes that then begins a process of where we should spend money. It goes the other way around. What are we spending money on? Where can we spend a little bit more money or a little bit less money? Exploring that in relation to the national outcomes or the national indicators. A full wellbeing approach, whether for a portion of the population, as we were arguing for children and young people, or for the whole population, would turn the whole process completely on its head. It would take a considerable number of years to do it. We speak to colleagues in New Zealand and they have done a partial wellbeing budget. They would describe it themselves as the first step, probably on a 10-year process of trying to turn their budget around from where they are now to being one that is fully based on an assessment of, as they call it, their living standards framework. However, they are doing that consciously. They know it is a 10-year process and they are improving every year. Whereas the approach in Scotland is much more to, well, we want to do something, and this is something that we are able to do this year. Therefore, they are applying the language and some of the processes around wellbeing on to pre-existing budgets without that long-term plan of how they are going to get from where we are now to a better quality outcome-based budget for the future. We could explore that longer, but I will leave that to convener. Apologies, Liz, to be followed by Michelle. I wonder if I can explore two themes, the first of which is based on some comments that were made to us, I cannot quote the names because it was a private session last week, but it did include very senior officials within local government and within some of the third sector. They all agreed that the national performance framework in principle is a good thing, but some of the adjectives used to describe its workability were ethereal, anodyne, top-heavy and theoretical. A couple of people added that they did not feel that it was necessary to have the national performance framework because if they were doing their job properly, they should be doing the things that have already been set out. I thought that that was quite an interesting comment about the reflections from people within local government who were on the front line of having to put into practice a lot of policies and we were particularly talking about improvements in addressing the children's poverty issue in Dundee. I wonder how you respond to that. It picks up a comment, Dr Elliott, that you made about the principles of the Christie commission and what is supposed to be happening here, is that if we are really going to get better outcomes and better wellbeing, is it not something that we should be doing through local initiatives because they are the ones who know best what is working well within their local community? Rather than having this big scaffolding, there is this dilemma between too much on a general national framework, but what we really want are things that are working very well locally. Could you give us your reflections on that? There was a lot in that, so I will try to formulate my thoughts. First of all, should they not be doing it anyway and aren't they doing it anyway? I think that the evidence suggests that, yes, in terms of their work to improve a particular outcome, but not their work to improve a range of outcomes. It comes back to the Christie conversations about how we create collaborative and joined-up public services. If we are where a number of commentators and other organisations believe we are, where we have got to diminishing returns on being able to work in detailed silos to improve people's lives, the next set of improvements will come from joining up, finding the connections between public health and education and active travel and so on to make all of our lives better. We need to be able to work across those silos a little bit the way that Dr Friend was describing was so problematic in Northern Ireland. The evidence is that they are not able to do that because there are structural barriers to them doing that. What we have done is create a performance framework that is trying to overcome those barriers and not really giving it the tools to allow it to do that heavy lifting, and that is what I mean by the scaffolding. What else does it need in order to help it to deliver that? We sometimes get this reference to the motherhood in apple pie response to a wellbeing framework. Is it not all blindingly obvious that that is what we are? Yes, it is. Actually, if you look across, as I have had the great pleasure of doing multiple Government frameworks, they are all remarkably similar. Scotland does not stand out particularly differently from that. It has a value statement that we believe is incredibly important about who we are right now as the people of Scotland, and that is important. That sets it apart from the others. When you look at the content, there is very little difference there. What it is trying to do is articulate a vision and a space for whatever country is using the wellbeing framework that says this is what we are trying to achieve, this is who we believe ourselves to be, and this is what we are going to organise around. It is very specifically focused on trying to get away from a model of trickle-down economics that says that if we just focus on the economy, everything else will get on better. Most of the literature comes from that background. It says that the economy is not the answer to all of our problems or certainly GDP growth is not the answer to all of our problems. We need other things to happen, and that is a particularly important method from the environmental stakeholders. I am sure that many of them will have submitted to this inquiry as well. On the issues around implementation, the problem is that the story is not really very well told about what we are asking people to do. They can gather around the general concept, but the guidance on what they are being asked to do differently does not exist. If we were able to give people more of that guidance, it would help to resolve those issues. To pick up on that point, can that be localised? They were very clear in a couple of the groups that I was speaking to that they wanted more local autonomy to decide on what was doing the best thing in terms of wellbeing for their area. The only caveat to localisation is the equalities point. In order to localise that, it has to be about the wellbeing of all the people in that area. If there is a small area and statistics are limited, it may be difficult to get the views of people who are particularly marginalised in society through statistics. You may want to add in to that model some deliberative democracy participatory engagement so that you are not relying solely on what can be at a local level quite small numbers, which we know exclude some of the groups in society who have the worst wellbeing outcomes. There is an issue there. The more local you get, the harder you have to work at really understanding what is going on. The principle of subsidiarity, which is what we were really talking about here, is one of the principles that, as an organisation at Canadian UK, we have identified as one of the key drivers of wellbeing. The more local decisions are taken, the more likely those decisions are to be taken in ways that improve wellbeing rather than in a one-size-fits-all across the nation. They were pointing out that it is better for the ownership because people in that local community feel that they have devised the policies that are actually working. That was successful in A, getting people to understand them better and B, getting them well motivated to deliver them. I thought that that was quite a strong point that you made. There are now international examples coming forward of the relationship between local wellbeing frameworks, localising of it and activities such as participatory budgeting. We did some of that activity in Northern Ireland over the past few years. I have seen this morning some work in New South Wales coming in again. We are trying to make that connection between subsidiarity, participatory budgeting, participatory democracy mechanisms and the outcomes approach. I will lead into my second question. Do you accept some of the comments that we got that ring fencing potentially can be a bit of an issue in this? These were largely local government people saying that it was almost too much ring fencing that they would actually like to have some autonomy to spend some money on areas of policy where they knew there had been good effects. That was actually not to do with economic statistics. That was much more to do with social wellbeing. Would you accept that? Yes. Our history of research at Carnegie UK has a significant body of research and evidence about asset-based community development in rural communities. The importance for small communities to be able to determine their own priorities and then feel engagement and ownership came through all of that work. It adds to the sustainability of those initiatives if people feel that there is local ownership of them. They go deeper. They have asked longer as a result. A lot of our work was about community ownership, so it was about community ownership of land and community ownership of energy. However, there is a sense that, at a very hyper-local level, those things can come together and create very local wellbeing approaches that are specific to the needs of those populations. I am going to touch on a couple of themes that have been peripherally discussed already. I am interested in the complexity that arises from accountability, Government accountability versus subsidarity. Possibly the unwise or uneducated might say that it must be this or that, because that means that they are not recognising the complexity. If I can give you an example, as we know, there are many areas of critical change that the Scottish Government has no say over fiscal levers, monetary levers and so on, and it is on the record that the Scottish Government will try to do something about child poverty in terms of child poverty payment, which is then clawed back via another route by the UK Government. That is well on the record. My concern about that is not just from a political perspective, but from an accountability perspective, because the Scottish Government is accountable for all those outcomes while not having the control and the power to deliver on them. That is on the record already. However, I would appreciate some of your thoughts about the complexity of that and how we can start to square it off. Given the examples that have read all the contributions that you have made, what you have seen of that happen elsewhere in Ireland and Wales, particularly in reference to the softened hard powers that you have put out, perhaps just fleshing out some of the complexity a bit more because it strikes me that it is too simple to say that it is this or that. Jennifer, maybe you would like to go first since you were looking at me nodding. I do not think that anything in complex public policy is ever this or that. It is always a mixture. The example that was given earlier was about childcare hours. If you interpret childcare as core infrastructure that allows a country to function, then it may not be something that is up for local negotiation, but that is a conversation that we probably need to have as a country. Where do the differences lie in that right now coming out of the pandemic and are they different from where we started when this Parliament was established, which they may well be? I think that there is an issue about local democracy and the local democratic deficit. The connection between local people and councils is not particularly strong, and we have layers of accountability over and above that, which sometimes appear to be accountable and sometimes are not accountable. There are and have been in my career in policy in Scotland constant calls for reviews of the relationship between Scottish Government, local government and that community or hyper local level. I am not sure that the national performance framework or the national outcomes can resolve all of that. I think that what is difficult is that it sits alongside an imperfect system, and it is either trying to resolve that problem in the soft power way that Dr Friend refers to, or people put so much weight on it that they think that it is that top-down approach. It is neither of those things. The Scottish Government itself is not accountable for the national performance framework, but it might feel to be that it is only accountable in the act to having regard to it. There is not the clear and linear line that some people think is there. No, it definitely does. A couple of comments from Dr Friend from Dr Elliott would be useful as well. I think that my thoughts on this are with regards to accountability and how that can be reconciled with the extra-organisational nature of outcomes, which is sort of intrinsic to an outcomes approach. It has been grappled with in two ways. Either we try and hold people accountable regardless of that, and what happens there is the Northern Ireland experience of here is your indicator and the expense of all the other outcomes and indicators, that sort of narrow tunnel vision, or it results in the gaming behaviours that you see for instance with the work programme, the Troubles Families programme, the flagship UK government policies of the Cameron Clegg era and the history of performance management, just littered with examples of these issues. My view, and I think that this is the sort of emerging academic consensus, is that you reframe your vision of accountability from attributional accountability. Here is the movement, and we assign this attribution to you, the government for the movement and the national outcomes, to contribution. You are held to account for your contribution to a broad range of national outcomes or indicators, and that is done in a contextually specific way. Again, I refer to Wales in this. What they have done is the way that wellbeing plans, assessments and objective setting, a localisation process is conducted is based on a community planning partnership, a public body, a local government, thinking about stretching attributional plan, which sets out not just, for instance, if you were a housing association, they are not held to account in Wales, but not just my repairs and my occupancy rates and such forth. How can I contribute to anti-social behaviour in my area? How can I contribute to child poverty? Are there areas that I can think beyond the boundaries of my organisation creatively and collaboratively to make a broader impact? The sort of stretching contribution plan that is the start of that process has to be an accountability procedure. That is conducted through the collaboration between the Future Generations Commissioner in Audit Wales, and then there needs to be a post hoc assessment of that, not just on what has gone according to plan, but taking account of context in the intervening factors and trying to assess a link. Has that creative stretching process been genuine in that context? Accountability is situational and it is based on contribution, not attribution and not top-down assignment of accountability, but there are examples of that now in practice, which show that that can be done. The other thing that I would add is that oversight function that the commissioner has provided is giving them all the information that they need to feel to make a valid assessment on who stretched themselves and who hasn't, and to meet out that balance of soft power versus hard power, who needs the coercion, who needs the encouragement and the support, and who needs to be potentially taken to that next level formal investigation and review. Before you come in, Dr Elly, you specifically have mentioned it about the role of the auditor general in terms of accountability, because that also is getting more complex, where we have policy decisions being bypassing the Scottish Parliament and going direct to local councils, but without yet a clear line of sight of scrutiny and accountability, just a promise maybe to look at at some point later. I wonder if you had any thoughts about the complexity around that, although I note with interest what you are saying about the attributive accountability. I think that that is an interesting theme. First of all, I would just agree with what Max had said. When you look at this idea of a wellbeing framework, there are a lot of similarities with, for example, the sustainable development goals and how they have been set up and the philosophy of underpinning those is that each nation-state demonstrates how it is contributing towards the sustainable development goals. They are not expected to be ticking every single box and being held to account in a way that a performance framework might even think, but it is more about how you are making that contribution. There is a similar philosophy potentially here for a wellbeing framework. There are complexities around all that and how different bodies contribute to the different goals or outcomes. That is where collaboration plays a key part. A number of Audit Scotland reports have highlighted the need for more joined-up thinking within Scotland. The need for more collaborative leadership has highlighted issues with health and social care integration, for example. Coming back to something that Lizard said, we definitely need that because an individual agency or an individual council might say, well, we are doing this anyway. That misses the point. It is not about what an individual organisation is doing, it is about what the collective system is doing and how people are working together across that. Sharing learning and sharing experience between Scottish Government and local government, between local government and different allios, between the different parts of the system, ultimately. There is something there around learning as well that needs to be taken into account here about how we encourage more learning and more shared experience across the system. Indeed, between the Scottish Government and the Welsh Government and the Northern Ireland Executive and the UK Government as well, there is a great opportunity here because that has been developed in slightly different ways across the devolved administrations. There is a real opportunity to learn from a lot of the good practice that has taken place. I am not sure that that is happening yet. Scotland is almost as a silo in itself. Can we learn more from what other administrations are doing? I think that the answer is an absolute yes. My last question is, which again we have been dancing around, references a concept of agency. I think that we have alluded to when you have a potentially top-down and bodies taking ownership in various different ways. If we think about it from a bottom-up perspective going back to what you were saying, Jennifer, about subsidiarity, rather than thinking just at the very bottom of the triangle of the person in the street, how would you go about if we were reflecting the national performance framework to ensure that agency is instilled in every single touch point of the framework, just thinking it from a completely different perspective, because we have not used the word in the session today, but it is kind of jumping out at me. As you can see, you are all thinking about that, which is probably... Who wants to go first? I think that there are really interesting developments in Scotland around, for example, a creating space to reflect and to learn together, and we are getting better at that, hopefully at sharing experiences. I think that there are ways of encouraging that by asking the right questions. Maybe this is one of the things where we have a role for an agency of some kind to help them. How would you go further with that? How would you improve your contribution there? How would you develop that further? Not in a scrutiny way, but in a learning together methodology, so a much less hierarchical, a much flatter approach to say that we accept, as we were discussing earlier, that we all adhere to the principles of wanting to improve lives in Scotland and we all believe that the outcomes-based approach is a way of doing that, then setting ourselves a set of reflective, complex questions and say, are we doing enough on this? I just come back to one final point, because we haven't really talked that much about the environment or net zero or climate change and all of this, but the solution to our contribution as a nation to the climate crisis is not going to be in SIPA or in the environment project, it is going to be all of us doing our piece. That will be a case in point of exactly that. How do you go from the personal responsibility through to community responsibility agency all the way through the system to allow us to feel that we are making our contribution, but also to have that contribution respected by society, so it goes in both directions? I think that it is a really interesting question and a really worthwhile way of looking at this, because the outcomes that the national performance framework sets forth cannot be achieved through top-down imposition, it cannot be achieved without agency, and the success to a large extent will be on whether it inspires that at various levels, so that is why a branding and marketing approach to awareness raising is so important. A similar question can be asked to the Scottish Government, why has it been seen as important to adopt the UN sustainable development goals and to seek to align that? There was nothing forcing Scotland to do that, but it took it upon itself and it used its agency because it saw it as an important thing to do. I think that that is an important element to carry forth with the resultant implementation strategy, the on-going approach to implementation, keeping that as an active approach. That would involve taking stock of the public mood and really focusing on our people responding to this as an opportunity rather than a threat. I would add that as a general point. From my perspective, agency links back to leadership. We need leadership in order to take on that agency and to do things and pick up on some of the things that Jen had said about learning together. That is another key aspect of my research, particularly on public administration. I think that there is a gap there that needs to be addressed in terms of the wider development of leadership within Scotland and specifically in relation to public administration. I know, for example, that the UK Government is developing a national leadership centre for civil servants. To what extent are the Scottish Government involved with that and to what extent is that going to be used as a tool to help to develop more learning across the devolved administrations and with the UK Government? Max and I come from Northumbria University, but there are many more local universities that arguably have a role to play in that as well. Arguably, again, there is a lack of capacity there. As I mentioned in my written submission, we only have one MPA programme in Scotland and one MPP. Where do we get that investment in leadership in order to make it happen? Where do we get the learning opportunities and how do we embed that, not just within the civil service but right across the public sector? That is another aspect of that, and I think that it needs some consideration. I will be brief because I know that time has gone away from us. It was just around the whole scrutiny and accountability again. I would imagine that one of the key recommendations coming from us will be whether we have a commissioner or the role on Audit Scotland. It is really just to get an idea in my head. What would they do most of the time? How many people are we talking about? How big a unit would it be? What else is about 15 people, I think? Is it gone up? Max has more recent information than 30. It is a reasonable-sized office and it has a research function. It carries out primary research on how the act is being implemented. It scrutinises plans, the wellbeing assessments, local wellbeing assessments and so on. It scrutinises those to assess the level at which they are meeting the ambitions within the act. They have a media role, which is one of the things that I mentioned briefly in my submission that Scotland has not done well at explaining to the media that this is a new way of doing business. However, the commissioner's office in Wales has quite a prominent media role to talk about the wellbeing of future generations and trying to encourage that way of thinking about social progress in the round through their public engagement and outreach activities. You would see that as the cluster of things. They very rarely use the hard powers. A lot of it is in soft power, but the ability to use the hard power to call in information to make public statements about the information that they receive is powerful. Even if they use it very rarely, it does have an effect. The most obvious one, being the motorway that was the decision on whether or not to build a motorway around Cardiff, was changed on the basis of the assessment from the First Minister who changed the decision ultimately, but it was on the basis of evidence from the research team within the Future Generations Commissioner who worked out how many years it would take for future generations to pay off the debt for that role. Putting that evidence into the debate changed the nature of the conversation about whether or not the cost-benefit analysis was worthwhile. That is the type of activity that they do. Scotland has a number of commissioner offices already. We have Audit Scotland. There are a number of models that you could look to develop from Scotland rather than starting out completely from scratch. We just spoke about decluttering earlier. I was just concerned that this would add more clutter to the landscape. Another function that the commissioner provides is that it provides the know-how. A lot of things that we have talked about are accessibility to the framework. It provides support guidance, workshops, close personal working. It supports the Welsh Senate in using the act and it provides training programmes. It provides an outreach function at a much larger scale than we have in Scotland. You could imagine in Scotland that we have done well is that we have a lot of people knocking at the door of the national performance framework and wanting to use it. Third sector organisations that are interested in it do not know how. Providing guidance, support and shortcuts from a position of expertise and how that is embedded would have enabled those organisations to adopt and implement the national performance framework much better than they currently have. It is normally this. After the fact, it is a signposting thing that we have talked about. It is scaffolding that process and providing a lot of the infrastructure there. The other thing to talk about is that they have been operating at their capacity for a number of years and they have noted that to their Parliament and to the Welsh Government. The scale of support demands that come into their office really outweighs the resourcing that they are provided with. They get over a million and a half pounds per year, which is the roundabout budget figure. It is slightly more than that. Still, the scale of demand coming into them is pushing them beyond capacity. More than half of that demand comes from Government. Government requests for support from civil servants and the other half comes from not just public bodies but also organisations that are not accountable within the act. They have a wider movement-building approach. That shows that you can start at the scale of resourcing that would be needed to really move the MPF from a Government performance framework to a societal wellbeing framework. The first one is to remind committee of the previous evidence that, to the best of our understanding, there are about five people currently involved in the delivery team within Scottish Government, so that is six times more people involved in it in the Welsh office. It is a significantly increased capacity. The other piece of historic information is, of course, that the sustainable development commissions were closed now well over a decade ago as a result of UK Government decision. The Welsh Office of the Future Generations Commissioner was established as a direct response to the closure of the sustainable development commissioner. Indeed, a lot of those commissioners were involved in the establishment of that office, so there was a clear line between the sustainable development commission closing and the setting up of the office for the wellbeing of future generations. There has been a longer time period in Scotland, but you can still say that, actually, of the organisational history that a new office would be coming from, it would be related to the work that the Sustainable Development Commission used to do prior to its abolition. The commissioner in Wales does not carry out an audit function, that is still carried out by Audit Wales. I am just trying to think what Audit Scotland would play in all of this. You have the sort of before and after the fact, the sort of pass-over process in Wales, with the commissioner taking charge at the start and then Audit Wales coming in after the fact to review process and review progress. They have both got long-term scrutiny roles on the sort of broader picture in Wales, but there is this sort of pass-over approach that is used there. In one last piece, you mentioned that road in Wales. Who would have carried out that function if there was not a wellbeing commissioner? Nobody. In the absence of a sustainable development commission, while there were a number of academics and NGOs campaigning to draw attention to the fact that this was not something that they considered to be in the best interests of Wales, there was nobody of the status within the system who was able to sort of get into the decision-making process, to call that to account until there was an office of future wellbeing and that relationship with that. As I say, it was the First Minister's decision, but he did it based on evidence provided by the office. Thank you very much Douglas, and I'd like to thank our witnesses. We are now at the end of our evidence session, but your detailed evidence was greatly appreciated by our committee, and we'll continue with our evidence-taking on the national performance framework next week. We'll now take a break until 12 minutes past 11. Our next item of business will be an evidence section with the Scottish Parliamentary Corporate Body. The next item on our agenda is to take evidence on the Scottish Parliament's website. It's part of a scrutiny of the Scottish Parliamentary Corporate Body's budget. We are joined today by Jackson Carlaw MSP, member of the Scottish Parliamentary Corporate Body, and Mr Carlaw is accompanied by Scottish Parliament officials Michelle Higgarty, Deputy Chief Executive, Alan Balhari, Group Head of Digital Services, and Susan Duffy, Group Head of Engagement and Communications. Good morning, and I welcome you all to the meeting and invite Mr Carlaw to make some opening remarks. Jackson. I know that there's been quite a detailed correspondence between us, so I don't think that there's any particular need to repeat all of that, and we're quite happy to move into questions, but I suppose I would just start by saying, I'm not a website designer, I'm not an IT specialist, I'm probably one of those dinosaurs who is the last person you would consult in any of these matters at all. I imagine, probably like many other lay people, you can go to PC World and buy a website for a sort of £5.99, and that that will probably suffice. But of course, in an organisation as complex, as huge, and as of such public interest accessed by so many people from, not just within the Parliament, but from without, and from around the world, it's important that we have a website which, after 10 years, bear in mind the previous website, was launched basically when the iPad was launched, and so much has changed in life since then, not least mobile communications and the ability of people to try and access all these things in quite different ways to the ways that they were accessed historically, which I'm afraid the previous site, which was largely obsolete and capable of being maintained, and certainly was not something which was able to be accessed easily required to be upgraded. This was very much a feature of the commission on parliamentary reform that Ken Macintosh established on which I was pleased to sit in terms of the public engagement meetings that we conducted, the inability of the public to access the website, something that perhaps familiar users here in the building felt they could do with these, but for those people trying to engage from elsewhere, it isn't, it wasn't. And so that was the justification and reasoning when the corporate body took the decision some years ago to commence the establishment of a new website. So that's how it came about, and I know you will have detailed questions on what came about and possibly the process that led to that as well, and we are very happy to take them. As it goes, Michelle, Alan and Susan will variously assist me or, in fact, prevent me from contributing as we go through. Okay, well thank you very much for that opening statement, which was very helpful. I have to say that there is great interest from members of the committee in this particular issue, and therefore I will try and restrict myself to not asking myriad questions on the issue, but I will obviously kick off with a few. We do obviously have a focus on a couple of areas. One, obviously, is the scrutiny function that this committee undertakes and how the website relates to that, but also the cost itself is, of course, a major issue for us. And I don't pretend either to be an expert on websites, but a simple googling on websites, as you said, you can get websites for only a few hundred pounds. We do realise this is complex, but then again major organisations understand that the kind of cost for introducing websites is considerably less than what the Scottish Parliament appears to have paid by a number of zeros, and also the timescale to develop it is obviously a matter of concern. We were obviously sent a list of the cost of other websites, for example, from various UK departments, and those vary from £14,000 up to millions of pounds, but we don't know whether we're comparing apples with apples with oranges because there's no great detail in there. The first question that I would kick off with is that the Parliament's project began in 2017, with costs and delivery spanning subsequent years until 2021. In each year, no specific costs are provided against the web project, and I'm keen to ask and find out why that is. The web project budget was put forward by the Digital Strategy Board. This is a response from the Presiding Officer. It's part of the overall project portfolio bid, which comprises part of the SPCB's annual budget bid from 2017-18. SPCB approves the Parliament's annual budget bid and its indicative bid for the financial year, as we know. We were also advised that officials have recognised the need to provide increased detail on major multi-year project costs as part of the annual budgeting process to the SPCB and the Finance and Public Administration Committee. The obvious question there, as part of the question, is why were we not previously, and since when did that realisation dawn that we should actually get the detailed information, and when were SPCB members themselves aware of the on-going costs of the project? I've heard a number of views that they weren't necessarily OFAE with these questions. I mean, Michelle will come in, but can I just say that the corporate body was made aware of the overall project at the point that it was launched back in 2017 and the years over which it would run. The budget for the website was not separately identified, and each year it is accommodated within the overall IT and digital budget, so there was never any exceptional item. It's one of those things, along with other parts of the maintenance of the building that have to be planned for on an annualised basis over a period of time. The corporate body was aware of it in the sense that we understood the case for the new website. We were given an indicative idea of what the website development would likely be and the timescale, but given that lead times for different bits of it can be compromised any one year, it wasn't necessarily the case that any one part of it was scheduled within the budget and identified separately as the bit of the website development that would happen in that particular calendar year. You're right. I think it's a reasonable thing for a Parliament to point to the other government services websites, because we're not just a commercial business, we are a Parliament and with an obligation to both the internal users and to the external users to ensure that we have a website that is fit for purpose and has capacity to serve the Parliament not just for a year but for a decade ahead. Michelle? Excuse me if I don't cover all your questions, please repeat. In terms of building on what Jackson said about the corporate body's involvement, which is obviously a strategic level on the advice of its officials, I've proven the budget bit, which comes to scrutiny in front of the finance committee each financial year. Behind the scenes, the clerk chief executive has obviously got delegations set out under the Scotland Act for the day-to-day functions from in the Parliament in turn. There's delegations to the group heads like Alan and Susan for responsibility of their areas of the business, which includes all projects and programmes that are being undertaken. So, at the time when this project began, the budget bit was obviously created and developed during 2016, put forward to the finance committee in the evidence session of the December of that year. It went through a process where the digital strategy board, which has an actual strategy for the organisation and how it wants to take forward digital investment, prioritises the range of projects that it feels needs to be undertaken to address that strategy. That, then, is put forward to the strategic resources board, which is myself and the clerk chief executive with the input of our chief financial officer. We look at the prioritisation of the entire Parliament budget, but within that we look at prioritisation within the project budget, which is usually about £4.5 laterally £5 million, as part of the overall Parliament budget, which is £112 million excluding capital costs for this year. In terms of that scrutiny, we then determine the prioritisation between the different portfolios, because digital is only one of the areas that we have projects spent in that can range across a number of other areas. The other one that is most significant is facilities management, which is a lot of the infrastructure and services to do with running the building. We make priority decisions across all of that, and we look at the detail of how the prioritisation has been undertaken by the different business areas. That goes forward to our leadership group, where we again scrutinise the budget bed before we advise it to corporate body. The corporate body has a few different cuts usually at looking at the budget before it is signed off, and then it is put forward to scrutiny by the finance in front of the finance committee. That is the process that would have been undertaken under the various delegations that come from the Clark chief executive down to the group heads and through the governance that we have as officials before the corporate body is advised by officials. From here on in, you are looking to give us the breakdowns that have not been provided previously. Would that be right? In the budget at the moment, when we bring our budget bed forward to you again in December, we set out all of the various components of the Parliament's budget across members' expenses, office holders, etc. We also have a schedule 3, which gives more of the detail on what we expect to deliver under projects. What we do in there is give a high level overview of the key elements of the budget, but not every project. Obviously, there are numerous projects undertaken in the organisation, so we pull out the most significant areas of spend. The thing that we have been doing over the past four years, which we are keen to now look at how we can build in for both the corporate body and for the finance committee, is that where we see there is going to be a multi-year spend on a project or a programme of work, because we have started to develop a project pipeline that is looking over five to six years to try to even outspend on project spend. When we say that, I think what we are going to try and do for both corporate body and for the finance committee is not just to say that we can meet this within our indicative budgets in the annual cycle, but to say up front that this is likely to be a two or three-year spend, a high-level estimate is x, y or z. We will seek to do that going forward. The corporate body obviously does review these things, but in the sense that they are approved projects within a schedule, it is monitoring and understanding of how the work is progressing. Where the corporate body might be more exercised is where a project comes before it that comes out of left field, if you like, for which there has been no provision whatsoever. For example, the recommendations for Police Scotland in relation to security in the building, none of which had been anticipated in a five or a ten-year plan and which are required to be progressed with a degree of urgency. That would be a project where the total costs and the recommendations even of Police Scotland would be considered by the corporate body and may or may not be accepted as we decided what we were going to authorise and then the exceptional item of expense that we would have to approve. Let's move on to the tender. We were advised that specialist technical staff are procured through existing framework contracts and the technology behind the site was also procured through an existing contract in place at the time, but why not go out to tender? Can I invite Alan to answer that on behalf of the group? We have current contracts in place that we wanted to use. They obviously let us move forward at a faster pace to get things going. Excuse me, at a faster pace, it took three or four years to develop this website? It would have taken an extra year if we had added on a procurement exercise on top of that as well when we didn't really need to. The contracts were already in place and we could actually utilise those. Before we would even start just to get through a procurement phase for a high-value project, it would typically take between nine months to 12 months to use following our standard procurement policies and procedures that we have in place in the Parliament. The frameworks that are in place were let through either the Government frameworks that are in place or many competitions in those, or there is a framework there that includes multiple suppliers. Each time, for instance, we were going out using the interim services contract. We were approaching, there are seven companies on that, and we are looking for them all to submit the responses to give us the best value for money from that contract when we are looking for a contractor on that position. You are saying in the real world outside this building that normal procurement and for a website, because when I look on websites I say that I can produce one in a week or a month or something like that. I do not really see any that says, well, this will take three or four years. There seems to be a lack of reality compared to the rest of the world out here. Jackson talked about a website that makes it easier for users that are less familiar with the parliamentary process, which is a public to find and understand. I have to say that, as an MSPU, I find that the current website is actually more difficult to actually manage than the one we had before. The very fact that you have to go on to the old website to look at the official report to me is a complete nonsense. What do people in this Parliament actually want a website and the members of public for to look at the official report, committee reports, questions, motions, parliamentary bills, to tell us that this is an incredibly complicated and sophisticated technological solution to some indecipherably complex problem when people just want to know who submitted a motion on what or who has asked a question on this or what bills are coming up. I mean, are we really expected to believe that we got value for money in a £3 million project which took years to come in and, frankly, is like a camel in a horse designed by a committee? Just to me it is a mess and I know other colleagues feel the same way. I mean, is this really the best that we can do for the huge amount of investment that is went in? That is a suitably pejoratively raised question, convener. I suppose that, as someone who does not access websites and things, I might find these things complicated as well. It certainly is the case that if you were an internal building user, you understood how the previous website worked and you were familiar with your way around it. It was a hugely rigubrious website. It had hundreds of thousands of documents within it, which were slowing down its ability to operate efficiently, and that is very unlike, for example, the website that you would expect to come across for any Government department, elsewhere, or any other Parliament. We were very, very dense in terms of content, and if you were not one of the internal building users familiar with the site, you would really not actually have had a very good experience. Have we got the final product and is it not capable of evolving further? I am sure that that is not the case. I know that Susan is very much involved in engagement in relation to how the ongoing development of the website will progress, and it might be worth Susan saying a few words at this point. Yeah, I mean, just to say that the model that the website was developed on was one whereby we wanted to develop our in-house capability so that it was not a case that we would just build a new website and that was it. It was then incapable of being improved, which were lessons that we had learned from the previous website back in 2010. The idea is that, yes, the website was handed over to us in-house in a state where we could basically take it forward, but what we have done through the course of the project is to develop our in-house expertise both within teams, in my group and in Alan's team, so that we can constantly develop and improve the website because no website is ever finished. We very much appreciate that there are certainly mixed views in terms of the website. Just one thing I did want to say, just following up on Jackson's point about people from outside not being able to find their way around the website as much as they perhaps could do, and from an engagement perspective. Previous, just as an example, under the old website, if you wanted to engage with a consultation that a committee was running, really you needed to know which committee was looking at which issue because you needed to go into the website through that particular committee, whereas under the new website you go in, there's a list of consultations and it's done by a list of issues that are being looked at rather than by committees. Now, I know that that's different to the way in which it was done before. I used to head up the committee office, so I had a really good idea of which committee was doing which piece of scrutiny. I don't have as much of a knowledge of that now and I do find that particularly helpful. I would suggest that that's a relatively minor point, given that there are other such functions, etc., which people would probably use most frequently, but I only want to touch on one other area because all my colleagues have keen to come in. One of the things that we looked at was areas of spend that I found quite interesting. For example, in 1920, the salaries for people working on the project were £928,000 that year. The actual money spent on software was £4,000. The following year, the salaries were £940,000, with only £2,000 on software. What kind of salaries are people getting paid that are actually working on this? How many people were working on this, Alan? The costs that are really attributed to those salaries that you are talking about are day rates, so we employ contractors with specialist skills on day rates. Throughout the project, they vary between £300 per day up to £840 per day, depending on the skillset and the market value at that time. Even £840 a day or £300 per day still took years to produce the website. I notice that, apart from the challenge of managing co-ordinate and a large team virtually, members of the team dealing with issues also had childcare in homeschooling. We are spending all this money and we have also got to accept the folk who are earning these huge day rates. We have got to juggle our homeschooling and childcare. Is that not taking into consideration when things are taken forward? I will clarify that point. Alan is referring to the specialist day rate card, which was obviously part of a contract that had been let at scale in the public sector. The reason that we go into those framework contracts is because they offer better value for the public sector. The website was delivered largely to the timeframe set out. I think that what you are referring to about homeschooling was because we hadn't envisaged the pandemic hitting towards the end of the project. There were naturally a number of challenges in delivering the project. We have set out for your benefit the fact that we had to introduce two new services to the Parliament. One was the remote voting, another was the hybrid parliamentary business platform, which were not projects that were envisaged, but we had to address those in order to keep Parliament running. Some of the specialist staff, as I understand it, were deployed on to those contracts. We also had staff having, as your own staff, would have had to go home and then deal with the fact that the schools were shut, etc. For a period of the pandemic there was illness, people had Covid. We were balancing, as you would have been with your own staff, a range of factors that naturally impacted on capacity levels in the Parliament, coupled with the fact that we had in the last two years of the session a demonstrable uptick in parliamentary business, which meant that also for staff who were not specialist staff in terms of delivering the website, but who were going to have to be trained up to support the website whenever it went into business as usual, clearly there were capacity challenges there because of the support required for parliamentary business, but all of those things were just challenges that we would expect, but maybe not the Covid one, we wouldn't have expected that, but those are challenges that we have to try and navigate as part of delivering Parliament running business and running projects as part of business. Okay, so we people earning up to well more than £200,000 a year it seems on a day or eight basis for this, but how many people are actually working on it? I mean, if you actually look at 850 a day, it comes up to 1100 person days, if it's 300, it's 3000 person days, so imagine it's obvious that somewhere between that, so how many people were actually working on this project on an on-going basis when these specialists were needed? I don't really have that figure to hand, convener, I'm sorry. I mean, I think it was roughly about 10 to 12 contractors were employing at a moment of time, and we're also engaging our own staff as well. I think the difference in salaries you've indicated from the table that it's worth pointing out that the backfill is actually recruiting two parliamentary staff, so they were under our terms and conditions, so the contractors that we brought in, the high-day rate contractors, when they were off for childcare issues or whatever they were off for, they weren't actually receiving any payment for those days that they did not attend or could not attend, so they're off their own holiday or through childcare demands through the pandemic, they were not actually paid during that time by us. Okay, thanks for that. I'm going to open out a session to colleagues around the table, so the first person to ask questions will be Daniel, to be followed by Michelle. There are two, I think, primary issues with this, both, I think, one in terms of the transparency around the decision making up front, but secondly, I think, around the level of information provided about the project on an on-going basis. So, Mr Carlaw, neither you or I are website designers, that's a fair comment, but we are both business people and in business it's a good idea, and indeed if you're a large business, a requirement that if you have undertakings, which have multi-year obligations, that you specify those adequately in your profit and loss account. Now, if I look at the 2017-18 budget submission from the SPCB, the only indication that the Parliament was undertaking a £3 million website contract is a little line in schedule 3, which is the description against IT digital services projects, and it reads as follows, information technology digital change projects include rolling out the MSP case management system to more MSPs and staff, delivering a new Parliament website and internet, software, Windows 10 and Office 365, and hardware upgrades for SPS staff, replacing the Parliament telephone system. So, what was in there that would have alerted our predecessor committee that the Parliament was undertaking a significant website because that's pretty hidden, isn't it? Well, in the year you referred to, I gave evidence to the committee and your colleagues asked me questions on the subject of the development of the new website, which are included in the official report. So, MSPs were aware on the committee of the development of the website and interrogated me on the development of it. But there was no indication there, really, that there was a £3 million undertaking that was being carried out. Do you think that that was sufficiently transparent in terms of the significance and scale of the website project? In terms of the development and delivery of multi-year projects of core services that are provided by the Parliament, there will be a number of different examples that you could point to. The maintenance of the lifts probably being one that you might take a look at and ask how much do I know about how much it's costing to replace the lifts in the building or to maintain them. Those are all incorporated within the particular line item elements of the budget. I think that the corporate body would be more concerned. Were we to find that there was a significant and we would be alerted to if there was a significant problem evolving in the development of a project that was now spiralling out of control or cost in some way. In each meeting of the corporate body throughout the year, the different departments of the Parliament scheduled fairly detailed and extensive reports, which we consider many of those, I think, are subsequently available as minutes to the public. In terms of the overall portfolio within a £100 million budget—I don't know what it was at the time—maybe £80 million budget at that time, I think that the level of detail that the committee was seeking to have or to ask us about was accommodated. As I say, I was asked questions in that year and in subsequent years about its on-going development. I appreciate that acknowledgement. I would suggest that £3 million and £100 million budget as a percentage would warrant further detail. That is not £3 million in one year, but in the 100 million years annual. Indeed, but just as an expression in terms of, as compared to the overall budget, it is a significant project. Indeed, to understand whether it is going out of control, certainly you need that level of detail. The problem with transparency goes a little further. If I look at the detail that has been provided by the SPCB about where the costs have gone, all we have been provided with is a schedule of resource costs set out by technical and non-technical contractors, but there is no specificity about what work they were doing. I would expect, in any IT project, to see phases split out so that you understand where those efforts are being applied, whether that is the initial analysis, design, build, testing and user acceptance phases, for example. Those are very basic things, and yet we have none of that level of detail. Can you explain why that has not been provided? Yes, I can come in. What we provide to the finance committee is an overview of the entire Parliament budget, which is £112 million this year. Within that, we separate out members' expenses of the holders. We provide you with an overview of the Parliament budget within the bit that is the project spend, then we group and give you some idea of the key elements of spend within that, but not the detail of the schedule of how it is going to be delivered or the methodology, which is something that officials are charged and delegate it with doing in terms of running the Parliament from the corporate body. What I do not understand here is that you have described in table 2 in terms of your submission to the committee that there are areas of spend, but that is not what they are. It is a time profile broken down by resource type. In order to get a good handle on any IT project, you need to understand that effort by phase. Do you not? That detail is available to Alan and to the digital strategy board, and then there is reporting against the milestones in that, which comes into the leadership group on a quarterly basis, and then that is reflected up to the corporate body as well. There is reporting against the key strategic milestones of the actual delivery plan for the project. That goes on with the officials. It is reported to the corporate body. The digital strategy board takes regular reporting on the project as it was being developed. Beneath the digital strategy board, there is a project board and a project team that we are doing regular reporting as well. It is officials' jobs to get on and run the project. What we are providing to the finance committee every year is key areas of spend that we expect to see within those grouped programmes. Usually, IT, as I said, can also be facilities management. In other years, it will be broadcasting. In other years, it will be security, depending on what projects we as officials have advised the corporate body need to be undertaken to run Parliament, address risk, replace ageing infrastructure. One of my key concerns, looking at the detail that has been presented, and indeed some of the answers that we have had today, is just the level of granularity that this project has been monitored and managed at. Mr Carlaw said that it is not possible to specify which bit of spend would occur in which year. However, I would suggest that that is precisely what you should be able to do in any well-run IT project. Any well-run IT project should, as I said before, understand what levels of effort required by phase have a project management office that would track those things using things like gant charts and projecting spend. However, I would be grateful if that detail could be provided. In particular, I am very concerned by the category of non-technical contractors. That is almost a third of the cost in total. That is a very non-specific category of work. Indeed, can you just explain why you were using third-party contractors to deliver this? This seems to be a very complicated way of managing a project like this, managing individual contractors on an individual basis, because that seems to be what has happened here. Can I just comment, before I move on to Alan, on the governance point, on the things that you refer to in gant charts, where a project might be following the typical waterfall approach. However, there are other methodologies, including Agile, which is now a commonplace methodology in the digital world, and Alan might pick up on that. However, yes, estimated costs, phases, milestones, et cetera—all of those things—were in place as part of project governance. That is part of the delegation to the group heads from the clerk-chief executive that they need to manage their budget. They have to have proper administration of their budget, and they provide assurance to him that they are doing that. In addition, there is performance reporting at various strands of project team, project board, digital strategy board, all the way up to leadership group and the advice to the quarterly performance to the corporate body. To assure you that we follow standard governance whenever it comes to projects and programmes, but that granularity sits with officials. If I could hand over to Alan on the contractor spend. I think that, as I have stated earlier on, our strategic talent in this project was to make sure that we built the capability in-house. The reason we went out through the contractor route was in getting in specialist expertise, including the non-technical side, which includes things such as product owner and user research. There are not true technical skills such as developers and technical architects, but they are part of that project environment for producing your website. We were learning from them as we went. When we eventually switched over, we had enough knowledge then to run the project ourselves. The elements that you are talking about were all available at the project level. There were sprints that were used rather than the classic waterfall approach that Michelle mentioned. When we were using those sprints, there were four weekly sprints with clear plans in there about what was going to be delivered. There was a backlog created about what was not being in there, there was risk management and all of that sitting in there. Those were elevated up to the project board that were required from the project team. From the project board, they were relevant to DSP or to our digital strategy board or to our leadership group. They were reported on quarterly to the corporate body through our quarterly reporting mechanism sitting in there as well, but at a very high level by the time it got up to that level around the individual elements of what was being delivered day to day. Can you just finally clarify—I am still not clear on the basis of that answer of why you took the decision to manage to deliver this using third-party contractors rather than a single contract. Surely that is inherently more complicated to manage because you were saying, acknowledging yourself, that there wasn't that expertise in-house to build it. Was the expertise in-house to manage the variety and number of contractors that you have clearly been employing? We could replace our own project manager. We did have project management capability at that time and our own SRO, the senior responsible owner for the project as well, with experience in digital projects. We put those in place to govern the project where we needed to bring in skills that we did not have around the development. We did so through the contractor route and passed those knowledge on that was one of the key parts of the whole project was to make sure that we were developing our internal people so that at the end of this we have the capability to continue to develop this. That capability has been used elsewhere in the organisation. There were a couple of other websites built by the running house team around business and Parliament around the professional politics. Also, the same skillset that we developed through that allowed us to develop the voting application in a very short period of time using the same tools and techniques that we used throughout the project. We passed on a lot of knowledge within that. I could ask further questions. I do not want to use up time, but I would be grateful if somebody could write back to clarify the number of different boards and management groups that have been mentioned. I think that there is documentation about how they interrelate and how the governance works. Secondly, if the agile methodology was used, I would be grateful if somebody could explain why. It strikes me that this is a project with a very clear functional footprint and I would wonder whether agile is really appropriate for it to deliver a website project like this at all. One general observation and perhaps a constructive point. I think that, as Michelle was identifying, in terms of the high-level information that the finance committee receives, we are looking here at one particular project and I think that there have been questions as to why there was not more information in relation to that project. If we were to drill down all of the projects that the corporate body is overseeing and progressing and to provide the finance committee with that level of detail on them all, then I actually do not think that that would be helpful or productive in terms of the progress of the budget or the finance committee's understanding of where they wanted to focus their attention. If, on the other hand, there is a particular area within the development of the Parliament and the responsibility of the corporate body that ahead of the budget presentation you would like to have that more detailed information that you are perhaps seeking to have today in relation to the website, then that is something that the corporate body, through its presentation in advance to the finance committee, could supply. Thank you. We will certainly discuss that in private sessions. Subsequently, M Michelle, to be followed by Douglas. Thank you and good morning everybody. Just following on from a couple of points made earlier, given that Jackson, first of all, a question for you, I appreciate that the corporate body sits at an accountability level and there is a multitude of projects going on, as M Michelle set out, but given what we know now in terms of its fair to say that the costs for the development of the new parliamentary site has caused at least raise of my eyebrows and caused some interest, what recommendations will you be looking to put in place as to how the corporate body can fulfil its accountability considerations more carefully going forward? I am sure that you have reflected on that. I joined the corporate body just after the discussions had taken place on the website, but I found that I was handed the portfolio and had to go straight to the finance committee and then sell it, which was interesting. Those of you of longer memory were member after the death of Alex Johnston, which is what caused my accession to the corporate body at that time. There are obviously a whole series of projects that I have been involved with in the years that I have been on the corporate body, three years then and then again since the election last year, where to someone like me who has even been in business, I look and I think, crikey, this is a pretty eye-watering way of achieving things. I am reminded at times that we are a Parliament and, yes, I suppose my mother would take the view that she would not change the light bulb in her kitchen until it popped because she was going to get value for money for it, but if we took that view about the infrastructure of the building and just waited it for it to fail, for the website to fail, for the lifts to fail, for everything else to fail, the telephone system might be the next one. I do not think that people would thank us for it, so there clearly has to be a long-term projected look at wear and tear obsolescence and the need to upgrade and monitor the things that we need to plan for over a number of years. In this case, I know nothing about websites and, given the alacrity with which my SPCB colleagues agreed that I would represent them this morning, I take the view that there wasn't a great depth of expertise amongst them in any of this either, so naturally, therefore, within the corporate body, in terms of our on-going monitoring of it, we have an idea of what the original estimate was going to be, what the timescale was going to be and, yes, we do receive in relation to all the projects that are expensive, some of which we can relate to more easily than others, detailed information. Will we reflect as to whether there are lessons to be learned just from the way this has emerged and the kind of concern, perhaps, that's being expressed about it? I hope so, and we have already, I think, suggested that we will look to have a further discussion in the corporate body about whether there are any other protocols or natural points in the year where we should be having a further look at these things, just to ensure that we are offering the level of scrutiny that we are able within the portfolio of responsibility that we have. I think that you've now got on to the bit that I was specifically asking about, that where it may be, in effect, your neck is on the block and we've heard a lot of talk about, well, I've never run an IT project, various people have said, that to me indicates that that is a much higher risk in terms of scrutiny and the need for governance at the SPCB level. And if you equate it, it's almost like you're operating at board level. Any typical board would say, well, these are the projects that carry more risk and therefore we need greater scrutiny or capacity for scrutiny. So I suppose you're reflecting that you'll certainly look at it more carefully. I now feel a wee bit alarmed that the, for various reasons, that the SPCB isn't able to fulfil its scrutiny role of the workings of the Parliament, because to me, I'm not surprised at this, guess what, IT projects are always complex, they always go over budget, they always take longer, I know this because I've run IT projects. So I suppose picking up on what Daniel said, I personally would like to see out of this a report produced on behalf of the SPCB that they've signed off that said, from a governance perspective, what specifically will change in the light of this project and what we've learned from it? I'll ask Michelle to come in, but that, to some extent, does exist. We have an amber traffic light warning report on all the risks and developments within the Parliament and the progress that's being made against them. It's not that these things only come about if we ask for them, so we are proactively alerted to that. This website project came largely within budget and on time, subject to the caveat that the pandemic did complicate things, because resources had to be diverted away from its development to things like the hybrid working of the Parliament, for which we had no advance notice or provision or additional staffing that we could deploy other than that, which was currently working on the website. I think that I would take issue with the idea that the corporate body isn't being advised of all of these things, but Michelle? Yes, on your point about governance, I want to just reiterate that there was clear governance in place. The governance that we would normally follow in terms of projects and the various strata of that that I've outlined, I'm happy to provide a note with my colleagues on what that entails. Clear delegations, two group heads for their operational areas of accountability for budget performance and administration of that. Clear checks in our financial systems. Audit with an external audit board with Audit Scotland on it. We have a risk register, which the leadership group regularly review in terms of a risk. We have quarterly performance reporting against delivery of our delivery plan and our strategic plan, using the RAG status that Jackson has noted, so that we are constantly reprioritising tackling issues, addressing risk, seeking to mitigate risk, and also ensure proper financial and performance management of the organisation. Everything you've said I'm entirely familiar with, and I've no doubt it's happening, but it's clear that, given that we've got this session today, and what's come out of the concerns of the finance committee, there's clearly been a mismatch, and it's that mismatch that I am pulling out. There's clearly been a mismatch, and if it were me, I'd be looking carefully to avoid that mismatch happening again. I want to move on, though, because I know that every colleague wants to come in. Again, just picking up a comment that Daniel's made. I wouldn't necessarily agree about the methodology used for projects. We know that the world has moved on from waterfall projects, but, in the figures where we've got an agile methodology, which, of course, is sold by the IT consultants as much more flexible, what it often can mean is that you have increased costs where you're developing multiple prototypes and costs can mount in that area, and it's sold as a benefit that you don't do this huge, big analysis project upfront that then is out of date. I do notice looking at the figures, and I'm not entirely sure what's fitting in where, but my thing has just died, and I have to log in again, so I may be able to quote the figures in a minute. Between the alpha and beta phases, there clearly was quite a number of prototypes going on, which looked to me disproportionately high, and could even suggest that different personnel were coming in and going out. A whole point of agile is that you're developing a prototype, you're testing it, and it's iterative as compared to the old water flow approach, so perhaps Alan, you'd like to give us more context about that? Yes, certainly. I mean, what I would say about the approach very quickly, just to show people, and this is the approach that is used now by the Government Digital Services down in the UK Government and by the Scottish Government for the digital projects. I mean, waterfall is still used, but the terminology that they use is almost a bi-modal approach, depending on what the project is, so it's quite a familiar approach. In terms of the iteration, I think that the iterations that we delivered were lots of different additions to the initial prototype, so we were adding more services as we went. We did do user testing as part of those and refined some of those. In fact, some of the feedback that we received from the corporate body around things like the postcode search and these sort of things meant that we changed the order of presentation of things like that, so it wasn't just that we were polishing a particular set of functionality, we're continuing to add functionality, moving towards that goal of actually moving across to what was the beta site to be our actual website and then let the old site wither on the vine at that point. I'm not disagreeing with what you're saying, I'm just saying that based on my experience, this is an area in using methodologies where costs can incur, because you've got a multi-layering effect, and again, from a risk perspective, that's something I'm very much aware of as a former IT project manager many years ago, I have to concede. My last question, though, is in terms of the typical project continuum, you always have this trade-off between cost, time and quality. I'd like an honest reflection, perhaps from all of you, is in those areas, what did you trade in terms of time, cost or quality, and knowing what you know now, what you would trade, and if anyone says time, cost and quality was all of equally high standards, all the evidence tells it that that is never the case for IT projects. I'll start with you, Jackson, although I appreciate you'll need to bring your staff in. Time, cost or quality? I don't know that I came to those value judgments in the development or any more than I do over whether it's time, quality or cost on the lift refurbishment programme either. Clearly, I think the key thing I would say in all of this is that the corporate body is acutely aware that this is public money, that's what we start from in our examination, and that what we have to have is a product that matches the quality and expectation of the Parliament, but we defer and delegate to those people charged with that responsibility to take it forward. We don't interfere in an executive way in the operational decisions that you've just asked about, so I will invite— But knowing what you know now, you'll certainly be asking in the future about what is being traded in time, cost and quality, Michelle. The Finance Committee has its variable interests from one year to the next. It very often used to be the commissioners that I was interrogated at, excruciating length about, but the interest of the Finance Committee can shift in any given year given the priorities of the public understanding of the work of the Parliament. The question that you put has to go to those people who are charged with the delivery. You rightly recognise the difficult balance that we face with any project really around the time, cost and quality. We were very conscious of the fact that if we gave on too long in developing our staff and getting that up to speed, the cost would increase because of the model that we chose to use in terms of using the contractors. We always thought that it was going to take about three years, and we put a large push on despite Covid, to make sure that we were finishing roughly on time for that one. The time and the cost element was there. At that point, we had the capability then, built in house, to improve the quality. If anything, we gave a little bit on the quality. We are trying to serve multiple masters on the quality in terms of the different users of our website. There are quite a variety of users, members, staff and the public for our engagement needs and those sort of things. Also, political commentators and journalists, they all use our websites. I guess a little bit on the quality of anything that we gave on. There is always delicate balance. You are moving those things and judging those things around on a function-by-function basis. Perhaps my last little question off the back of that. I perhaps take a different view to some of the people in the committee in that I was surprised that this work hadn't been started in a period of 10 years when it was known that the operating system had become obsolete. I.T. and provisioning for I.T. is invariably expensive and only goes one way in any organisation. My question is that it seems to me to have been quite a long time to have started the project. I would appreciate your reflections, Alan. I suppose that that goes back to my comment about maybe everyone needs to understand that this is an on-going, risky, built-in cost to the Parliament all the time, because that is the nature of I.T. and digital. I would just like to understand what took so long. I think that we would have liked to have started it a little bit earlier, to be honest. I have been able to maintain it. When we put the previous website in place, it was done by a contractor who basically completed that task and then moved on. We were left looking to support it. We went through quite a lot of resourcing issues within the I.T. team. We still struggle to recruit and to retain people to actually get things going on the project. There were also changes at the senior management level in terms of ownership of the website and the services around that, with Susan Lattley taking on halfway through the project, or quite late on in the project as well. In an ideal world, we would have moved a little bit faster, but in terms of resourcing and finance available and those sort of things, this is about the time that the Parliament waited to do. We were getting by, I think, for quite a few years, and we were quite content with that at that point in time, but it just got to the point where we were worried about a catastrophic failure. We had been notified by the National Cyber Security Centre that, because it is no longer getting updated and improved, it was presenting a cyber risk. It identified that someone had had an attack on it, part of a Government attack against the UK Government, but it also probed our website and looked for spaces in there as well. I think that those were the final straw that really got us moving on that. I am not surprised to hear that. Susan, I would like to bring you in here at this point, because I think that this is my final question. There have been a few areas where we have talked about what is in effect, intracom, in terms of the various Government bodies and also to the committee. Is there anything that you would like to add to the record that you would perhaps consider doing differently? I think that the point that you make about the continuous improvement is a very good one, because you were asking earlier on about the balance, and Alan was also talking about when we have to make that judgment. We judged that in March 2021 that the technical platform was in place, and at that point we would be able to hand that over internally. We knew at that point that there were still aspects that we wanted to look at immediately to try to improve further. We will always have a rolling programme of continuous improvement as things evolve and as we get more feedback. At that time, we took the decision that we wanted to move across at that point, because we felt that we were in a position where we could take this forward. We wanted not to incur any further costs through keeping external contractors on for longer than we needed to. Also, because that was the end of the parliamentary session, we also wanted to have the new website in place for the new parliamentary session. It is not quite the question that I asked, but I will let it go to let other people come in. Thank you. Thank you. Douglas, we are followed by John. Thanks, convener. Questions are not around the functionality of the website, but it is back to the whole project initiation process and governance. Some questions are, was it an agreed project and who signed it off? Was the overall cost of the project budget approved and by who? Was the scope of the project agreed and by who? Was the timescales of the project agreed and by who? I think that the answer to that is largely yes and by the corporate body, but on the advice that we received. Yes, so the digital strategy board is the board that puts together its various projects, more than just this project or other projects, which then goes up to the strategic resources board. I chair that board currently. What we do there is look at the entire Parliament budget to advise the corporate body on what is going to be taken to run the Parliament, keep the lights on and improve the Parliament or address risk where we have things that are going to fall over or provide a cybersecurity risk. At that board, we have an additional challenge function based on all the challenges that have gone on before in the business areas to look at the nature and shape of the different pots that make up the Parliament budget but also within that the different pots that make up the project spend. When we are satisfied with that, we advise the corporate body at a number of different meetings on what the Parliament's overall budget is. The SPCB is the ultimate body that signs off the bid that goes forward to scrutiny by this committee. Would the corporate body know that overall cost of this website project being over three million, or would they just be presented with, this year it's going to cost 100K and next year it's going to cost a couple hundred more K? Usually the corporate body look at, we're working on an annual obviously budget and cycle, so the corporate body look at that. As I said a while ago, what we've been doing over the last four years is trying to improve the project pipeline, which looks ahead, obviously, to where we might have multiple years of spend on delivering something like the lefts or the web or other things like that. We've improved that, so the corporate body now is starting to get that overview. At this time, the corporate body wouldn't have been aware of the overall cost of the project. The finance committee wouldn't have been aware of the overall cost of the project. It's like me going out to buy a car. I don't buy the wheels one year and the engine the next. I need to know how much a car is on the cost. The corporate body would have known that the cost was within the indicative budget that was going up for each financial year, but looking back, there wouldn't have been a total cost of that spend that the finance committee would have seen. That's the thing. That's a complete failing then. A project has been approved without people knowing the overall cost of the project. It almost seems that there's frame agreements being in place, which has almost been handled by, it's almost like a maintenance project of a website, just adding a bit more on every year. It didn't go out to tender, so you don't know what the total cost is going to be, or maybe somebody did. You don't know what the scope is going to be because you haven't drawn that up because there's no tender process, or maybe I'm wrong. The tender process that Alan addressed was using existing frameworks that had been tendered on behalf of the public sector, so there was a tender process. We were using a framework. There was a tender process for the resource that was going to be doing this project. There wasn't a tender process for the overall project, so nobody could know what the overall project was going to cost, or maybe somebody did. No, there was an estimate for the overall cost, which the digital strategy board had for the overall cost, based on estimates in each of the financial years, which I think we'd set out in our response to you, and there was the number of years. Then, when the committee took evidence, we obviously also talked about the fact in that evidence, in one of the two evidence sessions about the fact that it was a multi-year project, but spend was within each indicative budget level for each of the financial years. The spend was contained within the annual budgeting cycle. There wasn't additional spend, if you like. No, but no elected members were ever told of the complete overall project cost of this project. So, in 2016, when we put up our evidence, we set out a number of things in the project schedule, some of which would be contained as projects within an annual year, some of which were going to be things that we flagged at the time might be across financial years, which we did with the website. But what we didn't have was that this was going to cost £2.9 million in totality. Now, that is the thing. I think that now that we have the project pipeline, we can go back and start to address for you in terms of saying, this is what we are going to spend our time on in the current financial year when you are scrutinising our budget, but we expect it to run to be a feature in quite a few of our budgets. So, I guess from Michelle's point, then that is a sort of lesson that has been learned that there is an overall project cost in house. We've matured our project pipeline approach over the last few years, and we can start to then bring more of that to the table whenever we are coming in front of you. In terms of the scope of the project, was that agreed at the outset, or is that just something that has been added to over the multiple years? I'll let Alan come in, because he was involved in the scope. I undertook a discovery phase within the first year, and we outlined what we were going to do in terms of the scope. At the same time, we further identified the cost of the project. Who was that then agreed with? Was that back to the corporate body? The scope did not come up to the corporate body, as far as I am aware. There was an update to the corporate body after that, outlining what we were going to be doing, but it was a high-level update that was presented in the May of that year, March of that year, before we kicked off the financial year of 2018-19. It was presented to the digital strategy board prior to that. The project board would have taken that information. It did not seem like the corporate body had any real... They were not presented with the facts, either, of what the project was actually going to do and what the overall cost was. Maybe they seemed to have been as much in the dark as the finance committee at that time. Can I just come in on that? Obviously, we have gone back and looked over the various documentation over the years. The delegation to officials is to develop and do all of the rigour around business cases being approved, challenged and assurance that those can be delivered, and doing the budgeting and putting that advice up, where the strategic resources board then looks like the strategic shape of the overall Parliament budget juxtaposing the various elements that we need to balance. The business case would be something that would sit at senior level with officials to improve the detail of the business cases. We have various checks and balances on that as well, but the corporate body would not be signing off detailed business cases for projects of which there are, as Jackson has alluded to, many in different areas of run in the Parliament in any given year. The corporate body is putting forward a budget that has been then approved by the finance committee without us knowing what the overall cost is. We do talk to the corporate body about the key areas of project spend, what they are going to deliver strategically for the Parliament, be that address, risk, etc. We put forward the budget saying that this is what is accommodated in our project schedule, but what you are talking about is forward look. Exactly, because they were not given the complete project spend, so it is difficult for them not to approve it, because they do not know how how much it is going to be in total over a number of years. What we were saying to them was that it is affordable in each annual budget, and it was affordable in each annual budget, so that is our assurance to the corporate body. However, I take on board your point that we can set out that forward look and report it on that a bit more to both you and the corporate body. There are a few things that I can afford, but it does not mean that we necessarily have to buy them. John, to be followed by Liz. Just a couple of points. To be fair, I am slightly more sympathetic towards yourself than some of my colleagues have been. This has come in roughly on time and roughly on budget, which I think is a good result, given that we have much experience of other IT projects that have not. First of all, I will just mention something that I find a little bit difficult about the new website and for searching for things. I do not use the website that much, but sometimes I am looking for a motion, and I was looking for a motion. I knew that I had written a motion on councillor pay, and I searched councillor pay and it gave me no results. Then I thought, well, maybe it was something else, so I thought councillors pay and it gave me one result, which was my motion. I just wonder if the public might struggle with that kind of thing, because unless they know therefore the exact words that have been used in a motion, they would struggle to find it. I will leave that with you, because you have said that the site will be developed. My other point is probably to reiterate what others have said, because I do not think that I was on the finance committee at that point when this started. Ms Haggerty, you have already said that you are going to present information slightly differently in the future, but I just like a re-emphasis and acceptance that it is very important for the finance committee to know if a project is going to run on to several years. That did actually happen with some of the security enhancements that went over more than one year. I remember being on the committee and we did ask and we did get the information that it would go over more than one year, because the risk, as others have just pointed out presumably, is that if you came back to the finance committee in year 2 and money was tight, there is a risk that they could have said no to the budget in year 2, and that would presumably have meant that all the money in year 1 had been wasted. I think that it is very important for the finance committee, whoever is on it, to know what they are committing to. Effectively—correct me if I am wrong—the finance committee is committing to a five-year project when normally we only commit budgets one year at a time. There is a risk to the corporate body that your second, third, fourth and fifth year budgets might get knocked back if the finance committee does not understand that they are making a commitment for five years. Am I understanding that correctly? The finance committee is scrutinising the overall Parliament budget in terms of the totality of the Parliament budget and all the various bits that make it up. Project spend is only four and a half to £5 million out of £112 million. Considering other organisations, we maintain it at that kind of level over many years at that level. That involves choices. Officials have to scrutinise different projects. We have to take a risk-based decision back to the convener's point about affordability. That is one of the considerations. We eek out the life of things in order to push out where we say that we can do that without catastrophic risk for the organisation. For example, the last time we replaced all the consoles, microphones and chamber, we harvested a lot of that equipment so that it could eek out the life of what you have in the committee room, so that we were looking at affordability in the round in terms of how we use our project spend. That is because we have to make choices. We have to make priority decisions every year. On top of that, as we have had for the past couple of years, we get a bit of a crisis situation, which means that we are having to peddle quite furiously to be able to address the increased resourcing costs to the organisation, such as enhancing cleaning and new service deliveries so that we can operate during the pandemic additional equipment for people to work at home, so that we can manage that. Those projects, like the website, are within our project budget without asking for more money, which is what we manage to do. We constantly reprioritise within our project budget in-year, and we seek to also look at spend across the years when we look at our medium-term financial planning, which is what we will be speaking to you about when we come back in December again. I am convinced that you are trying to keep costs down and not spend. There is a balance between preventative maintenance and reactive and so on. I stress that it is very important that the Finance Committee understands if a project is going to be lasting longer because, effectively, the Finance Committee is pre-approving what would normally just be a year-by-year budget. Can I just follow on that point? Can I draw the panel's attention to the letter that the committee received from the Presiding Officer, in which she says just towards the end that, as I quote, officials have recognised the need to provide increased detail on multi-year project costs as part of the annual budgeting process, and that this committee should note that, because it is one of the Parliament's larger investments, the project is scheduled to be reviewed as part of the internal audit programme. Can I just ask when that audit is due, and secondly, can I just seek confirmation that the results of that would be passed to this committee? It will be within the current financial year. I will get back to you on specifically when we anticipate that in the audit programme to happen, but it will be scheduled for the current financial year. I believe that it is a matter of public record, so those documents are available. I will follow on Mr Mason's point that, when it comes to multi-year projects, it is very important that we can scrutinise that. Obviously, we have to look at things on a yearly basis, but just the forecasting for projections is important to the scrutiny of this committee, so any information that we could have at that time of that internal audit would be immensely helpful to us, if that was possible. The second thing that I want to ask, which again is in the Presiding Officer's letter, where she says that the investment that has been made has achieved certain things. She says that it is more resilient, stable, flexible and a robust website. We have obviously heard varying views on that, but she secondly says that it has reduced the possibility of a really awful cyber attack, which we all have to accept as public bodies, sadly, these days can happen. We saw recently what happened to SEPA and how that problem made life very difficult for at least two of the committees in this Parliament, the Environment Committee and also the Rural Affairs Committee in the last Parliament. Obviously, that had implications for the work that the Parliament could do. Without giving away any state secrets, can I just ask about why we are confident that the possibility of a cyber attack has been addressed by this new system? It is a very important point that, if we were to have a cyber attack, it could cause us huge problems. I begin by acknowledging and thanking the officials who have tirelessly worked to ensure that the sustained efforts that the corporate body has been advised of external parties who have sought to break into and corrupt our network have been frustrated and defeated on each occasion. It is something that the corporate body has taken quite seriously. We received briefings, I seem to remember, from GCHQ, which was obviously very exciting, but there has been a genuine and sustained effort. We have worked, and I know that the team has worked incredibly hard to frustrate that and have succeeded. I know that in developing the new website, as Alan will now explain, the understanding that that is an on-going issue is very much to the fore of our thinking. We have built in good cyber practices from the off in this project for the design, which is one of the major risks of the old site. There are two things that we are really doing to reduce the risk. I am not going to be complacent and say that we have eliminated the risk, but we never can eliminate it. We are always, hopefully, going to be one step ahead of the bad people who will do this, but you cannot guarantee that you will always stay ahead. Two things that we are really putting in place. With the new website, with the team that we have on board now, we are able to update that and maintain that and keep that up to date. Just as you would do on your mobile phone, you would get app updates and those sorts of things. We get that for the actual system that runs it, so we make sure that it is applied in timely fashion as fast as possible, so that that helps us to prevent known faults becoming an issue for us in terms of cyber attack. The other thing that we undertake regularly with the website is what is called the penetration test. We will test it to see if there are any faults and we will address those faults or weaknesses that are identified within that. We are coming at it from two perspectives. One is the preventative side and the other is trying to tackle the cyber attack ourselves to see if we can break that. We have also taken advice and guidance from the Scottish Government Resilience Centre and the National Cyber Security Centre around our security for this. It is a key thing to us. That is very helpful and very encouraging. My final point, which again comes back to what the Presiding Officer was saying, which you have alluded to this morning, is that one of the reasons for the redesign is to ensure that it is more accessible and user-friendly for members of the public and people who are outside of the building. What are we doing to measure the responses of those stakeholders so that we know that what we have done is helping people to understand the place a bit better and to contribute to it? Throughout the course of putting the website in place, we tried to do a lot of consultation with a number of stakeholders and, going forward, we want to continue with this. On what you are asking about people who are external to the Parliament, on the website, we have constantly something at the top that we ask people to give us feedback in terms of what they think about the website. As part of the programme of continuous improvement, we take the results of that feedback together with feedback that we get from internal users, specifically members. We held a drop-in session back in March for members and their staff, and we will be holding another one in June. We want to hold that on a quarterly basis because, whether it is really important that we try to capture feedback from all the different users and that we use that to make improvements to the website. We are aware that we have a number of different users, and it can be a complex process trying to build something that will meet the needs of all those users. From an engagement perspective, what was key for us was making sure that people who did not know how the Parliament was structured and how it was operated would be able more easily to find things out, but we will continue to act on that feedback. Is that feedback relatively positive just now from people out with the building? As with most feedback, we can have a mixed bag. There are people who very much like the way that the website is structured. Some people have perhaps given us constructive suggestions, so I would not pretend that it is overwhelmingly positive, nor is it overwhelmingly negative. It is probably, as would be expected, a mixed bag. There have been a number of ways in which we have responded already to feedback, and there are a number of initiatives that are already planned. It is not just the range of users—one of the most significant things that the team had to accommodate—was the different ways in which people are now seeking to access the website, with a huge shift to mobile digital technology seeking to access it, which simply was not something that the previous system was capable of sustaining. Thank you very much. That concludes the question from other members of the committee. I have just got one or two to round up with. One, for example, on the issue of cost is the annual licensing support cost that has not come up, but I noticed that that has increased from £54,000 to £86,000, so I will have an explanation as to why we have that significant jump in cost. I think that I know that, but I think that you probably should ask that question. It is a different product that we are using. The platform for the website now costs more, but it is a better product. It gives us a lot more functionality for our engagement side. There is a lot more to it. It is a more fulsome product that we are using, and therefore it costs more on an annual basis. Having covered cost and scrutiny, I want to cover one other issue, which is quality. It is in response to a comment that Alan made earlier on. We had the referendum in September 1970 to set up this Parliament. 20 months later, the Parliament was established with a functioning website, which served us for many years. I recall that there were many IT problems. For example, we only had a capacity of 50 megabytes that are in-box and there were constant issues. I do not remember many difficulties with the website, I have to say, but I think that we would all accept that, nevertheless, we have to evolve. It is more than 20 years of elapsed, obviously, and there have been some upgrades, but this was a mega one. In response to Michelle, when Michelle Thompson talked about the trade of sweet and quality, Alan said that we had to give a little in-quality, but surely, if you are taking more than three years to develop a project, more than £3 million has been spent, you should not have to concede anything in quality. Where have we conceded that quality? I think that we were conceded a little bit in quality to make sure that we did not keep the project running, as you rightly highlighted. The project was costing us money every day, basically, for the contractor costs that you have talked about. We wanted to finish that as quickly as we could, so we could move that on to our own team. In doing so, we accepted that not everything was perfect. We could have kept polishing this for many years, but we really did not want to continue on with that cost. We knew that we had changed our whole model across the organisation to a continuous improvement, continuous development model for the website, and that is exactly what we are going to do, is to continue to improve that, seeking feedback from different people on that. A basic thing that I referred to earlier on is the fact that you have to go on the old website to look at the official report. Surely, that is a nonsense. I do not think that it is just a coincidence that, since this new website came up, we now, for the first time more than 20 years, get sent the official report every day so that we do not really have to go on to it. That is my fault, I have to say, because I have become increasingly concerned since we stopped producing paper copies of the official report that parliamentarians have not been bothering to access the official report and potentially have less familiarity with the general business within the Parliament than was the case before. I specifically, for no reason other than that, would be of enormous value to me, requested that the official report be taken out of the general, kind of go and find it category to one that would be proactively sent to all members on a daily basis, as the old paper copy used to be. Okay, but surely if it was more accessible online, most people would just access it online and wouldn't have to have it actually sent to them on a daily basis. I mean surely, but I mean we'll leave that, we'll leave that then. We'll leave it at that, but just one final thing, and it was just basically from Susan, she talked about it, we have to eke out the life of equipment, and yet this very day IT people are in my office, both constituency and upstairs, telling me that they have to replace laptops, which I think some of the staff have only had for less than a year, et cetera, and all sorts of other technology. We had some discussions about retaining some of it because we didn't want to lose any of it. All works pretty well, as far as we're concerned, certainly a lot better than it used to work a few years back. What is the necessity of that project and what's the cost going to be? It's the overall cost, because we'll obviously maybe want to scrutinise that further later in the year. We're refreshing the technology used by return members now, we've done it obviously after the election. We refreshed the technology, it will give out technology to the new members. What we are doing with that is we're making sure that you have the latest version of Windows installed on that. This is key for us to maintain our estate. We need to be working on the latest version so that it's compatible with other applications, but also maintaining our cyber security stance so that there are no known issues with that. It would be fine if everyone worked in Hollywood and was hidden behind a hard shell, but that's not the way things have worked out over the last two years, so we are refreshing those. The easiest way for us to do that is to replace all of the equipment at that time. That equipment, if it's not in more than four years, will then be reused elsewhere further down the line, so it will actually bubble down as we go about. We've got a fighting stock, if you like, that we put together so that we can actually replace equipment for members, and then we'll actually reuse the equipment that still has good value to us. We do try and eik out. We're very conscious of the tension, I guess, between the sustainability angle and the speed of equipment and the longevity of the lifespan of the equipment sitting in there. It's easier for us to upgrade the whole thing at once. It's more a cost-effective way for us to do that, and then use that equipment that we've taken back from you if it's still serviceable to pass on to somebody else, but again, it would be fully updated before passing on to them. Thank you very much for your evidence today. The committee will consider any next steps arising from today's session at a future meeting, and we also look forward to considering the SPCB's budget bid for 2023-24 towards the end of the year as part of our wider budget scrutiny process. That concludes the public part of today's meeting. Next item on the agenda will be to discuss private paper and consideration of work paper. We'll now move into private sessions. We also have a two-minute recess while we allow the official report and witnesses to leave.