 Mae gennym ein duol ddewid, dweud i gŷnod ddechrau ddechrau gŷnod ddechrau i'r ddewid i Gŷnod Dedechrau i'r Dweud i Gŷnod Ddechrau i'r Dweud i Gŷnod Dedechrau i'r dweud i Gŷnod Dedechrau, a bèl geitref agorag yr athgoffaidd gwybod amdano, i ddewid i'r ddewid i'r ddewid i gŷnod oed. Agender item 1, awg comfy yn gwirionedd gennym gweithias yma. Agender item 2, ar~)r o ad знаюr, rwy'r Gwyrdd Skotland. ond y bwysig iawn o'r ffordd digital ac rwy'n gweithio Ffrazor McKinley, Y Controlor of Audit, Gemma Dimon, Senior Manager, Morag Camps, The Audit Manager a Lucy Jones Auditor, All from Audit Scotland. Rwy'n gweithio bwysig iawn o gweithio Ffrazor McKinley a Gemma Dimon. Thank you, convener, and good morning members. A very brief word from me, and then I'll hand over to Gemma to talk a little bit about the content. I think that the main thing that I wanted to say is that this is a slightly different report for us. As you know, as this committee knows more than most, we've done a lot of work over the last few years on IT projects that have not gone well. In discussion with the committee and others, we felt that it would be helpful to try and pull together what we think some of the main themes and principles are from our experience of doing that work. The team have pulled together a set of principles that Gemma will touch on in a second, which we think are both the things that aren't in place when things go wrong and therefore the things that really need to be in place if those major IT and digital programmes are to be delivered successfully. As well as drawing on our own work, you'll see that we've drawn on work from colleagues and other organisations around the world and indeed other organisations as well. What's interesting is that there's lots of commonality around the themes. We are very happy today to bring this report to the committee to help you to consider how you might take forward some scrutiny of the whole question of digital public services in IT and hopefully to inform a debate more widely in the public services as to how those things can be done more effectively in the future. With that convener, I'll just ask Gemma to add a few words on the content, if that's okay. So what we wanted to do with this document was really to have it as a service to simulate discussion within public bodies about what some of the learning has been from other organisations and what weaknesses and what things they might need to look at to be able to learn from that. So what we wanted to do was rather than to set it out in a checklist format, we set it around a series of principles. Now there's five main principles, but they all intertwine with each other. You can't look at them separately, you have to look at them all together. Throughout all our reports, we've always pulled out skills and experiences as a really key thing for organisations to have in place. What we've done is we've intertwined that throughout the whole document because it affects all of the principles and we've used a little icon to draw attention to that because it's really such a key factor. So really what we want to do is use this document to help organisations to move forward, to learn from the past, and we've used a series of case studies and quotes to help to illustrate our points, to bring real-life examples in of where these things have happened. As Fraser mentioned, we're very happy to take more detailed questions on any of the principles and the document. Okay, that's very helpful. Thank you both very much. I wonder whether I could kick off just so that we set the scene because committee only obviously considers IT projects where there have been problems. So I'm keen to get a sense of the overall picture. Could you give us an idea of how much public money has cumulatively been lost as a result of the IT failures we're aware of, and can you give us a picture of the kind of ratio of successful to unsuccessful IT projects? So I'm not sure I'm going to be able to give you that just now convener, but we can certainly try on the numbers, we can do a bit of work there. I think on the balance of successful and non-successful might be a wee bit trickier to get just because there's an awful lot of IT development that goes on on kind of under the radar, but we can certainly see what we can do and write back to you if that would be helpful. At the same time, just to get an idea of how that compares to maybe other countries around the globe and also how that compares if it's possible to get these figures with the private sector in Scotland, because I'm sure not everything is as wonderful as it might seem, so it'd be useful to get that context. I'll invite colleagues in, Alex Neil. Very quickly to emphasise that last point. I think it would be a useful exercise to benchmark ourselves against other countries. Estonia being a very good example of a country that's well advanced in its use of IT, but not necessarily for data purposes, but certainly the Government is literally paper-free, and there may be others as well as benchmarking against the private sector. I think that it would be a useful exercise for all of Scotland to give us a sense of what we need to do to get to a position where we are at the top of the league for performance and all of this stuff. Just very briefly, I guess absolutely, Mr Neil, happy to do that. We've done a bit of that in doing this work in that we've looked at reports that have looked at places like Estonia and we know that the Government is looking at Estonia and other places themselves to get a sense of that. I think that the experience of Estonia is an interesting one, because my relatively limited understanding is that they, in a sense, had the benefit of going from virtually no digital infrastructure to fast-tracking to an entirely digital, rather than trying to unpick legacy systems, which is where we are. That said, absolutely things for us to learn. The private sector, again, is a really interesting example. Some of the reports that we link to in our report have examples of private sector that have gone well and some that really haven't. To take the convener's point, I think that it is too simplistic to say that the private sector does this well and the public sector doesn't. Basically, you just don't hear about it as much in the private sector. Again, we can have a think about what more information we can get on that, recognising that, inevitably, that is going to be more difficult for us to get to grips with. We will certainly have a look. That is very helpful. The European Commission does some benchmarking of European countries and their digital. It tends to be at a UK level, but that might be something that the Scottish Government itself can explore in terms of how they benchmark themselves within the UK and against the European Commission, because there is that information available at the EC level. In our further reading list, we link to one of the benchmarking reports. I have to say that when I got this report, I was a little bit surprised that it wasn't what I expected. The report is an aggregation of best practice as much as anything else. I had sort of anticipated from what the Auditor General had been talking about that what we were going to get was a better analysis of what's happening in the public sector IT procurement within Scotland. In view of the well-publicised problems that we've heard with NHS 24 and other systems that have been installed, I had expected that we were going to get a better overview of what the steps of government has taken, the new structure that they've put in, whether it was working and how all this is going to be pulled together in the future. Obviously, that doesn't quite do it. Is there an intention to do that analysis of the public sector in Scotland, which is really what this committee is looking for? That's very helpful, Mr Beattie. I apologise if there's been a mismatch in the expectations there. The short answer is yes, and I can say a little bit more about the work that we have planned to do that. It's also worth reminding the committee that we are revisiting some of the individual projects that are in here, most notably the CAP Futures programme. We're publishing an update report on that in the next month or so. Morag, do you want to say a wee bit about the work that's planned in a couple of years' time? We've just published our five-year work programme, and as part of that in 2018-19, we're planning to do some work looking at digital and health in central government as well. We've not fully scoped that work, but we're looking at things like the assurance framework that was just brought in and the digital first standards and the digital strategy, how the Scottish Government is doing against that would be part of that work. I think that from discussions that we've heard in the committee previously, we've obviously been concerned about individual projects, but we're also concerned about the overall health of IT procurement. You can easily get bogged down on one particular project that's going wrong and dig deep into that and get into the morass there. We were looking for a rather more overarching picture as to how we're going to avoid that situation in the future, about the steps that the Government has taken and the structures that it has put in place. Obviously, I personally would be very keen to see that. The work that Morag has described will do that. I think that there's an issue of timing for us. A lot of the stuff that's just been implemented is quite new, and we have reported to the committee and elsewhere about those and you've taken evidence yourselves. The reason that's in there for 2018-19 is that those frameworks and that approach will have been in place for a few years and will be in a much better position, I think, to provide some conclusions and judgments on the extent to which it's working. Is it practically speaking that it would be too soon to do something on it now? I mean, we could describe the arrangements as they are now, but I'm not sure that we'd be able to reach many conclusions on how effective they are because they are relatively new. I think that what I would say is that the point that we make in this report is that there are many things at play here to do that characterise the success or failure of an IT project. Procurement is a really important one, but, as we've said here, we think that there are lots of other things that we've seen that mean that those things don't work. What we're trying to do, I guess, is to pull those things out just now as a useful step for people to look at and reflect on, and then there will be a more, if you like, traditional audit product coming in a couple of years' time when we can actually see how the new arrangements are working on a practical level. What sort of expertise does Audit Scotland have internally in respect to IT projects? We have a team of ICT auditors who spend most of their time working in individual public bodies, councils, health boards and others, looking at the IT arrangements in those bodies, and then, as we do this kind of work and as we look at IT projects more widely, we draw on external expertise sometimes to help our own understanding of that. The team here has obviously worked quite extensively now on IT projects, so we develop our own expertise as we're doing the work, but we would recognise that we're not, in that sense, IT experts. What we bring is that audit expertise in asking the questions that we think need to be answered, and hopefully, I guess, what we try to do is try and simplify what can quite often be very technical and complex situations and jargon and things. That's what we try to bring to it, and we make sure that we get the technical expertise that we need to ensure that the judgments we're making are credible. Given that you've produced this document, what do you expect happens with this document now? Do you expect the Scottish Government to adopt it? Yes, it's been well received by the Government and others. It's being picked up in all the trade press, so computer weekly, all those kinds of things are running stories on it, so we would expect it to be picked up through that route. More specifically, locally, we would expect our auditors to ensure that individual public bodies are looking at the report and reflecting on it, and we'll be keeping an eye on how that's actually responded to. The officers on the ground will be using this as they are? Absolutely. And where there are big set-piece IT projects, one of the reasons we thought we wanted to do this kind of thing now is that we have things like social security coming down the line, which is going to have a very significant IT requirement to run the new powers that the Parliament has around social security. One of the reasons we wanted to publish something quite quickly around good practice is to ensure that this kind of stuff is fed into that, the planning for those big systems, so as well as looking at it in individual bodies, we know that there are some pretty big IT requirements coming down the line, and hopefully that will help to avoid some of the pitfalls that we've experienced in the past. Now, we understand that the Scottish Government official has been commissioned to develop an assurance process for major IT projects or IT programmes—over £5 million. How does this document fit into the assurance process that the Scottish Government is developing? Yeah, I mean that's very specifically, Mr Beattie. That's one of the things that we want to be looking at in some detail when we come back to the next piece of work, because the new assurance framework is a really important part of this. I guess in terms of this report and our principles, it touches on a number of things, not least the planning, governance, leadership and strategic oversight elements of this. We think that the assurance framework should be helping with some of the principles that we think need to be in place, but clearly we'll be keeping a very close eye on the extent to which that assurance framework is actually working. I'm still on that sort of theme. I understand that there's an Audit Scotland checklist that the Scottish Government uses or is putting in place. Again, how does this fit into that? Is checklist an existing one? Is it being amended or updated to take into account this report? You might remember a couple of years ago that we produced a report on managing ICT contracts that followed up an earlier report, which again was looking at the Scottish Government bringing in the new assurance framework and what their role was. What we did at that time was that we appended with that report a checklist of things for senior management and boards to consider as they were implementing an ICT project. That report works with that. It's supplementary to that and provides a bit more detail on where some of the pitfalls have been in the past from projects, where some of the areas that boards and senior managers really need to look at to give themselves an honest assessment of where their weaknesses are. It all works together with that. I found the report quite useful in capturing many of the issues, but I feel that it's like a second edition publication of something that I've read years ago, other than the currency of some of the projects that you've mentioned. It's the same kind of messages again and again year after year. Where on earth are we with things like software, project management, methodologies, quality assurance processes? I've asked this kind of thing before. While we might sit here and say, oh, the Scottish Government has accepted this and the public sector is looking at it in source of rails, it's the step change, what happens next that delivers what we think should be delivered that really interests me. I'd be interested to know just across the landscape from your own experience, who embraces formal methodologies, either in the Scottish Government or in the public sector? Who does that? If they are embracing those standards, why do we still get those software projects wrong? I'll kick off on that, convener, and then I'll ask the team to come in. I suppose that this is the head-scratching thing for us, Mr Coffey. In some places, some of those things are in place and yet it still doesn't work, which is one of the reasons that we thought we tried to do something like this, to try and pull together what we think are the key ingredients, if you like, of success, but you're absolutely right that when you look at even taking some of the ones that you've looked at recently, whether it's NHS 24 or I6 in the police, they were using methodologies, so it wasn't a complete absence of they hadn't thought about it and yet, in fact, in the case of the things like I6, we reckon that the procurement process, just looking at that bit specifically, was actually pretty good. Pretty sound. And yet, it still didn't work. I think that where I get to with that, so I don't think that there's a concern or an issue about people not being aware of or not using industry-recognised approaches to these things. We see that everywhere. I think that there is an issue about industry approaches moving on, so you'll remember one of the things in the I6 example was using what they called the waterfall approach, that very kind of top-down approach, which these days is used much less often and used, as we describe it in here, the more agile approach, so smaller phases of programme being the way forward. I think that the thing we've tried to emphasise in here is that you can't just get one of these things right or two of these things right or three of these things right, they all have to be in place. I think that when you look at the six examples that we've drawn on at the start of this report, they have some of those principles in place, but they were also lacking in one or two of them. So I think that the main lesson for me, I would draw, is the absolute need to have all of these things in place and to ensure for me that the leadership is right, and importantly for me that there's a realism attached to leadership that is both engaged and committed, but also a little bit independent, a little bit separate. I think that what we've seen in a lot of those examples is that people get so drawn into wanting the thing to work that they lose a bit of perspective. I think that what we have tried to draw out here is that importance of being clear about different roles and responsibilities, so there is a mechanism to take that step back. One of the things that the assurance framework is designed to do is improve the gateway process. You've heard evidence on a number of occasions that some of those projects that haven't worked have gone through a gateway process and have been told that it looks okay. The assurance framework is designed to have a much more robust kind of stop-go mechanism. Sometimes, recognising when we're better cutting our losses is something that we need to be better at as well. That's a long answer to a short question, Mr Coffey, but for me it's that sense of having all of these things in place rather than just a few of them. I think that's perfectly true. Even when you mentioned I6 there, even if you look at the cap system, is it fair to say that the errors or the mistakes are occurring at the front end of these projects? In my experience, it's convenient that if you don't get it right at the beginning, you're hardly likely to get it right at the end either. What that boils down to is understanding what the users or the customers require in investing time, energy and resource into getting that right. Usually, if you do get that right, you get the project right. If organisations are embracing those standards and methodologies, it's still a bit of a mystery as to why they're getting it wrong. Perhaps it is a skills, experience and expertise issue, because it's easy enough to say that we use an embrace, a quality standard, but if we don't really know how to use it then it's not going to be much good to you. Is there something that we need to learn and understand about this whole basket of five principles? If we were to ask you or anyone to apply those entails across the Scottish Government's public sector landscapes, whether they are truly embraced, what kind of picture do you think we might get from something like that? I think that it's a very variable picture. What we see is that what works in one organisation won't necessarily work in another organisation. There's not a one-size-fits-all solution to that, and that's really because of some of these softer issues, like the leadership and the culture, the different cultures of organisations. You'll see it when you have witnesses in front of you that the culture of different organisations is very different, so what works in one won't necessarily work in another. It's very difficult to know which principle applies more in different ones. They'll all apply in different mixes in different places. I think that you're right that the skills and experience is very important, and we remarked on that in all our audit reports and very much throughout this. At the very front end, as you described, that's where if you set off on the wrong track it's very hard to change tracks then to pull it back, but what you really need at the start is the skills and experience of somebody who's done this kind of for their experience to say, well actually what is it that we need? What is an honest assessment of the skills of our organisation? Because it's very easy if you've not done something before to not know what you don't have. If you've not done something, you can assume that you have the skills and experience, but actually until you get halfway down the line you realise oh actually we don't have what we need, so that's where the sharing of the learning, the talking to other organisations, the critical friends and the mentoring across the public sector is really key so that a chief executive of an organisation gets some realistic expectation of actually what it's like to do one of these projects, what the skills and experience he needs and how they can fill the gaps that they think they have. Do you think there's an extra dimension, an extra issue in the public sector in terms of time critical nature of some of these projects? You may have mentioned CAP and I CAP is incredibly time critical and I don't know about I6 really, but I suppose it was too, but the new software for the social security system in Scotland is going to be pretty time critical as well and that already gives me some concerns that there'll be huge pressure in there to get something working and often that is the first mistake we make in not accepting that some of these systems are very complex and they need the time to be developed and to be add in. If you hurry a solution you don't get one, you get something that you don't really want and isn't any good to anyone. I feel that we collectively have to keep saying these messages, but we have to try to persuade those who make the decisions about procurement and development of software to really invest and give the developers the time to put these packages together because the talent is there to write this software, there's no doubt about it, but you can't do it in a hurry and you can't do it unless you completely understand what the customers or the user's requirements actually are and I feel as though I'm singing an old song again, convener, and this has been said before, but we need to make that step change to try to improve this whole software development process and I think it can be done, but I think that investment at the front end and giving developers the time to do this properly is absolutely essential and I look forward to the next piece of work that Audit Scotland will do on this. Just falling on from Willie Coffey's point, it's important to put us into perspective, it's not just talking about a couple of computers and a backroom somewhere, we're talking about £4 billion that's been spent on ICT in the last five years and a huge sum as well on procurement alone and when it does go wrong, it can have a huge impact on the public and on businesses, so it is quite important, but I guess, like Willie Coffey, I'm looking for some reassurance that people are going to start to get this and I don't think there's a shortage of guidance and it sounds like there's clear methodologies and industry standards, so is this really about taking ownership within an organisation because it feels like when people say, oh, we're having IT problems, is it something that's happening over there somewhere else, it's not really a management problem, it's a computer problem, but everything that I've read so far is about management, it's about having the right skillsets, so again, maybe repeating Willie Coffey's points, but are we going to see this big leap forward because it's not just a problem in Scotland, I appreciate it, but in terms of where we are in Scotland, is there going to be a slight bulk moment? Well, let's hope so. To be fair, I'm not sure it's a case of people not really trying to get it, so to be honest, one of the things that we grappled with with this piece of work is trying to avoid just motherhood and apple pie and frankly stating the bleeding obvious, so you'll come to your own judgment about how we've done with that, I suppose, but there is something for us about trying to pull out those things and to come to your point around leadership. I think that what's very clear is that people are struggling, I think, a little bit to go from a world in which it was about IT to digital public services, they are not the same thing. They require quite a different mindset, they require a different set of leadership skills and behaviours. Digital is about how you run your business, it's not about an IT project, IT's an important part of it, but it's much wider than that. So I think there's a big transition, you'll have heard all the stuff about it being the next industrial revolution and all that kind of stuff and there is definitely something in that. I think just to very briefly use an example of where in NHS 24 we updated the committee towards the end of last year and I think that's an interesting week case study in the NHS 24 experience and my sense of it is that for quite a long time they and to some extent we were very focused on the IT project, it was about trying to get the IT project to work and then a new chief executive came in, had a look and said actually do you know what there are other issues at play here about the role of the organisation, how it's staffed, how people are trained, what its job is and I think took quite a brave decision to say we're not going to commit to another date to implement the IT project because we need to really go back to first principles and in a sense I think do what you're describing which is to really figure out not how do we get the IT project in but what is it we're trying to achieve with this thing and in that context hugely important thing that they do in terms of out of our care and everything else. So I think at times and it comes back to Mr Coffey's point I think about the environment and the I6, police I6 was a classic case as we reported about the political with a small p environment being such that people were absolutely desperate to make that thing work and in that environment it is quite a difficult thing to say you know what we're going to press the pause button because we're not sure this is right. So I do think there is something in there for leadership and management that recognises it's not about IT it's about how we run our business. I'm quite interested by the point you made about you know being brave and basically saying you know maybe this isn't the right thing now but in your report you talk about this a political context with a small p which contributed to I suppose the misplaced optimism throughout the I6 programme which we know has been a disaster. In your report on page nine I think it is you mentioned that legislative and ministerial commitments can reduce flexibility in timeframes I just wonder could you maybe flesh that out a little bit more. Yeah I mean that I think as Mr Coffey said there's no doubt that there are some projects where there are really that when there are proper hard deadlines that are not entirely within the control of the government or whichever organisation is doing it and cap would be would be one of those and this is why I think it's really important that we learn these lessons in advance of the social security powers coming in for sure. I think the lesson there is that so so there is a hard deadline what is what is the minimum we should be asking what is the minimum we can we need to deliver to make this thing work by that deadline I think what we do too often this is where the optimism thing comes in is that we say well that's the minimum and it would be really good if we could do this other stuff as well and maybe if we added in this bit of functionality and while we're at it we'll just and then before you know it it's an even bigger and more complex thing then it was then it was even then it needed to be in a sense. So I think that's where the realism comes in is recognising what the deadline is recognizing what needs to be in place for that deadline and then taking a more phased approach potentially to adding things in beyond that and too often we we tend to throw the kitchen sink at these things. Again when looking at these reports I always try and think about you know that the benefits to the public and we talk about transforming public services health and social care would be one of the examples that we've talked about on this committee before but I think all too often IT systems are still pretty much detached from each other so I know there's some progress being made but how optimistic are you that we're going to see systems properly integrated so that public services can be joined up and people can get the best possible experience. To be honest I think that's I mean we've pulled out users as one of our five key principles really to kind of to make that point about how important it is to involve the users in the project. Essentially they are one of the key success factors you know if people don't like the system or if it doesn't do what they want it to do they won't use it and then it can't be judged as a success. I think what we are seeing is as Fraser mentioned in that kind of shift to digital public services is a recognition of the role of the user and how important it is to get them involved. So we are seeing that shift in culture I think one of the things we'll be doing when we do the next piece of work is to look at that and to look at how users are being involved in things and what difference that is making but I think we are starting to see some of the signs of that shifting culture about how important the users are and how they need to be involved. Are we getting any good examples of that where the user is not just on the margins and a little bit of consultation they're actually there at the design stage and all the way through. Moag, have you got? Yeah, I think that we sort of pulled out Regers of Scotland obviously had issues in the past but they are very much focused on the users and that's like internal users that are actually going to be using the system in a wider business as well but also like solicitors that are going to be using the systems as well and bringing those people in to sort of test ideas out with them and then actually test the software out with them as well. So you know there are examples out there of organisations that are doing that. Revenue Scotland would be another example that learned good lessons as they were setting up the systems there. I mean I think you mentioned health and social care. I mean there's absolutely no underestimating the challenge there because trying to get council in any chess systems to talk to each other is going to be a mammoth task and we are as you know reporting, we'll be reporting a lot over the next few years on integration and similarly again to the police what we find is that the public service reform bit, transforming services bit, is running ahead of the systems bit which actually does get in the way of people trying to do the job on the ground whether you're a police officer or whether you're a care worker or whether you're somebody working in a hospital. So again I think there's a recognition that that's really important thing. I think in the case of integration it feels like they're quite still quite a long way away from grappling with that. A lot of the attention has been on getting integration joint boards up and running. So again that's something that we'll be looking at in terms of how the plans and how what they're actually doing on the ground to help to join up data, to help to join up systems all with a view to ensuring that the service users are getting is the best that it can possibly be. Thank you very much convener. When reading the key principles, particularly around about planning, it was amazing in some ways that it seemed like it was stating the obvious around about having the right skills all the way throughout the project and how often those were lacking and we've seen that from when we've looked at i6 for example in other projects. The reason for that being, it is a cultural thing almost because I know from being in Aberdeen City Council working with officials, there can sometimes be a resistance to getting external advice in because it's external. There's always this assumption that it can be done in-house, it's better off doing it in-house. So how do you think that is a problem in terms of the public sector and how do we kind of break through that? It's a key issue and there may be a bit of that. I think that there are other things at play like genuinely the market for good IT skills is very, very tight and expensive and there is an issue I think about what the public sector is able to pay for the best people in IT. That's both in terms of bringing people into the organisation but also to be honest paying daily rates, some of which can be quite substantial and people are understandably in the current environment not that desperate to do that. They think very carefully before doing it. The key thing for us is that it comes to this thing about being an intelligent client. You need to make sure, particularly when you're working with some of the big technology providers, the big technology integrators who do this stuff day in, day out. At those initial planning and particularly through the procurement and negotiation stages, it's so important that the client side, the public sector side, has people on their side that can engage in that negotiation and discussion as equals. That's not an easy thing to crack and it almost certainly does mean that you're bringing people in from outside. It's something that the Government are aware of. They're investing quite a lot in this whole agenda around skills. They're training up digital champions who are senior people who are not techies to use that horrible word but are going to be leading organisations into digital age. They are recognising that, but they are playing catch-up a little better. You've answered part of my second question just there about what the Scottish Government is doing. I appreciate that they're taking the time to ensure that people are trained up, skilled up. When we looked at CAP, we saw issues about staff morale as well. It wasn't a good situation at all. There's the risk of losing people. Do you know if they're looking at addressing those issues so that you don't just see people with the right skills going out the door that we're actually working to retain them? In our last overarching report, the managing ICT contracts report, we mentioned about the digital transformation service. The Scottish Government was setting up an effort to try and plug those skills gaps to have some centralised skills and people that could then be used to go around different organisations within central government to try and help to fill those skills gaps. That's again a way of trying to stop those skills going out the door so somebody is getting great experience working on a project and then leaving to go off into the private sector, for example, so that was that effort to try and keep them in. Again, that's something that we're very interested in and we're going to be reporting back on how that is working in our next audit report as well, because we're interested to see how much that's taken off and how much it's being used. One of the other key principles is about leadership. Again, it was interesting to lead that read that often we have failure in ICT projects because of weak leadership. Again, that sounds familiar from experience in council when often you can see other projects with us, infrastructure projects, capital projects. Often sometimes there's issues because the project management isn't there, the leadership isn't there. I see that those issues have been flanked up in the report but do you have a recommended model? How should that work? What are you saying to the Government that they should be putting in place to ensure that there's strong leadership on all of those projects, to ensure that it's delivered on time, on budget and appropriately? We've not gone as far as recommending any specific models or methodologies because we think it's important that the individual organisations pick an approach that's suitable for them and for the project. That's why we've taken a principles-based approach. As I've said, some of the stuff that the Government is doing is designed to help the most senior people in the public sector to think about their leadership in a digital age. It's not about how to lead ICT projects necessarily, it's about what does this mean for the way that we run our business and there is also work, as Gemma has described, in terms of the actual delivery of ICT projects. It's interesting. It does strike me and maybe there's more we could do on this, whether there are lessons to be learned from other big infrastructure projects. We seem to be managing to build a big bridge across the Forth River. It's very different, but I wonder if there are some parallels that people could draw on around that leadership and programme management approach and apply to those kinds of things. There might be something in there for us to think about as well. There's an issue where sometimes we just don't have the governance structure right and the reporting structure right, but is there also an issue with that sometimes we just don't have the people with the right skills to actually project manage as well? Yeah, I mean certainly there's a strong body of evidence in all the reports we refer to in here that, for example, if in some of the examples we've seen down south, if you've got four senior responsible owners in a year, all of whom are bringing a different approach, then that's clearly not going to work. To come back to your culture point, there is an interesting thing I think about the public service and maybe the civil service in particular where we expect people to be generalists, so one minute you're running an area on culture, next minute you're running a big IT project. Now they bring enormous skills and experience to that, but I think what we've said in the past and the committee I think was very clear in relation to NHS 24 is there is a question about whether that expectation of our senior people in public service is reasonable given the scale and complexity of some of these IT projects that we're looking at. Thank you, convener. Liam Kerr. Thank you. I find that line of questioning from Ross Thompson very interesting, actually. You talk about the lessons learned on other infrastructure projects. In some ways, when I'm looking through this, I'm saying that these are just failures of project management. Ross and I come from the northeast and you've got enormous oil related projects. On much bigger scales than any of this, going swimmingly. As I think Ross was getting to, you get the right people in and you get the right project going. I think that the leadership is an important part of that. Where I get to is that you've got a whole section on governance. Your section on governance doesn't go quite as far as to talk about accountability. Shouldn't that be addressed in the report? In the sense that, as you know from our various sessions, I have a big issue that very few people seem to be held accountable for these failures. The finger rarely gets pointed at one individual. What isn't that to be part of the report saying that somebody's head should roll if this goes wrong? I think that that's a very fair question. Now that you've mentioned that, Mr Kerr, we're talking in various places about roles and responsibilities in governance and those kinds of things. There should always be someone who is clearly accountable for the delivery of the project. I completely understand your frustration sometimes when you've got people sitting in front of you and you're saying, so who's responsible for this and who's accountable for it and you're not getting a clear answer. I think that I would also say that those things are not clear cut. They are big and complex and there are lots of different things at play, so sometimes pinning it on a person is quite difficult. Your point is well made. I'll take that away and have a think about how we might talk about it. We've talked a little bit about accountability in the leadership section recognising whereabouts it sits within the organisation. I've got quite an interesting quote in there from a report from South Australia, which talks about the difference between responsibility and accountability. That's an important thing for organisations to think about when they're putting their governance structures in place and where responsibility for delivering a project lies and where accountability is because that can lie at different places within an organisation. I thought that the South Australia quote was useful. It might be worth pulling that into the body to say that this isn't just an example, this is a principle. Moving on from that, the following page from where you've got your South Australia example. You talk about the senior officers and the stable leadership and reference quite rightly. Obviously, there's a revolving door often and that will be a cause of failure. You don't then go on to say how we prevent that. When I was reading through one of the things that I thought was why aren't we talking about some kind of performance related pay? Rather than people coming in and being paid vast amounts of money and saying, deliver this, why aren't we talking about some kind of performance related pay or golden handcuffs or something like that to stop the revolving door and deliver the success? Again, a very good question. There's a kind of balance to be struck, isn't there? My guess is that sometimes the revolving door happens because it's not going very well. There's a link between your accountability point and sometimes the revolving door point. How long do you give someone to try and get a thing to work? Your point is well made about how we ensure that if we are able to first of all recruit and attract the best, there are big questions in there about reward and other things. What mechanisms are there in place to make sure that those people are staying? A lot of folk will come to those things because professionally they'll be enormously rewarding and they're good things to work on, but there is also clearly in this market a question of remuneration and pay. The model for that is not really for us to comment on, I guess, but I think that what's helpful about your questions is that there's something that we can think about as to how organisations and the Government and councils and whoever it is are thinking about exactly that topic as they're beginning to progress this stuff. A slightly different line for me. How is your report? There are obviously some projects running, so the renewed CAP project, the NHS project. Do you get any sense that they're obviously in the process? How is this going to be taken on by those projects? They're clearly not going to stop and reboot, as it were. How are those principles going to be taken on by the projects already running? I think that it's a really interesting question and I think that one's like CAP, which is actually nearly finished and will bring you an update in a month's time on that. The conversation has so far moved on from this, but I think that it's really important for organisations who are in the midst of an IT project to have a look at this and to think, and we talked to her about that honest assessment of actually where are we against some of these things. What is our governance structure working for us? Is it given us the speed of decision making? Have we really captured all the risks and are acting on them? I think that it gives potentially at board level and at senior management level the opportunity to have that kind of stock take and to say actually if we use this approach, if we look at these principles and we were to assess ourselves against this, or to ask some challenging questions of those people leading the projects and of the project teams, to say actually where are we against some of these things and to have that kind of open discussion about it, then that's how we hope it would be used so that we can have that chance to have a little pause and to have a thing and have that assessment of themselves. I think that we'd be concerned too strong a word, but I think that the good organisations will not just hand this to their IT director and I think that there is a risk that people will look at this and go, it's an IT thing and hand it to their IT director. This is a conversation that should be happening at corporate management teams and at board level and that's where we would expect this to be looked at. Is it your view that if the principles are followed in this document to the letter, the project will be successful? If so, if an organisation were to choose to deviate from those principles and then the project goes wrong, should their head roll for that? We can't give any guarantees on any of those things, I guess, is the slightly glib answer to that, Mr Kerr. I think that there's no guarantee of success in any of this. I think that what we're saying is that if there's no guarantee of success, we're pretty sure that if you don't have these things in place, it ain't going to work. If you look at it that way and as I said earlier on to one of the questions for me, it's about ensuring that the principles for success that we've set out here are present in every stage of the project. This is not a sequence of principles, these are a set of principles that need to exist at every stage of a major IT and digital project. Certainly, if some people were to make an explicit choice not to do some of those things, we'd be asking some very hard questions about that. My sense is that sometimes people think that they've got all of that in place, but actually they don't. They think that they're doing the things around leadership, they think that they've got a governance in place, and they think that they're engaging with users. When we turn up in the middle of an IT project, it would be very unlikely for people to say, yes, yes, we're not engaging with users. The law would say, yes, we're engaging with users, but quite often it's not the right users, rather they're not engaging in the right way. NHS 24 is a good example of that. A lot of the activity was around the users of the system in NHS 24, which is enormously important, but there was very little engagement early on with out-of-hours services and health boards, which is the whole point of the exercise. That definition of user is enormously important. Now that we've done this, even if it is a little bit motherhood and apple pie, if people aren't looking at it and they can't demonstrate how they've considered it and put it in place, when they're delivering a big project, then absolutely we and you would be right to be asking some very tough questions about that. Thank you. Okay, Willie Coffey. Thanks very much again, convener. I can only mention someone that's quite crucial here. He said earlier, Fraser, that public sector reforms often run ahead of systems development's ability to deliver. I think that that's a key message in here. Liam Kerr asked if these principles are in place, will it be guaranteed to succeed? I think that the answer is definitely no. It's not guaranteed. Governments, public sector, at the very outset of any software development project, I think that they really need to engage with their IT staff as early as possible before commitments and announcements are made about delivery of systems like this, because it's the worst fear in the world as a software developer to be given a task and to be told, you know, get six months or a year to write something that could really, really take much longer than that. If there's early engagement with the technology staff and they have to be able to stand up to Governments, ministers or whoever and say, that cannot be done in time, so don't make that announcement. I think that's what happens right across these projects, not just in Scotland, but right across the world. You need to have technologists who will stand up at the beginning and say that it can't be done, but you also need Governments to be able to recognise that and to just caulk any a wee bit before making pronouncements and announcements about when systems will be delivered. If they don't do that, convener, those kinds of projects will continue to fail to deliver on time. Encourage back benches to do likewise. Mr McKinley. Probably not a comment on that, convener, but I'm struggling to remember which one it is now, but there's one of the examples in here that talks about the importance of the policy people and the IT people working really closely together. That's absolutely true in an IT project, and I would say that it's equally true, Mr Coffey, before the things announced. So I don't underestimate the difficulty of that sometimes for all sorts of entirely legitimate reasons, but yes, the more—and again, I come back to NHS 24—one of the things that they did there was that they actually not only did they say, we're not implementing on that date, they didn't announce another date. They said, we're going to take our time, and at the appropriate point when we're more sure, we'll let you know. Now at the time, there was a bit of backlash about that, and about old-year work that you don't know and it's strudderless and how do you not know when it's going to go live. But again, there's a judgment about whether you take a bit of flack for that at that point, rather than committing to a date that you then have to miss and then a subsequent date, and so it goes on, because our experience in places like NHS 24 is, once you're in that spiral of naming a date and then missing it and missing it, all the issues of morale and everything else come in, and it's very hard to get yourself out of that. On the basis that there are no other questions from members, and I don't see any indication that there are, can I thank you all for coming along this morning and giving evidence to the committee? I will pause the committee briefly to allow the witnesses to move back from the table, and then we'll move on to agenda item 3. Gender item 3, which is considering the draft annual report for the parliamentary year 12 May 2016 to 11 May 2017. Do members have any comments on the report? No, subject to the one amendment that's been raised with me earlier, are members content to sign off the report? Okay, that's great. Can I now move the committee into private session? Thank you very much.