 Gweithdoedd, i fewn i gyd. Felly gyn nhw'r ddweud y Gweithgatidd Gwyrddon o'r ddweud yn 2017, wrth gwnaeth gyd yn siŵn i gyd, mae'n gyrfaen nhw'n ddarlis. Mae'r inghylch yn ddigwyddol i ddechrau i ddarlis i ddweud yn Gwyrddon, ac mae'n ddigwyddol i gynghwych ar ei wneud. Arweol agriad. Our next item of business is an evidence session on Audit Scotland's review of Police Scotland's I6 programme. I welcome Caroline Gardner, Auditor General for Scotland, Catherine Young, Audit Manager and Mark Roberts, Senior Manager of Audit Scotland. I thank the Auditor General for accommodating the change of date for this meeting. I am aware that you have a commitment directly after this meeting, so I intend to have this session finished for quarter to two to allow you to leave Auditor General, because we appreciate you coming along. Can I invite you now to make a brief opening statement? Thank you, convener. It will be brief. In June 2013, the newly established Scottish Police Authority signed a contract for £46.1 million with Accenture to deliver a national IT system for Police Scotland, known as I6. This followed an 18-month procurement process, which we found followed good practice and included intensive engagement with potential bidders. I6 was intended to replace over 130 IT and paper-based systems inherited from Police Scotland's predecessor forces and to transform how Police Scotland records, manages and analyzes information. It was intended to be a key component of police reform. Three years later, in July 2016, the Scottish Police Authority and Accenture mutually agreed to terminate the contract. They agreed a settlement agreement of £24.65 million, which included a full refund of all the money paid to Accenture, a total of £11.09 million, plus an additional payment of £13.56 million. At the heart of the failure of the I6 programme was a disagreement about the scope of the programme, the interpretation of the contract and the extent to which Police Scotland's requirements were met by Accenture's solution. The disagreement surfaced almost immediately after the signing of the contract in June 2013 and permanently damaged trust, relationships and confidence between Police Scotland and Accenture. Other factors also contributed to the failure, including the method adopted for the development of the system, over reliance on an existing system that had been delivered by Accenture for the Guadiath Avail in Spain, and a misplaced optimism about the prospects of delivering the system, which may have led to a reluctance to consider terminating the programme earlier. The failure of the I6 programme means that some of the benefits of police reform have been at best delayed. Police officers and staff continue to struggle without of date inefficient and poorly integrated systems, and there are also wider implications for the modernisation of the justice system. It is now critical that the Scottish Police Authority and Police Scotland put a plan in place that sets out how the benefits that I6 was supposed to deliver will be secured. It is particularly important, given the emphasis on the use of technology in the recently published Policing 2026 draft strategy. Convener, the team and I will be happy to answer the questions from the committee this afternoon. Thank you very much for that. I refer members to paper 1, which is a note by the clerk, and paper 2, which is a private paper. I perhaps start by asking the auditor general for a bit of background information on this, because your report acknowledges that good practice was followed during a fairly lengthy procurement and discussion process, yet within three months of the contract being awarded, it was at red status, only achieving one milestone. Is there anything that could have been done differently throughout that procurement and discussion process to avoid the difficulties that emerged? If Police Scotland and the SPE were to embark on a similar process, what recommendations would you make to them? A very big question, convener. I will kick off and ask Mark Roberts to come in, if I may. As you say, we found that the process that was followed did fit with the good practice that we and others have recommended over a period. I think that the underlying problem was a mismatch between what Police Scotland was specifying in the contract and what Accenture, how Accenture interpreted that, and particularly the understanding that the system that had been delivered for the Guardian of the Ville would supply most of what was required for the Police Scotland system. That was the gap. I will ask Mark to talk you through that in more detail. As we say in the report, the programme for procuring the system did follow the good practice that we would expect to see and that we have recommended in previous reports. For example, the establishment of a programme board, the identification of one single individual as a single responsible owner, a dedicated programme manager to manage the day-to-day operation of the programme as a whole. In terms of that procurement process, it was absolutely following what we would have recommended and what others would have recommended as good practice. In response to your question about what could they have done differently to do something else, I think that it is very unlikely. The chair of the SPA has said as much that such a large-scale big IT programme would not be a route that they would pursue again. They are much more likely to have broken us up into more discrete bits and doing it in a much more modular rather than one giant programme in the future. That might assist considerably in trying to avoid some of the problems that I6 ran into. I struggle to understand how, throughout the process, nothing was picked up or identified that was feeding into the problems that seemed insurmountable. We say in the report that, despite what was a very intensive process, we made reference to 160 dialogue sessions that went on between the police and potential bidders, which were focused on what were the technical specifications required of the system, were about how it was going to be delivered and the detailed contract. Despite all that, still within a very short period of time after the contract was signed, the differences in interpretation of what was going to be delivered emerged very quickly. That is a challenging part of the history. It almost seems as if the discussions that were taking place were around a system that was being delivered, but the discussions were at a very high level, not at a practical operational level. Is that a fair comment? I do not think that we could really comment on the level of what was involved in those detailed dialogue discussions. My understanding of it is that they were at a fairly detailed level in terms of exactly how the system was going to operate and what was going to be expected of it. Clearly, for some reason, despite all of that, that left something of a gap once they got into the getting-going on developing the programme in terms of the difference in interpretation as to what was expected and what was going to be delivered. Okay, thank you for that. Stewart and then John. Thank you very much, convener. I should say that I spent several years lecturing on computer project management systems to postgraduates and I have gone back to some of my lecture notes that I used to use and there is one in particular that I want to just put as something that I said in 2002. Off-the-shelf solutions solve someone else's problems, not yours. What I was seeking to identify to my students was that you should be most wary in looking at the ways forward of the off-the-shelf solution. If you wanted to solve your problems, it would be solving someone else's. You could, of course, have taken the option of taking the off-the-shelf solution, which, if you thought it was a good solution to the generality of the class of problems, and then changed the way that you do things to adapt to that off-the-shelf solution, if it was an industry leader. Of course, what has happened here has borne out my point. Is there a general point that this illustrates that one must exercise particular caution when you are buying somebody else's off-the-shelf solution and imagining that you can adapt it, while not, in a major sense, compromising what you want to do to adapt to the system that you are buying? It is a very well made point, Mr Stevenson. We say in the report that there was overconfidence, as it turned out, on both sides, on the side of Police Scotland and Accenture about the extent to which the system that had already been developed for the Spanish policing service would provide the basis for what Police Scotland needed. In the event, it became clear over time that much more bespoke development would be required. There are a couple of things that we say in the report that I think are relevant to that. One of the central concerns that Police Scotland had about the Spanish system once they got closer to it was around the capabilities of the search function across the different elements of it and how far it was genuinely able to sort of integrate that information. The second is a comment that we make in the report about concerns raised by the programme board about the extent to which the Accenture team understood policing in Scotland. I think that both of those suggest that there is something important in what you say. Mark, do you want to answer that? I think that it is fair to put on the record that the I6 wasn't taking the Spanish system and directly applying it to the Scottish policing environment. It was built on that and was recognised from a very early stage that it was a much more complex product. It was always designed to be founded on it rather than being a sort of light replacement. It was, as we say in the report, it gradually became clear that significantly more was going to have to be bespoke, developed rather than being an extension of what had been delivered within the Spanish context. Professor Fred P. Brooks of the University of Southern California, forgive me, I can't quite remember. In his book, The Mythical Man Month, it says that you should never seek to modify more than 10 per cent of a system. It's cheaper and easier to write a new system. Would this break Fred P. Brooks' law of 10 per cent? The estimates that we had seen and heard about were that two thirds of the system could have been based on the existing system. In that context, that rule would have been broken. It's very well adrift. Fred P. Brooks wrote his book in 1974, it's worth making the point. John Befall by Ben. Can I ask about the genesis of the project, please? It takes place over a number of years. It's the extent to which your reporter feels that you're able to comment on what someone's referred to me as, in information that I've given to me, the missing year. I understand that the project actually starts in 2004 and occurs significant staffing and consultancy costs. Are you able to comment on that and are you able to say, for instance, if those costs were part of the settlement? The report in front of you today looks at what happened from 2010 onwards, which is when people started talking specifically about ICICS as this one large programme. Mark, you've had more history with this. I wonder if you'd like to comment on the longer term timescale. I absolutely recognise that this is the latest episode in a very long history of police at IT developments. Over the last couple of decades, we took it from the 2010 date that the Auditor General has mentioned, as that was when the outline business case was agreed by the then ACPOS Council. That was, for us, the starting point. We didn't look at costs prior to that date or any activity that had gone on prior to that date in the course of this. Obviously, there was contextual information. To answer your point about whether those costs were recognised in the longer term, I don't think that I can fully answer that. That's perhaps a question that might be better posed to the SPA. Our understanding is that the settlement agreement that was reached with Accenture recognised staff costs, but our understanding is that that reflects during the course of the actual I6 programme, not in the longer term, the early agenesis of it. The business case for I6, does that allude to any of the previous projects? There was one significant very costly failure, but does that allude to how it comes to that point, the collaborative working between what was then Eight Forces and two different organisations? Certainly, in the full business case, there is reference to previous attempts to develop different systems. I'm afraid of talking ahead, but I don't recall whether there was any specifics about the costs that had been incurred prior to that date in those other programmes. As I said, we were focused on I6 as I6 from that 2010 date. Mention the business case. The business case talks about potential savings of £200 million. Are they likely to be realised, or has that written out the equation? I6 has not come into existence, so the benefits that are supposed to start accruing from it are not accruing. Until we see what is the plan that is going to follow on from that and hopefully be able to implement the vision that is outlined in policing 2026. That is when we will be able to tell when those benefits will start accruing. At the moment, we don't see those as accruing. The part that I6 was playing in overall police reform, can you comment on that at all? I think that it is fair to say that it was intended to be a central part of police reform, both in terms of the £200 million savings that were a contribution to closing the funding gaps that we are now aware exist, and just as importantly, in making the way that police officers and police staff carry out their roles more effective and more joined up, fleeter of foot, because of the good use of IT. We know from Mr Flanagan, SPA chairman, that he has indicated in a quote that there will be no son of I6. Will there be any non-gender specific siblings or cousins or anything like that? I'll ask Mark to come in. I think it comes back to the point that Mark was making, that the intention is not to have a big bang approach like this, but instead, in the context of the draft policing 2026 strategy, to look at ways of achieving lots of those better, more efficient ways of working through the use of technology in smaller, more modular ways. We haven't yet seen a plan for that, but I think that's the approach that is intended. On that, our mechanism for following up on that is that the Auditor General has asked the Auditors of the Scottish Police Authority to monitor the developments of future plans for our ICT as part of the annual audit process. That's where we'll keep our ongoing monitoring of progress. To a non-technical person like myself, if we replace one big project with 10 smaller projects, does that spread the risk or does it increase the risk? Is there a general principle that is able to comment on that? At the generality, perhaps not putting all your eggs in one basket is probably a good idea. That would follow the more general practice in the IT world, which we've talked to interviewees about, in terms of a much more modular, more agile approach to software development would be much more likely to be adopted these days, rather than, as I mentioned before, the one giant programme. Many thanks indeed. Thank you, convener. I was also going to touch on the balance between a waterfall method and an agile method, but I think that that's been quite comprehensively covered. Just for absolute clarity, there was another point that was raised in your initial remarks, Auditor General, by Mr Roberts, on the procurement process and the fact that that 18-month process followed good practice, but I think that best practice was perhaps also used. I just wanted to absolutely check, are there any lessons that can be learned to enhance that process even further, for example, within the invitation to tender arrangements? Is there anything that could be improved going forward? In broad terms, we think that that process did follow good practice, and indeed the fact that Police Scotland and the SPA managed to recover not just the payments they'd made to Accenture in Fall, but also an additional payment in compensation for staff time and other costs that have been incurred demonstrates the strengths of the approach that was taken. We are in the process at the moment of pulling together the lessons from all of the work that we've done on IT systems failures and problems, and you'll be aware that this is one of a series that I've reported on. That's due for publishing in May, so we're planning to put it all together there. I guess I'm also reflecting on the remarks made by Mr Stevenson about the risks of relying on existing systems that appear to do what you want them to do, because we do think that overconfidence was one of the contributing factors here. Mark, do you want to add anything to that? No, I don't think so. I think I was going to make reference to the work that we're planning to publish, which pulls together a lot of the themes that have come out of our various reports on IT, so I don't think I'll have anything further to add forward to reading that with interest. I think that in terms of the resources and the implementation plan, which were again, as Mr Stevenson had highlighted, were based on—it was a hybrid of off-the-shelf and development, if you like. I think that some of the issues that they discovered later on were down to the fact that this initial implementation plan and all these resources were based on that assumption, so I think a lesson learned in terms of that is even though there was a very detailed tendering at bid and the business scenarios outlined in there looked at all the functionality of the system. Again, probably assumptions were made and the basis of that then, the resources, the implementation plan led then to some difficulties later on having to overlap some phases and issues around that. I think that in terms of a lesson learned, the assumptions are back to that hybrid, make no assumptions, I guess, around that hybrid solution. One of the things that indicates that projects died is when change stops, because as projects move forward, both the deliverer and the customer understand their own needs and what requires to be done better, because people don't know how their own lives work until they have to dissect them in detail. Have you got evidence from what you've seen that there was that learning process where the customer, in this case Police Scotland, were continuing to learn and understand? As we know, you cannot write a spec at the beginning of a computer project because it's the last 5 per cent of the work that delivers a project and the first 95 per cent is just working out what you need to deliver. Was that learning process still going on or was there a failure for that to progress? Wasn't there a good structure in which that could happen, both for Accenture but also for Police Scotland? I think that one of the positives that has come out of this is a much clearer understanding and documentation of the Police Scotland's business processes and how things were done. That was going on fairly early on in the lifetime of Police Scotland and I think that the process of going through that business pro-sync system mapping revealed some of the divergences between the various legacy forces that had been in how things were operated. They have now got that work and that understanding done and that was an evolving thing during the course of the process. In whatever way Police Scotland choose to take this forward in the future, hopefully that will have a clearer understanding of, as you say, you don't know your own life. I think that they probably do know that a lot better now than have a plan for how things should be done in the future. That will be a useful foundation for the next stage of learning as they move forward with developing systems that they need in a different model. Auditor General, you said in your opening statement that the failure of the IC programme had wider implications for the modernisation of the justice system. I wonder if you could maybe elaborate a wee bit on that and maybe comment on what you think needs to be done to put the police and justice reform back on track. Absolutely. Again, I'll ask Marta to give you the detail to it. Exhibit 1 in the report shows some of the key areas that ICS was intended to be providing these wider benefits over and above efficiency savings. A key part of that was the criminal justice system, making it easier to do things like full case reporting, to handle warrants and police citations. Not just what the police are doing, but passing that on through the courts at service. Marta can give you a picture of the impact that that is having or not having. The full business case for ICS was very strong on the importance of the police as a part of the wider justice system as a whole. There was a group set up to link in organisations like the Crown Office and Procuracy Fiscal Service, the Scottish Prison Service, the then Children's Reporter Administration and the Scottish Courts and Tribunals Service. As part of the making justice work strategy and the Scottish Government's justice digital strategy, it was a key component of doing that. Just to give a bit more colour to that, in recent weeks the Scottish Courts and Tribunals Service have published the latest phase of their evidence and procedure review. That gives an illustration of at the moment that, if police are providing photographic evidence to the courts, those have to be printed out in hard copy and handed over as photo books, rather than being exchanged electronically. That is just one example of something where there is not a connectivity between the police's IT system and the wider justice systems. Again, that is something in the plan for how police Scotland's IT develops in the future. There also has to be the connection to all the other elements of the justice system as a whole. Thank you. That is helpful. Can you just estimate the length of delay that you think this has all caused for that to happen, just on an approximate measure? That would be speculating well. We did not try to do that and it would be wrong for me to do so. Just following up on that or the general, you talked about the IT project being central to police reform. It is central, as you say, to policing 2026. As Mr Roberts rightly pointed out, as the justice committee has seen in terms of our inquiry on Crown Office Procurate and Fiscal Service, the digital strategy is very much central to the changes in the reforms that that service is looking to bring forward. Do you have confidence that the learning in relation to ISACs is, in fact, being shared not just within Police Scotland and the SPA but shared with partners in Crown Office Procurate and Fiscal Service? One of the impressions that we were given through that inquiry was that individuals were heavily involved in the digital strategy and understood it and were all with the detail. The vast majority of people were assured that this was going to unlock all those benefits but did not really understand how that was going to come about, which gave a bit of an impression of what appears to have happened within Police Scotland and the SPA that the detail is not necessarily properly understood or people have different interpretations of what it is going to deliver. Is that learning being shared across the public agencies? My starting point is that it is hard for us to say. As Mark said, the original business case both included that wider justice system perspective and had mechanisms for linking in to the key organisations that have got roles to play. The starting point was good. As the contract got up and running, the difficulties arose so quickly after the contract was signed in June 2013 that people's attention narrowed down to getting the system itself to hit the milestones and deliver as planned. We should not forget that the original plan go live date was August 2015, so very soon after the problems arose, we were butting up against delays to that date. If you step back and look at the new 2026 draft strategy, that is looking across the criminal justice system and the interchange of information about criminals, suspects, witnesses and all the different parts of the system that need to work together. What we do not have is any sort of plan for how that strategy will be delivered with IT as a central part of it. I think that it is hard to be sure that the lessons have been learnt more widely and that that would be an important action to take forward whatever the plan is for delivering 2026. Mark, do you want to? Just very briefly, we did not speak to external partners during the course of this audit and it is important to put that on record. Given our previous work in other areas, the justice sector is quite good and is quite well coordinated as a portfolio. The work of the Justice Board provides a really useful forum for bringing all the various bodies together. That does not guarantee that lessons will be learned and lessons will be shared, but certainly what we have seen is that it is quite an effective grouping and that because they are operating as a system, the potential for learning is probably quite good. In terms of the deficit, we talked about that in our previous session, estimated at around £180 million at the moment, but climbing. I think that the report very reasonably points to the political context in which those reforms are taking place. Now, while the relationship between Police Scotland and SPA, I think that we would all acknowledge that it is in a far better place now than it was when this project was getting under way. Nevertheless, I think that we have been asked to take on trust some fairly heroic assumptions at the pace at which Police Scotland feels that it is going to be able to turn around that deficit. That is combined with, even if it is not a big band IT project, but IT is very much central to the strategy going forward. Does that not create some concerns that, in order to try and hit timeframes that are perhaps unrealistic, we run the risk of not necessarily repeating the same mistakes, just repeating a bunch of different mistakes that end up with us in the same place as we are at the moment? I think that there is no doubt that the vision for using ICT in the 2026 strategy is a very ambitious one. That is not a bad thing. We should be ambitious about the way digital can transform public services. I think that policing is right for it because of the slow progress that has been made to date. Do you not caution us among the promise and over-deliver strategy, rather than an over-promise and keep your fingers crossed that you can actually hit the targets? What I would like to see in the report is a detailed strategy and plan for how ICT will support that change to policing across the piece, as well as how it will contribute to making the savings, which were an integral part initially of the police reform agenda. I think that it will be a challenge to do that. That is not to say that it is not achievable, but, in the absence of a strategy and a plan, it is hard to be clear how challenging it will be to do it, whether it is challenging and realistic or challenging, but really unlikely to be achieved in the timescale that is required. Are there lessons that can be learned in terms of the gateway process that was put in place to some question as to whether the waterfall approach was the right one, or whether an agile gateway process would have perhaps picked up on issues earlier and allowed either the project to be an agreement reached for the project to be abandoned or maybe fundamental changes made to it? Do you get the impression that that different gateway process is likely to be implemented this time round? I think that it is very unlikely that it would adopt the similar waterfall approach. I think that, as we have said previously, the agile methodology and a much more compartmentalised bit-by-bit approach to software development would be the way forward. The Government's gateway process to provide assurances at certain points and on certain elements of programme management, that system is still in place and still in operation. What we have seen from our wider work on the ICT programme is the Government putting in place an independent assurance framework that allows them to involve at earlier points in projects if there are any concerns or difficulties where an external partner might be able to help and smooth the process or provide a check. That is an additional layer of external assurance, external governance that is being put in place. I will go back to that point about the financial pressures that Police Scotland and the political pressures that come on the back of that. Do you believe that the SPA and Police Scotland are fully cognisant of the risks of responding to that pressure in a way that stores up problems for themselves down the line, either because they are being overambitious in terms of what they are trying to achieve overall or because they are being overambitious in terms of the time frame in which they can achieve those milestones along the way? To be honest, I think that that is a question in terms of whether their extent of awareness of the risk is perhaps a question for the SPA and Police Scotland. I think that the only point that I would make is perhaps that 2026, relative to some of the timescales, the development is actually not very far away and, given the scales of the financial pressure that we have previously seen underspends on capital budgets, that all ratchets up the pressure. I am sure that they are aware of the risks and cognisance that solve them, but I think that they could probably describe how they are mitigating those better than we could. How much of an impact was the pressure that the SPA and Police Scotland are under financially to make efficiency savings? How much pressure did that bring to bear on the fact that the contract was not or could not have been terminated any earlier? I think that we say in the report that we have a slightly different concern, which was this mattered hugely to the police because they had had the previous IT system failures that were referred to by Mr Finnie in his question earlier on. There was a real determination to demonstrate that they could get it right with this one. It was very important to Accenture globally as a project that they expected to be able to deliver and then to be able to win more business elsewhere on the back of, starting with the work in Spain through Scotland and further afield, so they had a stake in making it work. Beyond that, the political interest in the reform agenda and the particular operational problems that policing was facing at that point around things like armed officers and stop and search all meant that that optimism bias was sort of ratted it up, that people were so committed to demonstrating they could get this right, that perhaps the option of saying, should we pull the plug at this point, wasn't considered as seriously or as early as it might otherwise have been. Now, that's applying hindsight and it's always difficult to know whether there would have been a better time to do it, but I think our view as a team looking across the evidence available was that that may have got in the way of people deciding to terminate the contract as soon as they might otherwise have done. Going forward, will that almost very steep learning curve help the SPA in Police Scotland and whatever system is put in place to replace this? Because they'll be under far more pressure and they'll be a far bigger spotlight shown on whatever they do next. Will that learning curve help them to be far more aware of if there's a possibility that this might not work? We need to step back from this, we need to review what we're doing. Will that happen now, do you think? I think it's an issue again to explore with Police Scotland and the SPA. I'd agree entirely with the point that came from the exchange between Mr Stevenson and Mark Roberts that the process of going through this has helped the police to understand their needs better. I think the experience that this has been salutary for them, they absolutely understand the risks involved now, the challenges to turn that into a greater capability of delivering systems that will do what's required. I think we also say in the report that it's clear that relationships between SPA and Police Scotland are now better than they were at the time and the roles and responsibilities are more clear in formal terms. They're all positive things and this is still a very challenging and ambitious programme of work that will be required to deliver the 2026 strategy. William, was there anything else that you wanted to ask? No, I think that I've got a number of relatively short questions which hopefully will lead to lighter short answers. Was it understood what the whole life costs of the system would be because this is a system that might last a couple of decades? I don't know if they answered that question. I'm sorry. Right, well that's fine. Do we know who on the intellectual property that came from the development was it AccuCenter? Was it Police Scotland or Shared? Because, as the auditor just said, the supplier was looking to gain more business. I think that we would have to go back and look at the contract again and perhaps come back to the committee in writing on that. That would be helpful. Finally, similar area. In relation to the future maintenance of the system was the provision that a different supplier could take over the maintenance? In particular, was the source code the necessary makings of the system available by escrow or other means that another supplier could pick up and maintain it? The contract was for on-going maintenance by AccuCenter. As there are no further questions from committee members, can I thank our panel for attending today and for the evidence that they have given in a fairly short timescale? I think that we've covered quite a lot of ground, so thank you very much for your co-operation in doing that. We now move into private session and the next sub-committee meeting will be on 20 April. I'll suspend briefly to allow witnesses to clear. Thank you.