 Thanks, Ken. I think it's quite appropriate for me to follow up after Nicole talking about measurements because audit is probably the classic example of the measurements we hate. Just to tell you a bit about myself, I've worked for ThoughtWorks now for the last five years. My background in IT came out of quality assurance and as a certified information systems auditor and certified in risk in IT. There's actually a certification for that. I live on Vancouver Island today, best place in the world. Pacific Northwest, the world's best kept secret. Co-author of Lean Enterprise with my colleagues at the time, Barry A. O'Reilly and Jess Humble. I've spent the last two years basically talking about Lean Enterprise and how Lean Principles can be applied in technology to help us get better at what we do. Actually, I see it as probably one of the few ways that we're actually going to survive all the changes in the fast pace we're going through. I just want to say a little bit about my experiences. I came out of healthcare. My degree is in home economics. I am two years away from retirement and my goal is to go out in a blaze of glory. I'm going to say some pretty controversial things today. Let's start talking about audits. How many of you have actually hidden under your desk when you see the auditor come onto your floor? I was the auditor. I knew you were hiding. I would actually pretend to go and write a post-it which I would stick on your monitor and look down and go, oh, there you are. We all dread them. What exactly are they? Why do we have to go through that pain? First of all, they're meant to be independent usually by a third party. It may be an internal party for the organization doing an internal audit. It's an examination and that's probably why we all hate them because we always feel like we're being measured and we're going to pass or we're going to fail and we usually know we're going to fail for many different reasons. They should have a defined scope. This is the audits that you experience today in IT are usually a result of some sort of compliance obligation that the organization has and the compliance and the regulations says you shall be audited by an independent party once a year and the findings will be reported through to the board of directors. And that leads to some sort of report and usually recommendations that come out and if you've ever had to read those recommendations, you know that it's not good for your blood pressure most of the time. So that's basically what they are in a nutshell. They're not meant to be a measurement of pass or fail. They are intended to say, are we doing things going in the right general direction and are there ways we can improve? Why we do them, I already went through a little bit about the compliance part. Root clause analysis, when there is a specific problem, people know something's wrong, but they're not quite sure what it is. Sometimes they'll call in auditors to have a look at things who have specific expertise in forensics, for example. And they need a level of assurance. Sometimes the board of directors say, are we on the right path? And a good example of this is when I used to work for an airline and we decided we were going to build our own reservation system because we were so freaking smart, right? Let me tell you, that is very, very complex thing to do. And after two years and still having nothing to show for it except extraordinary bills, we did an audit on it. And the net result was, let's just stop this nonsense and move on with our life, get a reservation system that is in existence today and implement it, okay? But the goal of audits really should be to reduce risks to the organization, okay? That's the bottom line. And when we talk about risks, it's really important for us to remember about the use of technology. It used to be, back in the 90s when I started in IT, is that the technology was the bottleneck for the organization. It took a long time to develop software and implement changes. Today, that's not true. We have technology platforms. You guys are here, you know what capabilities are. I'm not gonna go in and explain it, but it's changed dramatically over the last, I'd say, seven years, right? What the bottleneck is today are these traditional management processes, the finance, the risk and compliance people. The way we procure things, even the way we reward people within the organization and our HR policies, those are the things that are creating the bottleneck for the organization to move forward and to actually deliver value to the customers. Now, the audit community, they're still stuck here too. We have a problem that sort of sums it up. And what happens is that because of the advances in technology, cyber crime and breaches to information have escalated at a correspondingly rapid pace. So the natural reaction of the business community and the old command and control structure that we use to manage particularly large organizations is that I'm just gonna start putting in more and more blockers to prevent these bad things from happening. And guess what happens is that we land up that we can't make decisions. We've got conflicting department goals because of the way we're siloed. We've got conflicting priorities and we're risk adverse. The natural reaction is always to say no. So you have customers, we have technology that can deliver really great things to our customers, but we get this wall of no from these other parts of the organization. Legal governance, risk compliance, finance security all have huge obligations, external obligations that we have to meet. And we're using audits based on old technologies. Here's a dirty little secret many of you may not know. Most IT auditors never worked in IT. They're accountants, Nicole. Yeah, and that was a natural progression of IT. A lot of IT and the automation came through finance. So accountants became familiar with IT at the time and applied a lot of their principles and practices to the auditing of the use of IT, okay? That's just historical. Things have changed, but I pose a question to this community. What are we doing about that? What are we doing as technologists to change that perspective of auditors? And right now running and hiding under our desk is not helping. So what is the biggest risk to the use of technology in today's business? Anybody care to call out? Do you know what it is? Ideas? Cats? Checkboxes? You build the wrong thing. Anybody been there? Secondly, you build it the wrong way. All right, got a good thing. But I'm just going to build it my way and customize it so nobody can use it. This is the risk that we have to talk about when we're talking to auditors. So there's a lot that we have to learn about the way they think, the way they speak, the way they do their jobs. They have an awful lot that they have to learn about how this new technology works. Four years ago I was living in Melbourne and I spoke to a group of auditors on continuous delivery because as continuous delivery practice lead for thought works at the time. And I, Jaws were like down on people's chest because they just went, oh my God, how will I ever audit this? I don't know how to audit this. I think you just couldn't wrap their mind around the automation and the capabilities of it. They're still at that today. I was just reading in an Osaka journal which is the ones that the information systems auditors use, a certification body for all auditors. And they go, how to audit an agile project? That was new and breaking for them, okay? I just, DevOps, continuous delivery just blows their minds, right? So the thing is, what I was talking to them about was that you have different methods, you need different rules. Some auditors are stuck in this mode where I've got a very slow moving vehicle on a road that's very seldom used. Others are in this situation where I've got extremely complex organizations with many different parts, all the corners, intersections looking different. Some are very slow, some are fast, some are quiet, some are very congested, some are not. And the rule set that I have for these is very complex as well. And it's a nightmare to audit, right? And I don't understand what's happening all the time. And then when we show them this, they just freak out, okay? We're on the race track. Yes, if we get it wrong, we're gonna die. But we also know that we need to be very, very careful, okay? So the thing is that if you're going at a very fast pace, there are things you must do as a technologist to ensure that you're reducing the risk to the organization. And it was funny when we were with ThoughtWorks and Continuous Delivery when we started for starting and we had something we called, well, I won't say it, but anyway, it was, there was a difference between continuous delivery and then adding the whole lean concept of the N10 flow where you had experience, design, and experimentation and feedback. We said, all the continuous delivery allows you to do is shovel stuff faster. It doesn't tell you whether you're shoveling the right stuff, right? So as technologists, we tend to focus on doing stuff faster, but we never stop and think about, is it the right stuff? Okay, now some of us are changing that when you get experienced designers in with your teams, but more to the point, if your organization has an obligation to external regulations, that's part of the stuff you have to shovel. Okay, so how do we resolve this conundrum? We have people pushing back against your efforts to move forward at the paces that you feel is required. The first one is that we need to create a common understanding, and I was at DevOps London last month on a panel, and Gene Kin was on the same panel, and he said something very poignant, he said, you know, the likelihood of an auditor learning the technology is much lower than the likelihood of a technologist learning the language of the auditors and the risk and compliance people. So my challenge to you guys is, you know, how many people here have ever actually looked at the legislation for SOCs or HIPAA or PCI DSS? Have you looked at the controls? There's a few people here, you know, kind of makes you quake in your boots sometimes, but, you know, you look at it and you go, well, most of this sounds reasonable, and the thing is that it is reasonable. Most of those tell you what you have to do. There's no indication of how you should be doing it, with the exception of PCI DSS, but get me going that on different conversation, because that would be another half hour. So my advice to you, first, learn the language, learn what they're talking about, because every time I wish I had a dollar or every time I said, we have to have this because of SOCs, and I'm going, no, give me a dollar. I don't know how they come up with that, but someone somewhere said, you know, we're doing this because of SOCs, and nobody really questions the intent. What are we trying to achieve with this control, you know? Basically, it's reduced risks of bad things happening. But if you can prove that a specific control doesn't actually meet the intent of the whole legislation, then you have a better basis to have a discussion about what is an appropriate control instead. Provide some documentation. OK, I know, I know, it's sad. If you can automate the documentation, that's great. If you've got tool sets that will do that for you. If not, you know what, just take the day, put together a little slide deck that says, this is how our team works, OK, geared towards a sixth grader. Because that's probably the level that most of your auditors are going to be at when they come in the door, particularly a new one. They need time and they need to get up to speed with technology and how your teams think and operate. And create that visibility and transparency. If you don't understand this, you're too stupid and I don't have any time for you. I've actually had somebody say that to me. That type of arrogance, immediately you're going to go, uh-huh, OK buddy, let's talk. Because I can make your life really miserable, right? Don't try to hide anything. It's not a test. You're not going to fail. Nobody's going to fire you because, you know, just say, look, we've got problems. We know we've got problems. Here's the things we're trying to do to improve it. Mind your access. Oh my god. Things I've seen. OK. Basics, basics, no account sharing. OK, pair one, pair two, pair three on my code base, not acceptable, right? I need to do this thing with access when we talk about access is the 3As. Authorization, oh no, authentication. Are you who you say you are? When I have pair one, I don't know who the heck an individual is who's on that pair, right? Then you get into authorization. What are you allowed to do? And then you get into accountability, which is who did what, when? That's what we think about when we talk about access as security people and as auditors, OK? And we're going to look for that, OK? Prove to me that you're actually doing that. I'll say in today's technology with the authentication, two-factor authentication is better. Temporary tokens are better. It's just like in everything, everywhere, if you can. Least privileges? And that's basically, if you don't really need to be in the back end of a database, you shouldn't be, right? Don't give everybody all access just because it's convenient, because that's the last thing they'll probably make the auditors dig deeper and make your life more painful. And this is about making it less painful for you. So if you just do these simple things, it gets a little bit easier for you. They go, oh, yeah, OK, I think I trust these people. They seem to know what they're doing. Guard your secrets. Keys. I don't know. There's no good way to manage keys across teams, I don't think. But do the best you can. There's many different ways you can do it. And I'm not going to get into all of them because I only have 12 minutes left. Guard your pipeline. When you're doing DevOps, when you're doing continuous delivery, the point of failure is your pipeline. Now, usually as devs, we're pretty lax about loosey-goosey, about who has access to do what, within the code base. All of a sudden, that includes all my tests and the results. It includes the configurations, not only on my dev and test environments, but also on my production environments. So all of a sudden, there's a lot of information in there that needs to be protected. And some people don't realize that has to be protected too. So not only do you have to guard it, you have to make sure that you have it backed up someplace. Because if you lose your pipeline, you're dead in the water, right? You need appropriate approvals. And the key word here is appropriate. Don't send everything up to the highest level. That is just a nightmare to manage. And it just slows you down. And really, it's like I was saying at dinner last night, it's like, when my boss used to go away on vacation, I go, have you left your Steve stamp, right? Because I'm going to have to approve something for you while you're gone. And then the other thing is I would bring papers and documentation to him. He said, should I sign this? I go, yes, you should sign it. I didn't have a clue what he was proving. So it's perfectly acceptable for one team member to approve another team's member's access, as long as, again, you have visibility and transparency into how that was done. Know your information. If you've got sensitive information, it needs to be protected more than nonsensitive information. There's all sorts of ways to get around that. But one of the first things you should ask when you're starting to talk about a new system is, what is how sensitive is this information? How does it need to be protected? It's one of the first questions you should be asking yourself. And oh, golly, clean up after yourselves. Finding stuff everywhere, GitHub, Treasure Trilogy, what they call it, the Disneyland for secrets, cyber criminals going in there, keys, passwords, configurations to production systems, everything. It just shouldn't be out there in the beginning. But you should probably be trolling through and seeing what your team is leaving behind where and cleaning it up after yourselves. So mind the access, provide evidence. The good news here, you can automate this. And when I was taught to be an auditor, here's the one thing they told me. If it ain't written down, it doesn't exist, all right? So auditors will always look for evidence. Use the audit features on any tool sets that you have. Turn on the logging. And again, protect your logs, because they're very important. So most organizations would say, move all of the logs to a centralized log server that has very restricted access. Because the first thing a hacker's going to go do is change logs if you haven't protected them. And then you have no forensics to figure out what the heck did happen. Use a risk-based approach to change. This is quite different than what most people have experienced in change management, where every change has to go through a change advisory board and be approved, and it takes anywhere from three days to three weeks, right? But you need to define levels of risk. And we did this when I was working at the airline. And we said, and we were actually using something similar to a pipeline, like that was eight years ago, okay? But we'd say, you know what? There's risks that, or there's changes where it's, there's no interaction with other systems. Very low risks. It's not in our financial world. It's not in the dispatch world for the airline. There's no regulatory and compliance issues with this. It's low. And we're gonna manage that automatically through the pipeline. And we'll probably have some peer review and stuff going on with it. But basically, the team's gonna manage those and make a decision, whether it's a good or bad thing. Medium, a little more interaction with other systems, okay? And probably require some coordination between two to three teams, okay? Those actually probably want some peer and product owner review and approval. You know, without, again, having to go to a whole cab to get it reviewed and changed. Oops, sorry. And then the last one, hi. Okay, huge, complex systems, lots of interactions. That needs a senior manual improvement saying, you know what, this is essentially gonna change the way we do business within the organization, right? Now, if your team knows about the obligations and knows about all of the, you know, risks associated with the information and acts responsibly and accordingly, you can move more and more changes from high to medium, medium to low. But you have to prove that you're able to do that, you know, get up and walk before you start running. And as you get a benchmark and more evidence and data that your team is making good decisions, then you have a better basis for the conversation with your risk and compliance people and the auditors. Like, this is an appropriate way to do it. So the last one, be disciplined, be reliable. Oh goodness, it's not all about speed, people, sorry. These are from Google. The top one in Google was certify everything, but I don't know that everybody here needs to certify everything, but they actually say instrument everything. All your builds have to be traced back to the source. So who made this change on the build and where did it come from and when was it approved? All production code is peer reviewed. Okay, this is how Google works with a single code base for all of Google, right? Stop when the build breaks. Oh my goodness. Times that go in and they go, tell me about testing and they go, do you have automated, yeah, do you run them? Yeah. What do you do with the results? Well, usually it doesn't work because they're testing something wrong with the configuration and the test environment. And you go, could you not just stop and fix that? You know? Well, no, we just keep building. And I go, that's sort of defeats the whole purpose of continuous delivery and DevOps, right? We're just gonna keep going. It's not about the speed, it's about the quality too. And in order to make it work, you have to be disciplined about those types of things. Manage the tests. Okay, it's not about the number of tests as Nicole, you know, in the conversation we're having. No, it's about the coverage and it's about the results. Okay? And are the results, are we measuring the right thing in the results? It's not about the numbers at all. Okay? And you will find that as you progress and your product gets more mature, there's probably some tests you can drop. There's more tests that you're gonna have to build in because as you add new features, there's probably more different risks involved in it. And then audit the build chain. So those are the things, those are the basics that every team at Google actually has to manage. And then there's a little bit more. So performance measurement. So again, when you're talking with auditors, it's not about, did I tick all the boxes? Okay, did all your boxes get ticked? Because that approach does not reduce risk. And you only have to look at the number of organizations that have been PCI DSS certified and how many credit card information breaches there are reported every year to know that that doesn't work. So measure things that matter. What you're really measuring, what is the outcome we have achieved? And talk in terms of risks. Don't talk in terms of the technology and the speed and like auditors are concerned about risks. You gotta relate everything back to the risk to the organization and the risks to the code base itself. Have we reduced the complexity of our technology? Unconvinced that complexity is going to be our downfall with the use of technology. It's getting out of control and there's no way that we can manage it unless the data scientists can figure out a way to help us. Quantum computing is scaring the bejeebers out of me. Now watch what the auditors do with that. And then are we getting better? That's the key that auditors will be looking at. Are we getting better at reducing risks to the organization? And risks are not just about can a bad guy come in and do something. I mean that's important but that's not the only one. Okay and the last one, take responsibility. So know your business. What is it our organization is trying to achieve and how do we fit in with that and what we're doing? Know the risks, okay? We talked about that and seek frequent feedback and act accordingly. Don't keep doing the same thing over and over thinking that if I just do it harder it's gonna get better. It doesn't. You need to change the way you work. You need to change the way you think. And when you're dealing with auditors, think about risks and talk about risks because that's what's gonna resonate with them. So in the famous words of Jesse Robbins, don't fight stupid, don't run and hide under your desk, don't have those passionate discussions and behind closed doors, just try and make it better. That's the bottom line. So I'd like to thank everybody. I'll be here till noon probably, a little shortly after. I have to run back to home, but it's been a pleasure and I always welcome the chance to talk about audit when it's particularly to people who are heavily into technology because it's such a painful thing for many of us.