 So I'm Katie Prizzi at Kay Prizzi on Twitter. I don't do Twitter a lot. Every time I come here, I say I'm gonna get better. So hopefully I'll get better. But if I don't, sorry, it's me, not you. Here we go, okay. So that's me. So I have a background in DevOps strategy and implementation consulting. I still don't really know what that means, but I went to companies and told them what they were doing wrong and then tried to help them, which I think is what we all do. But then three and a half years ago, I went to HCSC, which most of you will know is Blue Cross Blue Shield of a couple different states. And there I'm a DevOps tooling analyst. Please no one get mad at me for having DevOps in my title. I know that that's a huge point of contention. My title before that was Web Applications Development Specialist. So I feel like this is a real improvement. I live in Forest Park, Illinois, which is just west of the city, the end of the blue line, not O'Hare the other way. And I'm a speaker, Noob. So this is my first time speaking at an event like this. Thank you. It's very nice of you. So be nice to me. But also this is the third time that I submitted a talk to an event like this. So the first two were rejected. So for anybody who has submitted a talk before and we're not accepted in here or otherwise, keep on submitting because sooner or later, they get tired of saying no and say yes to you. So just quickly, what is HCSC? This is standard corporate slides, you know, because it's a lot prettier than my other slides. So Blue Cross Blue Shield of Illinois, Texas, Oklahoma, Montana, and one more I'm forgetting. We have a bunch of employees. So when Jeff asked yesterday how many employees we have, I was the one with my hand still up. So pretty big organization, lots and lots of people, hopefully a lot of people like and use their health insurance from us. If not, sorry, I don't know, call the help desk. But so it's super interesting about the healthcare arena in the last, I don't know, five years or whatever, is that we've all gone through a pretty massive transformation. So healthcare companies used to be like big old dinosaurs, right, that kind of did the same thing every year, didn't really have a need to change because everyone just renewed their group health insurance at the end of the year and everything was cool. But Obamacare really flipped all that stuff on its head. We needed to get to market faster. We had more people, more members using the insurance, more members submitting claims, more members calling in to the help desk, lots of stuff kind of all of a sudden, right? I don't know if anybody remembers, but when Obamacare came out, none of us really knew what was gonna happen. And so it was kind of this, we really need to change the way that we do things. And one of those things really was IT. IT was operating as this old dinosaur and that wasn't good, we were losing market share, losing people to our other competitors, competitors were bringing better apps, better products to the market faster than we could and things were not good. So we did a massive IT transformation, we hired a new CIO, new CTO, totally re-ordered the whole organization. But really that focus was on a couple of things. One is bringing development closer to the business, so trying to operate in a more agile way, getting, trying to get away from waterfall, trying to bring in best of breed employees, so the best technology people you could find, trying to attract them in to this new and exciting and changing place. What else? Oh yeah, and system stability and reliability, that's important. If you submit a claim and our claim system is down, that's bad day for all of us. So that's really important, now that we're bringing more and more people in. We also had a huge amount of tech debt, no one really paid attention to tech debt, so when you don't do that, it keeps on growing. And we needed to be less risky, which makes sense, we're insurance. So from my department's perspective, this is kind of what we looked like as the old dinosaur person. So we're doing yearly or maybe twice a year deployments at most. We were tracking requirements in emails and spreadsheets and post-it notes and sticking it on people's desks. We had eight version control systems across the enterprise. One of those accounts like a file system on some Rando server somewhere. What else do we have? Oh, people were doing Eclipse-based build, so right clicking in Eclipse and pressing build, no one had automated build scripts, which makes it really hard to implement like a CI CD pipeline. What else? Only a couple people were using code quality scans. So remember I said we had a huge amount of tech debt that was only on the apps that were actually using code quality scans, which was, I don't know, like 70 apps out of 500. So there was an over 400 apps that we had no idea what their tech debt was. The deployments were really fun. So either we're a big web sphere shop, so either you would build your ear file locally on your laptop and then you would put that into a folder somewhere that a web sphere admin had access to and they would pick up that ear file and then deploy it to the server for you, which was really fun when you used a folder a lot, so you had five ear files in there and it was real confusing what your file do you want deployed? I don't know, let's see what happens when it comes up and then I'll let you know if that was the right one. Or we had drop folder deploys, so you put it in a folder. We had a cron job that ran every half an hour. If there was a new file in there, it would pick it up and then it would deploy it to web sphere. So if anyone has deployed to web sphere before, you know that it takes JVMs, like 20 to 30 minutes to come up and down, which means that an app team wouldn't know if their file actually deployed for an hour. And we had very manual audit reporting. So once your audit would come and they would say show me all of your stuff and we would spend weeks or months printing things out and handing it to them and booklets and not doing, right? So all of these things very manual, super, super slow, not very cool. So with this giant transformation, we're moving people from Waterfall to Scrum or even XP development and we improved on all of this stuff. So quarterly, monthly, or even weekly deployments, so massive difference from annual or bi-annual. We actually gave them an issue tracking system, so things are all in one place and tracked. We reduced those eight version control systems down to three. None of those are local file system, so we made people take all their code off of there and actually version control it. We gave everybody Anthem Maven automated build scripts, which again, I think kind of sounds not so impressive, but think about where we were where we had maybe 50 applications actually using build scripts. So to be able to give everybody build scripts and have them do automated builds was a really huge win for us. Code quality scans for all. So everybody got code quality scans. Now at least we know what our tech debt is. And then we built automated pipeline-based deploys, push button deploys or triggered on a commit and source control that deploys all the way down through your SDLC all the way to production. So again, went from a company that never deployed to production in an automated way that was always a manual. Here's my year file. Was admin, please put it here. To push a button and it goes. And then with all of that, we built in self-service audit reporting so you could track from your requirement to what code was changed, to what build happened, to who pushed the button to deploy it to production and also what change order number because we're a highly regulated environment. What change order number went along with that deployment? Okay, we're doing okay on time. And because it's a big company, we did all of that work at scale. So 400 applications, over a thousand developers. We developed training for everybody. So everybody went through a two week agile workshop where they learned how to create epics and stories and put them in backlogs and all of those cool nice things that everybody talks about. We created computer-based training so that with all these new tools that were rolling out, people could take classes at their own time and learn how to use these tools. What else? Oh, and then we had instructor-led training so if those CBTs weren't enough, you could go talk to a human and ask that human questions and get answers and more insight or whatever that you were looking for. And then also, we hosted monthly lunch and learns. So come into a room, bring your lunch, and we'll talk about anything you want starting from what's an epic, what do I do with it now that I have one? So what's a branch? How do I branch? After I branch, how do I merge that code back in to, I don't know, I put in a change order number and it's not working. Why? So kind of all those things spanning all of that time. So that whole project I just talked about took a long time. It was a really big project. We started in January 2016. We literally just wrapped up like two weeks ago. And we're all feeling pretty good about ourselves. We have over 100 applications deploying to either UAT or production using that full traceability system. So to go from zero to 100 in 20 months seems pretty good. We trained 1,000 users, all 400 applications are on the systems. Customers are reporting that they have 50% improvement rates which I don't quite know what that means but hopefully we're gonna figure that out later. And people are also reporting that they used to take them three to four days to deploy. So I guess you start on a Friday night and you end on a Monday morning. That sounds awful. To a couple of hours. So yay, big wins for everybody. The thing is, is that this wasn't all nice rainbows and puppies and cookies. So let's take a step back to February where we're starting to feel really good about what we're doing. We've been doing this project just over a year really starting to hit our stride and have found out some things early on that we were able to fix as we were going along. For example, no one knew how to write build scripts. That was a big deal. But like we're really hitting our stride really feeling great about ourselves. 200 applications are on boarded and by on boarded I mean like a scrum master signed off on yes, we're using JIRA. Yes, our code is in source control. Yes, I see that you've given me my pipeline and I've deployed somewhere using it like things are looking good. And so then we said, well cool, we wanna see how many people are actually using the system to deploy to production, right? So I had someone create a report and they came back to me and that was the number, eight. Eight people were deploying to production using this system that we engineered the crap out of and trained everybody and was there for questions and were open for problems and we built a support team just to support anybody who needed stuff and we had eight people. I'm like, man, this is not good, I'm gonna get fired. This is terrible. What's going on here? And I couldn't figure it out, right? I checked all my systems, all my systems look great. I don't, I literally didn't know what to do. So as always, you learn things when you go to parties. So I was at work and we were having a Mardi Gras party. So we're eating king cakes, we're eating babies or having Virgin Hurricanes because we're at work and we're a professional. And I'm talking to this lady who I've known for years and ask, I don't know, just talking about random things. How was your weekend? And she's like, oh yeah, Friday was real tough. It took us 16 hours to deploy. Isn't that awesome? What great teamwork. And I was like, yeah, for sure, to get people to work together for 16 hours is good, but why did it take you 16 hours to deploy? Like did a system go down or like what happened? Was there some massive error? And she's like, no, this is what happens every third Friday or whatever. We take a 16 hour day and that's how long it takes to deploy the applications that I support. And I said, this is awful, get out of here. Forget these babies, forget this cake, forget these 16 hour deploy days. For sure, this job sucks, get out. You're too smart for this business. And she said, no, I love it. This is a badge of honor for me. And that phrase like really, really stuck with me. It still does. I can see her saying it, like a little bubble's coming out of her mouth. And I see the words like badge of honor in the air. And I'm like, honor, like why is, why are we proud of this? Why are we, there has to be a reason. It's not because she doesn't know like what's up in the world. She's just a really smart lady. So like what's happening here? And I think I've come up with it. And I think what she was really saying is this is how I'm valued. Or rather, this is how I value my work. And my value at this company is that I stood at my desk for 16 hours on Friday until this deployment is completed. And that is the thing that I do here. So I think that may be an answer to why we only had eight people coming on to my whole program, to my whole system that me and my team created. So I started thinking about this more. And I'm like, well, well, wait, if you find value in working 16 hours, doing something that these tools can do in like two hours, why is that valuable? And I think maybe this answers it. So I call this, this is where DevOps went to die, right? This is really what we missed in this whole system is that if inefficient work, lots of manual work takes lots and lots of hours and we reward people for working long hours, right? We stand up and we clap for them on Monday and we're like, you worked all weekend, good job. And then people are never incented to automate anything, right? No one gets claps for saying, I only worked two hours on Friday. They get claps for saying I work 16 hours on Friday. Okay, time is good. And if we're not incenting people to automate and people don't want to automate because they think that then there's gonna be less value in their job that they're gonna bring less value, then no one is ever going to adopt this DevOps culture that we're trying to push forward. And if no one adopts DevOps culture, then we're gonna continue to have inefficient work and long hours and just kind of this continual circle. So can we attack that thing? Can we somehow stop rewarding for long hours or change the way that we reward employees to kind of break this whole cycle? So here's some of my ideas. So one, we need a culture of hard work is bad, knock it off, stop doing all of that hard work. I asked you guys to be nice and now you're being too nice. So how can we make that culture, right? Get people to work less, to want to work less, to not celebrate these really long hours. So one, I think, and especially with this woman, recognize that fear is a real thing. People are scared that automation is going to get rid of their jobs. And so that's not to say, well then let's keep these manual jobs so that people aren't scared, but recognize this is a real fear and this is what's motivating some people to behave in the way that they want to behave in. So let's get people excited and involved in this automation. The way that we work is we have a DevOps team that kind of gives DevOps tools and processes as a service. And maybe that's not quite right, the approach. How do we involve people more in what we're doing so that they feel that they're a part of this automation culture? So one, let them choose. Let them find something in their day-to-day jobs or in their neighbor's day-to-day jobs, I don't care. That's ripe for automation. Who best to find areas for improvement than the people who are actually doing the work every day? So let them choose. It gives them a feeling of investment into the process and if they improve it, then their lives are already better. Let them learn new skills. So if automation is all development or scripting or whatever, I don't know, then let's teach people those things if they don't know them. It's okay to not know, but we're all real smart people. That's why we're in IT and we can learn this stuff. So go to classes, give them time, maybe on a Friday afternoon for self-lessons, whatever, it doesn't matter to me, but give them time, let them learn new skills. And if you need training, set aside some budget. A lot of this stuff isn't all that expensive and like I said, a lot of it is free on the internet. But set aside time, give them a Friday afternoon to work on their skills. And then also make it a part of employees' goals. A lot of you maybe don't work as at such a formal organization as I do. I don't know. But I have bi-weekly sit-downs with my manager. We have very formalized mid-year review and then end-year review where I have to write down my goals and then I have to report on them later. So let's do that. Let's write it down. Let's work with your manager, make a plan. And then when we do all those things, reward them. And recognize that they've set a goal, they've achieved it, and now let's celebrate. So you could do it with days off. So it's kind of the opposite. Where I work, we give people comp time when they work long hours. So people are insented to work long hours. So let's do the opposite of that. Let's incent people when they save time, when they save their own time. Give them some days off. If your company can't do that, maybe you have a very strict PTO policy, give them a bonus. This doesn't have to be five or 10 grand. This can be $50. This could be $200. This could be a $5 Starbucks card. I don't know. I don't care. People are motivated by different stuff. So find something that's appropriate. Maybe if they do a little automation job, maybe that's worth $50. If they do a big automation job, maybe that's worth $500. Superstar list for team meetings. This is the most motivating thing for me that's free. Is when someone stands up on a Friday and they say, you know what, Katie really helped me this week. And I was doing this thing that was this long, weird manual process. And she fixed it. And I want to nominate her as the superstar of the week. That happened to me once. Do you know how hard I worked the next week? Every time that team I am to me, I was like, yep, you got it. Whatever you want, no problem. Or like company swag, t-shirts, stickers, stuff that's like sitting around, probably in a storage locker somewhere, hand this stuff out. And again, it's different for everybody. Not everybody is motivated by the same things. But find ways to reward people for doing this work because it's really valuable. It's really contributing back to the company and people like stuff. Ooh, another way is give them more satisfying work. So give them awesome stuff to do. Now that you're taking away manual work, give them really, really neat things to do. Yeah, so that helps with the automate myself out of a job mentality, right? If we took the 16 hour deploy away and they only took two hours to do it now, what do I do with my 14 hours? Do cool stuff. We're all in IT and I think we all got into IT because we like this stuff. We like creating new projects. We like learning new things. And I think sometimes when you work in a job for so long, that stuff sort of gets lost, right? Like you're just focused on your day to day. You're not really thinking outside the box sometimes. So this is a great opportunity to have employees kind of relearn how to have fun in IT. So yeah, a new app that needs some smart people, put them on that. This person's real smart. They just automated a thing for you that wasn't automated before. Give them a new thing to work on. Give them the hardest project you can find and see what happens. Let them be creative. If they have some side project, give them some hours to work on that side project. Maybe it'll turn into a new app for your company or maybe it'll just make them real happy. Either way, I think we're all doing okay. Or let's pay off all that tech debt. So now that we're doing all these cold quality scans and we're learning about all this tech debt that we're accumulating, let's spend some hours paying that down. And then when you pay down some of your tech debt, give them a price because people like prices. Okay, so four minutes left. I can't believe I'm on time. This is working out really, really nicely. So if you remember nothing else, one, build time into your project schedule for retros and continuous improvement. This was a huge thing that we did not do. I knew that I needed to do a retro. I knew that I needed to do continuous improvement. I've come to DevOps days for three years. I hear these words all the time, but when it's not built into a project plan, you forget about it sometimes. So build it into your project plan. We had a, again, formal company, so we have a formal project manager and a project plan that's in Visio. I don't even know what that is. But build it in there. Make sure that you make time for it because if I discovered this eight number, six months into the project, rather than a year into the project, then my adoption rates could have been even better and I could have figured this stuff out earlier. People are your core. So again, I engineered the crap out of those tools. We're so shiny and perfect and beautiful and no one cared. And why? Because we had a people problem. So people are your core. You can do the best stuff in the world, but if people don't wanna use it or people are scared to use it or they have a million other reasons, why? They're not gonna do it. So realize that you gotta talk to your people first. And then last thing is that more people involved equals more DevOps awesomeness. So it doesn't have to be just your own little team. Get people to contribute to your projects. Get people involved. We all know that diversity of thought equals better thoughts. So get more people involved. It makes them happier. It makes them feel like more part of the process and it only makes your stuff better. And I think that's it. Thank you so much everyone. So most DevOps implementation strategies always talk about fixing the people problem first. I mean it kind of sounded like you did the inverse. You built kind of a technology platform runway and then solved the people problem out of because you had to. Yeah. But would you like, do you agree that that was the right path to take now or do you think solving the people problem first before you built a tool pathway would have been better? Yeah, so that's a really good question. And in short, maybe. I think, so I think in my situation we probably should have just done them in parallel. I think the real problem was that we really were operating on some real old tools that did not support these kind of scrum agile things. And so for sure the tools had to change. There was kind of no question about that. But I think we definitely shouldn't have built all these tools and tried to roll them out and then a year into it look at our people problem. I think that we would have been much better served if we had looked at our people problem right away and tried to address both. So when I've done similar things in much smaller companies my people problem is always that no one wants to do the magical thing that I just came up with. Someone wants to use this tool. Someone wants to use that tool. Yeah. How did you get everyone on board? Well, so one, I get this was coming from top down so that makes my job a lot easier. Two, I owned their other tools and so I turned off their access to the other tools. I can't use it now, sorry. But also I do think that like the luncheon learns I think we're very helpful. I think people want to use other stuff because they're comfortable with it and or they like it better and most of the time they like it better because they're comfortable with it. So how can we get you to be comfortable with this new thing which is what you're supposed to be using and there's good reasons for it. And I think if they really have a real reason why they want to use the other thing let's talk about that. Let's figure out what you like in that other thing. Could I put that in my thing? Does it already exist in my thing? How do I make you comfortable enough to want to make that move? That's how I would approach it. Hey, Katie, thank you for your presentation. So there's a spectrum of people when it comes to change. On one side you'll see the change champions, the change agent and on another side you'll see what we call toxic employees who are very resistant to change and they both have huge multiplier effect. So I wonder as part of your strategy did you identify or try to identify change agent, try to partner with them and did you identify the people on the other side of spectrum and developed kind of response to their position? Yeah, so that's a really good question and I think what ended up happening is so we had two pilot apps and we picked those because we thought those are gonna be our change champions, right? They're gonna be really, really happy about these new tools we're giving them, blah, blah. It turned out that was not the case. They were real sad about the new tools for a whole number of reasons. And so one, I would say don't pick the hardest app for your pilot, pick something that's not the easiest but like somewhere there in the middle because if you're picking the hardest app you're gonna mess it up the first time around and then you're already creating these voices of resistance, right? We went to this new tooling and it sucks and I don't wanna do it and blah, blah, blah. That spread's like wildfire and that's terrible, that's super hard to put those fires out. But I will say I guess then we ended up getting lucky like the fourth or fifth or sixth app into it where then we kind of started to hit our stride, started to learn some lessons from what we had done before and then people started to get happy with us and satisfied with the tooling we were bringing them on and starting to come out with some of the metrics that I showed of like 50% increase in productivity or whatever and so finding those your champions early on I think is like super, super crucial to rolling this out across to a mass audience because the good news or bad is gonna spread and it's whatever people hear first. So that's what I would say is if I could do it over I would have picked way less hard apps so that the first kind of experiences were really, really positive instead of us learning mistakes, whoa, sorry. Us learning mistakes as we went. All right, everybody give her a round of applause.