 Thank you guys for dealing with the delay of game. Thank you very, very much here. They've given me a clicker and a microphone. You guys are in lots of trouble. This talk's called Fearless DevOps, Committing to Code here, or we've all done it. We've all done it. Raise your hands if you haven't done it. You're a liar. I know you're a liar. So I've worked at a few places, associated content, Yahoo, Pearson, Ping Identity. Currently, I work at GitHub. I'm a senior database infrastructure engineer there. Deal mainly with Elasticsearch, MySQL. A few more things about me. If you're wondering why I look like a Super Butch 35-year-old, I have a really rare disorder called XXXY chimerism. I'm totally aware of this. I'm also a functional chocoholic chatbot therapist, and I have a love for high-commitment activities. That's actually me on the top there, completely botching that exit. Skydiving's a high-commitment activity when we talk about committing to code, right? You step out the door, you're going to hit the earth, dead or alive, quick or slow. Gravity is a constant as far as I've found. I have tested this hundreds of times. Gravity is definitely a constant. So when we talk about committing to code, what are we talking about, right? DevOps, there's a mantra. What's the mantra, right? Automate everything, everything as code. Everything is code. And that's a commitment we strive for. And it's this great idea. It's this really great theory. But like this year, I'm going to lose that last 20 pounds. It has real problems in implementation. Of course it has problems in implementation, right? Eating ice cream on the weekends is comfortable, really comfortable. Exercising and dumping carbs is super scary and annoying. Running things the way they are now is comfortable. Trusting your system to a bunch of code you communicate to through an erratic chatbot that takes things way, way too literally is scary. When we're trying to make everything code, there's a lot of rules to follow and there are many ways to break them. Don't get fired. That's a rule, right? Does that go through anyone's mind when they're doing a new project? Yeah? Don't get sued if you own your own business. If I lose all this data, I may have to talk to a lot of lawyers and not all of them will be mine. Don't get a bad rep as the person that doesn't complete their project or makes the wrong calls. Don't piss off your coworkers. And the list goes on. There's a lot of pressure in our professional lives and people think that I have it easy sometimes in my personal life and my recreational life but there's really only like one rule. And it's easy. I mean, it's kind of past fail, right? So bad things can happen in our professional life. Bad things happen to this guy. Don't worry, he's still alive. He has both shoes on. That's how we know. No, seriously, if you lose a shoe, you're totally screwed. So the weight of all these things in these pending disasters lead to fear of commitment, fear of commitment to our project, fear of commitment to our code. And we've all been there with that fear of commitment. These zombie projects that you start that you had the best of intentions and things went horribly awry. So when we think of implementing something new, our breathing quickens a little, our jaw sets, do we really wanna get involved in this thing again? And this becomes our default answer, right? No, I'm good. Got burned on the last one, right? Fear of new, fear of committing to a code-based infrastructure. This holds us back. We have our runbooks, we have our procedures. Some of it's automated, but not all. The stuff that's automated, maybe it's tested. Some of it, maybe there's enough tests so we can believe that it's tested, but we know in our heart of hearts, it's just not. And some of us has found this out, or maybe all of us. Sometimes that's worse than not even trying in the first place, right? And you're right, I'm not helping here. Now you're all afraid. This is actually the end of the speech. Go forth and do nothing. Live in fear. So how do we commit to our code? How do we control this fear of the unknown that stops us from fully committing to this code-based infrastructure that we're all dreaming of? So we're gonna break it down here, step by step, how to get to this DevOps promise land. However, first let's address this key foundational problem, us people. People are horrible. You will hear this in every one of my speeches. I'm horrible, you're horrible. People do stupid stuff and screw up. We get stressed out and we make bad decisions. Oh my God, I have to do something. I have to do something right now. When the answer is, you should probably do nothing. And you sit there and this cycle goes through your head like, oh my gosh, there's so many problems. There's feature creep, there's foundational stuff. There's this political battle. There's this person that always says that everything I do is wrong even though I don't know, maybe it is wrong. People are afraid. We get really afraid starting these kind of projects. But the problem is, everything we want is on the other side of that fear. It's on the other side of that chasm. We just gotta step off, right? And this isn't only true in DevOps, it's true in life. Who here is afraid of something in their professional life? Hands, hands, more hands. I sense lying. All right, no, no, keep those hands up. Hands up, hands up, hold your hands. There you go, show and tell. All right, no, no, you put your hand to get up. I saw that, take a glass, take a glass. All right, drink fear, what's your fear? Microphones? Yeah, human interactions, awful. Human interaction, I think that's a pretty common one in this crowd, right? Human interaction? Oh my gosh, you all put your hands down. Very bad at following directions. More hands, more hands, more hands. Oh, I like you. There you go, I'm gonna come find you. Yeah, you, you, don't look around. Van Gogh, Star Trek Van Gogh. Failure. Failure, failure is a really common one. Oh my gosh, what if I screw this up? Everyone will hate me. What if this project that's going to cost $3 million that I'm leading a group of people and I've hired four people goes down in flames? This will look bad. What if I fall into my friends at 120 miles per hour at 12,000 feet and hurt them? Actually, I do that all the time and they don't care. So, right, so everyone's afraid. I mean, I have a confession. I'm horribly afraid of heights. I cannot get on a two-story ladder to paint my house and I know this is very much like rain on your wedding day, ironic, but I truly am. So, how do I deal with my fear of heights? How do I deal with being afraid of this new project, of this new endeavor, of this new job? How do we get past our fear of rolling out this infrastructure when we think, I mean, we overthink it, right? This could define my career or at the very least, my current job. I mean, first day on my job, this job, I'm gonna deploy this chatbot script to list the size of the databases. It's the first thing people are ever gonna see. I'm so gonna be judged on this and they're gonna ask themselves, why are they actually paying me money? Maybe I should subscribe to LinkedIn, JobSeekerPro again, because I'm gonna need that, right? So, there's a little bit of a secret to getting past this fear. And the secret is one that like adrenaline junkies and professionals in life-threatening situations use before stepping into the void. It works really well when jumping out of airplanes. Works good when jumping off of antennas. Don't do that, it's stupid. It works in more extreme situations as well, like potentially start-up killing database migrations. We can't possibly restore this backup in less than 10 hours. I wish I'd made that safehold box. I'm an idiot. So there's three steps to controlling fear. Relax. When we get fearful, we get stressed. And I know that's like the worst thing that someone can say. It's like when you say chill out to somebody, it's 100% guaranteed someone will not chill out. And when you say relax to somebody, just relax. They will get more tense. Never works. But when we get fearful, we get stressed. When we get stressed, our heart rate goes up. Our neurochemistry changes. We make really, really bad decisions. I have personally seen people make stress-induced decisions that have been life-ending in front of me, because they did not control their fear and they panicked. We can't control our heart rate. We can't control our neurochemistry unless someone is a Buddhist monk that wants to give me enlightenment here. We can only really control one thing when we're stressed, our breathing. And it's super cliche, but it works. Hit the pause button, breathe. Changes our neurochemistry, we calm down. Controls our heart rate. Our heart rate and our breathing are linked. Especially if we're not undergoing strenuous exercise. And it gives us two things, time and space. Time and space is everything. It's everything in sports, it's everything in extreme sports, it's everything in devops, it's everything in life. The more time and space you have, the better decision you can make. And you always have time to breathe. You always have time to breathe. Does anyone think they don't have time to take a break? They've been on that, come on, yeah? Okay, you always have time to take a break. I can guarantee you that you have time to take a break. Every weekend I have about 25 seconds left to live and I go, and I save my own life. Before I make a large crater in the earth and my fiance becomes very, very rich. The benefits of group life insurance through your employer, ladies and gentlemen, they cover stupid things. So I'm certain you can take a breath when you're really stressed in the meeting, when you're coding, when you're project planning. Make some time and space for yourself. It's your time and space. Play a game, take a walk. Do whatever you need to do. Take it when you need it. The second thing is focus. Pay attention to what's actually happening right now. Not what you want to happen or what could happen or what could go wrong. When people focus, they're usually focusing on what they want to happen or what can go wrong. Not what's actually happening now and what the immediate action is to take. Pay attention to the reality of the situation and deal with it. Seriously, when you're focused, don't overthink it. Overthinking the problem creates worry and fear. And worrying is praying for bad stuff to happen in the future. It is even worse than not doing anything. It is actually like the black hole of productivity. Nothing happens when you worry. The third step is to flow. Be in the moment. Now that we have a workflow and had to handle fear and perform, these guys are performing. Arizona Airspeed, best four-way team in the world. They're not thinking about anything else. Mikhail's a good guy there, but they're flowing. And we need to flow. We need to think about exactly what we're doing and eliminate all the distractions. So we've got this workflow on how to handle fear and perform. So this doesn't sound like a self-help seminar and it has something to do with DevOps because I think that's why we're here. Let's go into what steps we need to take to actually committing to our code. It's also becoming really apparent that two cups of coffee was a little overkill up here. So don't overthink it, right? We need a few basic ingredients to commit to our code. We need a way to store the code. That'd be nice. A place to store the code. A method to know that the code works. A method to deploy our code. Code to actually deploy that does something. You notice how far this was down the list? Everyone starts with the code and then moves on. You need to lay the foundation for the code before you write the code, otherwise you just wind up with nothing useful. And a method to coerce people into actually using the code we've written, which sounds terrible, but we'll get to that later. And most of all, we need not to overthink it. So first, we need a method to store the code, right? Alexa, show me a version control system. No, wrong, also wrong. Now you're just trying to piss me off. There we go. Yeah. Place to store the code. Maybe if you Google around, you can find a place to store code and work collaboratively. Order some takeout, get a repo, have everyone sign up. There's lots of places to store your code. They're all good. All ASs that you use get, please, please, please don't use CVS. If you're using CVS, there's an intervention at the bar afterwards. We will help you. It's 2017. Now that we have a place to store our code that everyone can access, now that we have this playground to experiment, we can commit, merge, we can work with people. That's awesome, right? Next, we need to check in our code. We need to check our code, actually. Continuous integration. And we don't want a bike shed, right? Jump into chat ops. You should be using chat ops. It is also 2017, please. Ask everyone what CI systems they know. Whichever one pops up the most often, use it. Don't debate this for three weeks. It's not rocket science. It's probably Jenkins or Travis. So the goal is to get here, automatic merge, empower anyone to create a merge, a PR, get our review, and ship it. Frictionless, easy. So we've got that, right? We've got code, we've got a place to store the code. We've got a way to check the code. Now we just need a method to deploy the code, right? People are thinking like, sweet, I got this. Wrong. This is not the way. We need to talk to more than one server at a time. We also need to do something super important, right? We need to leave a record of what we did so that our team knows what's going on. Eliminate tribal knowledge and embrace scroll back. Use Slack, use Hip Chat, use IRC, I don't really care. Scroll back should be definitive for what has happened in your environment. I have documentation that's written in markdown, but 90% of the time I search in Slack to see what someone else did, and I look at their workflow. If you're thinking it, it should be in chat so people can go back and see what your thought process is, which is kind of scary, especially at 2 a.m. in the morning. But scroll back should be definitive for what has happened in your environment. So set up a chat bot. I'm really partial to HuBot, but there's others out there. You can roll your own if you want. Don't overthink it, just do it. Something is always better than nothing. If you don't have Hip Chat, Slack, et cetera, again the intervention is at the bar. We will help you, we will help you. Now we just need to ship it. Empower people to ship code so that you can ship your code right after CI runs, and maybe get a review depending on how important it is. Once CI completes, if it takes anything more than this in chat ops to deploy your code, I'm really sorry, but I have to say this. Your system is broken and your process is broken. That's all it should take if it doesn't happen automatically. If you can't trust your CI, it's broken. If you can't roll back quickly, it's broken. If you can't trust your engineers to deploy to production, we have a more foundational problem. It's a different talk or a consult. Now that we can ship something, we need to ship something, right? And here's where things go horribly wrong every last time, every time. Oh, I'm going to find you. Oh, I see you, I see you right now. The person nervously smiling. You're wearing blue Colorado shirt. I got you. Pick a language. Russian. No, no, pick a program language. Python. Python, okay. Anyone agree with that for the coding project we're going to do? No? No, anyone disagree with that? Everyone loves Python here. Ruby, okay, there we go. You, you spoke up. Actually, what framework are we going to write the project in? That's not a framework. You can make a repo though and make it a framework. What framework are we going to use? Let's do Rails. Rails. Anyone disagree with this? Anyone hate Rails? 500 people and there are no Rails haters. Liars. There are at least 10 of you out there. Okay, there we go. There we go. Brave man there. Brave, brave man, right? So let me tell you a secret, like, this is a huge debate. You start a big project, you're going to like turn this monolith into microservices. You debate over three months, what language it's going to take, what framework are you going to use, and it just becomes this huge nightmare. So here's the secret. No one cares. Two years from now, no matter what language you use, no matter what coding framework, people are going to be huddled around a monitor. They're going to be huddled around a monitor and they're going to say stuff like this, I guarantee you. Hey, look at this project. No wonder this person doesn't work here anymore. They wrote it in Ruby. Oh my God, we need to completely rewrite this, right? Right. And if you're there, we're going to do this. We wrote this in Python, there's a bunch of new programmers coming on board. Why did we do this? We don't know. I know, I don't know either. I was drunk at the time. We made this decision. Um, we need to rewrite this and hide our shame, right? Correct. All right, I'm going to order some pad tie and let's get to this. This happens, I've done it. A lot of people here have done this. No one cares. Jump into chat ops. Hey, what language does everyone know? Oh, Ruby? The project's now in Ruby. What language does everyone know? Oh, most of you know Python? It's now in Python. Congratulations. That's how we pick it. That really is that simple. Don't let someone write it in Scala when you're the only person that knows Scala. You wind up owning that project and then you wonder and you sit there and go, why do I have to do everything with this project? Because you're the only one that knows the language and no one can attribute. So that's it. Pick something then. Automate it. Something super small. Like listing your databases. This is a piece of redacted code that I've done. Thor. What language is this? Really? It's Bash. It's just Bash. But it works, right? Everyone can read it. Everyone can contribute. No one really cares. No one's griping to me and saying this is in Bash. I need to rewrite it in Spring. It took 10 minutes. Lather, rinse, repeat. Pretty soon we have hundreds of tools we can use in chat ops. Now we just need to get people to use them. How do we get people to use them? This is the bad part. Remember we talked about coercion, which is probably in the employee handbook under something you should never ever do. Coherse. Coherse them. Brow beat them. But Jess, this isn't very nice and it'll make people feel bad. And you know what, yeah? Do you know what makes me feel bad? It makes me feel bad when you SSH into a server that you don't need to. Cause I don't have any scroll back on that. I don't know what you did. And when that crucial production database disappeared and you didn't notice and you went to sleep, I have to do some archeology and I don't like to do archeology at 3 a.m. in the morning, especially after Friday night. So at GitHub we have this really robust implementation of chat ops. It also means if you shell in a machine, everyone gets an awesome Slack message saying, Jess Breckner, it's shelled into my SQL server. Then everyone says, hey Jess, why'd you shell into my SQL server? And I go, well, I wanted to list the databases. And they go, you know I have a tool for that. You can just go like, dot my SQL databases cluster. And I go, well, well, that would have given everyone a history and let them know what I was doing. I should have used that. And if the tool doesn't do what I want, which is a frequent excuse, it doesn't do exactly what I want. If it's in a shared collaborative infrastructure, the next statement is, well, that's great Jess. So we'll see a PR in an hour or two to reject those databases, right? Oh yes, absolutely. And then the tool gets better. One second, lost my train of thought here. So, and this is very empowering, right? If you have stuff in a repository up there in a language that everyone knows, they can contribute. They can contribute to that and they're empowered to contribute. So here's a really awesome story. Yeah, that I can share. I went to work at GitHub and you know you go to work at a new company and there's always something wrong with like HR, there's little things that don't work or some documentation that's out of date. And six months later, you're staring at this documentation getting really mad and you're like, I still don't know how to sign up for healthcare benefits because this was three portals ago and I forgot how they said to do it. And I really don't know how to get credentials in these servers either. Shove everything in a repository somewhere so people can edit it. I just went and checked out the repo, forked a branch, updated the instructions and got in. And at the end of that, I'm like, wow, I'm super empowered. I can fix all this stuff. It's like not only is dev ops version controlled, everything's version controlled. This is way awesome. So people use these tools and people get empowered once we have this whole scaffolding. Does that sound cool? Yeah, I guess not. So I think it works pretty well in my experience. So we now have a place, we now have a way to store code, right? We picked a version control system. Everyone picked CVS, yeah. Perforce, great choice, great choice. We have a place to store code. We picked one of the online services out there or we've gotten like an on-prem solution if we work for a three-letter agency or a bank. We have a method to see the code works. We've picked like Travis or Jenkins, which is really awesome. We don't have to manually go, I hope it works. That's just so 2003. We have a method to deploy the code where we're using a chat button, it just goes out. Or we have continuous deployment and it just goes out. That's another talk on whether that's good or not or right for your organization. We have code that actually does something out there. And we also have a stick to prod people to say, please use the code, please use the code. And leave it in scroll bag and leave it in chat ops. So everyone's got that, right? And before we take on this project of devopsing all of the things, we can control that fear that we have. We can relax, we can breathe. We can focus on the immediate issue that we're solving and not pray for that bad stuff to happen in the future that we like to do. Because we love to do it, I love to do it. I'm a worrier and it's so hard to stop. And we can flow. We can be there for the moment, we can be there for the task and not for anything else that we're doing. We can just eliminate all those distractions. Now let's go forth. Let's sprinkle some devops on it and just chat ops all of the things. Let's move beyond this. It doesn't mean we're not afraid. It just means we move beyond the fear. And we use that simple scaffolding method to avoid bike shedding and all the other pitfalls that we've looked at. And it's awesome. So that's all folks. How much time do I have? Yeah, as much as I want. Awesome. Yeah, I have a microphone. This is where it gets dangerous. So I'm gonna open the floor to like a general discussion for the remaining time. This prezzo is usually pretty quick so we should have 10 minutes or so. On anything? I have an opinion on everything, so ask. I am required by the legal team to let you know that GitHub disavows any of my opinions that are controversial. Yeah, come up here. I don't have a second microphone unless they have a second microphone somewhere. So I like the idea of having a little script that'll go out and chat everything that you're wanting to work on. But does that not get to be a lot of overhead after time, especially as you have more and more scripts and some of them end up just a little bit different or lots of flags or whatever that ends up looking like? You need to organize it decently, make a directory and make the shell script small. Don't create like this monolith service that has every chat command like list-databases.sh, list databases. You can also create separate rooms. Like we'll have a room for databases or for Puppet and then we'll have a room for database bots and all the database bot chat goes into database bots or operations or whatever. And then it's segregated there. You can also call that bot from within the main room if you need to really get people's eyes on something but otherwise it's just in the bot room, so. And then you can scroll back and see what the bots have done, especially as automated bots. Like I committed to our operations management system and it comes back and says, oh, your CI totally failed. Well, everyone can go in the bots room and go, yeah, your CI failed. Click, why? So it knows. No, no more questions, 500 people. I will find people and make you ask questions. No. All right, thank you very much. I'm Jess Breckenridge and this has been great. Thank you.