 Welcome to the Agile Pune 2014 conference. This is our first day of the conference and I'm going to hand it over to Josh For the keynote. I know we are running late by 30 seconds. So I don't want to eat into his time Josh and I worked together for over three years and we've known each other for maybe seven years now So it's a real pleasure to have you Josh over here to address Agile India community and I think this is your first visit to India. So must be special It was his birthday yesterday. So, you know, if you see him wish him belated birthday as well All right over to you Josh. It's on. Thank you. No rash. Thank you very much. It's a real pleasure to be here. I'm You don't know this but I live in Berkeley, California and I eat a lot of Indian food So coming here is like, you know, really amazing for me I've had an awesome time. I've been traveling all over the country So it's been phenomenal and I've been to your Taj Mahal. I have had your Italy in the south, you know in Bangalore So I've really Had a great time Let's get started. So I'm from the Bay Area of San Francisco and This is the famous bridge from, you know, that area. There's another bridge called the Bay Bridge And that leads you to my area of the world, Oakland and Berkeley But the San Francisco Golden Gate Bridge is really the most iconic bridge in the area world famous and I'm probably boring you to death about this bridge But there's a reason I'm talking about this bridge because we're going to actually discuss the construction of the Golden Gate Bridge Okay Joseph Strauss was the chief engineer and By that you should know his original design for the Golden Gate Bridge was rejected and they said, you know You're a great guy, but you're gonna have to invite some of your competitors in Collaborate with them and come up with a better design. So he did that and Basically it started in the 1930s this this bridge construction the 19 early 1930s was the depression in the United States So jobs are hard to come by people were on food lines It was a tough time And a matter of fact at the same exact time the Bay Bridge the Oakland San Francisco Bay Bridge was also being built And so that's what it looked like back then now their view of it as they started building and There's something that happened on this construction project Joseph Strauss said I am going to have the most safe construction project in the world. This is going to be Safety is going to be my the thing that I get known for so he basically went overboard He had people wearing helmets. They didn't really have helmets for workers back then they had helmets for miners people in the caves But they had to basically, you know adapt them for bridge construction They got creams for people because they'd get windburn up there way high up in the sky, right? They would make sure that they were harnessed so they had a harness that they're attached to and if you were caught without your helmet Or without your harness on you could be dismissed You could be let go and there were a line of workers ready to take your job this was incredible what he did and It really led to an incredible result, which was that they were For a thirty five million dollar project Okay, back then the average Fatality rate was one death per million dollars spent So you kind of expected to have thirty five people died during the construction of the Golden Gate Bridge It didn't happen. It took almost four years to build and For most of the four years they had zero deaths Now on modern projects today. We tend to have zero deaths Okay, but back then it was commonplace For the Bay Bridge, which was being built at the same time There were 24 deaths Now, what do you think it was like being way up there above the bay? building this thing What do you think they're thinking when they're looking down? I'll give you a closer view fear, right? I Mean you're way up there. You can fall to your death easily It's scary The thing is These guys They weren't scared Now you could say well, maybe they were men of steel They had you know all the confidence in the world, but actually they didn't they had something better They had this They had this net this was unheard of back then they spent a hundred and thirty thousand dollars Back then that's depressionary dollars. That's a lot of money back then Okay to build this net and the net caught a total of 19 people They were called the halfway to hell club and they Almost died and basically the net caught them so they were Really well protected on this project and so for the majority of the four years no fatalities Then a tragedy struck They basically had a platform with something very heavy on it it Some bolts came apart and it fell through Straight into the net Taking men with it and it went right through the net broke the net and plunge 12 men plunge down Into the waters 10 of them died to survive And there was one other fatality. I don't know how that happened, but in total 11 deaths And it was almost a perfect safety record up to like the last few months just before the bridge opened so We can learn a few things from this Safety is freedom from unacceptable risk Okay, it's freedom from unacceptable risk This is an important thing that I discovered which is that much of what we do in Agile is dealing with risk When we look back at how we built software over the years We can say there was too much risk in the old way we used to do things in The new way we're managing risks better Because some of them were unacceptable Now what is acceptable and what is unacceptable is based upon context right in a small couple of three four person shop Certain things might be completely acceptable in a giant corporation. They would be unacceptable This may be perfectly acceptable in India and In other parts of the world it would be completely unacceptable now. I have spent probably 16 hours in cars in the last, you know 10 days. I went to my suru. I went up to you know, Agra I Seen a lot and I really only saw one accident And everyone seemed okay So, you know, I'm not gonna judge this You know and these people were you know five people on one motorbike And you probably see this commonly, but they seem to be doing what they're doing Right. No one was getting injured. Now. I don't know your your rates But Naresh tells me people don't tend to go as fast as we do in the United States We have plenty of fatalities there Context is everything when it comes to knowing risk. Okay. Are we in the software field? safe Do we have a net like those construction workers on the Golden Gate Bridge? I Think we really don't I Don't think that the net is there for a lot of what we do So that's what I'm going to talk about next All right In our own e-learning software that we write we had something interesting occur. We had some production exceptions 237 exceptions we found in the logs surrounding screencast video downloads and things like that and We were a bit surprised to see this It was disturbing so we went and looked at the code. Here's what the code looked like And this is not going to be a quiz about you know what you can find wrong with this code or anything What's wrong with this code is not the code so much Right. I'm not that concerned with it. What I'm concerned with is The IDE There's a problem. The problem is it should look like this Right. We've had some accidents. We've had a near fatality In the code we've had 200 and some odd exceptions severe exceptions occurring and when we look at the code on our Developer machines, that's what we see Looks perfectly fine, right? It's just cold. There's no indication. There's a problem and yet There's a major problem happening. There's a problem. It's in production Why is it that we have this broken feedback loop between production and our developer machines Right severe exceptions occurring in production and we don't even see them in development That means we're not really working in a safe way. It means there's no safety net there It means we're not prioritizing safety, but wait a minute. We're going to get back to that in a minute Prioritizing the safety that was my earliest thoughts about this and I've come to learn that you don't prioritize safety This means that when a product owner comes to you and says we want to build this feature You say Sure, you know, we'll build the feature. You don't say wait a minute product owner. Look at this Look at this problem We got a major problem here. Do would you rather work on that or you want to work on your new feature? I think we need to prioritize safety I'm going to get deeper and deeper into why But remember this Golden Gate Bridge. They they finished it one point five million dollars Ahead of schedule and under budget one point five million they saved and they were ahead of schedule Incredible they said the workers said we were able to work without fear now Let's let's go deeper into software though. We're not talking about brain. We don't do bridge construction. So We see this kind of problem occur all the time, right Sure and utter exhaustion at the end of a project Health problems all kinds of issues burn out and Maybe the manager is celebrating, right? This happens all the time, right That's not a safe way to work. You can't do that all the time and be safe You're gonna lose people or you're not gonna get the best of people. There's no safety net there We build things that no one uses. We don't even instrument our software. So we don't know when that product owner says This is gonna be the best feature ever and we go ahead and we build it. Maybe we even finally crafted and Yet we don't even know if it's being used. Is that safe? I don't think so We have stuff like this that we produce complex SQL and It works, but maybe only one person knows how to actually deal with a multi Five six joins outer joins left joins right joins It's a problem, right? There's only one guy. We lose them. We lose her That's it. No one knows how to work with that code. We have crazy branch problems Branches and branches and branches emerging unsafe oftenly very unsafe We have 28 page functions Single function 28 pages long gigantic legacy deck People are afraid to touch this code We come to work and we're afraid. We can't even change our own code because we're afraid of it We have production outages right where Everyone goes crazy. Everyone tries to fix it. We're up crazy hours trying to fix it. It's it's a disaster We have this problem Amy Edmondson is a Harvard professor And she said that we are impression managers. We are all impression managers. What we do is we want to manage People's impressions of us So that means if we're thinking something Perhaps we disagree with somebody We're not going to say it necessarily because we're afraid of what they'll think of us if we voice our opinion right, I Like to you know, I know that your your form of yes here is this kind of thing here And I think sometimes what you really want to be doing is this Right the kind of negative. No, no, no, I don't agree with you It's a problem when the culture doesn't allow us to speak our minds because we're afraid We don't they don't no one gets the best of us. We don't do our best work Are we are we comfortable to express our feelings and thoughts? relevant thoughts and feelings is This person afraid to speak up is raising a dissenting view expected and encouraged Right We often want to have diverse views of something if we're going to really build something great We want to hear what people think but if you don't say anything because you're afraid you don't get that benefit so The the lack of safety is not just technical the lack of safety is also in the culture It's also in the way we interact with each other This guy productive conflicts challenges and disagreements again, can we have those? Or are we afraid to challenge people are we afraid to disagree? Can we be ourselves? In a lot of places you can't So I want to introduce you to mr. Paul O'Neill Paul O'Neill is An incredible guy He led a company he led several companies But the most notable thing he did was you ran a company called alcoa Alcoa is the aluminum company of america It's a global company all over the world. They have they make aluminum products They invented the aluminum process the aluminum smelting process back in 1887 Okay invented it The process of you know getting the precious metals out of the earth and turning it into aluminum That you can actually use to build things with and today this macbook pro here has alcoa aluminum in it and mining airplanes and cars and soda cans and All kinds of things are made with alcoa aluminum worldwide Paul O'Neill did something very unexpected as a ceo of alcoa Okay, first off. He was hired a ceo in 1987. That's a full 100 years after the company first started Right, there are 100 year old company. How deeply embedded do you think their culture was after 100 years? That's scary I work with companies are 20 years old and i'm a little worried that their their culture is going to be impossible to change Paul O'Neill took a company with 100 years And began shifting it What he said was I want to make alcoa the safest company on earth And he said I want to go for zero injuries zero injuries Wall street heard what he said at an opening meeting and said this guy's a nut job And the stock brokers went to their clients and said sell Sell sell the alcoa stock. It is going to be a disaster. They have a crazy guy Now as ceo who's talking about worker safety Ugh Worker safety He's not talking about revenue. He's not talking about new products. He's not talking about not paying taxes He's talking about worker safety What an idiot And a lot of people did sell the stock And it was a huge mistake for them Let's look at what happened first off This guy was serious about Worker safety. He was not it wasn't like, you know, some CEOs have what's called the flavor of the month This month, I'm interested in this next month. I'm interested in that. No This was core He realized that everyone in alcoa could agree on one thing We want to be safe So we have things here called recordables a recordable means if you get injured And you have to go to a doctor then you have to record it right if I get a mosquito bite No big deal necessarily unless it's you know This is a gang a fever or whatever you guys call that But if it's a recordable you have to write it down fine. So they write these things down And if it's so bad that you can't come to work it's a lost work day So they have a lost work day index This is what happened from 1987 to 2000 Right to their rates Paul was there from 1987 to 2000 Okay, so we had this incredible reduction in injuries Across the entire company Paul and Yale visited every plant And talked to all the workers and said if something is unsafe and you notice it You have the power to fix it. You have the budget You've got the authority you can fix that problem And if your manager gives you a hard time And doesn't let you or puts you in a very unsafe situation And you can't get out of it. You can't fix it. Here's my home phone number call me So a guy in Tennessee actually called him a worker There were conveyor belts three of them Right moving heavy equipment the middle conveyor belt broke The manager had the workers moving stuff from the first conveyor belt To the third conveyor belt And one guy said you know what? This is really hard. I think I'm going to hurt my back And the manager disagrees wouldn't wouldn't you know relent kept making them work And so the guy called Paul and Neil I think it was 11 p.m. At night for Paul And Paul said stop doing it Have the other workers stop Paul calls the manager calls the plant manager Stops all that work and says get the conveyor belt fixed And don't make the workers do this Now a story like that spread and there were many other stories like that many other stories. I love one of my favorites A guy used to have to climb these buildings And shimmy to like difficult places just to read a meter on some You know machine It was dangerous work when it rained or it was icy or snowy. It would be very dangerous Somehow another I know whose idea it was maybe his idea Maybe his colleagues they got him a pair of high-powered binoculars And he could just basically read the meter right from the ground. It was faster. It was far safer These kinds of things happen across the entire company They said if a problem happens we want to fix it once so that it would never happen again at any plant And that they said was a major advantage. This was something that was a competitive advantage so What do you think happened? In 2000 when paul left left paul left he became the treasury secretary of the united states of america What do you think happened any guesses What Did it stay this way? Did it go back up? How many of you here think it went back up because he left Joe hans How many think it continued or stayed level? Okay. Oh, okay well actually He left And it continued it continued to decline He wasn't this kind of leader who was uh only effective when he was there. He changed the dna of alcohol Absolutely changed it more, uh, you know statistics, uh These are just incredible things that that happened there Revenue. Yeah, if you sold the stock You were really kicking yourself because it just basically climbed and climbed and climbed and it continues to climb today Yes, there are little You know recessions and areas where they go down, but for the most part it's an incredible stock One quick story. They bought a aluminum smelting plant in russia It averaged five fatalities a year It looked like this very unorganized You could come to work and you could die Uh after Alcoa took it over even in the first year Zero injuries zero deaths Incredible safety record and that continued year after year safety Is not a priority. This was my first early mistake I said safety must be a priority and I talked to paul o'neill and I've talked to paul o'neill's colleagues And they said josh, it's not a priority safety is a precondition a precondition It must be present It's not a priority because priorities change It's a precondition. It's much deeper than a priority That's how he looked at it Okay All right. I'm going to play something now for you. Um, this is something that uh, I've had the great fortune to interview paul o'neill and Study him and one of the things I discovered was that He has these three questions Which you know, if you're in an organization and you ask people these three questions Uh, you can learn a lot and you can move really towards safety towards making safety a precondition So we ask these now in our own company my company industrial logic We ask these three questions and i'm going to play for you the three questions Okay, I'll try that again. I'm going to need that sound up Try it again This doesn't guarantee it but has them Okay ready So in the organization, I believe that has the potential For greatness doesn't guarantee it but has the potential for greatness The people in the organization can say every day without any reservation or hesitation Yes, the three questions. Here are the three questions Uh, I'm treated every day with dignity and respect by everyone I encounter Without respect to my gender or my nationality or my race Or my educational attainment Or my rank or any other discriminating qualifier That's question number one Treated with dignity and respect Every day We ask this in our own company and this was the answer we got Okay, and we're starting to ask this pretty regularly now in a survey And I show these numbers to Paul and he said not bad, you know, there's there's work to do but not bad Second question I'm given the things that I need training education tools encouragement So that I can make a contribution This important now That gives meaning to my life Make a contribution that gives meaning to my life given all of the things that training All of the education all of the understanding and support To do work in such a way that it gives meaning to your life These are these are Pretty high bars, right Apollo Neil is not just any old leader. I would think it was a maverick. He's an incredible leader Doing okay there too Still some work to be done And you know these things change there could be a bad day You know and then the numbers will change So it's not like no these things are static So the third proposition is pretty simple. It says Every day I can say someone I care about And respect noticed I did it In a word it's recognition regular Meaningful sincere recognition So you're you're recognized regularly for your work Appreciated for your work These are things that you know We don't you don't necessarily think about as we're building software And yet these are the questions that paul asks And says that if you want to be habitually excellent Habitually excellent not just occasionally excellent, but habitually excellent Then you can answer yes to all three questions. Imagine that All three questions, right? Treated with dignity and respect regardless of your standing your education your gender your Anything always treated with dignity and respect Proud of your work proud of your work in a way that it adds meaning to your life given the training and the support And everything you need to do the kind of work that adds meaning to your life And getting recognized for it getting recognized for the work you do on a regular basis these three things paul says lead to habitual excellence And these three things are all about safety paul says i'm most interested in habitual excellence and the only way we get there is through safety And that brings me to anzen Anzen is a japanese word and The word basically means physical safety even in japan most people who think about safety think about it in the context of our physical safety i'm Broadening that term a little bit to mean complete safety psychological safety Safety in the way we build products safety with our customers will be keeping our customers safe Are we keeping our shareholders safe? Safety across the board I like the term because i've had problems with the word safety When i use the word safety in some organizations many of our clients Not only do they build software they build big hardware things In fact, they're very Focused on safety around the hardware that they build some of them build for example tractors and The tractors are Difficult to build they're dangerous. It's dangerous. They're just like building a bridge. You can hurt yourself So in the united states, there's an organization called osha the occupational safety and hazard administration That kind of monitors whether or not you're you know protecting your workers, but that's physical protection There's no osha for software And so when I started talking about safety and software and hazards in software they said no no no you can't talk about this with software We could get sued We could get sued by someone for putting them in a hazardous coding situation I know it sounds funny, but I said fine. I won't use the word safety. I'll call it on zen So on zen just being safety and On the nearing is the process of Of making safety part of your everyday work Okay, so I have these six shields that we produced When we thought about what are we protecting and who are we protecting? We're protecting A bunch of things, but there's six core things that we tend to be protecting and that would be money Time Information people's information Um reputations Relationships and our health And that's the case whether it's the workers whether it's the customers whether it's the shareholders And so that's what we're protecting back in um the ancient Days of battle when um the trojans were you know doing battle they had shields And the shields were considered to be the most important Piece of equipment you could lose your helmet you could lose your breastplate Okay, it happens, but if you lost your shield you were in big trouble You're in big trouble because your shield not only protected you it also protected the others It was considered a major problem if you lost your shield you would be punished So these shields work together to provide safety On the nearing the practice of protecting people That's what it is It's not this This is over protection. We have this uh plenty in the united states now with uh new parents Kind of over doing it over protecting All right, this is uh the this is not real safety This is And this is a problem too This is a story from I think the 15th or 16th century one of the big battles between the french and the english And at the time the english Uh were still not wearing heavy armor, but the french had the armor that had reached its zenith of of uh heaviness It was really really heavy It was so heavy that if they fell off their horse during battle they couldn't get back up So they were free game at that point to just be captured or killed Um the protection was actually making them unsafe So you can go way too far with this stuff Uh, they just built a portion a rebuilt a portion of the bay bridge where I live rebuilt it Right it was not considered earthquake safe And it cost a ton of money and it took a lot of time Because their approach to safety now is very different than it was then there were zero deaths in building this span this modern span but There were environmental impact studies They they spent 11 million dollars on a way to make sure that no fish were injured when they were drilling the new Pile irons into the ground No fish were injured I mean You can look at all that they had to relocate certain kinds of birds that were going to be injured It's pretty intense what they're doing now You could say they've gone too far and that's why it costs Way more money than it cost back in the 1930s It's all about context what real safety is so Here's one of my daughters. Um, we're at a climbing gym in berkeley She is incredibly safe. I have a harness on I am Trained in how to do this in fact in the gym There are people walking around looking at my harness and making sure it's on correctly One time I had it on backwards and they told me I was embarrassed She's got a harness on she's roped up. She gets up there and swings around. She's perfectly safe She's safe to fail. She's safe to fall. She's safe to take chances Safe to have fun There she is up in a tree right um My wife took that picture and she was there at the time and thought, okay You know, she's safe Not taking risks is risky Right, it's riskier than playing it safe If you don't take risks You know safe risks You're getting into a very risky situation Starts with your competitors So We're talking about genuine safety here. Not this is not all defensive Right. We're not living in a cave It's It's both offense and defense when it comes to real safety There's a partnership between production and protection. This is one of the icons or one of the Big great thinkers of the safety world There are experts in the safety field and these experts actually work across industries Kind of the way I do and some of you do we work building software in many different domains many different industries Well, these guys work in safety across many domains And james reason notice something. Here's what he noticed If you basically Spend all of your time and money on protection You will go bankrupt If you spend no time on protection and safety You might have a catastrophe Something goes really wrong And so you have to find some kind of a way to navigate that middle line there The parity line So a lot of us start out with a certain amount of safety But over time we stopped thinking about safety and We see that That's fine until a problem occurs And once the problem occurs we say oh, no, no, this is no good. We have to be safer. And so It happens again And again, we go up and say no need even more safety And then all over time the organization forgets and then we have a bigger problem This is this kind of constant Approach we have to safety. It's not something that Like polo meals company Alcoa it wasn't really in the dna. It wasn't really considered a precondition It's just a priority. Then it's not a priority. Then it's a priority that it's not a priority, right? Very common when it comes to the tension between Production and protection This swiss cheese model very famous same guy james reason invented this We have defenses And sometimes holes develop in those defenses And then we have a problem. We have a problem if all the holes line up And a failure occurs This is from the space challenger explosion And you probably can't read all that stuff, but lots and lots and lots of problems swiss cheese model analysis said Yeah, there were defenses in place and they were compromised Safety is the presence of defenses not the absence of events This is another expert in the safety field todd conclin Safety is the presence of defenses He wrote a book called pre accident investigations where they're starting to actually Do a lot of safety work before the accidents occur We see a lot of mixed messages in companies Right you say one thing And you mean something else We value your safety. Oh, but you know work into 2 a.m. At night. Don't write any tests Don't speak your mind and get the job done But we really value your safety. We care about you Mixed message, right? Say one thing do something else Mixed messages, they manufacture mediocrity and mistrust They're really a problem in our organizations If an injury occurs Right this fellow here. He's working on a band saw. He injured himself. What happened? Well, gosh He's working on a band saw on this this rotating saw and a something hit his eye and now he's got an injury What an idiot why isn't he wearing any glasses? Why isn't he wearing safety glasses? Hmm Let's investigate Oh, you know what the safety glasses are all broken They're broken. They're old. They don't work That's why he's not wearing them. There's no culture of even wear. No one wears them anymore Why's that? Let's look a little bit deeper Oh Management leadership, they're kind of focused elsewhere right now. Maybe they're focused on Increasing revenue or they're focused on competitors or they're focused on other things. They're just not focused on safety Remember that tension between production and protection? Well, it's gone back to focusing on production So boy This is interesting. We can say that there was an active failure the guy hurt his eye and we could blame him But we could also say wait a second Rather than blame him. Let's look back and try to understand What were the latent conditions? What were the conditions that led to this active failure? Etsy anyone ever hear of etsy etsy is a site for selling arts and crafts people make sweaters and They make beautiful pictures and all kinds of different arts and crafts and selling worldwide It's done very very well great company And they're doing really well, but they had a problem their their site went down They hired a brand new programmer. It was his I think Eighth day on the job And his eighth day on the job. He did something really unfortunate He deleted the css file for internet explorer It went to production because they do continuous deployment And when the server tried to bring up that page It couldn't so it tried to show the error page Problem is the error page relied on the same css file So it went into an infinite loop And this whole site went down Oh Eighth day on the job. What do you think happened to him? Anyway, what happened? What's that? Fire He was given the three armed sweater award This is the award for the most spectacular, you know, uh Problem that occurs But it's a reward He was patterned on the back and said thank you Thank you for finding a hole in our defenses You just made etsy a safer place because we're going to make sure that You know, this never happens again. The error page will never be, you know It'll be an independent page static. It won't rely on other pages Thank you for doing that. You're welcome to the company Here's your reward Etsy is a place that has a blameless culture They do blameless retrospectives twice a week Blameless, it's very clear. This is a blameless retrospective. No one is going to be blamed We're always trying to find always trying to learn from Failures and increase our defenses Todd conklin against it never place a worker only one defense away from failure This is an incredible saying Never place a worker only one defense away from failure That programmer at Etsy was one defense away from failure He deleted that file and boom There was a problem Let's increase the defenses there Let's use this as a way to decide what needs to be improved Where do we need to increase our defenses these days in my company? We look at Areas where we're not fault tolerant areas where there's a single point of failure Right so that we can sleep easy at night. We know we're protected Other companies like net Netflix and Various other companies are basically doing chaos monkeys and things that intentionally knock down servers in production to see that everything works correctly They're making sure their defenses are present They're not blaming each other This gets you nowhere When something goes wrong and your culture is focused on blame It doesn't work. It doesn't lead to learning It shuts people down the culture is no good. The protection is not there You're going to do really badly On paul o'neill's three questions You're not going to feel respected You're not going to feel meaningful meaning what you're doing meaningful work in your life Turn it around for a minute though and say wait a second instead of blame We're going to go and say what could we learn and how could we improve that? Wow, that's awesome south godin We're not afraid of failure We are afraid of blame That's what we're afraid of. We're afraid of blame. Blame is a real problem. It's toxic in a company Understand failure for what you can learn not who you can blame Now quick question for you. Who makes more mistakes? The well-led team with good relationships Or the poorly led team with poor relationships Who makes more mistakes? Who thinks it's the first one show of hands? Who thinks it's the second one? Okay, many more of you think it's the second one this time. You're wrong. It's the first one Why is it the first one? The what the the people who work will make mistakes They're allowed to make more mistakes and learn from them Well, does the poorly led team even mention their mistakes? Are they are they potentially afraid to even point out that they made mistakes? Right, we talk about sweeping things under the carpet. They're like, oh, you don't want to see this You're not going to see it. We didn't know no mistakes. We didn't make any mistakes They hide And so yeah, they make fewer mistakes This is research. She did by the way. This is not something she made up Amy Edmondson who wrote this book on teamwork on teaming She did this research. She was hoping that the answer would be this And it wasn't and she had to figure out why and turned out well Yeah, it's clear. These people are comfortable to point out their problems and mistakes The old view blame discipline retrain fire The new view learn Build better defenses much different We have a very safe environment for deploying we do continuous deployment. There's all these automated tests makes it really safe to deploy The production exceptions I mentioned Bimmer we had some exceptions for the screencast thing We got rid of them. We focused on that. We said we don't want this ever happening. This is important. It's more important than some new feature So let's get rid of that stuff. Let's let's totally make it safe. So we're not bothered by these things and our customers aren't interrupted We built these things called stop work authority cards. I've been learning I've been going to safety conferences and safety workshops. In fact, I'm the only software person there They're like, what are you doing here? I like well, uh, I want to bring safety in a software development And I learned a lot of them. They carry these things are called stop work authority cards Stop work authority means that you have the power the authority to stop work if you think it's unsafe And people have this in my company. Everyone has a card This has got the six uh engineering Shields on one side and the stop sign on the other People in the company have used this on me People in the company have used this on each other It's been phenomenal for helping build a culture of safety I have a whole bunch of them up here if you want one come up after the talk and I'll give you a card Very powerful in terms of helping us change the culture We do tailboarding Tailboarding is a process that a lot of uh people do in in non-software fields, right? I I learned from these electricians folks people that supply electricity To whole cities they will go to a site and they'll do tailboarding before they start work Tailboarding is a process of discovering. What are the risks that could occur on this job right here right now? Um, and it looks like this we have a column in our con bond board, which is tailboarding So nothing can go into into work in process until it's been tailboarded until we've had a group of us Talking about what are the possible risks and how can we work safely? We do something called job safety analysis for any given job that's potentially unsafe We analyze it and try to make it really clear What are the risks and how do you work safely for that job? That's generic, right? That's not I'm in the specific situation for a specific story card. This is just in general Right, you're teaching a workshop. How do you do so safely? You're going to a new customer. How do you do it safely? And we give regular recognition We use this tool. It's a website called do props And it has all these funny little icons where you can pick Something to recognize someone for doing something great great customer service Um, a great idea You know working on something that's important These are things that we never used to do and so we're actively Trying to do things that will make us habitually excellent and that comes back to The things that make us safe And that's on to nearing Thank you very much So I think we have about 10 minutes five minutes Do we want to? Yeah, any questions Question Thank you, Joshua. I think that that's an eye-opening session Um, one of the questions that came to my mind is you said that in software, you know, we don't think of safety as A priority or we don't precondition precondition As a product owner the challenge that I had is that now Quality seems to be safety To software. So what's the difference between focusing on quality versus focusing on safety? Good question. Very good question Is you know different in everyone's eye what one thing one person thinks is quality and other person thinks is terrible, right? I think ultimately it comes down to Studying where are the hazards? Where are we potentially going to hurt ourselves? Where are we going to hurt our customers? Right, if we hurt our customers, right, uh, you know, I always I talk about the um The iphone right jennifer laurence, you know had her pictures Access she doesn't care about the design of the iphone the quality of it the quality of the materials She couldn't care less what she cares about is iCloud Allowed someone to hack into her account And and so there's no you know safety there for her. So when we're talking about this I think we have to look at it comprehensively for our customers For ourselves, right if we come to work and we're afraid of our own code then clearly there's not enough quality there in terms of A safety net right Um, but it can go too far just like we saw the over protected kid, you know with all of the stuff on You know, you could you could go too far with it So it's not easy. You have to find that balance Of where people feel acceptably safe, right those those people on the five five people on one motorbike You know if they're going at a decent clip, but not too fast and they're used to it and The driver's very very attentive. Maybe that's acceptable. So you have to find it, you know find that acceptable risk and I think quality When quality is good, it's it's making us safe. It's protecting us I use Dropbox for my photos Right, I would say, you know, and they're they're lying their logo their tag line is you know, keep keep your memories safe or something like that, right It's high quality code, but it's there for my safety so Any other questions Josh so uh software development is not just coding. It's like uh from an idea you have to uh Take it to the software engineers. They built it. They called it Somebody test the code and then you deploy it So it it there are a lot of phases. There are different people who are working on them So the meaning of safety probably is different for each of them So the strategy of safety of how you implement safety. Is it different for these different groups? Is it the same? Is it a one thing that one size fits or kind of? You know, I mean it it's basically like anything when you're prioritizing You have to look at a bunch of different things and say where are where are the highest priorities? Right, if it's something really minor like A programmer is looking at some code and says that variable Is it's totally unclear what it it's what it does. The name is terrible. I'm going to rename it So it's clear to other programmers That's a tiny little safety improvement You can say because they've now made it safer to modify that code because that variable this name is clear now move to dev ops and Someone in dev ops says well, you know what we don't have fault tolerance for the databases It's not quite fault tolerant yet We thought you know thought it was is that you know something we should focus on Rather than doing something else and a tester testers might say, you know We don't think we're doing good job of automating our testing the tests break all the time They break and the team doesn't care when they break How can we change that to make it, you know, everything stops when the tests break in the build Everyone's going to come with added from a different perspective What we like to see is that constant activity towards improving the defenses across the board Right in our company. It is not just the software people making us safer. Our coo sandra is focusing on our safety We are administrative assistant. She's constantly focused on making things safer. We have Students of our e-learning two weeks before they take our e-learning We give them a sound check album to check and see that everything's working correctly before we even show up Right. So it's it's basically going across the company in our own ways. We think about how to make things safer And occasionally I think you might find something where one person thinks it's adding safety and another person thinks it's It's actually making things worse or going too far. That's going to happen. You're just going to have to work those things out There's no magic formula For that so But there's everyone has their perspective on it. You all have to contribute One more question So time My question is that, you know In countries for waterfall versus agile in agile when we going in there We are not going in there with two months of analysis Then you go code and then test we go with a perspective. Let's get in there See it for ourselves break in small pieces. So we are taking the risk there So if we think about safety in as compared to waterfalls, do you think it's a bit more We have to put in more time more thought when it comes to safety and risk in context of agile versus waterfall Yeah, so is is basically is agile more risky than waterfall because you could think in waterfall you had more time to analyze And to design and then to write the code and to test the code But I think the biggest problem we saw with waterfall is that We might have built building the wrong thing Right now we've done all the analysis all the design We started building and then discovered it's not only what the customer wanted What I look at with agile is it's The feedback loop and the learning loop a lot faster Right so that we can quickly get feedback from people potential customers Product owners or is this really what we should be building right and that actually I think reduces risk quite a bit Right because you're going to fail faster By building the wrong thing quickly you learn. Oh, we don't really want to build that Let's let's pivot. Let's move to something else So to me it actually brings a lot more safety. I look at a lot of the agile practices Which I've been doing since like the late 1990s And I say that safety was a common denominator there Safety really was a common denominator in the technical practices in the practice of getting early feedback and building Minimum viable products right these days. We don't we don't guess what our customers want We build the most minimalistic little thing Get it in front of a customer get feedback early We spend very little money on this very little time just to get feedback and they might say That's not what I want at all or or yeah, that is great But I needed to go do this and so we move in a direction we didn't expect Right. So to me it actually feels like it it reduces risk But for you if it feels different, you know, you're going to have to find a way to Make it safer right make it safer for you and your team It's it's all about analyzing that and understanding. Where are we feeling at risk? Does that answer your question? Hi Josh, we have heard security as a word more in IT development than safety How you define safety versus security? That's a great question safety versus security. In fact, I was just in brazil And they don't really have a word for for security and safety is one word Right It would be easier if there was just one word But I think for most of us we don't really think about Security enough did apple think about security enough for iCloud? Right, you know what happened there, right the person someone tried to Basically hack into Jennifer Lawrence's account and it didn't stop them from trying You know That username password over and over and over and over again You should have stopped them after five tries or 10 tries or something didn't They weren't making safety a precondition. It wasn't Considered that way there. So to me they're they go together, right? In fact, we are doing more security checks and you know, basically running You know analysis and things like that and patches and all kinds of things when when there are security Problems out in the field much more than ever before so we've made that more of a Focus for us, but I think it's it's related. I think for many programmers We don't even know how to code defensively to avoid, you know the hacks and the things that can occur We just have to be careful of not going too far because if we go too far towards, you know Learning the you know nuances of all ways to be to write secure software Maybe we're going too far from that that parity line, right and we're going to get Closer to the bankruptcy here. I don't know but I do know today I don't think most people building software are doing enough around security And that can really harm you Definitely related Thanks a lot, Josh. Thank you. I hope people enjoy that that's a very new idea in many sense I've not seen many people even touch upon this topic. So I think it's a very novel idea and as also as always Josh is always up with something new. So thanks for coming and sharing that with us. Thank you, Narash