 Hi, this is Brian Grayson with Wikibon. We're here at theCUBE, here with theCUBE, here at DevOps Enterprise Summit in San Francisco. Excited to have on Jay Paul Reed with Release Engineering Associates. You guys are talking about what you've been doing at Salesforce, so you're helping Salesforce make a transformation. Salesforce has been known for a long, I mean, they're cutting edge. What are you helping them to get better at? What types of needs do they need in terms of DevOps and moving faster for releases? Well, so it's really interesting. I mean, they are, have been doing a lot of work, a lot of great work on DevOps and releases and that kind of stuff. Where we've been working with them is on their retrospectives. So they have a very big, very complex system that they operate for their customers, and sometimes it has service impacts or outages due to changes that they make. They roll out a new feature or whatever, and there's an incident, and there's a team that works on those incidents. And then the real sort of challenge that we were kind of working on, and it's still our working on, is how do we take that data, collect it in an actionable way, talk about it in a healthy way, and then feed that back into the system to get improvements so that those sorts of incidences become less frequent. Okay, and is this really, is this focused on, you know, we hear a lot about feedback loops and monitoring and getting data out of the system. Is this really focused on that, or is this more on sort of postmortems when there's failures? Well, so it's actually kind of both, right? I mean, in some sense, a good actionable retrospective. You hear a lot of the term blameless postmortem. We actually talked about why that term has some problems, is problematic in a few ways. So we like actionable retrospectives, but the actionable part is sort of, it's very core to that DevOps feedback loops, and then you tighten those feedback loops. So it's the same sort of thing, slightly different language. So how do you, let's talk about that a little bit. You know, postmortem, people tend to think something died, you know, there's that natural. How do you, because there's a technology aspect to that. Obviously, you know, we're automating things, we want to collect feedback on them, and then there's a cultural thing. Like, give us the basics of how you help people not make it about blame, not make it about, you know, it's accountability sometimes gets misled. It's accountability so I can blame somebody. Talk about the types of things you try to help people understand. Right, well, so we could spend an hour talking about that. But what's interesting, you talked about accountability. There is a difference between accountability and responsibility, and one of the things that's important to make the distinction is that everyone is accountable. They are held to account for their behavior, but that doesn't necessarily mean they're responsible for the particular failure. And so we do a lot of work, you know, we went through in 15 seconds sort of the structure of a postmortem and the presentation. It's actually a pretty simple structure. It's really on creating the space, describing the incident, coming up with a timeline, and then talking through it, and coming up with ways to sort of remediate not only the incident, but improve the response of the organization to that particular incident. So that is pretty kind of boring and pretty kind of follow the eight steps, but it turns out all the complexity, the really interesting stuff, comes into what you were talking about, the sort of blameless part, the sort of having an empathetic culture, being able to be vulnerable in that space with your colleagues in a way that is healthy and results in better business outcomes in terms of operating whatever particular software service that your organization might be operating. Now, do you guys find this conversation is better to have right after an issue or is this something you want to do proactively? Because the struggle has got to be when you talk to executive staff, they're going to hear P&Ls and bottom lines and something didn't make a number and so they want to associate it with something. What's the right time to have that conversation and who should be in the room? Right, well so these conversations you don't tend to have right after a big outage because everyone from the people in the trenches that were working on the problem all the way up to the execs aren't going to want to hear it and that's understandable. They were just sort of in this incident and they're in that mode, that firefighting mode, right? So it really needs to be a shift in a conversation that you have sort of outside of a particular incident that you're worried about. And to Salesforce's credit, again, talking about innovation in the space, the stuff that they're doing, that's very front of mind of them. They came to me and they engaged with release engineering approaches to really look at that outside of any particular specific incident because that was really important to them leveling up their game and the way that they do with that stuff. So certainly don't do it in the context of an outage but we talked a little bit about, you know, you're right. People talk about the PNL, the money associated with that sort of thing. That's actually not quite the right framing. It's the thing that we talk a lot about. We talked about this, the one of the takeaways is that it's not the outcome, it's sort of the response when you're talking about an incident. You have to shift it from how much did that incident cost us to how do we respond better to the next one because if you're looking just at the outcome that has no impact on the next incident that you face. But if you look at the response that directly feeds into all the next incident and any future incidents that you're going to have. Yeah, so, you know, Salesforce is a little bit unique in that, you know, not only do they have direct customers but they run platforms that people will build applications, you know, theforce.com and Herokee and so they're, to a certain extent, they're direct to their customer and then they're intermediary. How do, is there a way to think about it differently if you were at a different customer who's more direct versus if you're working with somebody who's got an intermediary? How do you think about, you know, who is the customer in that space and does it change depending on how far away you are from the customer? Yeah, so, you know, I don't think it changes. Actually, one of the complexities around Salesforce and we've had some really interesting incidents and interesting conversations is that because their system is sort of a platform that people can build on, you will find people throwing all sorts of crazy things at their platform. So that's always fun, right? But it doesn't really matter. You can kind of look at it as a scaling thing, like how close do you want to go or how far away do you want to go. If you really think about Heroku as a great example, those companies, they have customers of their own, they should be doing their own post modems around retrospectives on how, the way they operate that service on Heroku and how their development processes and activities contribute to success and failure. And that's actually one really big thing that we want to point out, is that in this comes up in conversations with, again, operations engineers and executives, humans contribute to the success of the organization and the successful operation of the technology just as much as they contribute to the failure. So it's really important to consider that as sort of a holistic thing. And again, that helps with the conversation. It really is a sort of holistic reframing on human error, complex systems, and operating them. Right, and we're hearing that over and over this week, is technology is a piece of, technology isn't the thing that's, you could probably pick a lot of technologies and they're going to work. It's getting that people piece right, the culture piece right, and so forth. And actually, I think you're seeing a lot, the science part, right? So Nicole Forsgren has done all the numbers on the math that makes DevOps work from a profit and loss statement. So there's science there. We were talking about entropy. If you look at large scale systems, Mark Bridges, he's a physicist and he talks about quantum physics in the context of large scale systems like the scale that Facebook and Amazon run, their system sales force runs their systems, right? So what I love actually this year that we're doing a lot of talking about is it's not just technology, it's not just people and culture, it's actually science. Yeah, no, and that's sort of a great wrap up with that. You know, it's, it's a great show. We're seeing so many different angles. We're seeing lots of different customer types, lots of different things, but we're seeing it from the psychological perspective. We're seeing it from the human perspective. Really a different show. It's great that it's so community based. A lot of you are learning. So with that, we're going to wrap it up folks. Thanks for watching. You can watch all the videos at siliconangle.tv from DevOps Enterprise Summit in San Francisco. Thank you very much. Thanks.