 Well good afternoon. Good morning or good evening depending on where you are in this world and welcome to today's devops.com webinar I'm Charlene O'Hanlon managing editor of devops.com and moderator for today's event and I welcome you before we get started with a What promises to be an absolutely fabulous webinar today? We have a few housekeeping items We do need to go over first First today's webinar is being recorded So if you miss any or all of the event know that you'll be receiving an email with a link to the webinar on demand post event Also, we are taking questions from the audience during this webinar So if you have a question at any time for either of our speakers today Please use the control panel and submit your question and then we'll take about 15 minutes or so near the end of the presentation and go through the audience questions Okay, with that we'll go ahead and dive right on into today's webinar, which is avoiding the devops tax our speakers today are Mark punsack who's the head of product at GitLab and Christopher condo who is senior analyst at Forester. Welcome gentlemen. Thanks for joining me today Thank you for being here right mark. I'm gonna go ahead and turn things over to you and let you do your thing All right, perfect. Thank you very much So today's big topic is avoiding the devops tax but to get there we're going to start more generally with digital transformation and current trends and devops and then lead up to the devops tax and Finally, we'll close with some best practices And with that I'm actually gonna hand this off to our guest speaker Chris condo from Forester to start Thanks very much mark and thanks everyone for having me here today So one of the things that you know, we hear about a lot at Forester and we study and you hear it in the industry is this whole idea of digital transformation and the idea that digital transformation is you know causing companies to do all these various Automation or devops or whatever and so the whole idea is like, you know, what you know, what is digital transformation? What does it really mean? And so, you know, one of the things that you can look at even from this slide is the idea that you know Digital transformation is all about moving ahead quicker with not only just, you know Delivering software faster but delivering value faster. So when we think about digital transformation We think about and talk about delivering value to your customer quickly repeatedly and with high quality Next slide please So one of the things that You know, we that I like to talk about is whole idea of customer obsession and Customer obsessed DNA if you want to actually execute on a digital transformation initiative You need to keep the customer in mind and that means, you know going from customer aware like I know we have customers I know they come into my store. I know they use my website or I know they want to interact with You know our application to being customer led allowing the customer to sort of guide your product planning a grant guide your strategy and That means staying connected with the customer. So, you know exactly what's going on how they're using your product taking the Information that they're giving to you and putting it directly to use in your product The next one is being data rich. How many times do you have this situation where you've got? Loads of log files loads of information, but you're really not doing much with it So we're so you got to move from being data rich to insights driven You got to go from being perfect, which is you know, I don't want any bugs I want a hundred percent code coverage to being fast But we find out time and time again. They try to be perfect just doesn't work No one ever releases code. That's a hundred percent bug free So don't try to make that your mantra try to do something that you can get the out the door fast and Recover fast and repeat that process and then finally go from being a siloed Organization or a siloed team to being connected being connected with your customer being interconnected within your team and with your organization Next slide So one of the biggest Factors that's really changed over the last few years is this whole idea of you know And putting the customer first kind of came about with what I would say is like the kind of the mobile Experience and the whole customer experience it really kind of changed the game And so when we think about the customer experience we think about the complete end-to-end user experience the customer has with your product I'm not just talking about you know, you know Do they go on to the website and and buy something from you or do they have an iot bracelet that maybe somehow They walk into your store now and they're completely connected with you Is it your automobile that you not only sit down and hold on to the steering wheel but it connects to your cell phone and You know connects the user more intimately with that experience and so customer experience is much richer than it used to be Customer experience is really a complete end-to-end experience the customer has with your product Next And so here's a here's kind of an example of what I mean by digital transformation impacting the customer experience Let's say like you know you want to do your taxes you go to the tax site and you download your 1040 form Whatever form it is you got to fill it out You got to drop it in the mailbox, and then you kind of like you kind of have to wait You don't really know what's going to happen. So from the business perspective This is perfectly optimized. You know the forms are available That you know, what's the big deal right the the person doesn't have to go to the post office anymore They can just go to the website and download a form. The problem is that the customer experience really hasn't changed Hasn't changed at all. It's an afterthought. You don't know how if that customer is enjoying this experience You don't know how long it took them to fill out the form And then most of all the customer doesn't even know if they filled the form out correctly And so they have to wait they have to hang around and wonder You know is my tax return going to come back did I do something wrong? And so, you know that whole the whole sort of like Direction of delivering value is in the wrong direction The value here is that the business made their life easier, but the customer's life still stinks So what we want to do is we want to change that around and this next slide kind of helps you out and understand What I'm talking about we want to go from you know a digitally transformed experience where the user just says you know And this is borrowed from TurboTax obviously not trying to you know Trying to try to give them any Here right, but you know this is a really this is the main idea right take a picture of your W2 It automatically validates the form automatically uploads that to the tax site And then if everything goes okay They even put the money right in your bank for you So the customer, you know gets an instantaneous response. They get a great reaction from filling out the form They know if it's right or wrong. They have options. They have help along the way They're not just fumbling around trying to figure out if they're doing the thing, right? Where's my number two pencil, etc It's all done. It's all nice and tidy. They get their bank and they move on So in this case the user's experience is optimized by a direct connection to back-end capability and processes Previously those back-end capability and processes were the realm of the business and the business managed that by exposing those and opening up that processing capability to the front-end and Integrating the customer into that experience. You've created a complete end-to-end experience That's optimized for the customer and delivers an excellent customer experience So that's what we mean by digital transformation and this is why it's so important and to further that point Let me show you what we're seeing in the industry as far as how customer experience actually impacts a company's performance Next slide, please. So in this particular case Forrester actually did research where we discovered that the leaders who excelled at the Forrester customer experience index and Forrester has this Index for measuring your customer experience the goodness of it versus the badness of it. Those customer experience leaders Outperform the customer experience laggards by on average of 29 points in the stock market. That is absolutely Remarkable, so that means that these people who are have you know, and we did this enough as a portfolio Perspective now we're not trying to say that there's a causality here, but there's definitely a correlation here The people with the bad customer experience their stock is lagging those companies that have an excellent customer experience So that's showing you that customer experience really matters It really does make an impact on the bottom line and digital transformation is a real force to be reckoned with Next slide So, you know, what else is going on? Let's talk about for example rideshare and self-driving cars Now everyone is using those these days or most people are anyways, but you might think all right What's the big deal? We'll take a look at the industry that thought it wasn't going to be That thought it might be impervious to that the taxi industry. There's an industry that had Heavy government regulation it had a very high price of entry You had to buy a medallion like that sometimes cost thousands of dollars to be a taxi cab driver You had to be certified yet to go through all kinds of regulations and have you know a certain kind of car These folks thought that they were buying into a regulated system that was going to be protecting them and their business investment Didn't turn out that way turned out that uber and Lyft and other ride-sharing guys just entered their market and didn't care about those regulations And it turns out the government's turn sort of a blind eye to it doesn't care They think it's actually healthy and what is that leave that leaves this industry completely decimated and there's already been you know Some very interesting and sad articles about people that have taken extreme measures to kind of bring that to light Take a look at healthcare healthcare Administration just the fact that Amazon talked about the idea of bringing their own healthcare plan of market caused all the other Major healthcare indexes to drop think about the fact that you don't have to go to your doctor anymore to get a Flu shot you can go to CVS you're going to see more and more disruption like that And that's going to put pressure on not just large healthcare industries It's going to put pressure on your mom and pop in your regular doctor's office because they count on people coming in the door Not only getting a shot, but maybe getting something else done while they're there maybe a blood check or who knows what? I mean it might be good health practice to see your doctor once in a while when you do this But the fact the remaining fact remains those are revenue streams for these small businesses And they're going to be disrupted in ways that maybe they don't they don't expect right now How about Amazon entering the grocery business you think like that seems like a reverse brick-and-mortar kind of move But now all the other grocery stores are thinking like do we have to digitize? Do we have to make our stores more amenable to digital? Experiences should people be able to just go around the store and throw stuff in a shopping basket Check it off with their cell phone and walk out the door. That's causing a great disruption It's also causing disruption as far as supply chain management and other types of things that go on in the background that Amazon excels at So expect more disruption there and the whole fact of the matter is you know It's like who's next the common thread is placing the customer first If there's a place where the customer is not being placed first and some company can come along with an innovative way to do it It seems like the government is open to it and customers are certainly open to it as well So the bottom line is really no business is impervious to this type of Disruption of of their business So how do you how do you you know manage that? How do you wrangle that digital disruption? How do you you take that that energy that's being placed there and bring it all into your software development team? Well, we talk we call this you know Modern application delivery the whole idea of taking the those those four tenants We mentioned earlier being customer-led insights driven fast and connected and tying them to your modern application Delivery of technology structure culture talent metrics and processes You need to take those things into consideration and build your software development process around that So let's see how we can maybe elaborate on this and think about the ways you can you can put these things into place So the very first thing that I always tell people and it turns out every time I go and do an advisory with a client It's always about understanding what we mean by this concept of agile plus DevOps I I never talk about just agile alone and I never talked about DevOps alone I think these two things are needed together to really make The team hum and operate at top efficiency and we're talking about the idea of having people People organized in team models. So the organization structure supports these product teams managing their own Individual components. We talk about platforms the idea of having like a you know a DevOps platform Not just the collection of tools, but a platform that can be instantiated and repeated over and over again You don't want people hand crafting all their tool chains all the time. You don't want a situation where every time An engineer changes teams He has to learn a whole new set of tools. You want there to be platforms that can support these Engineers so that like, you know, if they go from one team to another They can expect a similar experience, but you also need to encourage, you know, innovation within that as well And also the process agile and lean continuous delivery Integrated governance integrated compliance integrated security. You need to take all these components and integrate them into these pieces of your software delivery process So let's take a look at the next step so One way to understand Your process is to do what's called a value stream mapping exercise and what's really exciting is this comes from the lean Manufacturing manifesto. There's been there's been books written about, you know lean software development And what's really interesting is we're now seeing trends where Vendors are starting to offer value stream management tools to capture these types of metrics It turns out that many connected tool chains have these metrics and you can actually capture them and understand these things anyways, but the whole idea of it is Actually understand all the steps in the software delivery process. This is what's known as the value stream It's like how do you get an idea from the very beginning from your business team? How does it go through all the various locks and gates out the door to production? And you take a look at all those steps Whether it be, you know, you have to wait for approval whether it be the code needs to be checked in and be code reviewed whether it needs to be tested or Signed off by QA or signed off by compliance or governance or whatever the various steps are a lot of times Just looking at the tool chain might not be enough. Oh, I checked code in it goes through continuous integration Then it goes testing and then it goes on to a server Well, what are the things that happened before and after that? How did that work item get approved? How did that epic and story get developed? All those things need to be understood because you want to be able to automate all of those things from end to end And you want to shrink out all the wait time so that all you're doing is process time You want to get it as optimized as possible doing a value stream exercise Understanding your full CI CD pipeline is the first step The next step that you want to take is think about integrating product teams. So we talked about this Couple slides earlier The whole idea is that you want an integrated product team and these folks, you know They're all we're showing all different colors here But the reality is that they need to they can be one team that encompasses a lot of different skill sets They need to be sort of deep so you don't want to be in a situation where I was just at a client the other day Where their concept of agile was they develop a bunch of code then they hand it off to a security expert That security expert lets it sit on his desk for three weeks Decides what he's going to do with it and then hands it back to them and then they think about all right now We're going to do a few more steps and they hand it off to a compliance expert She takes a look at it after a couple of weeks and then she decides, you know How do I decide if this is compliant? Maybe I do a few things and then she sends it back to them That is just like the sort of like Pardon my French ass backwards way of doing things What you really want to do is you want to do things where the team owns the responsibility for security The team owns the responsibility for compliance. They have these other Entities around them that can advise them on how to do best practices They want to be able to check in with a security expert and say here's our design. Here's our architecture Here's how we're handling these problems. What are we missing? What do we need to be doing next and so all of those teams sort of act as shared resources They don't act as you know blockers on a particular project the product team really should own its metrics It should own the the the capability to bring that product from the idea all the way to production and along the way They have these other entities that help them and sort of like you know Share in the whole responsibility of bringing the product through the door, but in the end It's the product team that kind of owns that Next and so one way that you can enable these product teams and this has really been a really big enabler is the whole idea of adopting a Modern application architecture which incorporates microservices and incorporates containers There's nothing that's been more revolutionary in my mind than the whole idea of moving to a microservice architecture It kind of all hit home when I was at a DevOps meet up a few years ago I used to think that microservice architecture was just going to be really chatty really filled with latency and issues Such as managing APIs and whatnot and when I sat down and listen to this retail company talk about how they actually broke through Their problem with managing a giant monolith and how their whole team You couldn't release a simple bug because they had to wait for the whole thing to be deployed The whole thing to have to go through the big giant pipeline It's just a big fat monolith trying to go through this pipeline said it took forever They started just breaking off chunks of it and loosely coupling it using APIs and now You know their search engine on their retail site and their type ahead Functionality on their retail site in there, you know, what's happening now on their retail site and what's on what's on sale today? All of those components can all be managed released independently of each other and they can all work loosely together And so the microservice piece wasn't about just making things small It was actually about helping teams work independently from each other So you could release components you could release architecture as the business needed rather than as the slowest link in the chain Required and containers are another thing that are really revolutionary It really helps drive home that separation of concerns between the developers realm in the operations realm You know dev ops was kind of messy people who had thing their fingers in each other's business sort of like but containers Sort of helps define that it sort of helps the ops people and the people that worry about operational efficiency understand like hey I know my realm. I know where I need to automate I know how to do the SRE practices and automate how containers get delivered and how systems of Operation get managed and the developer can say great. I understand my realm I understand how all of these things are going to fit into the container I worry about how I automate my development process and it kind of makes It allows folks to worry about what they're best at rather than trying to have everybody, you know know everything next and So what we talk about, you know, you know all of these things they all work best when there's a complete and Integrated dev ops pipeline and we're talking about from the backlog and the ideas all the way through, you know CICD testing configuration management deployment capturing metrics capturing KPIs and just doing this in a circle rather than a straight line Because you want to always take what you just did Learn from it take the metrics take the KPIs and decide how can we do better? How can we innovate? What new tool do we need? What new process do we need? What can we eliminate from this? What should we automate? What should we you know no longer be doing anymore because it's redundant Those are the types of things you learn from a continuous and integrated tool chain But you can't learn from a disjoint tool chain or a non-automated tool chain Next so these are some of the trends that I that I see happening I just ran a wave on continuous integration tools and customers told us loud and clear that they are looking for a complete Integrated tool chain because they're tired of integrating their own tool chain That's it's great to have the integrated tool chain But it comes at a cost and it comes that you know Basically what I've been hearing about a 10% of their engineering team is spent maintaining these dev ops tools And they're trying to find a way to shrink that You know application development delivery professionals are looking for integrated solutions or SaaS solutions to simplify tool chain management The idea of managing all these servers on your own is it's complex You know, you know, there's so many servers running under people's desks that you know Just chewing up compute time, you know If you could have a self provisioning system or a SaaS based system where the developer just can instantiate it or the testing can instantiate it Much better than having to request an IT team to provision something for you. We were also seeing dev ops vendors Sorry, I haven't gotten I'm going a little bit too slow I guess dev ops vendors are consolidating providing complete tool chains out of the box And the ability to model integrate and manage and monitor dev ops tool chains end-to-end is also a growing requirement So in across the industry, these are the trends we're seeing as dev ops adoption is growing next And what we're seeing is you know, it kind of varies public sector and health care are the slowest ones to adopt Utilities and financial services seem to be the quickest ones to adopt and kind of in the middle is the retail and wholesale But not too surprising, right places where there's a lot of governance can make you go slower Places where there's less governance allows you to go a little bit quicker Next and so the bottom line is you know building and maintaining a dev ops tool chain is complicated It's a lot of moving parts And so anything that you can do to kind of simplify that and make it easier on your team to manage it It is usually a good thing because having a Complete and integrated dev ops tool chain is part of it needs to be part of your digital transformation strategy and so Again back to my 10% of effort These are all the sort of integration points that come about from having to you know Integrate if you think about it every single tool in your tool chain Can can create like a friction for your team to manage that so anything you can do to minimize that 10% tax On maintaining your tool chain is usually a good thing Give time back to your engineers and so And so Interestingly enough that tax and that and that and that complexity sort of ends up translating into this sort of Overestimation of dev ops maturity by executives We went out and surveyed executives and we found out is that you know usually by about 10 or 15 percentage points difference Executives and sit in the C-suite thought they were doing all the right things to automate and implement dev ops But by and large when we talked to the practitioners It was about 15 percentage points less that we're actually doing it and the reason is that this complexity of implementation Is not accounted for and the complexity of sort of adoption is not accounted for because doing dev ops isn't easy It's harder than people think What's great about this slide is we just launched our The results of our dev ops survey or wasn't a dev ops developer survey But there was also this huge discrepancy between what the managers thought and what the developers You know said they were actually doing yeah interesting interesting industry time Yes, and it's good to uncover these kinds of differences because it can help you realize You know when you're talking to these different folks at these and these companies a lot of times You know when we're when we're evangelizing or selling, you know You're talking to different personas. You're talking to the implementation persona Who's usually the loudest, you know voice in the room and then you're talking to the actual buyer who needs to kind of be educated? And they both sometimes aren't talking to each other and you know It takes folks like us to actually get them to talk to each other and understand that there's a gap And so again, here's another you know interesting nice graphic You know 66% of developers expect at least some teams to use dev ops by the end of 2017 So, you know, everyone is expecting this trend to come In six but you know 64% of development shops integrate, you know Dev and ops staff on at least some teams But only 38% you know site an excellent working relationship with ops So even though everyone says we're gonna do dev ops and you know, we we have to have some integration They're saying there's still you know some rough spots between them And so I mean the other interesting one is on process and tools here 23% of development teams are automating builds But one only 27% you know are doing continuous integration and 39% of developers use open source and release management tools And so I mean it seems like you know There's so much buzz in the air about like, you know that two-thirds of the world wants to be using dev ops But the reality is that like it's only about 25% that are actually implementing it or implementing it in an effective way That they feel that they're actually getting something done So there's still a lot more work to be done and a lot more help needed to help Increase adoption and make it work a little bit more efficiently. Yeah, and we saw that same thing on our survey as well It looked like you know something like 60 or more percent of people wanted dev ops They're only 23 people 22% self-reported. They were actually doing dev ops, right? We want it, but there's this leg Yep, exactly Because that actually wraps it up. These are now my slides. So take them over. Thank you Chris That was fantastic. I'm really helpful stuff. Thank you for reminding me to that the very first slide you had there You know, it's a subtle thing that you said delivering value faster Because as an industry we do kind of get caught up in delivering code faster and it's really good to Do reminded that you know code doesn't matter value does and it's all about what you deliver to your customers You know that they actually find valuable and that is what we're all trying to optimize for it's not about lines of code It's value But anyway, so by now it's pretty obvious that business Businesses depend on the speed of their dev ops life cycle But despite spending billions on dev ops software last year You know, there's tons of wasted time and a lot of organizations are disappointed with the results Sure dev ops has improved things and we're better off than before But the true promise of dev ops hasn't been fully realized yet at least for most of us And the dev ops tools have brought improvements to the life cycle speed, but the tool chain itself limits how fast we can go And this is another example of what chris was talking about. This is a typical dev ops tool chain Tons of different tools tied together to deliver dev ops You've got different tools for planning and code creation and ci and security testing and packaging and release and deploy And configuration management and monitoring. I mean that's a lot to cover there But the integration complexity, you know, administrating all of these products and connecting them together It's not like you just snap them together in a sequence The interactions are actually complex with multiple connections for each component You know, for example, your ci needs to dock the version control But also your code review and your security testing and your container registry and your configuration management And the permutations are staggering. It's not just a one-time configuration either It's not like some administrator does this and you're good to go for the rest of the year Each new project needs to reconnect all these pieces together And that has you know, this huge this burden on the developer and the development teams So the time spent on integrating and maintaining these complicated tool chains is costing dev ops teams Which brings us to the dev ops tax, you know that little bit You know chris mentioned the 10 that gets skimmed off the top and it limits your efficiency But while it might seem small, you know 10 percent, right? Hey, it's just 10 wasted But like any tax the real impact is on how it changes behaviors So when it's a pain to integrate security, how many teams just don't bother? Or when it's a pain to share information between teams, how many organizations Overcome that burden and find a way to work together How much impact does this tax have on collaboration? With separate tools and separate processes We're naturally encouraging separate silos where functional teams work in isolation And as chris mentioned one of the key things here, you know, vertically integrated Product team needs to work really tightly together. But if the tooling gets in the way, that's a real problem So is it possible to live with that tax and still achieve true dev ops? Sure, it is but the tax makes it harder than it needs to be Now the sequential dev ops toolchain tends to be opaque inefficient and uncontrolled. But what does that mean? Well, the different silos between teams and their tools Make it hard to understand the whole instead of the individual pieces of the puzzle It's really hard to gain an overall visibility of the problems that need to be solved Individual silos also mean that teams still rely on manual or semi-automated handoffs Even if they're fully automated one team can't start their work until the previous team finishes their work You know the chris talk about handing it off to the security people and then they've got to schedule it And just it's on desk for three weeks before they even pick it up That kind of you know, even if that's automated doesn't matter if the next team doesn't pick it up for a long time And also, you know different permissions and security policies across the team leads to an uncontrolled environment where it's difficult to establish trust Now to put this in context in a different way There are a few stages companies typically evolve through on their dev ops journey On the one extreme there's nothing but source code management and everything is manual And then you add ci and then you add cd And then you get your first dev ops toolchain But it might still be disjoint and disconnected, you know a bunch of pieces hover together Then you get an integrated dev ops toolchain and that feels better But as we just showed it still has its problems the dev ops tax is still there So the next step is a unified toolchain And here usually it's the the pieces that come from the same vendor with all the same look and feel And they're easier to integrate together or maybe even a couple of them are prepackaged together But it's still not quite seamless So what we need to get to is concurrent dev ops Because the entire dev ops lifecycle when the entire dev ops lifecycle is seamless magic starts to happen Teams can work concurrently not sequentially Silo's are broken down and teams can work together at the same time seamlessly collaborating They're empowered to act without waiting for permission but with full accountability And with security embedded in the process not just an afterthought So put it another way When continuous integration testing code quality performance testing And security testing are all baked in from the very first push Developers can deploy with confidence And when when you're shipping continuously your changes are smaller. So any mistakes are smaller And when everything is visible Problems are caught sooner and more people can help fix them And when deployments are easy and continuous errors recover faster All of this comes together to turn that 10 percent efficiency improvement Into a three times faster dev ops lifecycle for software development teams But when I say teams I'm not just talking about dev and ops because the dev ops lifecycle depends on getting input from teams outside of development and operations The more efficiently dev and ops and security and business people can communicate and collaborate with each other The faster they can deliver value to the customer With concurrent dev ops each group gets an experience tailored to their needs but shares the same data And it interfaces everyone else So collaboration is easy Imagine an ops person finds an issue in production and they drill down to find the problem And see that a recent deploy caused the problem Or simultaneously a dev gets alerted that their recent deploy triggered a problem in production Goes to the code and sees performance change right there where they're looking When dev ops biz and security talk they're looking at the same data, but from their own point of view Now of course git lab offers Concurrent dev ops with a single application for the complete dev ops lifecycle With visibility of all projects and relevant activities and teams can see everything that matters Changes status cycle time security and operational health are instantly available from a trusted single source of data Information is shown where it matters most. For example, the production impact is shown together with the code that caused that change And developers see all relevant security and ops information for any change With git lab, there's never any need to wait on synchronizing your monitoring app to version control or copying information from tool to tool Git lab frees teams to manage projects not tools These powerful capabilities eliminate guesswork and help drive Accountability and give everyone the data-driven confidence to act with new certainty So bringing it all together Here's a recap of some of the best practices. We were from chris that to maximize your digital transformation You need to optimize your ciCity pipeline Create integrated product teams and modernize your application architecture with microservices in a cloud native approach But the tool chain still has dev ops tax So to help avoid that reduce the number of Integration points integrate as deeply as you can And strive for a single conversation across development operations security and business And as a final parting tip if you're just getting started Start with continuous integration automating tests and building confidence in your code will pay dividends many times over If you already got ci then move on to continuous delivery Automate deployments and make them less scary If you've already started the dev ops transformation then embrace the culture You can only go so far when there's a wall between dev ops And with that I'd like to open it up to questions awesome, awesome Wow, I have learned so much so far And I'm sure the audience has as well There have been a number of questions that have come in already But if you do have a question for chris or mark, please use that control panel Go ahead and get your question in We should have plenty of time for questions here and with that. I'm actually going to get going on those questions. Okay Um, first question dev ops has been traditionally about dev Any perspectives surveys or examples on the ops the ops part of dev ops like production automation Continuous monitoring environment provisioning things like that That's a great question. Um, I think you know a lot of dev ops did spawn From you know, sort of the you know, the development side of things and so You know developers want to you know deploy to production. That's sort of the first thing and you think oh, I'm deploying to production So I must be doing dev ops, right because uh, whatever I it went into operations But if you stop at the deploy, that's not really Covering, you know operations at all. Um, you know, because there's there's more to just deploy, you know, running things of production There's the performance of the thing in production the security The scalability, uh, you know the cost, you know, there's so many things that factor in And um, and again if you just stop at hey, I deployed so I'm doing dev ops. That's not really it So, you know being able to bring back monitoring being able to say hey this Change, um, you know caused uh, you know more You know, whatever more memory to be consumed or you know, there's a higher error rate or any of these kinds of things Um, you know, that's that's really important to bring into that um getting Also on that it is not just Like if you're if you're still sort of doing all your development Thinking you're agile thinking you're really great at deploying to production But then you throw it off to an ops team to own afterwards and if they get paged afterwards That's that's really only going so far with dev ops. Um, you know magic thing happens when the developer that deploys Is actually responsible for the thing running in production when they get paged When you don't have dev and ops, you know and handing things back and forth But you've got just one team that's together You know in support of that that thing and that's a that's a it's a really magical thing when it happens Chris, did you want to add anything there? Yeah, so, uh, you know my experience Was basically that exactly that so, you know, you write the code you own the code You know and the interesting thing is that when we when we switch to a dev ops mode at my former company We um, you know thought we were going to do it all and then we realized and we started talking to these Operational folks that there was a lot more to it that we hadn't thought about like, you know How are you going to geodistribute static images? Are you going to use a cdm? How are you measuring latency? Are you worried about, you know different types of configurations? and so It turned out that, you know, the developers didn't know everything we thought we knew and um, we could certainly learn a lot from our operational, uh friends And you know, they were working hard to um to standardize on automation platforms But you know at the time I was working in the industry like four or five years ago There weren't always a lot of tools available to do that that we didn't have, you know, standard container standardized Containers and kubernetes and things like that, you know These guys were basically dealing with virtual machines and configurations and so forth and so, um The idea that, you know You know, the developer can own everything they really should own everything about the the application The application needs to perform on the system. They need to collaborate with the operations engineer to find out What are the parameters of that system? What's the best way to optimize it for that system and have it be like a collaborative approach? and then when something goes wrong The operations team's job is really to make sure that the information that the developer needs to debug that Investigate is made available to them in a timely manner So that means about providing tools that allow you to investigate You know log files or other types of issues or cat capture metrics that can tell you before the site goes down That's something bad's happening. And so yeah when when developers own code and when developers are on pager duty That you know, that's when things get a little bit more automated and they really start understanding that Their software runs in an environment and that environment needs to be understood by experts who understand how that environment should be optimized And so yeah, there is a lot to the off side of it I think the other interesting thing that's happened is google published that book or About sre and I think that's having kind of an interesting uptick And folks are asking us a lot at least a forced or a lot of questions about how does that apply to me? You know, I have a legacy business. Should I be thinking about this? And so it's actually been a great conversation starter whether you can adopt all those principles or not It's very interesting very intriguing about how Operations individuals can think of themselves as developers as well. So I think it's like about a mindset. It's very interesting Okay, great. Next question. This one is for you mark. How is concurrent dev ops different from dev ops plus agile? good question so I think concurrent dev ops is really about Sort of the know the depth of the integration and you know, it's a lot to do with the tools but then it's a lot to do with what the tools enable and so it's about making sure that your your teams work really collaboratively and have a conversation and And you know can work seamlessly as a team again You can you can technically do that with a hard tool chain But it's easier if you've got tooling to go with it So when those things come together when you've got the tooling that enables it and the culture That enable everybody to work together That's when you really can see this concurrency where people are working in parallel together collaboratively and I think that that's really what that's about now You know, you have to Probably start from you know agile and there's you know, there's agile with capital a and agile with a little a you know You don't have to actually do agile practices to be agile, you know the adjective And I think In a lot of ways, you know lean startup is the other kind of methodology that's out there The point is you need to move fast you need to be reactive But you also need to work collaboratively together and that's where the concurrent dev ops really kicks in Okay, great Still time to get your question in if you have a question for mark or for chris Please go ahead and use that control panel Go ahead and do that. We have still have about 15 minutes or so But we have a lot of questions. So next one. Can you provide an example of what deep integration means? I think this is for you mark Yeah It's a little hard to to visualize with just words But like one of the One of the the things I like to talk about most is the The developer who goes and pushes up a merge request in get lab parlance or pull request if you prefer You know some chunk of code that they're they're requesting to be changed and people are reviewing it and um and normally traditionally You just look at the code and then um, you know Somebody reviews it. Maybe at most they check it out locally and run it and test it and see if it works But otherwise, you know, they just say, yeah, that looks good You merge it into master and then at some point later it gets to deployed to production Then you actually see it live But that's really kind of the end of the story until some ops person comes back and starts screaming and says hey Something went wrong. So a deeper integration would be You know, you create this merge request And we'll create a review app immediately And so you don't have to pull it locally to see what you're doing You can just see this review app and basically review up is like an ephemeral application That is very production-like Um, but is you know, just lives there as long as the merge request lives But you can actually see and interact with it. So qa And business people can see what this merge request is actually about to do But that's that's great But then we go further and say, all right. Well, since you've got this app there, let's um Let's run security tests on it Let's actually try to do penetration tests on the application to see if what you just did introduced any security vulnerabilities I mean, we also do static analysis of your code But we'll do dynamic security tests on that review app and then give you the results right in the review app So you don't have to wait for security to go and do their analysis And you certainly don't have to wait until it's in production to find that there's actually a bug You stop it before it gets into production So then we go further and we'll run a performance test on it And we'll you know, if the simplest thing we'll just hit the home page and measure how long it takes and analyze You know, how long it takes to download and blah blah But we do like a speed analysis of that page So again, the developer can see immediately if what they've just done is going to slow down the application Now we go further and say All right, let's say it did deploy to production and it did cause a problem But again, the merge request is merged So traditionally you'd be like, all right, I'm done with it as an ops problem But we don't we go back and actually say, hey, we detected that memory usage went up in production And we You know, we associated we know that it was this merge request that was just deployed So we'll let you know on the merge request that, hey, this is the impact on production So, you know, it was great that we did all these testing in the review app But somehow it got past the review app and now it's actually in production, but we still tie it all the way back So that's that's an example of deep integration where this thing permeates So many different stages and it's not just like, hey, we ran a test It's we ran all these tests. We brought it back to the developer that made the change so that they are aware of what happened All right, great Okay, next question. I think this actually would be appropriate for both of you Most regulated industries such as financial services and health care mandate a separation of responsibilities that prevent dev From production access and only ops have access to those environments I'd appreciate your perspective on companies that have successfully bridged this construct I don't mind taking that first So we had this very same problem At microsoft when I was working on office 365 and we had to develop, you know, secure development and operations practices to be fisma compliant and so the way we did this was we basically um First of all in order for a developer to get, you know, access to first the developer did not have access to a production system They would upload to a pre-production system and then the operations person would actually Transition the pre-production systems at a production box The other thing that we had was the fact that in order to investigate You couldn't do a live site in order to do a live site investigation We designated like the software engineering leads only could have access to it So in this case, uh, the structure of the organization was, you know, every engineering lead had like say five or Five to seven direct reports. Those leads were responsible for Sort of owning their teams, you know investigation primary investigation They would be the ones that held the pager and they would be the ones that would be the one They'd have to go into a live site But a live site investigation would be like the last You know The last thing that you'd want to do you don't want to do it if it's at all possible And instead what you do is you'd request, you know log files from a certain period The operations team would have a scrubber to eliminate pii or hippodata And you would take that system and put it on to another system that was as close to identical as the The system under test so you could investigate And so The idea was there was a separation developers didn't push code directly to production There was sort of like a firewall between the two and that's how we managed it for regulated industries And so that's how we did it for all actually just because it was like one process was easier to manage the multiple Mark might have other things to add since I've been I've been out of development for about four years so you know At least in that kind of capacity Now that actually pretty much that was that was a great example. I think that's the same kind of stuff we're seeing There's definitely when it comes to compliance and things like that people can take a really hard line And you know go into an extreme and say oh there needs to be physical firewalls or you know no connection between these pieces Or you can have you know a really good integrated approach that just has security baked in that You know we just enforce this via process and you know regulations and permissions and other kinds of pieces So you know the technology can help there But yeah, it's a really interesting lines in where people interpret how to do compliance All right great Okay, so many questions to get through um, please know that if we don't get to your question during the event Mark and chris will actually get a copy of all the questions that come in So i'm sure they'll be more than happy to follow up with you post event But let's uh, let's keep moving forward here Next question can monitoring be used as a pivot point to help create the culture of devops communication That's a really good question. I do see monitoring as this sort of um, you know this tipping point like when you Like you said, you know deployed a production You know cd people have been talking about that for decades really um But once you start adding monitoring into your system, that's when you start to really feel Like you've bridged, uh, you know, like a virtual wall, but you know a real impactful Difference here of actually tapping into okay. Now i'm responsible for The operations of this and that's when when developers feel responsibility, you know, because they see it Um, you know, there's lots of other ways too, right? They could you can add insecurity and then they'll feel that part of it too But but once you've made that transition you're like, oh, i'm responsible for the actual performance of this thing You you feel like that the walls are gone and now you're really working with Whether the ops team is still separate or whether it's an sre team that just helps but whatever it is It does go to break down things So I personally think that can be really good, but i'd love to hear chris as So one one of the interesting things that happened In my engineering days is we had a manager who decided he was going to do a monthly business review And I thought all right that doesn't include me, but then lo and behold It did include me I was the engineering lead for a particular software component and we had to say he goes I want you to tell me every metric that's going on with a live site And I'm like, well, what do you mean? And he's like, you know, I want to know how many 404s are there? How many 500 every's how many you know exceptions are there in the log file? What's the average latency? You know, what's the average this one's the average? He just wanted all these numbers and I was like, what's he bugging me about? So we started collecting all this information and this is back when you know, it wasn't really automated And every single month we'd get in front of this business review board and we report on these metrics And you know, it was really interesting. I was like, geez There's a there's a bunch of stuff going on in the live site that I didn't expect to be happening And it was detrimental to the customer experience And so the idea of monitoring and understanding what's going on with your product and not just sort of like, you know I realized we were just throwing it over the wall We were actually releasing it going on to the next thing and you know We'd let customer support manage what you know, what was coming in and not really care about it But the reality was we were putting out some some code that was you know, less than high quality And once we started getting an idea that the fact that You know, what we're writing is impacting our customers You kind of get a lot more connected with them and you care a lot more about what their experience is And when you can actually move the needle on some of those experiences and realize that There's no reason for people to have some of these crappy experiences You can actually make an improvement You can feel good about what you're doing and you can actually see a return on investment From the customers being happier and to you seeing, you know, less live site investigations You can focus more on just getting stuff done and making improvements So I think monitoring is a really important aspect of tying the developers To the product and letting them understand that You're delivering an experience to customers and and that experience is very important All right, great Next question How much time does it take an average team to go from Evolution SCM or CICD to the concurrent DevOps way of work? That's a great question and it varies a lot. So averages are really hard You know companies that have Really deeply established DevOps or not DevOps, but development practices and that are making this transition You know, frankly, sometimes it can take them a year or two you know, usually becomes a top level initiative and They plan out this thing and it takes them a really really long to make that transformation I've certainly seen it happen faster Um, and I've certainly seen it happen slower Uh, you know, people can talk about it for a long time and then not actually pull the trigger But you know with a concerted effort it doesn't have to take that long I think um One thing that can help there is you know to to start with sort of one project Especially if you get like some new project start the new project in new way and you can start from scratch And it's a lot easier and then you can test things out And then build up some experience and comfort and then you've got these points You can talk to other people in your company about and be like look, hey, this is the success We can drag you over there But um, but you know not every company can really do that Chris, do you have any other do you have any hard numbers on that? Any chance? No, I don't you know what I was going to say is that um, it's a struggle right for even folks to go from You know source code management and CI CD to actually just accelerating it what I'll see is um You know some some it usually happens in legacy industries It doesn't really happen in like you know some of the newer companies Uh, but um, you know, they they they start a pilot and they start going down this whole dev ops approach But then they find out that like it's not working everywhere and a lot of the times um, you know It comes to like uh compliance and regulation where there are people sort of in the middle that feel like You know, we need to follow a certain process or we're not really, you know Following the rules and regulations and I'm going to say it just comes down to culture You need to sort of break down culture and get everybody on the same page understand Here's the new way that we're going to operate Here's the new way that we're going to integrate compliance and and and governance into this process So that it's integrated as mark was saying not sort of like ancillary or bolt-on You want everything sort of integrated, you know What I tell people that are in compliance that say, you know, I need to do 10 audits I'm like, you shouldn't be doing 10 audits What you should be doing is making sure that everybody that's along the value delivery chain Is making sure all the check marks are being done And you should just be be making sure that they're following those rules and regulations And then they're providing you a report that says they did everything it's told them to do You're not your job shouldn't be getting in the middle of all these processes And making sure that, you know, interviewing all the developers are looking out their code They have responsibility to deliver those processes and deliver them in an efficient way These other teams that are working with these organizations need to help them Not kind of get in the way and so the problem is that People are responsible. They have their jobs to do and they and they don't want to get fired for not doing their job And so unless the company sort of like starts from the top and says here's a new way of operating And oh by the way from the bottom up, we want to do innovation We want you to be creative until those two pieces sort of come together It's hard to break through any kind of culture gap And so it really takes sponsorship from the top and it takes sort of a willingness from the bottom You know that the people the worker bees to want to be innovative and think about putting the customer first And think about ways they can break down these walls and move ahead Kind of it takes new thinking and sponsorship of that new thinking to kind of make it all work That's that's been my experience and observation Okay, great. Yeah, that that whole culture aspect is is seems to be a sticking point for so many companies Quick time check we're about four minutes to the top of the hour. I think we can get through one or two more questions Actually, the whole culture idea is a perfect second to this next question Which is how and what steps do we take to initiate the culture transformation and also change change team culture? Yes, so I don't mind taking that one I think it starts with that whole value stream approach where you look at all the steps it takes to get product out the door and then you know Capturing metrics it's almost the same question about when you monitor something and you realize that the process that you're currently using Or the software you're releasing has all these bugs in it You want to make it better and when you monitor the process and you actually put like, you know Voltage meter on every single connection in your whole entire tool chain and you realize there's so much time lost There's so much, you know effort being wasted on meetings and whatnot And mark also talked about, you know big agile versus little a agile, you know Stop following dogma stop following religion and think about what's going to work best for you How do you break down these walls and silos and get things working together and getting them getting them to work better? And nothing helps more than just having facts put in front of you How long does it take to get something from point a to point b? What's holding us up? What do we got to do to make it better? Okay, great Next question is for mark. How do we get users trained as a git lab administrator prior to adoption of the commercial product? Oh, that's an interesting question. Um, I guess there's a lot of ways you can you can do that. I mean, there's uh We have the the free Community edition, uh, what we're calling Libra now You know, you can download that and try it out and try it out on a small project It won't have all the functionality for your large teams, but to get started and to get the administrator playing with that It definitely works um Also, of course, we have the dot-com offering. I mean, you don't even need to install anything Uh to to start to play with projects and set up your csc pipelines and all that kind of thing I mean, we even provide free shared runners for your ccd So up to a certain point you can do a lot of things for free on dot com You could ask for anything more mark Exactly, you know, it's a pretty sweet deal if you ask me Um, but yeah, we I mean we're obviously built on open source We're open core company and so we really give back to the community a lot Um, so there's a lot of ways you can start to make this really easy Um, but you know, of course, it depends on What your what your concerns really are, you know, if you want to talk to a salesperson You can certainly arrange with us and we'll have somebody help you through it But it's really really easy just to get started on your own All right. Great. Well, we are One minute to the top of the hour. So we're going to have to close out the questions now, but uh, um Uh, thank you to everybody who did submit a question And as I said earlier if we didn't get to your question I'm sure these guys will be more than happy to follow up with you post events So, um, don't uh, don't feel like your question is just hanging out there And it's going to be completely ignored. Um So, uh, Chris condo and mark punsack. Thank you both for joining me Um, I I I know the audience got a lot out of it and I certainly got a lot out of it And I also want to remind the audience that if you missed any or all of today's webinar It is being recorded. So, um, you will receive a link post event that will take you to the webinar on demand You can also find the webinar on the dubops.com website We also have a listing of other webinars on the website. So, uh, take a look see if anything piques your interest Um, Chris mark. Thanks again for joining me. I appreciate appreciate your time. It was a great, uh, presentation Thank you. Thank you. And I hope everybody has a great day