 Hi everyone, good morning. Welcome to our talk on standardization and security. My name is Vinnie Carpenter, and I am the vice president of engineering at a company called Northwestern Mutual. I lead the cloud and DevOps organization at Northwestern Mutual. I'm honored to be joined by my colleague, Ravi Devanini, who is the senior director of engineering and it leads our DevOps engineering group. By the way, I totally feel overdressed. I should have listened to Ravi, but you know, I didn't, but I'm gonna persevere through, so. So, today we're gonna tell you this story of the journey we're on, of standardization, simplification, and how that journey has led to improvements in our overall security posture. But before we dive into the meat of the topic, I just wanted to spend 30 seconds to tell you a little bit about our company, just to sort of do some context and level setting. So we're a Fortune 90 company that strives to be at the center of our customer's financial life. We do investment, insurance, advisory services, planning. Our goal is to provide financial security for our clients. We have about five million clients and a revenue of about $34 million annually. We're a privately-held, 165-year-old company, right? So that's the key part. We've been around for 165 years, so as you can imagine, that means a lot, so just putting that out there. But we're lucky to be privately held with great financial strength. I think we're the only two companies that have a triple-A credit rating from Moody's and Fitzgerald's rating agencies. I believe Microsoft may be the other one. So on the agenda today, we're gonna start by talking about and discussing sort of what I think is a seminal work of Barry Schwartz, who is an American psychologist, author. Ravi's gonna talk about it a little bit. And Barry talks about this notion of paradox of choice. So then we'll dig into our experiences with sort of unlimited choice, unlimited flexibility, and then sort of pivot into how standardization has helped us simplify, and more importantly, secure our environment. As part of this, we'll talk about the cultural implications of standardization and what that means for us and share some of the outcomes and the lessons we've learned as we embarked on this journey. So without further ado, Ravi, take it away. Thanks, Vinny. Hey guys, I'd like to introduce this concept called paradox of choice, right? As Vinny mentioned, there's a lot of these ideas are borrowed from this book by Barry Schwartz. We highly recommend this book. So the first and the most common example of paradox of choice is found at a grocery store, right? There's a target nearby, I think it's this way. Try walking into any aisle. You will see 14 varieties of Cheerios, 23 varieties of olive oil, 40 varieties of toothpaste, 15 varieties of bottled water, right? 55 varieties of shampoo. If all of this is giving you a headache, don't worry, there is 80 different varieties of pain relievers in the over-the-counter medicinal, with aspirin, acetaminophen, ibuprofen in the form of capsules, liquids, tablets, you name it. That's a lot. Next example is TV, right? Because in America, we spend most of our time doing two things, watching TV and shopping. The number of TV channels you get from cable TV is 1,761. And let's look at some of the streaming services, right? Netflix has over 4,000 titles available. Prime Video has even more, millions of hours of content. Just scratching the surface, right? And many choices in the providers, way too many shows and way too many movies available. This is practically true for everything we see in real life, right? Choice is a luxury. More number of choices is considered as a symbol of privilege. Too much choice is essentially the current reality of our society. But wait, isn't that a good thing? Having a number of options will help us choose what's good for us, right? Not really. According to the research, look at the graph here, right? Like you will see as the number of choices are small, the satisfaction is low, right? But as the choices increase, the satisfaction increases, but more the number of choices, as the choices keep increasing, people tend to feel more pressure, confusion, and potentially dissatisfaction with their choice. This can then lead to anxiety in decision analysis, paralysis, and dissatisfaction, right? There are several names for this same phenomenon. It's called fear of missing out, FOMO, right? And law of diminishing returns. Whatever you wanna call it, this is quite real. There's another more scientific angle to this too, right? There's this research by American Psychological Association. It's very official. It says that the more decisions an individual makes in a short period of time, the more each decision decreases in quality. It creates what is called a decision fatigue, right? Because we have a finite amount of energy to exert, the more energy we use for trivial everyday decisions, the less energy we have for critical ones. Steve Jobs, right? He used to wear the same kind of dress every day, every single day, black turtleneck and jeans. And that's all you ever saw him in, and everywhere he went, he was like that. President Obama used to wear blue or gray suits, and that's the statement he made in an interview with Vanity Fair. He talks about paving down trivial decisions because he has a lot of other decisions to make. There are many people who eat the same kind of breakfast every single day, including Obama, five eggs and a toast every single day, right? Now the paper talks about multiple ways in which we can fight decision fatigue, right? But we don't need to go into that. This is not a psychology conference. Instead, we will talk about how this idea applies to us in the cloud-native world. So this is a quick look at the tool landscape. This is just a small sampling of a plethora of tools available in the market. We try to organize this into logical categories, right? All of these logical categories are high-level steps in the software development life cycle. At each of these stages, you will see how many options we have that provide a similar capability, right? For example, source code. We have GitHub, GitLab, BitBucket, code commit from AWS. All of these tools provide a way of version control. Similarly with coding, right? There are hundreds of programming languages available, and I only listed a few here. I'm not talking about all of the options available in the market, right? I'm talking about how companies typically end up with multiple tools that they actually use for the same capability, right? For example, a company may have GitLab, possibly some GitHub, an older version of SVN, and maybe a version control for mainframe as well. That's like four different tools for the same capability called version control. But the question is, why do we end up with so many tools, right? There are two main reasons for this. Technology evolution. A company like ours, Northwestern Mutual, we have been doing technology since the 1960s, Vinny, and we're big on experimentation too. And we have experimented quite a lot with technologies that were in vogue at the time, and that footprint of that technology never actually goes away. And then you try something new, and some of the teams adopted, so it creates a tool sprawl, basically. Next is what I call as technology islands, right? This is basically also called a shadow IT. If a company is sufficiently big, there's a lot of technology teams that are spread over several departments, right? And these teams tend to create their own set of tools, and hence these islands are created with very little communication between these islands. So shadow IT is also created sometimes when a team decides that they're gonna do something of their own, because just to work efficiently, right? Because everything else is slow, we wanna do our own thing. So what happens, right? When if we have so many options in the tooling? First is high cost of maintenance. You're probably paying multiple licenses for the same capability, right? Or you're probably logged in with multiple vendors. The automation that you create is reusable only within your technology island and not broadly. So your technology islands are creating these islands of automation, right? Too many tools, and hence the complexity is high. And you're creating standards, if you wanna create standards that apply to the whole enterprise, that becomes a challenge. Toolspral leads to data sprawl, to your data is all over the place, hence capturing metrics that measure your success, that becomes a challenge. Security is a challenge, obviously. This is a security conference, and with so many tools in our supply chain, our attack surface is really high. Supply chain attacks are one of the most common ways in which attacks are happening these days. It has, if you look at the stats, two years ago to now, it's like it has risen by 430%. And you need to be constantly patching all of these tools too, right? This adds to the cost as well, and obviously that results in a lot of lost productivity. All of these tools mean that your skills are siloed, your team members can change from one team to another team very easily, and Vinny's gonna touch upon that concept a little bit more later on. So what do we do? So what I've described so far is like a buffet, right? What we say with a buffet is, here's all the things that you want to eat, and go make your own plate, right? When we do that, every plate is different from every other plate, right? Very few people think about nourishment and health when there's a buffet, like people wanna eat. I would load up on dessert, and then there's a lot of wastage too. With too many tools, that's exactly what we're doing, right? We're saying here's all the tools and go create your own applications. What we wanna say instead is, we will provide a set menu, limited options, but good options. We think about health and nourishment. We wet and select these tools very carefully, very intentionally, and we wanna streamline our process, we wanna streamline our tools. We say that we wanna improve efficiency and free up resources to invest in things that are actually differentiators for our business, right? Because four tools for source control is not gonna add business value, right? Basically, a purposeful evolution of technology is what I'm talking about. We say, we wanna create a golden path, a paved road, a standard approach for creating applications, right? With all the standards and all the god rails built into this paved road, and all you have to do as an application developer is to drive on that paved road. So an application development team basically doesn't think about how to actually deploy their application, or what supply chain tools to use, or what CICD security god rails they should use. All they do is develop and deploy their application. This is an example of what a standardized platform might look like, right? You can see this is very different from the other diagram, very limited number of tools, right? And these are standardized across the enterprise. This is our set menu. If you need to write code, you use GitHub, and maybe a small number of approved programming languages. That's it. One tool to do your builds, one place to store artifacts, and one tool to orchestrate your deployments. One tool for infrastructure as code. Maybe a couple of monitoring tools, right? Intentionally standardizing your tools landscape. Now compare that with this, right? This is a buffet. This is definitely much cleaner. Now, once we standardize on the tools, right, we want to create a single way in which application can be deployed all the way to production. All the steps, for example, that a change needs to go through from commit all the way to production. That's your pipeline, right? Like we want to standardize that pipeline. One pipeline to rule it all, right? This basically provides a standardized build, scanning, testing, artifact storage, and signature, and all the way to the deployment as well. If you do this, right, a lot of different development standards can be enforced. Things such as code reviews, artifact security, if you're an application developer, you don't need to think about how to deploy your application, right, or what security steps you need to take. You just write code, you just write your unit tests, and everything else is ready to go. That's your payroll, you're just driving. I often think of developers as cooks, right? They're good at making a meal. Not necessarily good at taking that meal to the customer's table. That needs to be somebody else's job. So if, one of the examples we talked about, we said we want to use only a few programming languages, right? So what happens if someone just ignores this mandate and chooses to use the programming languages that's more modern and more fun, right? What can we really do? So what I've showed you before is a carrot approach, right? What we're saying is if you want to deploy faster and in a more secure way, right, use our standardized pipeline, that's a carrot. You want to start off this process by offering carrots, right? But only carrots and no sticks, it's not going to work. We need both. We need to enforce certain standards, standards such as code reviews or the fact that you need to create a change ticket for every production change or security scanning in your pipeline. Even things such as setting limits on how much cloud spend you want to have you want to have in your organization or how much CPU limits you want your applications to use, right? Pretty much anything your organization wants, we need to be able to enforce it. And this is the key to success. Sticks are as important as carrots. I can see some of you are frowning. This sounds like way too much policing and less freedom and stopping everything, cutting down everything. There's obviously a lot of cultural and organizational implications to this whole idea. We need to going to touch upon a few of those. Thanks, Ravi. So I think Ravi essentially laid out the case where unlimited choice and unlimited flexibility isn't really what it's cracked up to be. I think having unlimited choice, at least it's been our observation, leads to cognitive load, anxiety. And I think really thinking about like the broader picture of the enterprise, it leads to variability and the complexity in the environment, right? Decision fatigue as Ravi described is a real thing. I worked at organizations where developers had unlimited freedom, right? So you could write and go or Rust or Java or JavaScript or Python. You could pick Spring or Spring Boot if you wanted to or you could make your own. I mean, unlimited choice led to when new engineers would start on the team, especially decision fatigue, they're like, how do I get started? What's the standard way that I can do it? What's the majority of developers in this organization doing so that if I need help, there's somebody I can go to? And so I think decision fatigue is a real thing. And so I'll talk briefly about sort of the process we followed on choosing the right tool. And I'm gonna start with this quote from Alan Kaye. If you're not familiar with Alan Kaye, Alan was a computer scientist at Xerox Park in the early day where I think literally everything that we use today in computers came from Xerox, including the mouse and graphical user interface and all of that, right? And he's a pioneer in the world of object-oriented programming. And I just love this quote. His quote is very famously quoted saying, simple things should be simple and complex things should be possible. And I think that idea sort of enforces our approach when we're picking tools. Robbie talked about, we don't wanna pick a tool because it's new, it's shiny, it has all these things. When you're going through this tool selection process, think about the end goal in mind. Have you looked at the pros and cons of this tool? Have you thought about how this is going to integrate into your environment? Robbie described earlier the sort of perfect world we have this notion we internally call our ideal pipeline. And so what we give our developers is the ability to write code in a prescribed number of languages and frameworks. All they have to do is they commit their code to GitLab and then magic happens. The pipeline kicks in. So whether it's container scanning, so we use Sysdig for container scanning, right? All of that's done for you, vulnerability checking, coding standards checking, automated testing, automatic artifact checking, automated deployments to production, automated, like all of that is built into the pipelines. And so it simplifies the role of the developer so that he or she is just focused on writing code and doesn't have to worry about, how is this going to code get to production? He described the chef making the meal and then somebody else taking it to the table. The engineers are focused on just writing code and not having to worry about all of the things that go around making this, building it, testing it, securely checking into an artifact repository, making sure it goes through the right communities environment, making sure it has the right scaling parameters, making sure cost is managed, all of that stuff is done for you so that you can end up in a world where you're just focused on code. One of the things Rob and I were talking about, as we're kind of discussing this presentation in this slide specifically, I recall going to a presentation, I think it was 2015, the CTO of Etsy presented, I think it was in 2015, and the title of the topic was very provocative. He said the title of the talk was Use Boring Technology and it's a phenomenal, it really is a phenomenal talk. If you get a chance, Google Use Boring Technologies and please do it after this talk, don't do it now, but really the premise of his thing was, sometimes we're all attracted to try the new and shiny things and we wanna do that, right? We wanna experiment, we wanna allow people opportunities to try new things, build new things, but the premise that he's making is, if you're using Java and Spring Boot, for example, if you're using JavaScript and Node, or React, or Python and Flask, these tech stacks are being used by millions of organizations across the globe. Tens of millions of developers have sort of battle tested these tech stacks in every single scenario that you can think about. So there's not a use case that you're gonna come up with that somebody hasn't already gone to production with and so all of the issues, all of the potential known issues are known, either resolved or documented, so that you're literally standing on the shoulder of giants because they've done the validation for you. Maybe you start with the Rust and this is a bad example, Rust is an awesome programming language, but I'm just gonna say Rust is a relatively newer language is so all of the issues, maybe not known, right? So if you're going to production with a Rust application, you might be discovering things that maybe somebody else hasn't discovered yet, right? So this notion is, focus on business outcomes and not really on undifferentiated technology choices, right? Technology is cool, we're all engineers and I think technology is critical, but focusing on undifferentiated technology, like if you already have a source control management system, you don't need another one, unless it does magic for you, right? Unless it has chat GPT built into it and just writes your code for you, right? Maybe then it's worth migrating to. So talk a little bit about this notion of tools consolidation. As we pivot to kind of focus on business outcomes rather than undifferentiated technology choices, I think the path becomes clear, right? Because you have a vision in mind of how we want to get there and as you embark on this journey, we went on a journey, we used to have GitHub Enterprise as our source control management system, it did everything, all of our pipelines ran in there and we migrated to GitLab. You just have to know it's a multi-year journey and as you're doing this migration, it can't be a big bank thing, right? You have to figure out a way to do this where you're offering incremental value so that you don't have this. We're gonna go do this, we'll see you two years later and there'll be this new shiny thing in two years because technology will change, the business will change, the organization will change, the needs will change and so these things have a long life but you have to provide incremental value as you go on this journey. The other thing, obviously, pretty obvious, like you wanna do it in a way that you're not disrupting the business because obviously that's the most critical thing. Data migration is pretty critical, right? Like again, I'm gonna beat to death this example of moving from GitHub to GitLab as an example, right? Like the commit histories, merge requests or pull requests and comments and all of that stuff, issues have to come along so that the new system essentially has all of the data that the old system has. So you could safely go and say, I'm on system B, I can cleanly turn off system A, retire the hardware that was running on and not have to pay for licenses, right? Because what we've observed being a 164 year old company is, I think we bought our first mainframe in 1950 in the late 50s, I believe and a lot of the technology, unless you're aggressively investing in decommissioning old technology, there'll be a facet where you migrate 80% of it and then this 20% just stays around and then now you're responsible for care and feeding of it, securing it, making sure it's not vulnerable. So the last note is, as you pick new tools, training on engineers and training the engineers on the new capabilities becomes critical. So one of the things that comes up often when we talk about standardization, right? We talk about standardization, governance, like they all sound like developers instantly revolt and go, whoa, whoa, whoa, what do you mean? Like, you hired me because I'm an awesome developer and I just want to write code and so this question always comes up is, is standardization detrimental to innovation, right? I think the answer is no, right? You can innovate and live in an environment where you have standards, right? Like you think about technology decisions, I've been doing this for almost 30 years and think about every single technology decision is a set of trade-offs, right? You start on a path forward, you typically want to have a bias for action, you don't want to spend weeks, months and years spending time just doing analysis paralysis. So based on the information that you have, you've got to move forward, but you need to be able to pivot as more information becomes available, you sort of change your path. Historically, a lot of the organizations I worked at, we've given our teams a lot of decision making at the local level and I'm gonna describe myself as a recovering software engineer. I wrote software for 20 years and as a geek, software engineers love building stuff, right? And I know I've done this in my career where you're building a new application and there's this great five, like five to six great caching libraries that are in the open source that are used everywhere and you're like, yeah, but I could just write my own and so you do because that's what developers do, right? And so I think it's crucial to know what's worth building and what just is a commodity, right? This is not to suggest that we shouldn't let people experiment. We want to encourage people to experiment. We want people to try new things. We want people to build. Like we encourage people to write software at Northwestern Mutual. We encourage people to open source stuff, build communities, participate in open source projects, give back to that, right? Not to suggest all of that, but like focus those energies on things that differentiate your business, right? Don't focus on commodity. Like building another caching framework, you're not gonna make anything better than Redis or Memcache or the sort of leaders that exist in that space. Is it a good thought exercise? Great, do that in your 20% time if you want to just to learn and grow, but think about like building things that are differentiating and not sort of commodities. And the last thing that I think is really important and every organization has to arrive at it based on how they work, but it's the incentive model. What is the incentive model at your organization? I worked at places where if you open source something and you got a bunch of stars on GitHub, like that was celebrated because it's like, hey, you've done a phenomenal thing and it is a good thing, right? Like we want more of that again. I keep repeating myself because I wanna make sure we should encourage people to contribute to open source. We wanna keep doing that. But if that's the only thing you celebrate, that's the incentive model you're setting for the organization that, hey, if you do something cool, you'll get accolades. If you do your job and do an awesome delivery on a project and have phenomenal business outcomes, well, that's great, but like, we won't celebrate that publicly, right? So you have to have this incentive model where you celebrate both things. You celebrate people doing really cool and innovative things, but you better also celebrate phenomenal business outcomes because that's really why we're here, right? To build and deploy and secure our businesses. So talk briefly about outcomes. I know we're a little over time. I'm gonna speed through some of these things. So just a couple of quick items. Centralized governance, I think we talked about, is possible. I think Ravi described the, I think we call it the one pipeline to rule them all is like, we encourage our application teams to use kind of the centralized pipeline. So all of the things that need to happen, whether it's built in a certain way, code quality scanning, checkmarks for vulnerability scanning, assistive for container scanning, X-ray for open source vulnerabilities, like building a build a mature, like all of the things that you can think about and future needs as our security team thinks about more things to do to secure our environment could be built into that pipeline and the developers don't need to change anything because they're just leveraging our standardized pipeline and they're getting the benefits of all of that. I think one of the other benefits that I briefly wanna touch on before I turn it back over to Ravi is this notion of standardization, right? If all of the engineers on your team are working in a similar way using similar programming languages and tools, using similar frameworks and processes, it's easy to essentially swarm engineers to the biggest opportunity, right? We talk about this notion of fungibility, right? Like if you have 50 engineers and they're across, let's say, six or eight different teams and they all work differently and use different programming languages and use different ways to sort of get code into production, if we had a new business initiative that was gonna be transformational and we wanted to put all those engineers on it, there's a lot of ramp up time to say like, okay, what are we doing? Like are we building in Rust or Go or JavaScript or Python or Java? Okay, let's pick one or maybe we just build APIs and it doesn't matter what we build in but there's a lot of sort of organizational structure and friction that comes into play. But if you're all sort of building in a similar way, leveraging the standard pipelines, the standard tool, deploying in a standard way with the standard set of security controls, it's a lot easier for us to come together and make that next business initiative successful. So Ravi, I'm gonna turn it over to you talk about some lessons learned. Thanks, Mini. So in all of these, there were a few key lessons that we've learned. And I'm gonna try to sum it up quickly. First, single source control is an absolute must. If you have multiple SCM systems, your code is fragmented. Some source code is living in one place and other source code is living in another place. Your developer permissions are fragmented as a result. Like some developers have access to some repos and other developers have access to other repos. That affects your developer collaboration. If, for example, a developer creates a common library that could be reused across all of your software, which of the four tools would you put that in, right? Like how would you make sure that is not being duplicated? Not to mention the high cost of maintenance, double licensing cost, quadruple licensing cost probably. It's a nightmare. So if you don't have a single SCM system, please fix that first. Like one SCM for your entire organization is a must. Now we talk about DevOps as a culture and not a role, but every application team is a DevOps team. If you think about the idea of DevOps team, a DevOps team is an application team that is cross-functional, which means that team does everything that an application needs and every skill that the application needs to be deployed to production is within the team. But we don't want to end up with several applications optimizing their application teams, optimizing their process locally, because more often than not, local optimization leads to global degradation. A team that takes a global view of things, right? Of the entire process of all the automation of the common libraries, right? Creating an overall vision for your company and centralized metrics and centralized governance. All of this is essential, right? Like you can call it a centralized DevOps team, you can call it a tools team, call it a platform engineering team, whatever the name is, right? But such a team is essential. Next, we need to talk about this idea of picking the tools with end in mind, right? And we have to understand that when selecting a tool, we don't need the best tool in the market. We need a tool that fits the business needs, right? If a tool is pretty good and it fits all your requirements, that's all you need, right? And choosing one tool that is pretty good is much better than having multiple amazing tools that do the same thing in your environment, right? Consistency in the environment is that it leads to far better outcomes than having multiple custom solutions. And finally, we talk about standardization and governance and security. It could be considered as boring and too much policing, but it doesn't have to be, right? It can be made fun when we can implement a lot of gamification techniques and build excitement and make it fun, right? This idea of gamification exists everywhere in the real world, loyalty programs and badges of achievement and reward points and all the big companies do this. Like a leaderboard, for example, right? If you want your teams to reduce security vulnerabilities, create a leaderboard that ranks the teams that reduce the most vulnerabilities and reward them every quarter. Or if you want evangelizer and DevOps practices, right? Measure those and create incentives that reward people for such behaviors. Goes back to the point that Vinny was making about incentives aligning to your business outcomes. And this is a great way of making standardization and security super fun. That's all we have. Thank you, folks, for coming. And I want to thank CNCF also for giving us an opportunity to come here and present some of our ideas. You can see the QR code. You can scan it if you want to send us any feedback. But please, let's have some questions. Go ahead. Yeah. Yeah, so the question was, we couldn't hear it, is let's say your enterprise standard is C++ and a developer comes up and says, hey, I've been learning Rust and it's a phenomenal language and I'd love to sort of start doing work and incorporate that maybe into our standard. So great question. So we actually have a formal process, right? Because as I said, every technology decision that you make, even standards that you make are based on the information that you have at hand, the needs at that time, needs of all things change, technology changes, right? So we actually have a process. So all of our standards were considered essentially living documents. So we have a process, we have a centralized architecture team where you can come in and say like, hey, here's why I think Rust should be an approved standard language and walk through the pros and cons of it. And as I mentioned, the formalized process will take that into account, figure out the value proposition and if it makes sense, actually make it a standard. So we absolutely have that kind of process where our standards evolve. And we very intentionally want people essentially to submit, you know, a merger quest essentially is what we say to our standards because standards can't live in a vacuum. They have to be kind of living, breathing documents. So we absolutely do that. Just to add to that, for every business process, right? If there's a rule, there needs to be an exception. And in this climate where your technology is evolving constantly, you cannot be married to a rigid process and say that everything is fine, right? Your process needs to be flexible as needed. So we all, we definitely love an exception process. Yeah. There's a question in the back. Yeah. Yeah. So you want to talk about Dora? Yeah. If you look at the Dora metrics, right? There's lead time to change and number of change deployments per, you know, whatever quarter and time to restore. All of these four metrics are defined basically looking at speed and stability, right? And there's a lot of, if you look at the book on Accelerate Nicole Forsgren and Jess Humble, right? They talk about several behaviors that actually lead to such outcomes, right? These are business outcomes, but a lot of different behaviors actually contribute to that behavior. There's like 24 behaviors that are listed in that book. And collaboration is definitely one of them. How your people collaborate, how closely interactive they are, reducing the silos and being able to release. If I want to do a release to production, I don't need to talk to a separate team, right? I don't need to ask their permission. My team needs to be flexible enough, you know, or we need to have the right set of tools that so we can release the application to production. So definitely if you look at Dora metrics, they provide a lot of business outcomes and you're actually measuring business value also along with a variety of other behaviors. Thank you. Great question. Any other questions? All right. Thank you so much. Thank you for spending the last half an hour with us. Thank you again. And if you got a chance, please send us some feedback. Thank you.