 This is my second live talk since 2019, so it's really actually quite pleasant to have to drive somewhere even through the Canadian border, which was actually very nice this morning, to be here and give you a chance to talk. I am cognizant of the fact that I am between you and a coffee break and some demo sessions that are out there, so hopefully this will be a little bit entertaining. I'm not quite up to live demo status yet. I'm working towards that in another week or two. You'll be doing live demos, but I'm getting back on my feet as it were in the public stage, so that's nice. I wanted to start with a little bit about where Kubernetes started and where it is today, and I think that one of the things that it started with actually was deployment and continuous delivery, and what really motivated me when we were starting out a long time ago at this point was the fact that I was trying to put together these reliable deployments, and they worked, and we were using the DevOps tools at the time, a mixture of Puppet and Chef and Salt and Ansible and Bash, obviously, and it worked, I would say, about 85% of the time, and then you'd massaged it and everything else, and it eventually would work maybe 95% of the time, and so Kubernetes started effectively out of this idea of deployment, and the fact that it was too hard, too unreliable to get to a place where you could actually deploy things successfully. I think it was combined with a lot of interesting things that Docker was doing in the container revolution around hermetically sealing and packaging up of an application that is a necessary prerequisite to thinking about how you then deploy it, and I think what's amazing that's happened over the last decade is that it's actually become the way that we do things. It's moved from an idea that I might be talking about here to something that is just the way that it works, and it's just the expected way that it works, and admittedly, everybody's sort of at different varying stages of their evolution towards containers, but I don't think that it's something that I need to go on at length about why we should be doing that, and I think what's nice now is that we're actually, with things like GitOps, we're starting to actually take another crack at the process of deployment, not just the thing that leads into the deployment that Kubernetes orchestrates, effectively how do I get my configurations from source control all the way through and out into the world, into a place where Kubernetes can do something with it. So I'm going to step through a little bit about what was motivating us at the time, and then hopefully we'll tie that back to what I think GitOps is doing today, and why I think it's an important piece of the toolkit when we're talking about reliable delivery. One of the most important things that we talked about was declarative, and I think at this point hopefully declarative configuration is not something that is controversial to anybody, but honestly at the time it was. There was a lot of scripting that happened out there. There was a lot of programs that ran to do various things, and honestly when you look at sort of a CICD pipeline, it kind of actually is a pretty imperative thing at sometimes, even if it's been described in YAML, it's a workflow diagram that's effectively executing a program. One of the things that I think is nice about some of the move sorts GitOps is it really reinforces that declarative nature of Kubernetes from the GitGo. Another really important piece of what we were thinking about, at least what I was thinking about was how do we empower people to do more? How do we take away a bunch of the burdens that they were having to deal with on the day to day of whether it's unreliability or just thoughts that they had to figure out all of the list, I mean in honesty all the list of the things that a junior software engineer needs to go through that was referenced in the earlier talks. Some of that is absolutely things that you need to learn, but a lot of it hopefully we can take away with the right kind of processes and the right kind of sort of analysis of what is confusing and what is hard for somebody to figure out. So that sort of sense of empowerment is another important piece and I'll hopefully tie that back to where GitOps is coming from. Another really interesting piece of this is automatic, right? And I think that one of the analogies that I like to say is then we talked about this at the time is that Kubernetes is sort of the autopilot for your system. And I think that the move towards GitOps and the move towards some of these models also helps things be a little bit more automatic which in turn makes it easier for everybody to not really have to think about what they're doing and simply have, you know, I set a thermostat, it doesn't really, I don't have to sort of constantly be thinking like do I heat it a little bit more? Do I cool it a little bit more? It just sort of takes care of it in the background. It takes what I want to be present in the world and it turns it into the reality that surrounds me. And then finally separation of concerns and I think this is an important piece of everything that we've done really in DevOps, in modern distributed software development, in Agile development, all of these pieces, I mean we've all been sort of like moving around this idea that we're better off if we separate our teams out into relatively small groups, relatively focused on specific things and have really clean lines of separation so that I can sort of know where my world stops and where your world starts. There's this great Robert Frost quote that from one of his poems that good fences make good neighbors and I think it's really true in the systems that we build, it's very, very helpful to have a concrete understanding of sort of where my zone of control is, where someone else's zone of control is and it's not about like, you know, keeping people out necessarily, it's really about understanding responsibility and the ability to actually let go and say, yeah, I'm gonna care, you know, what happens in my space but I may not care quite as much about what happens in their space as long as that divider line, that contract that I have with their neighboring services or with anyone else, as long as that contract is being upheld. And I'm gonna start with that into the real meat of what we're talking about when we're talking about GitHub's by branching into something that was sort of referred to earlier in the previous talk around the fact that actually in modern software development and in modern application deployment, there's a bunch of different personas that are coming in, right? I think this is not something that, this is something that I didn't really appreciate honestly when I started in the industry and even honestly at some level when I started, when we started working on the Kubernetes project, even you're a developer, you kind of like, I don't know, I will say that you take on a little bit of a mono focused view of the world that like I build programs and I deploy programs and I guess there's other stuff that happens somewhere else, but except for when they bother me, I don't really think about it too much. And as you talk more and more to people who are building and deploying software, you actually realize that there are a whole bunch of these personas that come in and I've got three of them up right here. We talked, the previous talk talked a little bit about sort of the security persona that is responsible for ensuring compliance is responsible for ensuring baseline security that you're all using the same certificate authorities if you're still using certificates or you're still using the same identity providers. All of that stuff is one of these personas that comes in. The platform persona is one that I think is becoming a more thing that people are thinking about over the last two, three, four years. This idea of platform engineering is something that is becoming something that people are talking about where people have sort of said, hey, Kubernetes takes you to a certain level. It actually really doesn't help you become a better developer of distributed systems and for most of our developers, we're actually going to have to put another layer on top, call that a platform and actually really have a team that specializes in how do I build that platform. They're not really building applications. They're building sort of the framework that makes building applications easier and then of course, obviously there's developer personas, many different teams deploying many different applications. Now, the challenge of this is they all come together in the same cluster. I need to have intrusion detection software. I need to have monitoring and observability. I need to probably have storage systems or other kinds of platform level componentry where I don't necessarily want every single developer trying to decide which version of a no-SQL database that they should be deploying. And then finally, obviously I'm going to have my application teams. There's a lot of problems that come out of the fact that these multiple personas have to mix together on this cluster. The first is that obviously, if you have to have people deploying into these places, you have to figure out who's going to actually get access. Am I going to let my security team have control over every single cluster that exists in my company? That could be hundreds to even thousands of clusters in a large organization, and it's definitely a bit of a risk to have one group or one identity that has access to all of that infrastructure. Similarly, if I'm a developer, I don't necessarily want to provide access to the security team into that infrastructure, into my part of the applications because I don't want them messing around and accidentally deleting the serving application. They don't really know. The truth is that they don't necessarily, they're very focused on what they're doing. They don't necessarily know what's going on with my application. And similarly, for platform engineering, we need to have a ubiquitous platform all the way across for it to be effective. You need the same platform across the company. But you probably don't want them mucking around with the individual components on individual clusters. And vice versa, the security team probably doesn't really want to have to go to every single development team. In fact, I know they don't because it's increasingly painful to do this sort of stuff. They don't want, when they need to roll out a new version of intrusion detection or they need to roll out a new version of their own software for logging, monitoring, and that sort of thing. They don't want to go to every single development team and say, hey, by the way, we really need you to update, and here's the new version. That's a recipe for a lot of meetings, a lot of delays. And in my experience, literally months of time spent trying to roll out a new version of a particular piece of software. And so I think GitOps really helps turn this into something that's feasible. And the reason that it turns into something that is feasible is because if you install something for GitOps, like in the Azure Kubernetes service, we use Flux, although Argo and there's other solutions as well. There's great solutions out there in the CNCF landscape. If you install a GitOps solution, suddenly it's no longer the case that each of these personas has to manage a cluster. It's now the case that each of these personas has to manage a Git repository. It's important to realize that actually you have flipped your access control a little bit, right? I don't think we necessarily think of version control system as being access control to a cluster. It is in the world of GitOps, and so you have to be definitely very careful. Two factor, always a good idea, definitely a good idea in this context. Similarly, using true identity providers, using the same identity provider that you're using for your documents and email and everything else. All of these things are important considerations that may not be a part of what went into your source control decision making. But once you do that, then you have effectively partitioned out your cluster. So you can have a namespace that's associated with the security team. You can have a namespace that's associated with an application team. You can have a namespace or multiple namespaces that are associated with platform teams. They each have individual access to the individual repository that happens to represent the state of the world for that particular area of the cluster. But actually, if you go all the way to the extreme, effectively nobody has to have standing access to do anything in the cluster. Because the GitOps agentry that's running inside the cluster itself knows how to talk to that Git repository, knows how to pull things down and make it happen, right? So this separation of concerns and specifically taking that one cluster and turning it into multiple clusters connected to repositories where the state of truth is stored, means that there's a lot of power for those multiple personas without worrying too much about having them step on each other's toes. I wanted to highlight another really valuable piece of this that I think GitOps manages for people, especially in the security context. There was a little bit of discussion in the previous talk around software security, bill of materials, all of this sort of stuff. This is an area that has become increasingly important to people over the last few years, very much so. And I wanted to really highlight that one of the value propositions of GitOps isn't just the Git to the cluster part. It's actually what you can do on the repository side also. And again, this is stuff that is possible with CICD. You can set up Jenkins and you can set up GitHub Actions and you can set up any number of traditional pipeline systems to take the output of a pull request merging into a Git repository, turn it into reality on your cluster. But I think there's just something about the fact that it is so automatic in the world of GitOps that makes it, at least I've personally found that it makes it significantly more powerful. So this up here is an example of a PR. This is connected to my own home cluster where I happen to be running a Prometheus and Grafana to monitor some internet of things, things that are happening in my house. And honestly, previous to using Dependabot, which sent me this PR, and GitOps for managing the deployment, I fell way behind. I was six months behind, eight months behind, nine months behind. These sort of stuff that I wasn't really developing, right? Whether it's this sidecar, which is responsible for paying attention to and downloading configuration maps to configure Prometheus, or whether it was Prometheus itself, it just didn't stay up to date. Just because you had to sort of figure it out and you had to do a manual change and you had to merge pull request. It's just not that it was hard, it's just annoying. And so this workflow single-handedly transformed me from being six to nine months out of date to being less than a week out of date. Because Dependabot scans my repository every night, figures out if there's new images that are available for the various components, sends me a PR. And then once every couple days, if I'm bored, if I'm backstage, wherever it happens to be, I flick up GitHub on my phone, go through a bunch of PRs, merge, merge, merge, merge, merge, and then I forget about it. And in the background, the GitOps agentry that's running on my cluster connects to this repository, sees the change, merges the PR, takes the new PR, deploys it to Kubernetes, new images pop up, and I'm up to date, right? And so literally the cognitive load of me staying up to date in my own home cluster is three minutes of my phone backstage. So this workflow, and again, I don't wanna be like, my gosh, it's brand new because you could have done this with CICD, certainly possible. But for whatever reason I found, because setting up a CICD system is its own sort of cognitive load, this inversion of like, I can just point it to Git repo, Dependabot can merge PRs into that Git repo, has made it ludicrously simple, and effectively inexcusable for me to not stay up to date. And I just totally over talked that slide, so we'll move on. I think the other piece that I think we really see a bunch of value with in the GitOps workflows is actually in enabling that developer persona. And this is sort of where the platform engineer and the developer intersect, which is that for a lot of developers, they really don't want to be thinking about the specific details of how they get their code deployed into Kubernetes. They wanna push, whether it's Java or Python or .NET or Rust or Go or whatever your favorite language is, they wanna push some code, get a container image build out the other side, and then have that deployed immediately, right? And again, I think that the value of the GitOps thing here is that it enables them to do this at scale, right? It enables them to say, actually, you don't have to necessarily worry about the fact that there's five clusters in our organization. You don't necessarily have to even be really aware of where those clusters are, because when you switch from a push model of pipelines to a pull model of GitOps, all of those clusters, the number of clusters that you have, it stops mattering quite as much, because you're really only dealing with the Git repository. If a new cluster comes into existence or an old cluster goes away, you don't even actually necessarily as a developer have to notice. The same workflows that push code to four clusters, will push code to five clusters without any reconfiguration on your part. And so I think, again, that world of being able to keep the same basic workflows without any changes, without any modifications to my repository, even as I perhaps go from early pre-prod to staging to production in my application lifespan, really reduces the cognitive load that a developer has to put into understanding where their code goes once it deploys. It can continue to focus on that Git repository as the sort of sole source of truth. And then of course the platform engineering team can really focus on the initialization of that Git repository as being the act that empowers their developer to effectively forget about that stuff. I wanted to riff off of that same idea to talk about snowflakes. It's possible that this whole idea of a snowflake has passed out of our collective consciousness. But when we started in 2010 or thereabouts, the notion of a snowflake was a deeply concerning idea to a lot of people. And the idea was that a lot of our servers were effectively snowflakes. They were beautiful, they were ornate, and they were entirely unique. And if they melted, it was impossible to reconstruct them. And so one of the big campaigns early on in containers, in Kubernetes and orchestration, was to say, we're getting rid of your snowflakes. And indeed we have. We've gotten rid of the container snowflake. And if you look at a Kubernetes cluster, all of the images, all of the machines within an Azure Kubernetes cluster or Google Kubernetes cluster, or anyone's Kubernetes cluster, wherever it happens to be, they're pretty much the same. The trouble is, if you take a step back and you zoom out, we've created snowflake clusters. So yes, OK, all the machines look the same, but now we have 20 machines that are unique from another 20 machines somewhere else. And so I think what is really critical now is that we take and we realize that maybe the natural state of the world is the tendency towards entropy. And we need to take another crack at solving the snowflake cluster problem. And indeed, GitOps, I think, is a really, really powerful way of doing this. It's a really great way of ensuring consistency, because as I mentioned before, the flip from a push model to a pull model means that GitOps at some level doesn't care about scale. It doesn't really care how many clusters you have, because within each cluster, you're configured to point to the same Git repository. And also, because if you make the initialization of that pull daemon of the piece of software inside the cluster that is talking to the Git repository, if you make that part of the creation of the cluster, whether it's the configuration of a cloud managed Kubernetes cluster, or simply the scripts that you use to instantiate that cluster wherever it happens to be, it becomes wholly automatic to not just create a cluster, but actually to initialize it with the right versions of all of the right software, whether it's intrusion detection, monitoring, or anything else you want to sort of land on that cluster in an initialized state. And then, of course, keeping it up to date is equally simple, where it's just a PR, just like what depended about earlier, it's a PR to ensure that all of the clusters in your fleet have the same configuration. And so I think there's a real value, not just in terms of separation of concerns, not just in terms of empowering our developers, but also in terms of the consistency across the platform that you can achieve, you're not going to accidentally forget to include a cluster in the pipeline that happens to deploy the new version of the security software. You're not going to have to tell the development team, hey, we're taking away that region, and we're adding in this new region over here. You'll end up with a consistent environment, however you want to be. Now, I've talked a lot about GitOps and a lot about Kubernetes, I think one of the more interesting trends that we're actually seeing is the GitOps is actually moving beyond just Kubernetes. And in particular, we're seeing this being a pattern that people are applying to public cloud resources in general. Through service operators, whether it's in the cross-plane project or the service operators that are associated with all of the different clouds at this point, people are seeing the Kubernetes control plane as a consistent global control plane for managing not just their containerized resources, but actually the instantiations of a Postgres database or a blob storage system or whatever else you need to do to have your complete application. Because as much as containers are awesome and Kubernetes is a lot of fun, there's a lot of really great services in artificial intelligence and everything else that are part of a cloud that you're going to integrate as part of your application. And we shouldn't silo GitOps as being just about managing the stuff that happens to be in your Kubernetes cluster. It can just as effectively manage the stuff that's in your cluster as well as the stuff that happens to be in the cloud. I wanted to close with the last piece of this, which is to say, I've hyped this thing for a while. You might be forgiven for thinking that I'm telling you that this is the only solution. And in fact, it's not, actually. And I'm reminded a great deal of my time as a roboticist. I spent a long time doing robotics. And there was always this handoff and this interplay between control and planning. A controller like a thermostat does a really great job of following a wall. You don't want to use, trust me, you don't want to use if-else statements to have a robot that follows along a wall. Use a control loop. But a control loop does a terrible job of exiting an auditorium. Because I have to maneuver, I have to say, well, I'm going to go here first, and then I'm going to go there, and then finally I'm going to get to my goal. Controllers have a tendency to head straight for the door and then bump into things. And I think this is a useful analogy for understanding the interplay between traditional pipeline systems and GitOps. GitOps is effectively a fire and forget control loop. Make this thing into reality as quickly as you can. That's really fantastic for all of the reasons that I mentioned before, but it's a terrible way to roll out your software around the world. If you want to roll your software around the world, you need to move slowly. You want to move to Canary. You want to move to a 15% experiment. You want to take some time, because if it breaks, you don't want to break everybody. And that's where a planner comes in place. And so really, I want to argue that when we think about pipelines and GitOps, it's not two things in dichotomy. It's not a fight, but rather it's two very complementary technologies, one that is very good at easily making the state real and one that is very good at orchestrating stages of what you want the world to look like. And so hopefully, that helps you both understand the value of GitOps in a container context, as well as the interplay between maybe some traditional CI systems and the new world of GitOps. Thank you so much, and see you around.