 Okay, everyone here, the screen is on. Oh, the screen is not on this. Let me change that. Yep, you can see anything. Yep, good, I'm just fine, so welcome. Welcome to this talk about gate ups in motion and infrastructure as code. So first, I'm Vincent Gillett. I work for Zeneca, been in this business for way too long. And I was very fortunate four years ago. There was a very big French financial institute that came to me and says, hey, Vincent, we're moving to AWS, we're changing everything and we need to revamp our old CI CD process plus our continuous deployment. And you've got a wild card to do what you think's best to do this on the state of the art and that will remain state of the art for the next few years. So instead of having like yet another GitOps talk, well, they're gonna show you that you can deploy something on a Kubernetes cluster by just git committing something in a git repository. Here, we're gonna share with you the failure. Oops, sorry, the mask is a little bit loose. Yeah. So I'm gonna share with you guys three years of real production of doing GitOps, the success, but also the failure. Here we go. So buckle up because this Stokes is like 25 minute or something like that. And we've got a lot to chew on and a lot to talk. And before we start, I have a very strong disclaimer for these Stokes before you're trying to do this at home or at your company. This here and here is the picture of your SISO and its team. This is what they should look like. They should be working hard, sweating on their keyboard, working with you to enable you to do the job that you need to do. If your SISO is more and its team is more of the properly suited with the nice ties that show up in meetings and say no to everything you asked for, you have the wrong type. You need to change, send it back. You want this one. Because if you don't have this one, don't even bother proceeding with everything I'm gonna talk about. This is a very strong prerequisite to your DevOps journey. This is what you should have. So no, disclaimer's gone. We can go. So here's a few definition. The infrastructure as code definition should be pretty straightforward for you because it's been a long time so we're getting familiar with this. So it's code that is describing a desired state to interact with the API of a cloud provider. That's simple, right? But what's very important in this is the desired state work. We're gonna see that all along the stock, right? But this is very straightforward for you, I see. For GitHub though, this is a complete different story because this is like what for two, three years become the very buzzword. It's all on this bingo card of all conferences where you go to how are we gonna talk about GitHub or not or anyway. And I see a lot of blog articles in there and even like official pages from really big software company that describe the GitHub's way to do things and most likely I'm very disappointed, they're completely missing the point. For instance, the GitLab pages that says everything to do about GitOps is very misleading and completely missing the point. The companies that does this, right? It's a company called Weaveworks. They are the guys who did flux but they also have the guys who did EKF CDL. They did the Weave things for Kubernetes and they are actually the first guy a few years ago on a blog and a white paper put on paper their vision to how we should manage and operate Kubernetes payload in a cluster. And we would call it GitOps and then the release flux. And they have a way to define GitOps with four pillars that are extremely important. You have the first one here which is the entire system is described declaratively and not imperatively. Remember the desired state we had from infrastructure as code? Same thing, right? We don't tell AWS to say how they should spin up their EC2 for us, right? We're just saying we want an EC2 with this configuration, this networks, their business then, you know? It's AWS business to spin it up and hand it up to me the way I want it. I'm just describing what I want. First thing, the second one is pretty straightforward because we have code somewhere should be in Git everywhere, right? 2022 it should be fully common to have the things. The third one is where it comes a little bit tricky. It says that if we have a change somewhere that is approved on our Git repository, it can be applied to the system. I strongly suggest that it should be applied to the system and it should go because it's already been approved yet, right? So every change to the systems goes, every change in the Git repository goes to the systems. I don't want to say the word but that means there's some kind of pipeline somewhere here that listen to the Git repository, do stuff. And my resources either on a VPC or on the EKS get provision and properly configured. And usually people stop there and said, oh, okay, yeah, I'm doing GitOps. I have all of this. I have a huge pipeline on GitHub or GitLab CI's. I have all my code there. All my operation is managed through Git and I have the thing. So yeah, I'm doing GitOps, right? But they forget about the last pillar, which is actually the real game changer. This is what if you're not doing GitOps in 2022 right now you should try to build it ASAP because this is a real game changer. The fourth one they forgot is that there is an agent somewhere, usually inside your Kubernetes cluster that listens to this Git repository and stubbornly every three minutes, if there's a drift reapply the code that's on Git. So every three minutes 24-7 you'll have this piece of code that's stubbornly gonna cube cut all minus F apply, blah, blah, blah, right? All the time. This is a huge change compared to infrastructure as code because what I do infrastructure as code, I spin up my terraform, apply it, got my whole system, it works pretty well for a week. I don't know why, but the week after something breaks, what I do? Well, I SSH into the machine, I try to investigate, then I discover it's DNS because it's always DNS, right? And do the change and then forget about it, go back to sleep, right? This is not possible with GitOps because if I talk to the Kubernetes API to just change an annotation or edit a config map within the next three minutes, all my change is gonna be erased because the sources of truth is the Git. So if I wanna persist my change, I have no other choice but to commit and then flux will do with things, right? So this is what it's such a huge game changer into your deploy process in a company for Kubernetes and soon a code. The third kind of definition that will go through all of this talk right there, it's a principle that is very well known in IT, it's called KISS, keep it, it's stupid simple. Simple stupid, it depends, right? We want things to be very simple. We don't wanna build very complex system. We wanna build simple thing that handle complex problem, okay? If I had a dollar every time I heard in my career, let's build a framework, I'd be so rich, trust me. Okay, so altogether here, let's start our journey into GitOps and infrastructure as code and let's build a system but the first thing we need to do is actually think about the criteria we want for our system. We want our, before designing, we want our systems to be secured, we want it to be maintainable and you've got this laundry list of birth word, you want it automated, highly available or you also want progressive deployment because you've been on that meetup a few weeks ago that we're talking about category deployment and it felt so good and you really want it also, you want it elastic, resilient, blah, blah, blah, you want the whole thing, right? But trust me, if you try all of this as one, you will fail, badly. You will not have achieved a single thing, okay? So at first you really need to prioritize a few of this criteria because sure, for instance, being highly available is very interesting and you want it in the hand but do you really need it? D zero, right of the bat? We're gonna build a system incrementally, slowly, surely. So what we really need in there is to select those three things. First we need to design something secured because secured come first, secured come second, it come third, security, there is no compromise about it. Security should be the very first and the very last thing you think about when you're designing a system or you're writing code. So no compromise on security. Even third, it's the starting of our process. We want it secure already and we want it maintainable and easily understandable. Why is that? Because we're starting our journey, right? We're not very familiar with this. It's a complete change way of thinking your deployment into your cluster, your organization, your process, direct from Git to cluster, right? So you want things to be easily maintainable because you're starting a new things, you're gonna change things left and right, you know those bricks of Lego, they'll need to be updated very soon, you're gonna change stuff. So I don't wanna lose time with this, right? And the second thing is easily understandable because obviously at start things will break and I don't wanna spend a couple days on GitHub issue and stock of a flow on the other screen trying to understand what went wrong. And on top of that, you want it to be easily understandable for your team but it also helps, for instance, your management because you can clearly explain what you're trying to achieve instead of showing him a very complex AWS diagram with Lambda everywhere, you don't even know where to start for and he says, yeah, sure, okay, let's do it, right? So this is the first three things you should focus in your system, security, maintainable and easily understandable. And then when you have this, probably you can try with high availability, a little more automations along the way and in the end you will achieve all of the things but not everything all at once, right? So, oops. So right of the bat, I'm gonna show you the architecture of a deployment for infrastructure as code, state of the art 2022, be ready, donna, looks like that. It is kind of deceptively simple, right now for just such a state of the art things. I only have what Lake says, three rectangles at most but there's still a very interesting thing on that diagram. So you see there's a server somewhere that runs through things called Atlantis. It's inside a VPC, right? And it's running on an EC2. We'll see later on if there's any question why on an EC2 and why not and a fancy AKS we could see them out. But so what the things does, it's wired to your Git repository like GitLab, GitHub, whatever. Properly connected to a few repository in there and I work in there and I put my code and when I open a pull request and I push some code, this things will fetch the code and do a Terra from plan and nicely send me the plan inside my pull request as a bot, very nice. Because then I can only work in GitHub or Git Labs and I've got all my things, right? All the points. Until the plan fits my need, right? I keep on pushing, keeps on putting me the plan and when me and my colleague are satisfied with the plan we comment on a PR and said, oh, Atlantis, apply, poof. And then Atlantis in the VPC will actually apply the code. Resources spins up. That's it, you close the merge request, you get to go. But this is just a very fancy, I can be like nice feature because you're working in your pull request but actually the real change in there is very subtle is that you are inside this VPC already. This EC2 is running with an instance role with have a least privilege policy that halloween to spin up these resources no more, no less. Nothing new. It's from within the system that I pull the change and I apply it from the system. Do I need an AWS API key outside of the system? No. Do I need to give like a complex GitLab CI, GitHub Action, SpaceLift, you name it, all of the things that are external to the system? No need. The change come from within. From a security point of view, this is really good because you can have the key to this VPC, you spin up this Atlantis and then you swallow the keys, no need for it. Or you'll probably keep it on a safe lock, right? Don't delay it, right? But you don't need it, your developer don't need it, nobody need it because it is already from within that you push the change. This is extremely strong. On top of that, because on a note trade, because it's running on an EC2 in there with those instance roles for those that are familiar with let's say Terraform, in your provider you don't even have to specify the roles you're gonna use, the account you're gonna use because it's already on the machine. It's already in the Atlantis. So my provider is deceptively simple as well, well it has its version and that's it. Because the Atlantis is already having the role. So when I'm doing the Terraform apply, well he doing the Terraform apply, credentials are already there and they are within the system. I don't have to give them the credential. And for GitOps, it looks exactly the same way. Instead we're not in, well we are also inside of VPC of course, but we are inside an EKS and it's the same concept. I have this flock things that is running inside my Kubernetes cluster that is properly wired to the GitLab repository somewhere. And you configure it for this specific Git repository, specific branch, specific subfolder and every YML file is gonna find in there is gonna stubbornly trying to kubectl, apply, minus F all those files, nothing else. But we are inside the EKS. He's the only one that needs the service account that have the right access to provision, modify resources. You, your GitHub action and your developer, they don't need it. There is no need for right access outside of the cluster, which is from a security perspective is very strong. I'm not trying to push the change from a CI or a CD. It's an agent inside that is pooling the change to the system. And on top of that, that things does it every three minute. Okay. So, here's a bit of the list. I think I've talked to about everything in there, external access, yes, blah, blah, blah. Git audit for free. Oh, don't forget, Git is not immutable, right? Git push for still exist, still a thing. One very important thing in there also I haven't mentioned is that Evan thought we have so two new software but they're very lightweight and we don't have like a bunch of them. They are talking natively, Terraform and Kubernetes. We're not having an extra DSL. We're not having an extra also, say third-party airbag that sits on top of it in between. This is for those who likes so much or go CD, right? You need to configure access to the things with Flux. You don't need to because it's already inside your cluster and your access is managed by your Git access. Who can access Git can access the cluster, right? So, I think we're pretty secured. We're pretty maintainable also. It's just only a few rectangles here and there that can update very easily and they are speaking the same language that I already do, already do a few Terraforms and I do already a few Kubernetes object, right? And easily understandable as well. So I guess we're pretty good to go. But now you're gonna tell me, Vincent, this is very nice. We've seen this design, but what about the Git repositories structure, right? Because as you've seen in the previous diagrams, the Git repository were black boxed. You couldn't see a thing. So if you're gonna say to me, Vincent, but who has access to the Git repository has access to your environment, I'm gonna say yes, but you gotta watch out. Who's gonna access it? And how to architecture your Git repository? Because now you have some sort of a whizzy week what you see is what you get in between your Git repository and directly what you have in your VPC. So 99.99 of the time I see a team starting this journey, their very first idea to do their Git repository looks something like that, right? Looks also very simple, right? I have to admit it looks very simple. So there's this idea, we have like a mono repository here and there and it has specific branch for each environment, the Dave, QA, Prod, blah, blah, blah. Oh, thank you Git flow. I wish he never existed, but it still looks very simple, promising, right? But in reality, it turns like that. And this is base case scenario, right? This is really base case scenario. Because in the end what happens is that you will never ever be able to merge or re-base those branch together. Never. Maybe if you lucky a cherry pick here and there, good luck with that, right? But why is that? It's because each of your branch is having very specific value, specific parameters, maybe even specific resources for this specific environment. So you will never be able to just having this model repo and just promote all of this commits that you have in there. It's not gonna work. And usually when I said those 99% of the team started like that, we started like that, right? And there's always like within less than a week at most a sprint, people realize and said, oh, Houston, we have a problem. It's not gonna work. Very, very, very quickly. So they move on usually with the second pattern, which is something like that. And you can find that in some blogs talking about Github and the way to architecture your thing that says, okay, so we still have this model repo here somewhere. But instead of having those branch per environment, we're gonna have subfolders for there. For instance, we have here the project, but we have also our staging Singapore environments and stuff like that, right? And if we have a few or fewer, then we'll keep on having a different subfolder. So this is how we start, right? But three weeks later, it looks more like that and imagine in a year. And on top of that in here, there is two problem. Well, at least there is one problem and what big inconvenient in there. The thing is that Github, you cannot manage its access within the repository. It's kind of all or nothing, right? And there you have the access to write or merge a pull request into your Github and see everything in it or not. But there's no in between, right? I'm not gonna say, oh, you can see what's in the Singapore environment in there, but in my other subfolder, which is Tokyo production, there's no way you're gonna access that, right? It doesn't work in here. This is the pattern I usually find in clients that has this platform team, this SRE versus DevOps, right? DevOps decentralization, SRE platform team, get holder centralization. And they don't care about the access because they access everything. And if you're not in the team, you don't need to access. So they don't care about this pattern. And this pattern actually, it works if you are in this kind of company that have a very strong silos of the cloud providers. But if you follow the Conwellow and the companies, they often restructured, they change a lot. Projects change ownership. Environment change ownership. Projects getting restructured left and right. So know what happens in a year if I'm a very successful business, I have this and I'm hoping in Europe, but in Europe, they have their own team and I'm gonna cut in health disrepository and send them over and says good luck, right? It doesn't really work like that. So it is very, very, very rigid. And also I mentioned the execution context is that in there, you remember these Atlantis that is properly wired to this. So that means it's properly wired to one sub-folder of the whole repository. You better not miss the target, right? And I've seen that happen, the very usual P-prod, you know, with a typo and then one P disappeared and then suddenly you have pre-production going directly into production. Well, your client's not gonna be happy, your manager not gonna be happy and you're not gonna be happy with yourself, right? So this is very prone to mistake this kind of structure with this all in one in there, right? So in the end, I'm gonna say, no, we can do better, right? Not gonna work for us. So why is that? Just a little bit of academic things. So this is the MOF pyramid. For those who did UML, yeah, I know UML, right? Better a few years ago. So at the very bottom here, you have the instances, the real world, and just on top of that, you have the model and the metamodels and everything. And this idea of the mono-repo is not working because your mixing things that don't have the same life cycle. It's not because you wanna change the chart of the instance of a chart him that you need to also inconveniently have it change somewhere else in there, right? You don't wanna mix in them. Things that don't have the same life cycle don't belong to the same git repository. You want loose couple on this. Mono-repo is cool, but what really works, and trust me, I've been doing this for a long time, what's even better is trunk-based. And I'm very willingly able to sacrifice mono-repo for trunk-based, much more efficient. If you can have both, by all means, do it, but I'm not so sure it's gonna be very convenient to have a mono-repo trunk-based, think about it, it's gonna be a mess. Oh, trunk-based, it's much more efficient than those mono-repo. So here's what you should have here, the complete diagram for your things. So the part on the right, nothing's changed, the same Atlantis, the same resources, the spin-up, but here on your git repository, you have very thin git repository that are responsible for a very tiny layer. My KMS to uncrypt that rest, a few things, one layer, EKS control plane, one layer, a few of my nut groups, another layer. You multiply all the things, very easy. So you can do trunk-based very easily, and then suddenly what you're changing is just a line, a value somewhere, nothing more. At most one or two resources, and then suddenly you change from two deployments a week to 50 deployments a day. Definitely not the same thing, right? And for git-ups, looks exactly the same. Very tiny git repositories that are responsible for just one application in one environment. In here, for instance, you have a spring boot spin-up in one environment, just one git repository. It has, okay, one instance chart, a few config maps, and that's it, nothing more. So in the end, with these things we had in production, we're talking about a lot of machine, I handed up with close to 500 git repository. Was it a problem? Not really, especially when you have stuff like, I don't wanna make advertisement for them, but they do sometimes a good job, like GitLab. They are the only one to provide a logical container for git repository. I don't know why GitHub haven't done that already, but you can have groups, so you can create a structure of your localization, your environment, and then all this git repository for the things. And it works pretty nicely. I prefer having 400 trunk-based mini-repository than a giant mono-repository that I don't even understand the CI out of it, right? Okay, so that's all I had for you for tonight. If you've got any questions, feel free. Out is my Twitter QR code of my newly created Singaporean Twitter, so do not hesitate. I have open DM if you want. That's it. Should we, if you have a... No, no, he has a question. That's why I wanted to handle that. Sorry, I think we are waiting for the next speaker to start. I think we can hear about three questions, just three. Oh, we can do, yeah, sure. Thanks. My question is, how do you do production from the Zagis production with... If I'm not mistaken, there is a git repo. Yeah? Zagis in the group of production... So we have different git repositories, right? We have different subfolder, one for prod and not. And the thing is either at the beginning, eventhough it's gonna look very weird, but we did it manually, because what's you just changing into your repo in there? It's just a number of your image into your chart instance. But then after what we did in there, that was the first six months. And it was good enough, right? For three years, the first six months will proceed like that, no mistake. But then what we did is we wired from our CI directly a commit. And we would said the line, said an oak is always the great thing in IT. We would watch the line for the version and just get the we used to version or docker image with the chat of the commit. So we just put, replace this, and directly from the CI, so it's a technical user that would push the code. And it works very nicely. I'm gonna see if it commits the best on the site. I mean, you're doing like what commits to the station? Yeah, yeah, yeah. That sounds really good to me. It, trust me, it is, yeah, but it works. And it works pretty nicely. You need to start running it somehow. Yeah. I'll be, Gitlabs or GitHub, as they have really great API if you want to interact with them. Like you can create tons of technical users, tons of things. Okay. That's it. Just one last question. Anyone? This will sound weird as well. Do you have Gitlabs or serverless? So there is actually, I'll have a talk later this year about something called cross plane. So cross plane is this idea that you would use the Kubernetes control plane to provision your cloud resources. So you have, so you use the reconciliation loop of Kubernetes the same way flux does for a Kubernetes object for the cloud things. And it works very nicely with AWS. So you can provision your RGS that way. And every three minutes, every drift, still gonna be like replaying. So this is the next step for after infrastructure as code. It's called cross plane. You should definitely, should check the GitHub for this. That's it.