 I can tell we've gotten into the afternoon a little bit. Folks have started to filter out and fill the hallway track. But thanks for sticking around and sticking with us for the rest of the schedule. This is AppDeveloperCon for those of you just wandering in. If you were expecting something else, then you might want to look at the maps. I don't know. When I started, when I started, there were maybe two or three of these I feel like, and now it feels like there's a lot of these day-zero events, but at least they're all on different topics. You won't catch me at the networking ones, but let's see. So how many folks are going to take back one or more of the things that they've seen today already and try them with their team? How many folks are going to take back everything and try them with their team? Oh, good. Yeah. So there will be a survey at the end as well, where we're looking to find out what kinds of talks you want to see at the next one. So keep that in mind as you go through. We tried to get a good diversity of different topics here, but if you feel like we totally didn't give your favorite topic enough time and space, whether that's JVM performance tuning, which I'm sure most of us here are super excited about JVM performance tuning, but if you are, tell me, because I could have made the wrong call, or whether that is how much code goes into your application and how much code lives inside cars alongside it. I'm sure the Dapper folks would love to talk your ear off about that, if that's the thing you want to hear. But we are getting up to time for Kyle to start his talk, so we haven't talked at all, really, about CI yet, but if your code is not getting from your laptop or from your repo to production, I'm going to guess that the rest of the app developer stuff doesn't really matter. So I'm going to hand it over to Kyle to talk about CI Pipelines as code. Awesome, thank you. So, yeah, before we dig in, we'll do a quick experiment, so there's no raising hands, don't worry, but when you think about CI, what emoji face does that spark in your brain? Is it cool sunglasses? Is it angry faces? Is it upside down smiley? And that's going to be different for everyone, but hopefully it's not too bad, but some people I know it can be a real pain if it's not a great system. So moving on from that, if your CI is fine and all that, do the same thing with YAML. When you think about YAML, do you have a really cool sunglasses face? Do you love YAML? He doesn't like YAML. And then kind of think about the same association you have with writing code. So whatever your favorite language program is, writing Go, writing Python, whatever, what emotion does that spark in you? And is that something that you enjoy on a day-to-day basis or would you rather write YAML? And for me, it's code, so hopefully we can all agree on that. So today we're going to talk about CI Pipelines as code and in this context, code is code and I'm Kyle. So here's our quick agenda and you don't have to memorize this because I'm going to help along the way, so don't worry. I'm your guide through CI Pipelines as code. So first of all, what's up with all the YAML? So we're going to take a little journey of what a typical app goes through when you're starting with a brand new app and how you get to this place where you're drowning in YAML soup as we like to say. So day one of your application, you've got this YAML and actually maybe it's more like day 30 of a brand new app, right? So you've got some CI and it's like, what, 13 lines of YAML and whenever you push to main, it's going to check out and install Node.js and run this Node script. And so right now you're probably feeling pretty good. You're like, sweet, like I'm doing CI and it's true, like you are, this is really great. But let's take it forward a little bit. You're like, okay, day 40. We're still doing okay, it's still manageable but we've got more requirements now and our app is growing, we have more people contributing to it and so we also want to run stuff on pull requests and maybe we have some checks that go in. So you have a little bit more YAML and then the thing about CI, unless you're one of these CI diehards like some of us at Dagger, is it works and then you forget about it until it doesn't work again. And so let's fast forward a little bit down the road. Now you've got this little tiny snapshot of what all of your CI is. So you've got 1,000 lines of YAML. No one person understands 10% of it. Contributing to it is a nightmare and you don't want to be the one person that has to upgrade whatever action or whatever image you're running on. And I mean, go back to the emoji faces, like all the things that you might feel when you're looking at this right now. I'm sorry if it's triggering. But yeah, this is like just the reality of the evolution of CI for any application is that you just keep adding more YAML and you keep adding it and eventually you're like, what have we done? And so you get this kind of this core team like as you grow this, whether they're just a subset of your engineers or if it's a platform team or a lease engineering team that become the YAML wizards and they're the people that get to change CI because either nobody else will or I mean, looking at this YAML like is it super clear what's going on? If you work with a good actions, maybe it is because it's an easy example but like you get this set of YAML wizards. I used to be one. So I know what that life is like. And you end up with a lot of PRs like this where you make PR to update something in CI and you change a bunch of YAML and then the PR review is just like, okay, sure. Like I don't know what you're doing, but I approve. And that's really bad. But that's literally everywhere that I've worked on CI. That's exactly what happens. And so how do you get away from that? And in my opinion, usually you might have gathered from the title, that CI is code. So you're bringing this code that's doing these same processes but you get so many advantages of writing code and we'll go into some of them, not even a lot of them today but you get to compose your pipelines and your workflows using this code and that means when you make a pull request to change something, even if the person reviewing doesn't fully understand like your whole CI stack, they can look at that change and decipher more of like what the meaningful change actually is and when you're writing code, that also means this change that I'm making, I can validate it first. I don't have to push it to CI and then run an actual CI workload against it, right? Like that's such a backward system which brings us to our next point, the push and pray. And I didn't make that phrase up but if you come by the dagger booth, that's printed on the booth because that's a great way to phrase what CI has become, right? So like you've got these thousands of lines, YAML and this is like, it's funny but like how many of you have a repo where you've had something like this somewhere in your commit history? Like everybody, right? Like it's crazy. And so like how does this happen? It's because the YAML for your CI is this language that only your CI platform understands, right? So when you make a change to it, you can look at it with your eyes and try to be the CI platform and be like, oh yeah, that's gonna work and then you push it and it's like, oh no, you missed a space or if you can lint CI with your eyes, that's really great but that's still not enough, right? So you get this world where you push things up and you hope it works and even if it's not iterating on CI, even if it's working with your actual application. So like the stuff that you actually want to do with your day and not fixing CI, you know, you NPM run tests on your machine and it works, then you push the CI and it doesn't work. Why? Because your machine's not the same as CI but maybe we can fix that, right? And so you also get this horrible edit of the classic XKCD meme of this really long feedback loop because okay, if I can't trust my local machine to do these same tests as CI, then what's the point of running them? I just have to push the CI and wait to see if it's green and so that means we're relying on CI for so much but again, as we saw the YAML over time, that's more and more so now you push and wait like what, 30 minutes? How many people have a CI that takes longer than 30 minutes? Okay, it's too many, I don't wanna look. Put your hands down. But that's the thing, right? So you have this really long feedback cycle like I've made this really simple change. I can't trust my machine to validate that so I push the CI and so let's fix that. So we wanna make local the same as CI and that's not with mocks or running CI from our machine. Like we're not running a CI engine on our machine. We're not running remote access into our CI platform for our machine. We're not doing any of that. We just wanna run the exact same thing within our development machine as we're running in CI. In wherever, right? So how do we get there? So let's look at some code. So I've got this cool demo application. There have been some really cool demo applications today. So I'm jealous of the works that some of you all put into your demos. But we have this really, really cool one called the Greetings API that I use for a lot of demos. And so if we look at this, we've got this. It's written in Go. So we've got a Go API and it listens on the port and it returns a greeting and we'll look at that so I can prove that to you. But we've also got a front end that shows our really cool greeting on a website. And so if we look at, just look at our main.go. And we can see that this is going to return this function or it's gonna return the string hellocube.gon. Or yes, hello app developer.con. I would change it, but that would make our tests fail and that would be really embarrassing. So that's that. And I promise I would do as much as I could with one hand but how do you exit VIN with one hand? It's impossible. I'm trapped. Sorry, I can't. Twice. We're saved, all right. We can still do it with one hand. Okay, so I've also got this front end and the front end is a Hugo site. So in this fake application, we're developing Go. We want to use a Go toolchain thing on the front end. So Hugo is, if you're not familiar, it's a static site generator written in Go. The details aren't important for the demo but the main thing is we've got a bunch of Go stuff going on. And so if you think about the CI CD for this application, we've got Go toolchain. So we probably want to do like build test lint against some Go things. I mentioned the Hugo sites. We want it some way to build our front end site. And then we probably want to like deploy things at some point, right? So let's start with that Go toolchain stuff. So if we look at this, we're gonna be using Dagger to build our CI CD for this application. So that means we can write our CI CD in code and run it anywhere. So that fits our requirements that we looked at earlier. So I'm gonna start with this module just to work with Golang. And so it says that it's a module for lint testing build. So that's perfect. So let's look at what the code is here. One handed, look at that. So we've got some Go code and we've got a function that does a build, a test and a lint. And so the lint is a great example of like something that might work on your machine not in CI or maybe vice versa because like how many different linters are there and what are the differences between individual versions of those linters, right? So within this code we can say for organization like this is the way that we let go and this is gonna run the same thing when we push it to get of actions or circle CI or whatever but also on your development laptop. So we can say using the Dagger SDK for Go here we can use this lint image and again we're saying we wanna use the specific version of Golang CI lint and then we're running the specific command. And so we don't have to like copy and paste this between like get of actions and some make file and some other things. We just have one place that this is defined and we can say the same thing for like this function that gives us a base image like anywhere in our organization that we wanna run something on Go. We can define like one time what that looks like and so we have the same Go version, we have the same environment variables as in configurations and so on. And so let's actually use that in our application to do our build test lint for our back end. So I mentioned like in this project we got the front end and back end. So we've got the CI module here and we've got a back end and a front end directory within that and then the main.go that's gonna pull those together for like the total CI for our thing here. So let's look at back end. And again, we have that same kind of interface that I talked about. So like test lint build and again that test is gonna use that module that we just looked at. So this is we're referencing that Go lang module that I just looked at and test lint build and we don't have to think about the implementation details or you know how Go is being run here and then we can also add a few more functions to this. So let's say I want a way to get just my back end binary and so we can say that we're actually building this dependency tag between our functions here where it's gonna do a build and then give us the specific binary out of it and maybe we wanna be able to get a container for this too. So when we deploy it, we say this is how we build like a multi-stage image for our container or for our deployment. So we're gonna do that build just like we did before except this time we're gonna get a wolf image so we can deploy this thing of production without any vulnerabilities. And again, we don't have to think about how these different pieces are coming together. It's just gonna happen the same way. And as a bonus, we have a little functions that we can run this as a service too because if you think about all the things you're instrumenting in CI like all those processes or how you want to run and test the application to so you probably wanna run this service on your machine. So we'll take a quick peek at the front end as well and then look at how these tie together. So I mentioned earlier that like we've got this Hugo site and here we happen to have a Hugo module. And so with Hugo module we can build our static site and it's just gonna give us a directory that contains the compiled site. And same idea if we wanna serve this for our machine and we can just dump that into nginx image and we're good to go. So to pull it together, now I've got my test and again it's just calling to that backend module where we already defined our test and when we look at our build it's gonna look at our backend, get that binary, put that in a directory for us, look at our front end, put that in a directory after it builds and so on. So before we go on to like the deployment side of things let's actually like run this stuff and hopefully the internet cooperates. So using the dagger CLI we're gonna call test and we're gonna pass in this argument dir and that's a directory with our current working directory. If I go back to this real quick we can see here's our test function and the important bit here is we have this parameter called dir so that's where we're actually referencing here. And so if I run this internet, yeah. And so it's gonna compose all those images that we talked about. It doesn't care what's on my machine, right? It doesn't matter what my go tool chain or whatever I have but we ran this test and we can see it's cache and we passed our test and so now when we push it to CI CI is gonna run the same thing, right? So let's make things a little bit cooler here and go on to the deployment side. So actually bonus before we get there I mentioned we can serve things and run things on our machine so let's just do that real quick. So I showed that the way we could just return these containers and run them as services. So if I wanna run my whole stack on my machine I don't have to open 30 different terminals or maintain a Docker compose that's separate from the rest of my stuff, right? Because again, for those of you that are strict enough strict enough to maintain a Docker compose of your project like does it drift from your actual images and yeah, right? So with this, this is running the same code that you're running everywhere else. So when I run this again, fingers crossed for the internet. Let's go pull those images and it's gonna give us a service for our backend and a service for our front end and it's just gonna forward them both to our host here and so we can see we have 8080 and 8081 and if we go to a browser here and say localhost 8081 here's our super cool front end. This is a Hugo site. I didn't make that theme and then we have that little string that we saw from our backend API, Hello, KubeCon. So we're looking at our, both of our services running here, awesome. All right, so let's go back and look at some cool stuff. So let's say we want to release some code too. So in this case, we're talking about creating a GitHub release. So again, got a module for that and so we can see that we pass in a directory and in this case, we're gonna get that build that has our front end backend. We'll put that front end in a tar ball for this release. I don't know what you want to put in a release asset for your front end but this is what we're doing for this project. So we have, we call this GitHub release module and then call create, deploy. So now we're gonna deploy our front end backend to the backend of fly.io and the front end to Netlify. So here doing things a little more complicated but we could probably make a module for this too, right? But we're making a multi-arch image for our backend because I don't know what architecture I'm deploying to on my machine right now. It's ARM, but maybe the machine in fly is x86. So we make this multi-arch image, push it up to Docker Hub and then of course we got a module for fly and we call deploy on that and that's going to deploy that back end and the front end, got a Netlify module, get a call deploy on that. And so one of the cool things like we've been talking about go all day and if you're not a gopher then I'm sorry about that but what if we have a look at this fly module real quick? All right, I have to use my other hand, hold on. Here's our fly module, it's written in Python but as we saw before, we're actually using it in go and from the consumption side of these modules you don't really care what it's written in, right? Because you just care that it's doing the task you ask it to do. So if you've got this big ecosystem of Python engineers within your organization, they want to write all these components in their language. That's fine, you don't have to make them write go if they don't want to because you can use it in whatever language you want to use it from. And so from this fly module, I'm doing this super secure templating of a flag and fig and then pushing it up to fly with the CLI but yeah, the important part was like from this from this deploy function like I didn't really care that whoever wrote that module was not using go, I'm just using it. And then from our actual CI context or maybe from my machine before I get pushed or just as I'm iterating we want to wrap all those different steps to say my CI, I wanted to build test lint like we talked about before. So I just make another function and I'm just going to wrap those other functions that we talked about, so lint test and then another fun part. Aside from, with your CI configuration today aside from the YAML being proprietary the other part that makes it hard to be portable is that you have all this environment configuration that you've typed into the GitHub Actions Web UI or into wherever you're defining your CI environments and so that means that if you don't run in that environment then your settings are completely different, right? So this is separate from CI CD as code but stop putting your secrets in your CI. You should use a secret provider for that and so in this demo we are and so this one happens to be using something called the infysical which is just a secret provider, SAS and so I just have an infysical token and I'm saying in my code I want to go with this infysical module and get this GitHub release token and grab that for my dev environment and the cool thing with using that pattern is that obviously you don't want all your developers and your CI fighting over the same set of credentials and deploy targets or whatever but as a developer if my token gets to be one set of credentials that deploys to preview environment rather than the same place that my CI deploys to then that's all controlled through authorization rather than through some set of environment variables and in this case, yeah, we're just grabbing this GitHub release token and doing our GitHub release and then saying with like grab fly and now fire Docker hub all these things and so we've defined our secrets in a place that handles secrets and not in CI and then to take it one more step further we've got all this stuff in code and our CI can run that code but it doesn't have to we don't have to still run all these GitHub actions in GitHub actions because and that's an overloaded word so thanks GitHub for that one but we don't have to run GitHub actions still for putting this all in our code so let's remove the checkout step as well and just say I just want a command where I can give a commit for my project and it's going to run the CI on it and that way if you're reviewing someone's PR you don't have to go check out their code and run their tests or even look at the checks of GitHub you can just run CI with our commit and it's going to work right definitely so that's that and so we can run this all locally and we can also run this in GitHub so I'm trying to do this with one hand again boom got it so let's look at Circle CI first we've got a bunch of YAML and we've looked at enough YAML today so I'll just skim over that part but we just call that same Dagger call with that CI remote and like I mentioned we're just going to pass in that commit and that's it right and so if we want to do the same thing in GitHub actions same thing we're just going to call that function with that commit and I know that people here are going to like this one if we want to run the same thing Jenkins hey it's the same thing awesome so by doing this we've broken free of writing all that YAML we've written code that people if they're looking at this code and you make a change to how this commands run or the version of something you're not just making a getDiff that's changing some YAML you're making something that people can understand and write tests for and run themselves and all these things without pushing it and waiting for it and so on so that's that and if you want to go try it out yourselves check out Dagger.io we've got a booth here we'll be giving like a much deeper version of that demo if you want to see it with much more Dagger things attached to it we have this quick start guide and then also check out Reggie's book because he's actually got a section on Dagger in there so check it out thanks