 Alright, welcome back to Drupal South here in Wellington. It's really, really good to be here I'm here to talk to you today about getting the most from your CI CD experience. If you don't know, I'm Carl. I'm an opposite engineer at Previous Next. I work with our ops team on our Skipper platform. We take care of, well, I take care of the administration, business and development alongside the team and I help with the hosting of Drupal websites and our other customers applications. Today I'm going to be talking about why CI CD is important, just briefly touching on some fundamentals as well as looking at how we can delineate between the responsibilities of CI pipeline from, I guess, the scopes they cover, the responsibilities and how we tie that into having automation and processes, what that means for users and computers like people versus computers and how much work is carried out by the pipeline. I just want to talk about how, well, some of the cool CI things that we can do. So what opportunities are there out there? There might be some considerations for your own pipelines that maybe you can think about and we'll go into that. So why is CI CD so important? Well, it comes down to three primary factors, security, consistency and convenience. When it comes to security, we can trust that the CI platform is going to take care of a consistent environment amongst all of the pipelines we perform. So if you're having a virtual machine, if you're having a dock container, we can understand and know and trust that the environment is going to be consistent whether or not we manage it or if our platform is managing it for us. When the platform is managing it, there's a bit of an authority there where they have a responsibility to their customers as well as themselves to make sure that their platform isn't insecure. So there is a level of trust there. We also have measures we can bake into our own pipeline. So if we need a certain compliance level, we need to be able to measure something, we can put that in our pipeline and we can trust and work from that opposed to stepping back and losing that later down the line. Approval workflows are another big important step part of security. So you don't want your production deployers happening without your consent or without a delegated approval. So these are options you can bake into your pipeline as well if that's your intention. Consistency is one of the bigger parts of this puzzle where we can rely on consistent tooling, processes, outputs and our environments of course. Where everyone knows how to use the platform, everyone understands how they interact with the platform, where the logs are, where everything is. It's all very consistent. And probably the biggest point about why CICT is important is time-saving. We don't want to have to do all of these tasks again and again and wasting our time. Not that the important work isn't important but these tasks don't have to be carried out by a person at least not initially. Let's talk about the stages of a pipeline. We'll go into... I'm going to break down the types of work CICT can do for us and where that sits. So stages here are adoption, investment, optimization and hardening. Where adoption is where we're obviously choosing a platform, making sure it can do what we want it to do and the initial adoption. Making sure things don't blow up in our face, making sure our code can be deployed successfully. We continue to investment where we are actively investing in work to make our application testing. And we'll go into that. Optimization is how we can simplify a configuration and how we can make things more efficient. And hardening obviously is putting more rig around processes, what approval stages, what compliance levels, et cetera can be involved. We talk about these in four different categories. So descriptive, analytical, prescriptive, and... I forget, let's move on. So the first stage is adoption. Obviously we want to make sure the platform we choose has the features we need it to. Every CI system is different and a good example of this is GitHub Actions, whereas Enterprise only allows for workflow systems, so approval stages. Whereas if you don't have the Enterprise, you don't have that feature. And if you go to a platform such as CircleCI, this is an option for you straight out of the box. These are some considerations for platform choice. But at the end of the day, adopting CI is a significant helper to anyone. So having your foot in the door is the most important thing. But this is a long-term engagement for you. Investing is basically the stage we look at where we are making better code. Using the tools in place, writing tests to make sure our applications specific to our needs or our customers are doing exactly what they need to and not more than they don't. Optimization comes into being able to predict your outputs from given inputs. This can be first testing, this can be form sanitization. These are standard Drupal things. It's still a very good practice to have a system in play where you can predict what you're going to get from certain inputs. Front-end tests are a really good example of this, which can be done on CI. And you can produce a test, testing it front-end, making sure everything is in place for when the final deployment happens. Maybe you have a stakeholder which needs a certain button, a certain color. These are the things you can do. Image comparison tests have been very useful for us the previous next. We have a couple of customers who we go through each feature request making sure that we're testing a browser size and making sure that the differences in pages are not significant enough or they are expected. And hardening is scanning, compliance, approval stages, all of the important things that put rigor into our systems, but they make them last a lot longer and they give some sort of sense of security to people who are involved in the process. Looking at a complexity in the decision-making graph, it shows a lot of interesting things in this regard where we can see the human input is really high when you start out and when you don't really have anything telling your system how it should be acting or what it should be doing, whereas that scales up and the human factor really does vanish. Let's talk about some cool things that can be done in CI. Reusable code is probably the most significant innovation in the past few years. GitHub Actions, Circus CI Orbs, these are declarative piece of information where you can include it in your pipeline configuration and scale it out. Actions, Circus CI or GitLab allow you to produce something, you can parameterize it, make it as configurable as you need to. You don't even have to maintain this piece of code if you want and you can scale it out to as many sites as you are maintaining. You may have two, three hundred websites. You can reliably manage this information in one place, make it configurable and even use version tagging to prevent any unwanted changes from upstream. GitHub Actions have their actions as part of their platform. We'll look into that in a sec. I've got an example from a personal repository which shows really the real importance of having CI in place. I made a bit of a misstep a few years ago in not including automation, but it serves as an important lesson. Circus CI is something we use quite a bit of previous next and we maintain a couple of orbs ourselves. I want to talk to you about, go into the code of that, what it looks like and we'll go from there. And GitLab, I have a very small example. I don't use a lot of it these days, but it's been used in the past by myself. Before we start on this part, GitHub Actions went public in 2019. It wasn't long before Hobart, so I'll find my example in a moment. Circus CI orbs, same deal, have been around since 2018. GitLab 2012. You have other systems out there, such as Jenkins, which have been doing the same sort of thing on dedicated hardware for a much longer time. So it's just an evolving concept that has been materialized. Our GitHub Actions example, it's a bit of a personal project I had maintained before the pandemic. Obviously, I didn't have any automation in this project initially. And I wanted to talk about that a bit. Our configuration here just shows us what an action declaration looks like. It's a Circus GitHub workflow. So my example, it's a personal project, there was no automation, and it died during the pandemic. But I did bring it back and I wanted to look at some lessons in that. So looking at the Git history and the project at the time, I was maintaining upstream changes from some repositories. I was also making my own changes, hosting the application on a branch and obviously having a default branch. That became very convoluted very quickly and I guess I didn't foresee this happening. I didn't spend a lot of attention on this project. Coming back to it in 2023, I have done a couple of things of note. One is our workflow here. We're simply downloading Hugo, which is a static website generator, building it and deploying it. So this process will take half a minute. I'll show you a screenshot in a sec. But the other thing of note is using Git submodules. Fixing a couple of fellas on my part will result technical debt is gone completely. Circles to our OBS. Much the same deal. Here's a configuration where we can see how that would look. And we're going to look into a deployment scheduler in CircleCI here. So here's our configuration. We're packaging an application, making sure that we can deploy it and just deploy it afterwards. Here's our environment parameters and our version parameter. And we pass that through to the CircleCI orb. And that just comes through, prints the version, and deploys the application using Docker. This actually allows us to use this orb in any system which supports Docker, which is an important factor because if you use a Docker executor, you may not have access to Docker at that time. So these are some of the considerations when you design one of these pieces of reusable code. Similarly, in GitLab, you have this include system where you can include any supported schema file, URL, local. And as long as GitLab or the pipeline has access to pull that information in, you can include that information. Here's our persisting data. So this is a rather interesting idea I've done a lot of work on recently. I've had to learn a bit too. We're going to talk about cache, workspaces, and artifacts. Now, caching in a pipeline allows us to refer to a string that's associated to our checksum of our dependency file to pull in changes so that if we have to re-run a pipeline, if we have to have a parallel pipeline, we only have to do the process once and only when that file changes. It's a very convenient system of saving time and making sure that we are really nailing down on what we need from our pipeline. Our workspace works very similar to caches, but it allows us to completely isolate the environments between a build, test, and deploy. So we can determine that the stages are completely isolated. So here we simply have our user installation task and we are saving our dependencies to the workspace. And this workspace is attachable to any stage in a pipeline vertically. We'll have a look at a diagram in a sec to go into that. Similarly, if you have it saved to your workspace, you can attach to your workspace using something like this where you're attaching it to the project or the pipeline step and then you're testing your application completely independently of your build steps. Artifacts allow us to download information after a build has finished. This is likely in the form of test results. CircleCO has this concept where it will save the artifacts, go through your test results, it will be identifying failures, and it will show you, or it will be able to make use of those times in a later item I'll show you. Here's how the three concepts sort of intertwine. You've got your artifacts down the bottom after your pipeline is finished, they can be made available. Running parallel, we can see our caches are, we can use those between, and running vertically down, we have our workspaces which do isolate each stage of the process. Scheduling tasks, it's a somewhat boring subject, but it has a lot of opportunity for us. We are simply looking at a system cron. Here's an example, I did a blog post recently on this if you're interested. What we're doing here is we are triggering a penetration test. We're doing this weekly, and we are basically having updated information on our application, and you can see from the pipeline it's composer up, and it's going to be testing the application while it's running in CI. What this looks like in a cron system is just a trigger. Very simple stuff, but it's food for thought. What we can do with this, if there's anything out of cycle that we need to take care of, be it testing, build, or even deployments, and anything else on the screen you can see, these things can be supported by your platform, but they don't have to be. What this does, it gives you the power to see yourself regardless of what system you're on. And it's worth considering if you haven't thought about it before. If you look at how environments can be used, it's actually quite interesting in CI. We do a lot of things, we do deployments, we do synchronization. Here, for example, we are taking it back up and restoring our production database to our development environment. What this helps us do is it prevents the environment from going stale. We do have some sanitization work here, so we can elect to remove web form information, to remove caches to really know what is supposed to be in our development environment. And from there, we can deploy. This may not be a suitable approach for everyone. Environments are used for different things. There are content, there are other environments used for interesting things, it does become a problem after a time where an environment does get stale and testing features can be annoying, to say the least. You can also have purpose-built environments, so if you need to hand an environment off to a non-technical person for an approval, if you need to test a certain feature using a web front-end, or if you wanted to test the environment within CI, having a purpose-built environment can be very beneficial. This is also a sophisticated feature. Not every hosting provider allows for such a feature, but it's worth having the conversation and going from there. Canaries are an excellent way to test out features or test out major upgrades ahead of time. We recently had a Dribbleton upgrade for a larger client of ours. Using this approach, we could go through and eliminate each stage of the process of what is preventing us from making this upgrade. It was very successful and I'm very proud of the team for being able to go through this, but I've used this in the past as well. It's a very effective way of rolling out significant changes without breaking anything. Parallelism is one of our last ones, and here we are simply trying to run multiple pipelines at the same time, associated with the same process, and split the workloads out. We can do this in CircleCI by just declaring parallelism. This will start another machine operating the same load, and what we're going to do in that load is split it apart. Using the CircleCI daemon I talked about before, we're going to use our test timings and distribute that workload between each of the executors. This saves us time and potentially money associated to the running of those. Using GitHub Actions, I couldn't find anything quite as similar, but if you are aware of your workloads, if you have done some work around how much tests are taking and where should they go, you are able to distribute the workloads between each executor or runner. And GitLab also supports this, and it supports main processes as well, so if you need to test against multiple versions of PHP as an example, these are options for you. In conclusion, I was hoping today to try and inspire people to think about opportunities they may have not thought about in CI. A couple of examples here really took me by surprise, and I've been really utilizing them at work. I hope you've taken something from this. As for resources, there's no better opportunity than to get started, so if you need to, I would refer you to the documentation associated to the platform. Here are the CircleCI and GitHub Action stocks. I've also got a blog link to that blog post I mentioned about integrating Stackhawk and dynamic application security testing with CircleCI. Thank you. Have you any questions? So using a Makefile instead of the CI, what the Makefile allows you to do is also run it locally. Having a binary to try and run CI workloads locally, it's not completely transparent between the systems. There is a project called Act which basically imitates GitHub actions. It's great. It's expensive in data and it's not 100% reliable. It's not like for like. Make allows you to break the task apart. Declare them in the same way. You're not changing any code and basically having a consistent experience. That's the problem I don't have a solution for and it's something I've had many, many times. Both before and previous next and now we try to invest not all we can into the CI and go from there. The investment usually pays off longer term but it does take an initial investment. So I'm sorry I don't have a solution to that one I wish I did. Yeah, that's an interesting one. So for debugging, CircleCI does have a feature where you can start it with a SSH client. So if you need to run specific commands need to understand the system a bit more or there's a way for you to grab secrets from there actually which is not ideal but if you need to make a comparison and find that information it is an option for you and it can help. Canaries is just taking a branch from your default branch making a change you know it's going to break. Something you want obviously it might be a solo upgrade a Drupal 10 upgrade running the pipeline and seeing exactly where that breaks. We had a client where we contributed back a lot of core changes because we needed to get those upstream so those changes waved through we made the appropriate module updates and even if modules needed to be updated like contributed to to make that happen. It's a process of taking each time it breaks removing the barriers in which it breaks to get the successful result. What I would suggest is something which I used to do a lot more of it's making sure that you review all of the code that's in your pipeline. You don't want to be caught unawares so you need to sort of establish a bit of trust with the actions you're using and so review the code make sure it's doing what you expect it to and if it's not finding a different solution if you have a custom binary if you need to run something that you made it may be worthwhile doing your own like we do. We know that we trust it we maintain it and it works well for us. I must admit that incident I was in hospital at the time I don't have a lot of opinions on it I don't have any specific opinions I wasn't involved with the remediation maybe I should look into that more. Thank you for your time.