 Hey, everyone. Thanks for coming out. We're going to go ahead and get started up here. So we're going to talk about horizontal DevOps today. My name is Rob Baylis. I'm the CTO at Last Call Media. We're a full-service digital agency specializing in higher education and government sites. We also do a lot of ongoing support and we have a division of the business that does DevOps consulting. I'm here with Jeff. I am Jeff St. Pierre. I work at Tandem. We are also an agency. We have websites and some consulting gigs and we make a thing called Lando for local development. All right. So session title of this one is a little bit confusing. So we're going to stop and unpack for a moment just what we mean by horizontal DevOps. And basically we're working in contrast to traditional DevOps, which is kind of the act of building like a yellow brick road to production. So you have your local developers and you're going to be pushing code. You're going to be running some kind of testing process. That's going to end up at production. And as you're building this, you might start to construct a pipeline that looks a little bit like this. You might start with one project. You have integration with CircleCI and CircleCI does some work. Later on, you might add Gulp. Later on, you might add New Relic to that same project and then some testing or whatever. So that pipeline kind of develops over time as a single static thing, not static at all, but basically it's a one vertical stack. In contrast to horizontal DevOps, we're talking about we have to manage multiple projects, 10 projects, 15 projects, maybe two or three are active, maybe five are relatively dormant, and two are just hanging around or something. But how do you manage that complexity? So that's what we're going to talk about with horizontal DevOps. And just to kind of illustrate this, what we're really talking about is like in an agency or large corporate setting where you may be rolling out many projects across a time span, you might start your first project with a tool like Gulp built into it. You might work your way up to on your second project, getting CircleCI built in. And then on your third project, you might add New Relic kind of the same way we were talking about before. But the idea here is that we want to be doing this in a way that's going to be scalable. That's going to allow us to take these tools with us and carry them onto the next project. So we really want to be building that sort of upward momentum of the overall agency or organization knowledge base. DevOps. So one of the major themes here is we're trying to avoid the DevOps. So we're super focused on our new project. But we still have the other projects going. But we don't want those to fall over. The client doesn't want those to fall over. Nobody wants them to fall over. We want to keep them healthy. So here's our definition of the problems that we have when we're managing multiple projects here. So we obviously have clients and they have needs and budgets. And that's the real world. You can't develop everything for a finite amount of money. Everything's possible, but you can't do it on a budget. So you have to, there's some choices that have to be made there. Rob, you want to take developers? Sure, yeah. Developers. How many developers do we actually have in the room? Cool. A lot. Developers, we love new stuff. We love to work with new tools. We want React and we want all sorts of stuff built into the project. We might want, I don't know, deployment to S3 of your Drupal site. I don't know why you would do that. But we always want the new thing. So that's another kind of competing priority there. And then as DevOps people, we want uniformity across these projects. So if you're working with 10 projects, it'd be awfully nice if we were deploying to the same place, for example. And so these problems are always present in any kind of DevOps setting, any kind of development setting. But I think they become especially painful sometimes when you're working across maybe 20 projects. So we're just going to stop here and talk about how each of our agencies manages it. Jeff is going to talk about Lando and I'm going to talk about what Last Call Media is doing, just as kind of examples of how you might be trying to build this upward trajectory in your own organization. Great. So at tandem, one of the primary tools we use to manage this complexity is Lando. So here's what Lando is. Lando is for developers who want to quickly specify and painlessly spin up services and tools they need to develop their projects. So in that sense, we consider Lando a wrapper that gives us all the things that we need for project A and all the things that we need for project B. And they might be totally the same. They might be slightly different. They might be totally different. But they're packaged and managed by this thing that we use. So when we're getting started with Lando, the first command we issue is Lando and knit. And then we can... So this is in our Git repo. We choose a recipe. In this case we chose in Drupal 8. And then it's going to ask us some questions about configuring this application. So in this case, I'm telling it that where we're going to serve from is web. And then the result of Lando and knit is this Lando.yml file. And that's where all the configuration goes from running this Lando and knit command. So we're following the same pattern as Git and NIT, NPM and NIT, Composer and NIT. So we're following these similar patterns that are familiar to developers. And the result of this is this .Lando.yml file. So that Lando and knit command that we just run results in this single .yml file has a name of the application. The recipe that we chose, you don't have to start with a recipe, but recipes are a really convenient way to get going if you want to start with a Drupal 8 or Drupal 7 or Drupal 6 project that get you going super fast. And then you can extend it from there or you can go full blown custom right away. So we have a config key here. And the only thing there so far is that we're telling that we're going to serve out of a sub directory called web. After you initialize the application, you're going to do a Lando start and that's going to spin up your the Docker containers. Lando is based on top of Docker compose. So it's going to spin up all the containers that are necessary to run the application depending on whatever recipe or whatever's in your .Lando.yml file. So Lando start spins up those containers and then it spins up a database container, applications container. And then you have in this case a working Drupal 8 site that you're ready to start developing on. So the next thing that we recommend that you do is commit that configuration file that .Lando.yml file to your repository. This way you're sharing that configuration amongst all of your developers. So you have a team of three, a team of 10. Once your initial DevOps is done with this Lando init and you commit this file, they pull that down. They have all the configuration necessary to run this application. So when you do a Drupal 8 recipe, for example, that's pulling in Drupal console, Drush, things like that. So you have those toolings that you need to run that application. So with just a Lando init and answering those questions, you get a very simple file. This is an example of what I would call a sort of welcoming file. It's not overwhelming, but it's a little more complex than the first five or six line file that we saw. So this is intended to demonstrate some of the flexibility that you can add in besides just doing the Lando init and starting from a recipe. So here we're still starting from a recipe. We have a couple more things under the config key. So we still have web root is web. Now we have via engine X. So we can start creating production parity here. So if you're serving with Apache by default, the Drupal 8 recipe is going to serve from Apache. But you can specify engine X or you can specify Apache here and specify whatever you're using in production. So you're starting to build up that parity. PHP key, we're specifying PHP 7.1 here. We can go backwards to 5.3 and upwards to 7.2. So if you have one application on 7.0, one application on 7.1, super easy to swap it out. Lando rebuild the application, get a new container that has the right components for whatever application you're working on. Similarly, you can have swap databases or specify what kind of databases you want. In this case, we're specifying MariaDB. You can pop in X debug. So you can have X debug across all your developers. Lando is going to spin that up and have the things you need to use X debug. So here we have a services key and we're saying on the app server, we want to run a certain command. And what we're doing here is pulling in the platform CLI. So in the example that you're hosting on platform SH, you can pull in that platform CLI and have access to that right in the application across developers. And then the tooling key is going to expose that command. So we're saying expose the platform command on the app server and run this command when people have it. So that's going to expose the platform CLI to the Lando users on this project. If you just run bare Lando on an application that's using a .Lando.yaml file, it will give you a list of the things that you have available to you, the toolings you have available to you. So some things like in a PHP app, you're going to have, it's going to come around here. So I have Composer available to us by default on a PHP application. In the case of a Drupal 8 app, we have Drupal console and Drush available to us. And there's that platform command that we exposed in the tooling route. So most of those that you see up there are defaults for the Drupal 8 recipe. The platform one is an example of one that we piped in via that run command and exposing it via tooling. And you can add in as many other things as you might need with similar configurations. And there's lots, lots to say about that on the documentation. Rob, you want to talk about your... Sure. Yeah, I guess I would just mention that I think the really powerful way about, the thing about the way you guys use Lando is that that configuration file exists in the project and it's going to be more or less the same setup steps for every single project for you guys. So as the developers coming on, they're going to start with Lando Start. That's going to remain consistent. So this is a slightly different solution to a similar problem. Last call media had an issue several years back where we were starting every project fresh. And we basically were repeating things but not sharing anything. So we started kind of our own place to put this. It's called the last call media Drupal scaffold. And it's just a boilerplate Drupal 8 build that contains our PHP CS configuration, our Circle build, our ideal deployment process, that kind of thing. This is an open source project as well and you're welcome to check out the URL which is down on the bottom left. We have pretty explicit setup instructions for local development which are shared between all of our projects. So any developer that's coming onto our projects is going to see this to start with on the project page. And these steps are basically the same between all of our projects as well. So again quick setup, you know Docker compose is what we're using and we use additionally composer and yarn always built into our projects. So those are sort of the three things that we know are always going to be there and we rely on them from project to project. One thing I would point out is the last step which is this site import command that you're going to run. I'm going to go through that in a little bit more detail in a moment but for now just know it's the last step. We do most of our actual tooling inside the projects with composer scripts and we try to keep a few commands absolutely locked in and the same across all of our projects. So even though we're starting from the same point we may be able to deviate each project in certain ways but some things are going to stay the same. For example you have a composer build script and build is going to be compiling your static assets whatever that means for the site whether it's gulp or grunt or webpack or whatever composer build will always invoke that. We have you know linting so composer lint is going to mean the same thing across all the projects. Testing this is another area that can differ from project to project. We'll have like you know visual regression testing on one project. We might have B-hat. We might have PHP unit but the composer test command will do all the the setup and the invoking of all that stuff. And then finally the site import command down here and all that does is everyone sorry I should probably ask this first does anyone not know what composer scripts are? Okay no one. All right well just in case composer scripts are ways that you can specify in your composer file commands that can be invoked using composer whatever the word is. And so this site import command is just something that you can invoke via composer and it does whatever is written here and again this is going to be configurable on a per project basis. So you can see that on this one we have a refresh local pantheon is what happens when site import is run. So that's going to pull a database down from pantheon using the terminal terminus cli and import it into our local site. So you can imagine that on aquia that shell script might change but what's cool is that the developer only needs to know that composer site import works on in both cases. So again we're trying to keep that consistency from project to project to project to avoid the spin-up time as developers hop between projects. The other thing that we've done a lot of work on in our scaffold is the Circle CI integration and we've got three common steps that are going to happen pretty much any time you push code to one of our repositories. We've got a build step which is going to do all of our asset compilation all the vendor third-party dependency pulling in and then we've got deploy which is going to send it to whatever the destination is. In this case this is a pantheon multi-dive instance so don't worry that the test didn't pass not everything is broken and then we have this test step and again that might differ from project to project to project but as a developer coming into this project I know exactly what to expect when I hit Circle CI I'm going to see that there's a build deploy and test and I can see instantly what failed. So we share that across all of our projects again and it's been a really nice way to kind of make sure that we're always keeping that upward trajectory as far as our tooling. Great so why did we pick X meaning you know why does last call use Drupal Scaffold and tandem use Lando. I thought you were going to come here and get the golden road everything was perfect all your problems solved well it's complicated there's lots of decisions to make so the the thing we'd like you to take away from this is pick something think about it and make some decisions so it's good to think about these things and come to some conclusions so that's what we're going to offer you here some some guidelines. Yep right and I just want to expand on that and say that we're our message here is that as an organization you should be working to build that DevOps capacity and the organizational kind of knowledge knowledge base. So the guidelines that we're going to present here are in service of that. First things first we have to recognize that you do need to invest in DevOps as an organization if you want to improve your tooling improve your project first you need to actually spend some some time doing it. The way that we sort of suggest getting started with that is we're all working on individual projects today and as those projects progress you're going to hit a moment in every project where something breaks and you don't understand why but you have to spend the time to fix it. So if that's something has broken more than once or if it's something you anticipate having to to deal with again spend the time to actually come up with a good solution for it in the scope of that project and as you're doing that just kind of keep it in the back of your mind that you spent that time you should be considering after the project how do we actually take that and make it a universal solution for all our projects. So just as a concrete example of this during one of our projects we have we kept having an issue with CSS breakage so we would make a change and then we would push it everything looked great we would push it to production and there were pages broken that we didn't know about. So I think we've probably all had that experience before we spent some time during the scope of that project to implement visual regression testing and we did it in the way that enabled us to kind of go back later on and pick it out into that Drupal scaffold and really like leverage that going forward in a very simple way. Leverage it against more than one project more than just the project. All of the projects benefited from it. The other thing I would add to this slide is that it's really great and probably necessary to have buy-in from your the upper level of your company the the CEOs and CTOs because otherwise it's unlikely that someone from the ground up is going to carve out this time for this stuff to happen but if you're if you're top level of your company's encouraging that and they recognize that there's value in that time then it's going to be more likely to happen. Yep I think also as far as like identifying those things and pulling them out into this global namespace as we're calling it this is the practice that we'd like to encourage. Retrospectives are a really great chance to operate to to really like recognize it and pull it out and identify that it is a thing that could be fixed across many projects so as DevOps people we want to stay involved in those retrospectives and make sure that we're hearing what what people are saying. So why why invest in DevOps so these are probably pretty well known but we want to state them for clarity so when you if you choose to make this investment of time and money and people power you're going to increase efficiency you're going to be able to move more quickly per project you're going to become more productive so you're not going to have to worry about the annoying breakages you're going to be able to focus on the things that's making the project move forward and making your skills move forward. Rob you want to take over on developers? Sure yeah your developers are going to be able to move a lot faster if they have the confidence that something small is not going to trip them up so you know using the example of like imagine that on project number one for your agency you implemented a B-hat test that went through and checked that the home page bootstraps really simple check but having that in the future just that is the home page loading seems like a pretty simple way to like remove that from your developers brains and just make it so that they don't need to worry about that kind of breakage that'll be caught automatically they can focus more on the hard problems so it's going to let them move a lot faster and develop a lot faster and then sort of the next thing as you develop this tool set you're going to find yourself able to take on much more complex projects using the same resources so you have all that sort of like aggregated knowledge process technology whatever you want to call it you can carry that from project to project and really use it as a stepping stool to jump up to the next level in terms of what your company is able to deliver and then finally if you're able to actually develop this DevOps practice into more of like a habit that is well known as something your company does well then it can really become a good revenue stream or a selling point for your company if you're able to go into project pitches and say yeah we we do all sorts of testing we're excellent at implementing test pipelines and production pipelines that's something that that sells very well right so who's on my DevOps team I think this is an interesting question we did this session at a nerd summit in western Massachusetts and there was a person from Datadog there who basically called us out at the end of the presentation and said that's all well and good but I think that this doesn't just apply to DevOps people I think that everyone in the company should be a DevOps person and that's a really really good point we think that the people that are involved in DevOps they might not be DevOps people but they're really anybody that is on the team they're people that participate in you know the daily activities development quality control whatever and they're aware and mindful of the sticking points that they're hitting on deployments on you know regular development and they're just focused on improving like the technical flow of the projects making sure that those are going smoothly and also being responsible for either identifying or actually pulling those slight improvements that are made from project to project up to the higher level so that their surface to the organization and available for projects going forward yeah I would say that a large part of that too of including everyone in DevOps workflow is it is it's a flow so pay attention to where you know DevOps hands off to a developer and vice versa where a developer feels a pain point but they don't necessarily know how to solve it if you have your DevOps hat on on the background you can at least make a bullet point for that to bring that up to your DevOps people in the you know stand up or in a retrospective so that if you always have kind of this DevOps hat in the background and you're injecting DevOps into all your processes that's where you're going to get the flow increase of efficiency because you're communicating that to each other identifying those pain points and then giving yourself your company an opportunity to do something about those through communication building your global namespace so this is what we're talking about about building a tool set that can work across projects so what are these things so in in a super simple sense there they are Lando and Drupal scaffold those are definitely things that move across all of our projects to help us have similar and consistent workflows but we also want to see what else we can pull in and how we can manage these tools once we start building them up and they don't have to be tools that you built in-house either i want to definitely highlight that point that you know even if you part of this global namespace is figuring out how to use some external tool that's that's part of this too absolutely i would include you know composer and drush in that like those are those are part of the tools you need to build your application so they're part of your namespace part of the stuff you're managing how you manage that matters so one thing that we think is leveraging semantic versioning so if you have this in-house tool set they're controlling some some way either through like docker compose files or through lando configuration files version that thing some use semantic version even if it's just an in-house tool and you haven't released it as open source we encourage you to release it as open source if you have something that's useful to people but if you even if you don't and you still use semantic versioning against it you're going to be conveying information to your users which may be just your developers or they may be many developers if it's an open source project so minor point release if you go from 3.0.0 to 3.0.1 you're with no other information just your developer seeing that through semver they know that this should be a non-breaking change and i should feel comfortable doing it they probably have a safe way to test that and then they know if you go from 3.0 to 3.1 that something might be changing i should at very least check out the readme before i move forward with this thing so we think there's value in in using semver even on your in-house tools treat them as if they're an open source project or release them as open source yep the second point here package managers so we have some really great tools out there available to us for distributing code this is something that comes up as you're talking about how to actually implement this let's imagine that on project number one you have a shell script that you're using for deployment and you want to share it with project number two so the easy simple way to do that would be copy pasta that thing right in there the problem is that then when you go and update the script there's no central source of truth and there's no way to update the projects in the future so we encourage you to use things like composer things like npm to really distribute that stuff put it in a repository whether or not it goes public or not both composer and npm have mechanisms for private package sharing and it'll allow you to share that code out and if you're using semantic versioning for it it'll allow you to even control what gets pulled in if you want to pin it to a specific minor release that's fine documentation this one is absolutely critical across the board and it's something that even for something as as simple as a shell script the likelihood that the writer of that shell script is going to be the one that finds the bug and fixes it later is pretty low and you really want to make sure that everyone in your organization is enabled as much as they can be to get involved with this process and to contribute back to the devops tools that you're sort of aggregating here so keeping your documentation up to date is really important and we have a whole slide about that later on and then finally yeah just defining and sharing your release workflow for all of these tools so like if you have six shell scripts that are all in their own packages it would be awesome if each one used the same process for getting rolled out so it would be like imagine a change it's just a git commit git tag and then update the chain or update the change log and then get tag it probably uh so just try to keep that consistent across your tool tool set thinking in interfaces yes so this is a little bit more abstract and this is something that we touched on during our our demonstration of our individual tools but an interface is something where two systems kind of touch and in this case it's our devops tooling and our developers or our devops tooling and our pms or stakeholders whatever so this would be we're encouraging you to think about how developers and stakeholders and pms interact with your tools so is it a single command if it is a single command it's probably good not to break that command or change that command yeah so by thinking about the places that uh the stakeholders are touching these processes and how they interact with them we're establishing a contract with them so if we're going to change something of that level then we're going to have to go from three dot over three dot one we're going to have to document it because we're pulling the we're pulling the contract out from under their feet so it has to be communicated and rolled out in a slow enough fashion so rob and i think that you think this way that you're going to develop modular tools that can be pluggable and and come in and out of your processes uh with the least amount of harm possible and hopefully the reason you're doing that is for our actual benefit and not for harm so we're going to try to minimize how much we uh change that interaction that doesn't mean we can't change the implementation details rob mentioned earlier that you know uh composer build might run grunt it might run gulp that shouldn't matter too much to the to the user their interaction with it is the same and their result is the same they get a built asset pipeline um so just to quickly define that i think our interface of the last call media is the set of compo composer commands that i showed you build run uh sorry build lint test uh site import and for you guys it's essentially lando start is is your interface with your developers yep lando start and if you run the bare lando command all the tool and commands that are available or that's our interface yep cool yeah right and our goal here is to make uh new developers coming onto the project as seamless and uh easy as possible because we want to be able to have people hop from project to project all right don't be dogmatic yeah so i think in the tech world it's pretty easy to get locked into certain ideas and certain decisions and i think that that can be really dangerous um you know things like like i think drupal is long for a long time had a resistance to composer to the point where that became a little bit burdensome and then we we've heard injuries talk today about how we're encouraging like more widespread composer use so really check your dogma at the door as you're coming into this dev ops thing try to look at every solution or every problem as its own problem and don't envision solutions first so handling it outliers so it's it's great to say that we're gonna we're gonna make this beautiful tool set and everything's gonna be perfect all the time uh but that's not reality in reality you know that every project is gonna have you know some outliers so what we want to do is acknowledge that um and make sure that our processes are flexible to accommodate those things so that goes back to the the interface kind of paradigm how we're thinking about this and encouraging modularity so that we can accommodate these things um so we know that these differences are gonna they're gonna cost money because it's gonna be a thing you have to figure out so be frank with your clients you know be like hey like we see this thing and it's it's different than than everywhere else and it's gonna take some time and some money and the then the the client's answer might be that this is an absolute necessity this is a value add to us we we need this as a project requirement and then you're on the same page you're gonna you're gonna spend the time and the money and deliver the value to the client but the answer might be oh no we didn't realize that that was so hard maybe we shouldn't spend you know 30 of the budget on that one thing so put that out there identify the outliers in your process and see if they're worth it or not make some decisions based on that but then if you do move forward with an outlier you you might have an opportunity to abstract that back to your global name space once you solve that problem it might be useful to you to be able to use that on either all client projects or enhance the set of clients that you can work on one example that rob and i talked about a lot is like hosting like if you standardize hosting on like pantheon exclusively that's fantastic and that makes your development life and your operations life pretty easy but if you get a client that requires something else either contractually if they need platform s h wordpress.com wordpress.com and then and there you have to make a decision like is this worth it is this a value add to the client and to us and if it if it is maybe that's an opportunity to abstract that back out to your global namespace to be able to handle more clients of that type you had the example of the visual regression as well for you uh right yeah one client right and uh so i kind of talked about this earlier but we had like you know a client where they had some some very stringent uh i'm not going to call them pixel pushers but they were very good at qa and uh so like their their requirements were a little bit different from the rest of our our client set and it we use it as an opportunity to get that visual regression testing system in and uh that really benefited all of our clients in the end and us as an agency as well right documentation i told you we had a whole slide about this so we think that you need a lot of documentation but we think that you can do it in a smart way that's not going to be a huge pain to maintain so at the top level your company needs to read me this would outline you know the things that you have as part of your process it might have links off to external tools that you use it might have all of your best practices listed in there and it's a document that you could reference back to from some of this lower-level documentation so uh extending that idea all of your tools and you read me so in the case of the examples that we're talking about up here dupal scaffold has a read me and that's kept up to date and lando has documentation pages where you can go and read those and to figure out how you're supposed to interact with the tool what is the way the tool is supposed to work and then by extension every project is going to have a read me and of course your projects have a read me should tell developers how to spin up the project how to use the project and how to get productive on the project but we think the value in structuring it this way is that now you can have that project read me focused on the differences between projects because everything that's the same about them is upstream in your company read me and your tools read me and now you can focus on what what things are different about this project that you have to know so we think this makes sense to be able to extend your documentation and just keeping it up to date is so important it's such an easy step to skip if you're given the opportunity but just don't because you could be poisoning a developer downstream that doesn't know about this difference because it's not in the read me when something was changed and then they can't be productive and then they're stuck poisoning uh all right yeah so i want to take a moment to remember back to that slide that we had with all the boxes where we're stacking things up and i think that we made a very conscious choice when we started thinking about this presentation not to talk about some of the really hard problems in dev ops like hosting and logging and you know there's a whole host of things out there we think that agencies and large sort of corporate entities that are building out many sites at once should not be doing these things we think that you should be delegating these things to good partners so people like pantheon platform in aquia all of whom are sponsors at this conference are great fits for this you shouldn't be worrying about how your code gets from the get repository to the dev site like that's that's a solved problem and it's not really worth us spending a lot of time on what you should be worrying about is making sure that what you're pushing up is of good quality and that it won't break the site so we think that you should pick people partners that are aligned with whatever your goals are as an organization and also ones that are easy for you to work with so if you really like let's imagine pantheon their multi-dev service is just the way that your organization works then great go with them the thing is you're not always going to be able to get that so this goes back to that question of outliers and how you handle them when a new client comes on sometimes they come on with specific hosting recommendations and unless that recommendation is like wordpress.com then you should probably not force your your partners on them on the other hand we do have often the ability to make recommendations so if they come in and they say we would like to host on i don't know blue host you could say you know what that's that's awesome i'm really glad to hear that you've thought this through just so you know we work really well with pantheon and aquia both of whom are wonderful hosts and have competitive pricing and we think that development would actually move forward a lot smoother if we're able to work with one of those two and so i think that that's our role in this this conversation about hosting is to inform the clients about who the the kind of best partners are the ones that you work well with and just be honest about like what the limitations are continue to reevaluate this is also key this speaks to not being dogmatic and and many other points that we've made throughout you just you absolutely have to look at what's going on and seeing if you should continue in that direction or if change is needed rob you want to take the quote yeah right so i like this quote from steven king it's about killing your darlings which sounds a little violent but really what it's about is he's a writer and he's saying that when you're writing sometimes you need to kill the things that you love at a particularly dramatic moment in the story to further the story and as developers i think we kind of have or developers and ops people we need to really take this to heart and consider it because sometimes the tools that we build are not the best solutions for us and we're kind of blind to the fact that uh that that's the case so we should always be a little bit sad but a little bit enthusiastic to kill off any of our own stuff because it means less for us to do so as you're evaluating your process just keep this in mind and see if there are ways that you can reduce your overall technical load even if it means killing off your favorite i don't know project that that you built in right and so devops is all about iteration you know we're trying to move quickly we're trying to be agile and your needs are going to change you need to be responsive to feedback and you need to be able to collect that feedback as well monitoring is a huge piece of making sure that your overall process is working and that can be really tricky when you're talking about i don't know 20 projects or whatever but just start with what you have as far as gathering metrics maybe like just take a look at how deployments are going across all of your projects are things going smoothly are there um hiccups that you're hitting all the time that kind of thing and just start to quantify that and see if you can reduce it over time this is really like the reason that we're here is to to collect these sorts of metrics and to improve to improve them yeah so pick one thing improve that then pick another thing don't try to improve everything all at once it's too daunting yeah kind of if you want right so and just to kind of loop back here this is the model that we're working toward and obviously these box the the choices of what's in the box is totally irrelevant the idea here is that as an organization we really want to encourage you to be thinking about how you can move in that upward trajectory and build up that that stack of things that you drop on the table when you walk into your next pitch meeting for another project you have CircleCI you have visual regression testing you have whatever else so and then you can carry that forward to every single project that your your agency works on for your company that's it I think we'll take some questions if anyone has has them yeah actually if people could step up to the mic as they have questions I think the session is being recorded so that would be great question about your local hosting uh are those are do you guys both work in uh in uh Pantheon only environments because uh would are is it possible to switch land over so that it can it can work on aquia or amazon absolutely it's it's hosting agnostic you can deploy anywhere you like there are uh integrations built in with pantheon that you can leverage for uh easy getting code from here to there but you could also pull in platform tools and deploy a platform sh or anywhere really okay okay yes regarding including everyone in devops um sort of as you move that out to all of your projects do you have any strategies for dealing with uh people on your team who are not like 100 developers maybe they work 10% of the time on Drupal 90% of the time on content strategy okay and end up spending all 10% of their time fixing their local development environment to deal with all these tools yeah so I'll just ask the question um again make sure I get it so it is how to include people that are non-developers in the devops process or very part-time developers okay yeah uh I think that's a tough one I think that your goal there should probably not be including them in the sense that like they're not going to be developing new solutions uh but their feedback is very valid and it sounds like their feedback in this case was that they were having trouble with their local development environments frequently so I think that that is valuable and that is like the feedback that we want to collect from them uh so I wouldn't say that you have to put the devops hat on them but I would say that they are contributing to it in the sense that they're giving you that feedback and that's something that you can assess and say all right is this causing a big enough problem to be worth fixing for us so yeah I mean I I think there are different levels of involvement here and not everyone is going to be actually getting involved on the fixing things side some people are just going to be identifying problems and that's totally fine is this useful for setting up the local dev environments for the developers as well as the continuous integration and interaction with the hosting services uh are you asking about one of this tools specifically or land of tools yeah it certainly can package up all of your configuration so once you configure the CircleCI integration and make that part of your lando.yaml then when the second developer pulls down the project they do lando start it's going to spin up all the same integrations that you had for the first developer so they could do I say Drupal VM and any of those and put in some configuration for that yeah Drupal VM is an independent solution but it's also configurable so you pick one of these tools to manage your configuration for the project similarly Drupal Scaffold can manage the housing of all of these configurations great thank you the using Drupal project as the scaffold makes a lot of sense especially for all the Drupal 8 projects you're doing I imagine you still have Drupal 7 sites out there that you have to maintain that are probably way before composer and all that yeah you have any experiences trying to share your your DevOps infrastructure between Drupal 7 and Drupal 8 sites yeah it's definitely tough I think that you can still run composer on Drupal 7 and it doesn't even necessarily have to be hooked in with the project to be able to pull in like shell scripts or whatever configurations you need via like composer so yeah I guess that's the way I would approach it probably like just set up a dummy composer file in the root and pull that in pull it in that way of course all of our all of our scaffold stuff doesn't work so well on Drupal 8 or Drupal 7 but we're able to share stuff like you know the phpcs configurations and that kind of thing we actually ended up implementing a slightly different solution for the scaffold for sharing those like root files like phpcs.xml which just is capable of going and fetching them from GitHub based on Sember so similar to composer but doesn't actually require like the registration of the package so we can use that yeah Lando has Drupal 7 and Drupal 6 recipes so it's easy to switch between the different platforms it has other recipes too yeah so the two tools you mentioned seem to do a good job making sure that developers are working in the same local environment but how do you make sure that it mimics like what's running on production or test yeah right in the case of Lando we use the .Lando.yml configuration file to specify php versions and database backends that are similar to the hosting service so if they're using Solar 4.6 we specify Solar 4.6 in that file so we can granularly specify very close production parity that way yeah I think it's also worth pointing out that both of these tools are based on Docker and Docker is able to match production very well whether you take advantage of that by actually building your own Docker file is a whole other thing just quick plug for Lando that they have very good production parity with you know Pantheon I don't know I don't know about Aquia but platform like you guys are pretty on it as far as keeping up with those recipes yeah we have some examples that I think can get good question like so I've been using homebrew for like I don't know for forever and I've got you know I've got it all set up and like it's really fast and I've got Solar and I've got like Redis and Memcache and blah blah blah and I switched to Lando and it was like half again like it was just a lot slower and then you know so I'm kind of like you know and so how do you I guess when you've got a developer that's like you know and like I version all of that you know that you know like I do I do a lot of stuff to kind of keep it like up but how do you deal with that kind of situation you was like everyone should use Lando or is it kind of optional or yeah I think the value is if your team is you know more than three but you know five or ten then you have that shared configuration that can scale out across the team but if your development team of one or two and you have the ideal stack then I think you should keep your ideal stack you know because you don't have to manage that complexity across developers you're managing it for yourself but once you have to pass it left or right to other developers no no way yeah my setup is ridiculous and like I have to like recompile PHP when I upgrade things so it's just you know I mean like you know because it's there's but I know it like really well so I know like if I'm going to have to upgrade things so it's like you know because I'm yeah yeah it's just really fast yeah really hard to pass off to other developers it's really great for you to do yeah there's there it's not something that I right yeah and if that is a concern probably the probably the jumping point whether you have to pass that off or not I would say that uh yeah Docker does have some specifically like file access slowness to it um which you know you have to weigh honestly if you have multiple developers on a team I think it's it's already even if you have two it's already easier to write it in Docker and then like deal with the slowness but that's that balance is going to be different for everyone yeah you lose a couple seconds uh on dev cycles but you gain a lot by scaling up to a whole team oh I have the answer switch the Linux that's true that's true okay I'm not on Linux any other questions cool we'll wrap it up here feel free to contact us on twitter if you're interested in chatting and thanks for coming out