 Hi. This is this. That's me. Now, this is a bit of a sneaky title. Only slightly though. It's advertised as have a build and automate it with FING. But it might as well just be called have a build and automate. We'll definitely be looking at FING, but it's not and some details particularly as it applies to Drupal, but it's not a deep dive into FING. That would be even more boring than a presentation on automated build processes. If that's what you're after, then I'm sorry. There is a really good manual online though. That's what they're there for. I've built a lot of Drupal sites. I'm a Drupal developer in Sydney. I worked for Previous Next. I've been doing Drupal since about 2006 in some way, shape or form. I've kind of gone through the fairly common evolution of working by myself, working on larger projects that need teams and that sort of stuff. Over the years I've become more interested in things like how we scale Drupal teams and some of the problems that come up. Drupal's obviously not designed or doesn't work the same way as software frameworks. There's some sort of problems you run into fairly early as soon as you need to work with someone, which I'm sure many of you are aware of. We do a bit of everything as well. A lot of site building, a lot of module development, front-end stuff. We do with accessibility and cross-brows of stuff, performance. The beauty of working on Drupal projects is often you get to dip your toes in everywhere. And integration problems across that sort of stuff. This is in the DevOps track. I'm interested in this sort of stuff from a developer's perspective. Interested in DevOps as applying agile practices throughout the organization. From dev to delivery. Really focusing on shipping. We'll get into some of that. This is fairly applicable to a wide group. I don't know what sort of people would come to talk like this. Hopefully you have some involvement in actually delivering sites. That could be developing. It could be helping developers ship or testing or whatnot. This is a topic that kind of applies across the board. We'll go through it this way. We'll have a look at some of the problems that emerge as you scale teams up. This is definitely, I'm trying to sell you on build processes in general. Then we'll get into some specifics. We're starting at the beginning. Keep that in mind. Feel free to walk out if you've already got your build automated and your CI working. You can stay for the thing if you want to see that. It may not be that useful for you. If you're curious about build processes and you haven't really done that or you're looking to improve, then hang around. We'll cover some of that motivation and then look at some of the how. Some Drupal examples which involve thing. Then I'll give you some advice that we've kind of gleaned from making a mess of some of our build scripts and fixing them. Also some patterns we picked up from the community. Quick story. There is no one workflow. This is what we're dealing with with the build processes. We're trying to tighten up our workflow to automate it. This is what the DevOps is about. There's definitely no particular golden workflow for Drupal. Drupal is used in all sorts of ways. Scaled all the way up and all the way down. Single person shops to big teams and big hard problems. Even though there's no one workflow and the processes and tools are going to change depending on your situation having a process and having a workflow is still very important. Although I will give a caveat that a lot of this stuff is going to be less useful if you're working by yourself but if you have goals to expand then pay attention. Who here is familiar with this though? The very first bit and it says we are uncovering better ways of developing software by doing it and helping others to do it. Through this work we've come to value blah blah and the first one is individuals and interactions over processes and tools. This is all about tools and processes but it's important to as you're nerding out on this stuff keep your eyes on the prize. All about delivering value and if anyone saw the B-hat talk I think it was in this room that had a very good summary of the motivation. Keeping your eyes on the prize that we're delivering value and not get lost in the weeds of all this automation stuff. Let it serve our purposes. I used to work by myself if anyone else did. Life was easier then didn't really need things like configuration management maybe used version control. I was probably more interested in using version control to deploy which is not a great idea either but that's all fine and then you start to work on projects we need to work with other people and this is where the problems start. You do automation before. This evolution is not in one particular order but these are the sort of things that I kind of noticed as I worked on bigger projects. As you work on teams you start to use things like version control to communicate with people not just to deploy you of course use exportables and that's a source of automation opportunity. This sort of stuff becomes a reality. Different tools so as you work on different projects bigger projects you start to need things like solar and of course server stuff. Memcasty and all sorts of things that smaller projects are less of a problem and not only do you need them in production you need them in different environments and you need to track them and so this is the guts. This is where it starts to really become a problem because doing this stuff just by yourself is not that bad you have a production server, you put solar on that, you need solar on your laptop, what's the problem? The problem is when you have different versions on different laptops and different operating systems and that sort of stuff so is this familiar kind of territory? And then development ramp up of course is probably the other thing which I hear a bit of pain about every now and then and that just gets compounded as we get more and more complex technology stacks. So this is related to it so of course we start scripting the stuff away at various times to set up new sites, we have a script to deploy we have scripts, sync environments, we have scripts, manage our exportables maybe we type it in, maybe we have a script a bit ad hoc perhaps at the beginning but obviously we recognize that these are boring parts that are getting in our way so we often do our best to try to automate them. Then we think well maybe we can I've heard of this CI thing, maybe we can do some of this stuff on the server so we copy scripts to the server and we run them there so now we can maybe even automate it on checking, things like that and maybe some of you have even gotten this far which is you start checking scripts into your source code repository so that you can kind of share them, that's kind of the next stage and this is all sort of heading in one direction, I think you kind of play with this stuff for a while until you start to think maybe I need a build system, it has tools now. So some of the build systems around are things like make, rake is there a jake, lots of aches maven and ant and they build stuff build build build build build so what am I talking about it's kind of a hard thing to summarize like what a build is, it's not a script, what is it it's kind of a job, some people use it to just mean a task any ideas, it's the waste it's the non-development tasks that we do so anything that we have to do to get the code running that's not writing the code so it's, you know, we have a word for this in sort of agile in stuff and it's waste and it's not, you know, it's a necessary evil so it's not a terrible word but it's something that we need to be aware of, we need to always know how much waste we've got so that we can look at it regularly as part of our retrospectives and speed it up, automate it, make it smaller if we can sometimes we can't because it includes things like deploying which I mean you have to do that but it does need to be automated so boring, repetitive, automatable yes, it's a build, that's what it is so just in case you're confused it's sometimes hear it in context of compiling code and the reason is because for compiled systems it's not so much about compiling is not the problem that they're solving with the build tool it's compiling lots of things and then compiling things that take a long time and then managing all that together so dependencies between different things that need to get compiled and orders and then really taking care of the time problem there so we're starting to have these problems even on non-compiled projects so we do repetitive tasks and some of them are dependent on others and we want to shorten the time between making the solution and deploying it so that's what builds are about and a good build should have a few properties it should be shippable again, we're focusing on delivering value so things like auto deployment and continuous delivery this is on the road to that continuous integration this is really the glue that gets us there so if we have a good automated build process that we're maintaining and keeping healthy then the rest of this stuff kind of falls out and we're managing our waste not creating it so while we do automate lots of development stuff focusing on the pipeline to production is a sign of a good build process should be repeatable this is an important one so it should be a tool to control our complexity not add to it so it's a fine line with this stuff it can run out of control but it should be repeatable you should be able to take the same source code out of version control into different environments run the build and get the same output it's a simple goal but often goes haywire it's one of those things you had environment parity problems so you introduced an automated build process and now you've got two problems just a quick word on the environment differences they're inevitable right I mean staging is never going to be exactly like production developments definitely not laptops definitely not but we make them as close as we can we use the build process to try to identify the differences and manage that complexity so it's easier to reason about so when we do have environment differences we know where they are and this is the sort of thing you'll get out of an automated build process testable so just by extension of focusing on shipping to production it's going to be inspectable executable we can run it we can performance test it we can look at the final code and lint it and do all the things we may need to do and of course run all our tests which for us I don't know about you but that's a huge suite of functional testing mainly so these are slow tests and we want to run these before production so it's no use if we can only test it live and last but not least it produces artifacts things you can inspect and things you can deploy so artifacts is everything from test reports to the actual executable product so it takes us to what our builds made of get into the pieces other builds that's right they are sometimes made of other builds so I'm going to use some thing language here you'll hear words like script action stuff like that in build literature but just to keep it simple I'll break things down into the thing components so we've got our tasks we want to run a script we want to execute a shell command we want to copy some files you know just boring stuff that we need to do say we're installing dribble we need to change file permissions or it can even be things it can even be deploy type things set up a database in a development environment but it varies a lot this stuff is obviously going to change depending on your setup but that's the beauty of it so then we group those into targets these are the things that we actually run generally and they'll actually take us to our desired state so there'll be a few different tasks run the script copy the files check the code so that sort of stuff and we've got our environment so this is going to change different sets of external files, different servers and different properties and we'll get briefly into that too about what we call jobs so this is outside the build process generally but this is just running a target in a given environment so we do this with Jenkins we tend to do it using Fing so anyone here using Jenkins at the moment? Anyone using Fing at the moment? You're in the wrong room so Fing is not going to make so who's using Fing locally like in local development like a lot only a couple and using it on the build server only about half half interesting both so Fing for those who don't know is a it's called Fing is not going to make it's a build tool it's basically a clone of a patchy ant which is a Java build tool Java in the sense that it's written in Java it's extended in Java so Fing is to ant as say the hat is to cucumber very similar almost exactly the same and the build scripts XML sort of little DSL thingies so we'll see a few of them it's probably the worst part of Fing is that you're dealing with XML and it's fairly difficult to try to make a XML heavy presentation exciting in any way but there's a pass we can expand it with PHP so this is a big win and this is really why we went with it something like rake there's a lot of great build tools out there but the way we use it we run a lot locally we want to get the same build targets local and on the server so we want our developers to be actually using it to be automating their tasks and they know PHP so it's a simple matter of friction and this is stuff that works best from the ground up so your developers should be able to sort of stuff they would be doing in shell scripts perhaps if they know a bit of that they should be able to do it in this and then put in the version control and share with everyone so that's sort of the goal that's kind of the takeaway of why Fing and not something else but anything everyone's comfortable with is really what you should be using because these build jobs should go into source control and everyone can collaborate on them so no Java runs everywhere obviously you're going to run on your server stack so that's all good so I'll give some summary of just broad overview of the big pieces of Fing it's pretty straightforward it matches up with the pieces of a build so we have projects sorry if that's a bit small better so it's XML very exciting has hierarchy so that's the main feature so you got a lot of nesting going on so the project is the root node that's really all you need to know you give it a name and you stick the file in the root of your project and now you're good to go and you can put all your jobs in there and like we said before we have tasks grouped into targets so the targets go inside the project the tasks are executed in the target easy each target can depend on other targets so here we've got a build target that needs some sort of initialization target this is pretty standard stuff and you can specify default for the whole project so generally you'll find that what you're aiming for is like one good kind of ready to go download the source code maybe install some dependencies with Composer like maybe install Fing with Composer and then run the Fing default build and that should run everywhere so you probably want to keep your CI specific stuff in a different build and you have some development tasks in other builds but there should be anything that you absolutely positively need to get going you put in a default build and you just call it like that and you're done so tasks are a little bit more interesting this is where we start to kind of make our little DSL for our build processes so here's one that someone made for Drush so rather than just going back to shell script again it's a bit easier to manage kind of see where we're parameterizing and kind of injecting different environment setups so if you're not doing this I mean I know it's natural to get started with this and jump straight to just exacting out and running shell stuff but this is really where you want to be because that stuff gets unwieldy very quickly and the way you do all that injection stuff is with properties so we use properties to pass the context so we'll have different contexts when we're running on Jenkins different when we're running on our laptop often the same build tasks so here's a very exciting hello world type one I haven't tried these by the way so if you're typing them in I can't promise they'll work they're about right there so properties can come in from various places it's very robust you can pull them in from command line options directly which is great for CI in all sorts of contexts properties files of course is the main one so we ship with sort of a build properties file often to get people going it's kind of you know most people are on similar setups but if they're a special snowflake on their laptop they can just tweak that and run the same build jobs and you can also get properties in interactively through the command prompt so you can use it as an interactive tool as well so that's FIM it's pretty straight forward any questions on that so far easy, installing is very straight forward we use the composer method now which I think the website still lists the pair one but the composer one is definitely solid so you can even go down to locking in the version for the project I'm a vendering fan so I don't mind that at all and using it's straight forward so tasks can log stuff to the console if you haven't seen a build tool before like a regimented shell script and you can choose when to log stuff to the console or not and then there's obviously fail and success status and it bubbles up so the task can fail makes the target fail, makes the build fail, gets picked up by the CI server and they all talk the same language and there's a easy way to get through that so the output and any artifacts are there at the end and they can be used by the CI server also for reporting so there are two ways to get results there so is everyone doing this sort of stuff how are we integrating FIM in CI land these days, are you just looking at the console output or anything else console output or just pass fail so what are you doing with it so we use both, once we have a failing build we'll have some artifacts to inspect generally so that brings us to what we put in so these are the sort of things that we automate and we take a bit of a different approach so I think it's good to go bottom up with this sort of stuff so something that a developer would do by hand they might want to automate it, they put it in the build eventually it'll kind of sneak in upstream rather than putting all your logic into your Jenkins server that should be fairly dumb but we do a lot of the standard Drupally stuff we want to automate everything that isn't real development work, so I said it before but this is absolutely everything so if you find yourself doing something by hand then stick it in here so we sync databases, we run tests we do some managing of our exportables and front-end tasks and then that default build of course to kind of get people bootstrapped so I'll show you some examples I tried to tease out some of our specifics and kind of give you an idea of the sort of thing that might be generally useful but a lot of this stuff is environment specific so we sync environments with Drush anyone else, environments with Drush hand, dumping the database so you start doing it with Drush and then you get sick of typing Drush SQL sync over and over and over and over again so you make a shell script and then you get sick of that and you start to manage it this way so this is pretty cool so we can run the same sync on the CI because we've parameterized the source and destinations and we're dropping it first so this is often important so we're capturing those little details so that's what a real task looks like it has its own name and its own set of properties etc so they're all defined in PHP and I don't have a slide on that but if you're interested I can pull up a simple enough example, they're quite brain dead the task definitions are quite simple we run our tests so might look something like this again we're not calling out to the shell directly we're kind of capturing what's important so where are our BAHAT tests stored, where's the executable that's like the main one, this is the sort of thing that lets us run it in different environments because that's just another property where do we put our output, same build job different environments and parameterized and even stuff that's really dripply like dealing with features for instance I think I worked out that at one point the only surefire way to revert features cleanly is to clear the case twice list the features, revert the features, clear the case again and then do like a little dance but this works I think so you can see here we're calling out other thing targets so build it up into little pieces and keep it clean this is actually not how our current scripts look but they will soon, some refactoring required okay this is another interesting one actually that one of our developers came up with he got sick, we use sort of a feature branch based workflow so he got sick of going to another branch, checking the branch out, syncing the database for like a peer review kind of setup so what's this branch going to do on the current test instance that everyone's working against so you know there's obviously some repetitive stuff around that so he wrapped it up in a switch branch task and then made a branch target so now whenever he wants to test the branch or any now he checked it into the project now everyone's got the task and he just tells everyone about it all they have to do is type in Fing branch branch name or actually in this version it's interactive so it says what do you want to call the branch X if the branch exists like you're checking someone else's work then it will switch to that branch, sync the database, revert the features all in one go so type one thing and now you're looking at exactly what they were looking at so makes reviews a lot smoother this is the sort of little stuff that productivity improvements that are important boring, boring jobs solved, front end tasks even so this is becoming more common, is anyone here using SAS a lot yeah so you need some way to manage this stuff so probably the interesting one here is that we're using Bundler to lock down the versions of the SAS stuff so this is the complex annoying world we live in for instance if you wanted source maps perfectly required feature for dealing with some of these post processed star sheets you needed bleeding eds version check out of this compass thing and that or it wouldn't work one developer might battle with that and solve it they check it into the source repository in the lock file and then everyone else just keeps using the same task it'll depend on a generally we have a prep and that'll make sure that Bundler install or update I don't remember which it's the opposite to Composer I don't have to remember because really she's and they magically get the right version and everyone's on the same page so seemingly simple stuff that can go annoyingly wrong but you only have to solve it once and then we generally again put this stuff all together in a big default build pretty good there really I'm just going to go quickly over some patterns you've probably noticed some of them before a lot of them are common sense but they're kind of the important kind of takeaways if you want to have a healthy build script so we've got this is a favorite of mine so it's called a working skeleton in this book but it's an approach that I've often taken with this sort of stuff if you deal with things like Drushmake and things that can kind of get a bit complicated you don't do the whole you know to a large project or you don't do the whole thing in one go and we focus on deploying to production so it can be very difficult to retrofit a lot of this stuff and get it working smoothly so the way to do it is obviously start at the beginning if you can and don't just start at the beginning to take that thin slice approach and get it working local on the CI server auto deploying nothing auto deploy an empty site and do that first and then you'll kind of you'll know when it's broken and it'll be easy to maintain so it's a simple tip but I kind of see it done wrong and it's kind of the agile way. Location agnostic is a pretty straightforward one make sure it has access to all the files you need use relative paths that sort of stuff and version the script with the source so don't put the build XML on the Jenkins server put it in the project so that's sort of the way to go and that's you know it's like dependency injection script injection so keep the CI scripts on the CI server so you will have CI configuration or your job configuration and maybe some other stuff that's specific to those environments but don't we don't manage them in the source control that you might manage them however you're managing your server configs for instance but keep the app build scripts with the app one for each project you have a starter one but let it diverge project to project if you work on lots of different projects so it's fine and you know it's got to be something that you're free to improve and add to and that sort of stuff and then we just get into sort of some optimization stuff so you'll once you get into this you'll notice that the build times get bigger and bigger and slower and slower and they pile up so look into breaking out targets this is sort of stuff that we're just getting pain points on now and looking at ways of fixing so that's probably more a continuous integration topic but you can start at the build and kind of get your targets to obviously not be rerunning the same target when you have dependencies and get a bit more of a flow going if there's a slow target like find a way to reuse that you can put some logic in and you can kind of check for results and there's little hacks and tricks you can do to not run the same build over and over again if it's been successful move it forward that's this classic CI pipeline kind of thing that you're kind of going for and it's really about just the build jobs should actually get faster and faster the further down the pipeline you get and you do that by getting jobs that trigger other jobs and so that's called cumulative builds and those were actually taken out of a great book on build processes so it's not a CI talk but the beauty of this sort of stuff is that if you follow this approach of starting at the bottom I actually recommend people don't start with a continuous integration server like right at the beginning or take that working skeleton approach and have it very simple don't try to pile heaps of logic in there and this is a Travis file and that's a CI configuration I mean all the real work is done in the, sorry it's cut off it's done right at the bottom, thing build so that's about I think we've got maybe three minutes, two minutes for questions, sorry if I ran a little long there's some must read books if you haven't seen a red continuous delivery it's like the best book ever and that's the book that I've got the patterns out of so thank you, questions? It is so there's a statement, a comment that Ant is language agnostic for extensions, right, right, right I have a preference, again it was the fact that I can write a custom task totally from scratch with very basic PHP knowledge I mean it's dead simple, I don't need a java compiler or anything so if you're saying that I don't need that for Ant then I would guess that they're exactly the same and I'm confused but as far as I know the config syntax is, I mean I've seen like build configs and they look very similar, I mean it's XML Do you have somewhere work or something like this where you are published something like template for the built XML or something like this I'll probably blog this stuff but if you don't want to wait go to, I think there's something from a long time ago, Drupal project template maybe, but I'll put that in the blog as well, but yeah look if you're curious, I think that works, right, so question is is there an extension to use non-XML build language no, not as far as I know, that's really the problem with thing, but all of this stuff is applicable to any build tool, so pick one you like and run with it, but again I would lean towards things that your developers can actually use to extend and not make it in operations only or only one guy in the organization knows Ruby or whatever but other than that it doesn't matter, so go with what's frictionless I think is the way okay, there are many I guess I kind of brushed over that so standardize execution, so you've got one command to run, that's a big one, you could write your own meta shell script, but that's painful, what else the reporting, so often these build tools kind of use a reporting format or support a reporting format, or not so much build tools they plug into the CI servers straight away, so you don't have to do any extra work so that's a big plus, there's plugins for fiend, maven, ant, rake, etc, for all the CI servers, so it just glues together and easy, but I'd say that the maintenance of shell scripts gets out of hand for me, I mean, writing XML scripts is ugly, so you should try to make them less scripty and more declarative and smaller, and there's kind of a subtle advantage of using such a horrible language like XML, in that it makes you keep them tighter, whereas with shell I find my shell scripts can get out of hand quite quickly, because it's easier to get stuff done in it, so there's lots of reasons but let's have the main ones I really should have mentioned that yeah, that was a big, thank you very much, that was a big oversight on my part, there are hundreds of pre-done tasks for copying files, changing permissions, deploying stuff, dealing with, there's even like a Drupal update system like database migration system what else does it do, everything, I mean you don't have to write too many of these things, but they're easy to write as well, so that's also the big win for Fing and I'm assuming that Ant has a similar library, but Fing has an absolutely wonderful library of hundreds of ready to go tasks that have just work, so I think we're out of time, thank you oh, and two quick things, the survey thing, and the code sprint, code sprint don't forget the code sprint