 How many people use Drupal here? Nobody. How many people know what Drupal is? Can you tell us? Can you tell us? It's a common management framework which basically is quite similar to Jumla or something like that which lets you primarily build websites of kinds. So we're not going to talk about Drupal here or what Drupal does really because that's what we're going to talk about. And when I say the top of my title says zero-touch Drupal development because I primarily what I do is spend a lot of my time managing other people's Drupal code. And the use cases that I have to do this kind of work in is managing Drupal instances, several of them across a life cycle of a project which primarily starts at QAN testing, goes on to a staging environment to get approvals from clients and partners and before we move live. So of course the purpose of such a workflow is to maintain a same audit trail of change, reduce risk and liability and be able to roll back changes easily. So you don't want to go live and go, oh, we screwed up, let's go back. So a lot of this workflow, especially when it comes to Drupal and other similar content management framework is done manually a lot of the time because Drupal as a system likes to keep a lot of its configuration, a lot of the Drupal mechanics, whether it's content type, different data type or your views or your taxonomy or categories, all of that stuff in a database. And we all know what lives in a database into version control is not the same as that. So it's something that we've struggled with as Drupal developers for many, many years and we have tried and worked through this over the years and now we use something called features which is essentially a Drupal module which lets you export the mechanics of a Drupal website into code. So anything that lives in the database can be exported into code as another Drupal module which can then be pushed to code and pushed into version control. So that's a device of the first issue we have which is now everything, pretty much everything in your Drupal environment can go into version control. Now, the next thing to stage in life, the agencies who develop for clients I've worked with or I've seen do it, they primarily develop centrally where everyone's SSH and gain making changes to a Drupal configuration got your teams and modules and then taking your code based on and you might see a lot of those based on and putting it into a state site and then taking it to the... The problem with this is a huge chance of... there is also the fact that in this fatigue on the developers because they have to be involved in deployment of all clients. As system administrators, our job is to take that fatigue away from developers and let them just focus on development. Clearly that has a value to it. Developers then commit often, release often without having to worry about, oh, I need to get SSH in there get my code up there, text with their movements somewhere else. So the idea is to automate this stuff. All the developer needs to do is... use Git here. So all we want to... we push in to a certain branch and those changes go live onto the branch that is... onto the branch that is... onto the instance that is associated with that branch. So the methodology we follow... I've done pastries. We use Git. Everything's versioned with code in Git. We use separate branches for separate... separate stages of the lifecycle of a project. So I... usually what we do is we commit... anything that's QA and testing stage is committed. All developers are committing to the amount of branch and then we automate the process of how anything that's committed is detected and pushed to your testing or QA instance of... of boot. When you're on a stage, you want to stage it. So this time what you do is you're just merging the changes doing the Git merge into a staging branch. And again, we pick up these changes, automatically detect these changes and push it to the stage side. And the next process is of course going live, which is again merging your code from stage to live. Of course it still gives the developers the flexibility to have their own feature branches, hotfix branches, as long as they merge it back into the master. How do we do this? I have to be a bit patient with me. My laptop died last night. I'm using the machine I think was developed which was developed in like 1943 or something. So things might be a wee bit slow. So we just... we just talked about the concept. We use version and role for code code. We use features in Google. I'm not going to really talk much about features. It's quite sad to explain it to you. None of us really develop in Google yet, so I'm going to leave that out for now. If you do get into Google development, first thing you should do is get these features so you can manage your code and live in the database in version and role. One of the things about automating stage to live is making sure that you are consistent across your whole environment. So if you've got a QA testing server and staging server and a production server, just make sure you're always running the same operating system versions, the kind of software running across these instances of servers. Always use the same naming convention across projects for the trees and branches. Makes life easier in terms of if there is a naming standard. Always use the same workflow for each project. Sorry? Each environment has how many machines? Well, okay. So for the purposes of this example, we will go, we'll be dealing with, say, let's say three servers. One is your QA server, one is your staging server, one is your... But that can of course change. Eventually you're pushing to a staging environment or a live environment which can be one or multiple servers. That is, again, outside the scope of what you're doing. But yeah, so for the purposes of this example, I'm going to do a demo later which I think is a fun demo to use the same server which is the VTSI setup last evening for the service demo. We're going to just push to one server today but we're going to push our QA stage and live environments to the same server. But it can go anywhere, basically. So yeah, so as we follow the same naming convention, same kind of platform consistency across our environment, we should be fine if we want to automate these tasks. So how do we automate? What do we do? The idea is, for example, to log in to a remote staging server via SSH and run a git pool or custom bash script to apply a change, change virtual loads configurations, manually perform database modifications. We want all of this done. So what's the technology we use? We spoke about git. We spoke about different branches for different stages of the project. So I think this is where things start to get interesting. How many of us are familiar with Jenkins, yeah? So Jenkins is a continuous integration server. It's used a lot to run a lot of... It does a lot of things, but it's used by a lot of people to run tests with some applications and stuff. But one of the things that Jenkins is really good at is it's a reactive tool that can basically detect changes and then run any number of arbitrary tasks or commands connecting to local or remote servers over SSH or several other ways of communication. The fun part about Jenkins is that it's very good at determining success or failure when it's running the test and it's got an amazing modification in the end. So it can email unsuccessful or unstable build and because it's detecting changes from a git repository, it can even email the culprit. In this scenario, the developer is making one push to one branch. And in most cases, when you're doing that, you're still sitting at your desk watching your build. And when you get that notification, you know it's broken, you're in a good position to fix it. How do we do this? So what does Jenkins do here? It primarily monitors a certain branch of a git repository. I'll do a demo of this in a bit. It's going to watch your test branch. You commit your push, a change. Check as immediately it gets to work. It detects your push and goes, cool, I'm not going to do something. And this is what we do. How many of us know what Fabric is? Any of you guys here? You know what Kevastrano is. So Fabric is very similar to Kevastrano. It's basically a Python API which can run a number of arbitrary tasks on local or remote servers over SSH. And it takes a spring to connect to a certain server and runs that Fab file. Pretty similar to what Neusrano does. You could totally replace Fabric with a tool of your choice, or use Fabric, because I just understand Python. So what Jenkins now does is actually detects that change and runs some shell code. In this case, it is running a little script which is a Fabric command. Looking at a Fab file and going, hey, I detect a change. What I'm going to do now is take that change, push it to my test or stage or lies over based on what repository or branch the commit is going to and executes that. We use something called Admin. It's a hosting automation back into the front end and the back end tool which relies on a couple of different modules called Postmaster and Provision and Drush. Drush is primarily a command line tool to talk to Google instances. And what Fabric now does is actually sends a bunch of commands, which are Drush commands to my Provision API server going, I detect a change, pull that, have an existing site, make a new build, migrate that site to this build and it's necessary to the Apache version of configuration and your site is live, whether it's a test stage or live site and so on. Come on, that's good. Yeah, but the idea is you're still basically pushing to a kill repository and the build process here is primarily it's not actually compiling anything. All it is doing, as you will see, when I talk about a build in Drupal, what a primary platform, which is primarily your Drupal code base, the modules that you need to run your Drupal site, which are primary. So if I make a change to a, it could be as simple as that. And what happens in this case now is instead of updating that CSS file that exists on the server or a new platform, and Drupal has a multi-site hosting configuration so what happens is you can run one platform and you can run multiple sites within a platform. So what it's doing is creating a new platform, your site's already running on one platform, it takes that site from there, moves it to your new platform, but you're changing it. So the things that change here are the instance is primarily you have your site, a new platform with new changes and that's how they are. You can see the change to point to the new platform. All of that, which is done by protocol, agent and provision. So that's the same thing basically. So in your case, what you have is you will have a few different stages of development in your QA test, a QA stage, making for a certain branch, you can still use Jenkins to detect that change and run a command to just send those changes to a particular branch. The various combinations possible, I think when you see the demo, you probably start to understand how these things can, you can totally understand. But when you're managing one setup, you still have a process where you have to move from QA to stage to life. So if you're doing that manually, that means as a developer you are getting involved in deployment and that's the idea, eliminate development. And the other advantage here is that there's a whole security now sending changes because you don't have to access it into servers. Developers aren't accessing it into servers. So if there are five developers working on one code base, they don't need to have access to any one. All they're doing is pushing to kit and pushing to a certain branch of kit and then we determine where those changes are going. So everyone has to take precautions while handling any kind of automation. So if you create a new build, you need to make sure your previous ones packed up. With any kind of machine control, we can always revert back changes, but it can get complicated, right? Especially when there's like a database somewhere. So what we do in this case is when I create a new platform or a build, and this is done by AGL during the migrate process automatically, it makes a backup of the whole code base and the database into a separate folder before it creates a new platform and migrates inside. So in case something goes wrong, you can always just fire up your last backup and go back to your platform. Instead of struggling with reverting changes within version control and then you can deal with it. You fix things and push again. So the advantage of automating this whole process from test to stage to live is that you suddenly see more frequent deployments happening because developers are dealing now with smaller changes that they're not moving on a code basis and if your file is around from one place to another, all they're doing is clicking, pushing and getting to the rest of it. Of course it gives you a way to actually track every little change that's happening. You know the culprit. The cause of that developers become open to it. You know you're working with smaller changes, something goes wrong, it's easier to deploy. Also it gives them more ownership over their own changes because everyone likes to see a little gift coming from them. You don't see that if you're moving on code basis from one place to another. So people do start to take more ownership when they're waiting on little things and pushing off and automating all of it. Of course there's less potential for data loss because you're working with the workflow where you're going test this stage to live and you always know so you're not going to suddenly push something to your life side and break the limit. Like I said earlier there's just a lot less fatigue on development so you don't have to work with deployment tools on the server now if that's being taken care of. And when you do test now you can have a release time now. So that's a separate set of so there is this performance testing. So for more notifications now Jenkins like I said can send Jenkins also the notifications API is very flexible so you can plug in all kinds of things. There's apps or iOS apps which can talk to Jenkins and do push notifications to your iOS app you can plug in SMS in there so you can make people's phone boxes notifications to their iOS and Android phones. So that's... I'm going to do this okay so the only reason I made these slides is just to kind of keep me on task but what I will do is I will add a few things to this and put it up on the funnel where there are some links that you guys can look at that a lot of this work we do is a lot of this set up is actually based on a lot of work that this amazing French-Australian developer called Miguel Jark who I worked with quite closely has done he's been kind of like working really hard over the last few years to automate a lot of people's deployment and I'll put up links in these slides and put them up on the funnel to a couple of white papers that he's written and he's actually written around a lot of stuff where you can use similar setups without the people not being involved at all because use the CI server use Fabric Git branches to manage your whole work so we do a little demo there is the Git plug-in but you can write all kinds of stuff and it integrates not just the Git but pretty much any SCM that you can think of whatever you perform how the eager front-end looks like now is that I have a couple of sides here set up this is the actual eager instance that is it's basically making within eager and when you upgrade eager it just takes that eager instance to a new platform this is a test side that I set up for this demo we'll look at that in a minute so this eager does all kinds of things like I can just go in here go create content just like I put on a Google site to create an article I could just create a server or a site so I could create a server and go that's what I want to connect to that's the root username and password or whatever username and password and connect it up to another MySQL server or another Apache server it does it does clustered setup so you can have one eager master and have as many slaves as you want and deploy across across web servers and database servers it's it's pretty nifty actually platform and I say create a platform it primarily means I am just creating which is primarily a Drupal code base plus any installable files or any custom stuff that you might have so you can do all of this stuff from the web front end you can create a new Drupal site based on what platforms you've already got in your eager instance from the web interface but we haven't used the web interface but for a lot of setups where you want to manage a few Drupal sites and you want to be able to just quickly fire up across different versions of Drupal it works really well so you can have a platform for example running Drupal 6 and another platform running Drupal 7 now you want to create a new Drupal 7 site you just go create site use this platform which is a Drupal 7 platform and you Drupal site up in like the next million there's a so what you realize on is provision which which then relies on brush which does all the work in the command line it runs there's a cron job that runs every minute which does the dispatch and if you actually scroll down here I don't have it but primarily what it does is it has a task queue and you can tell it how many tasks it can it can execute on each cron run and what we do is we run a cron run every minute and run about 5 tasks at there 5 tasks so I can do things like within there I can do things like if there is a site running on Drupal 6. say 2.2 and 6.24 releases tomorrow I can just go into that platform and hit a migrate button and go migrate all sites in this platform to the new building 6.24 and 2.2 and so it will it will hand it over to provision run the cron and that's done it will tell you that so that's how the web front end works you know so that's yeah it's very simple to configure and get up and running in terms of like getting a multi-site Drupal installation done now let's go back to Jenkins you can see that I have you can see that I have a little task here going deploy test.honky.com from master what this task is doing is deploying from the master branch of my Git repository to test.honky.com which is the which is the the Drupal website I want to be my desktop website let's look at this task you can see that I have so that's the repository I want Jenkins to pull that's my that's the repository there's a bunch of advanced stuff that you can do so everything you can actually do things like by default Jenkins every time it pulls the repository will can even like so it's basically the same syntax I'm just pulling this so I'm running test.ufo.net is the server my test server that we looked at where the eagerness tensor is running now so what's happening here is deployment as such is passing these arguments to a file I'll show you the code in a second that's the site I want to send this build to if I make a change so test.honky.com is my test site that's an installed profile a Drupal installed profile is primarily a set of modules or themes or you want to use to install your site master now that's not referring to my Git repo that's actually referring to the fact that it's the Eager master server that I want to talk about localhost on master database is running on localhost and there's a build file now that is a build file that trush will use using trush make to actually build my site I can just open it it's quite simple here it can be more complicated primarily all I'm doing here is get the version 6.22 of Drupal code and then I have an installed profile which lives in a Git repository which is going to pull now that's a repository that's an install profile it's a simple install profile which has another make file in it which will then fetch a bunch of modules and install your site that's the build number so what happens here is watch this happen right so I'm actually in the folder that's my Git repository so I have this is my install profile that we referred to earlier and that's the repository that we pointed Jenkins to so we'll just try and make a little change it doesn't matter so I just edited made a little edit to a CSS file yeah let's push now let's go back to the Jenkins interface so Jenkins is now polling changes to that because you're free every minute so we should start see that little task come up here it takes that change it starts to it gets to work and it goes now it's building so we can actually look at the console output basically this is as it's happening so started by an SCM change it detected that something's changed in the repository it checks out your code and checks out the revision and then runs deployment SH that's the command that we asked it to earlier to ask Jenkins to run it's building the platform it's getting a bunch of modules based on my Drushmake build files done done and then tells me where it's made a backup to that's 26 yeah so it verified a new platform that's today's date and time and verified this so it's moved my site to a new platform that is created verified so what we can do now is try and see how we go to stage from test we just try and create a new Jenkins task to now watch our stage branch in the repository and try and merge what the change we just did in test and try and get that to create a new site and I think I will do this and then I will quickly show you the scripts which I use the deployment file and launch so I will create a new job in Jenkins and I will just copy the previous job and we'll edit that after about 10 minutes I have been done in 5 minutes there is no point working but you guys get the drip basically what we did with test what we do in this case is create a new Jenkins job and instead of past we got test and I will just merge or we can share it with a comment but to give it a simple I was just going to do a little merge and watch the same thing happen again with Jenkins detects the change in SCM creates a new build again runs my scripts creates a new build again and so that was the make file we saw earlier so these are the actual scripts that Jenkins was running that's the deployment message so what it's doing is the fact file we want to run is passing it a bunch of arguments that fact is when you run so we are going both side profile so the make file we want to point to and the build number and the date and the date is what we are using to create a new platform so every time there is a new platform it's basically a date and time this is the pitch so if the build number equal to one which means if it's the first time a task is running we have to tell the script that the site doesn't exist yet you have to actually create it so these are the actual fabric tasks I'm running I'll show you where these are defined in a second so it's building the platform it's saving the alias for the platform which is what we have to do in a year after we create the site already exists so it builds the platform and migrates the site, saves the alias from the site into eager which is what we saw in the eager front end that's the actual fact file and this is where I am defining those tasks so what am I doing here that's my build platform task in small size, migrate size, save alias so this is a bunch of trust make and trust commands so it's making the platform taking the arguments from everything and basically builds a platform install to site those are the fabric tasks which is defined in the FabFile and we get the employment SS to run these tasks and that's pretty much what we do so as you can see it's like you can completely take Drupal out of this equation and still use a CI server and a bunch of fabric tasks to pretty much do anything you want so the demo didn't go very well and we lost power and all that stuff that's pretty much it, thank you very much any questions yeah so what we do is we always commit to faster and then we always then we're not committing this