 Thank you for attending. We're really excited to host this webinar with Promet Source and we have our lovely guest speakers, Michelle and Alan, who are going to talk to us today about continuous integration, knowing it, doing it and doing this now. So we're really excited to have them. Just a little housekeeping before we begin. Make sure that if you're listening from your computer, select the mic and speaker audio option and you can wear a headset if you prefer or you can listen live. We'll have you muted during the call so whatever you're comfortable with is fine. We ask that if you're asking any questions, use the Q&A window. There's also a hashtag that we're going to be using and I can let you know that in just a second and you can ask questions there. Also there's a post-webinar survey that'll show up and we encourage you to use that so we can better improve our webinars for our attendees. Now a little bit about the Drupal Association. The Drupal Association's mission is to foster and support the Drupal community so they can collaborate together as they innovate the project. There's so many ways that we can help the community together. We host Drupal.org and we're building a tech team to improve the site. We provide grants for community members to find ways that grow smaller communities and further the project like starting new camps in new areas. We also host Drupal Cons. We just had one in Austin which brings thousands together to work on the project and bond as a community. We also provide scholarships to help amazing community members from around the globe attend the event and all of this is funded through our memberships and our Drupal Cons and our partners. Coming up we have our Drupal Con in Amsterdam. That's going to be in September. We also have our Global Training Days program. This is a program for training companies to or community members to host a low cost or free training for intruded Drupal members. It's a really exciting program to get people involved in your community and learn about Drupal. Then we also have another webinar coming up next week, June 24th. MetalTool presents securing Drupal and the tools to test it. We just want to take a moment to thank Promet Source and other supporting partners during our program. They're really a wonderful asset to the community and helps the Drupal Association put on things such as our Drupal Con and supporting our programs that we're doing right now. Without further ado, I'm going to switch it over to Michelle and Alan from Promet Source. Again, if you have any questions please let me know in the Q&A and I'll switch over and ask those questions. Everyone, thanks for joining us. This is Promet Source. Today we're talking about Drupal and continuous integration. The message today is do it yourself. Do it now for real. You might have heard of this before. You might have heard of continuous integration, but we want everybody to have a takeaway here. Whoever you are in this spectrum, whether you're just getting started, you're just learning about it, we hope that there's something, whether you're a project manager, product owner, developer, there's something here that you can take away and start doing continuous integration now. In this presentation, we're going to go over what CI continuous integration means to Promet and that has been something that has evolved over the years. We'll talk about why it is important, why this is something that people talk about. And then Alan, my co-presenter here, will go over his 10 principles of CI. And finally, we will dare to do a live demo. But first, we would like to know a little bit more about you, why Alan and myself are tittering on introducing ourselves. If you could go find the chat over on your webinar and tell Lauren why you're here. So maybe you're here because you heard CI, you kind of have this vague sense that it's important, you want to know a little bit more about it. Maybe you already sold on it and you feel really overwhelmed and you're just looking for a starting point. Or maybe you already are rocking CI and you just wanted to come and get some affirmation about how much better you're doing it than us. And we would love to hear from you too. Please let us know what you're doing, all the awesome stuff. We'd love to give you a shout out and to clue people in on other things that other people are doing. There are certainly many ways to do CI. Or maybe you're just here, you have no idea, someone made you come here, you were trying to make it look like you were doing work today, and you're going to take a nap. You're welcome. So while you're finding a chat and telling Lauren why you're here, Alan and I will introduce ourselves. My co-presenter is the great Alan. Alan Chapel is a Drupal Solutions Architect here at Promet Source. He is the maintainer of Views NaturalSort. He has built modules to help migrate into CiviCRM. He's done a ton of work with CiviCRM. A huge background in automated testing. He worked at all players previously before coming to Promet and has brought on a lot of knowledge about automated testing PHP unit. You can tweet at him at generalredneck, his blog down there, generalredic.com. He's all over Texas. Definitely check him out. Alan, are you there? I'll introduce myself, Alan. I, on the other hand, less interesting. Michelle Crecci. I'm a developer. I'm a commungin here at the Chicago-based office here at Promet Source. You can find me on dupal.org or tweet at me at devmichev. I have done a lot of work with continuous integration mostly because of micromanageness and wanting to make things work better. We also need to give a shout out to our dear friend, Will Milton, who's not joining us today, but he's a Solutions Architect, a guru, a wizard here at Promet Source. The best way to find out more about him is by looking at his GitHub account. We'll talk about Vagrant. We'll talk about Chef. Even some packages, composer things. Will Milton has really spearheaded it all. He's doing some very cool things in the dupal community. Lauren. It looks like we have a mix. We have a couple people that are just getting started and really wanted to get an idea of what is this and how do I do it and then also some people that have a little difficulty with CI and just want to hear a little bit more from the pros on how it can make their life more exciting and productive for their work life. It looks a little bit mixed as we suspect. No smug. No smug. No, no. That's too bad. Well, great. I hope then that our webinar today can be both too technical and not technical enough and just find a way to disappoint everybody. We'll look for that sweet spot. Once again, if you have any questions, you can tweet at us. You can also talk to Lauren. You know how to find her now. But you can also shoot us a question at this hashtag. We'll answer some questions at the end of the webinar, but Allen and I can take questions at this hashtag through the rest of the week, really. All right. Finally, back to that continuous integration stuff. So if you've heard about continuous integration before and you feel like this is important, you probably heard it in terms of Selenium maybe. Selenium somehow just came into the zeitgeist a couple years ago. Everyone was talking about Selenium or maybe you've heard of Jenkins. Certainly this was the way that I first heard about it. I had this sort of vague sense that continuous integration was Selenium, Jenkins, making stuff go, magic happened, wasn't quite sure. My very first thing that I needed to do when I came to Promat, my very first task was figure out how to test. This was two years ago. So I really started collecting the toys that are involved in continuous integration. I set up a Jenkins server. I ran a bunch of Selenium tests. I felt that CI to me was a collection of all these really sweet tools that are out there. And that was a terrible approach. That was the wrong way to do it. And I even did presentations about it where I went over all of the tools that you could use for CI. But in fact, what I learned is that CI is a process and it is a commitment to improving, which I hope that I can expand upon a little bit more. But it didn't make any difference if I had all of these tests running if I couldn't make it mean anything. So essentially what was happening is my Selenium tests were failing all the time. And some of that had to do with how poor the tests were. And I will own that. But also if something is failing and you cannot say with any assurance why it failed, you can't roll it back to anything meaningful, you can't assure that it won't ever happen again, you can't do regression tests on it, then it doesn't mean anything to say that something doesn't look right. I wasn't the only one who has come across this. I've met a lot of people at different camps and DuPleCon who got maybe 30% there with CI by implementing some tools but found that they were meaningless, they added more work, they were tedious. So I have totally changed my tune. When I started thinking about the problem we were actually trying to solve. The problem we're all trying to solve is we don't want to roll a big messy project onto the server and then test this shit. Which is essentially the mentality that we were in. So we would develop, we would add some configuration, we'd have some stuff in the database, we'd have some stuff in code, we'd have all sorts of bits of code that would come together and we'd roll around, Katamari style, putting this project together and we'd roll this big ball of stuff onto the server and just expect QA to tell me if this is okay to put on a production server somewhere. This is the problem we were wanting to solve. Just collecting big ball of stuff. So when I started to realize that continuous integration was not primarily about cool toys and I started thinking about it in terms of a process, we formed a definition of continuous integration for us. So here at Promet Source what continuous integration has come to mean is just finding opportunities to inject small bits of quality insurance into our development process at every stage. So rather than thinking about an enormous process all at once and how are we going to have all this automation and we're going to deploy and build, those pieces came later but when we just found small opportunities to start doing things that's when we really started to cook. So I could Google this for you, I could Google continuous integration for you. If you Google it yourself you would maybe come across this wiki page, this is from the continuous delivery wiki and you see a process here where you have your development team and they're using version control. If you don't have version control that's the first thing you're going to do today, that's your takeaway, go home, get, get all of your projects under version control. Once you have things under version control then you can start to work on building and locking down your build. We'll talk about how we've locked down our build and our deployment. Once you have that then you can start adding acceptance tests, then you can start adding user acceptance tests and then you can have automated releases. So we'll talk about how we've done that and yes the cool tools that we've used to do that as well. So knowing that this is how we have improved ourselves, I think I can start going in and showing you what to do. Ellen. There we go, finally. I've been trying to unmute myself for like, I knew that you wanted to talk this entire time but you didn't have to sabotage me. You got to introduce yourself. You got to be, I'm turned out to be the commungent here you know and that's the way you're introducing yourself. But no, no, no, I'm going to have my part of this conversation here and I'm going to let these people know exactly what CI is. You kind of went over it in not quite the detail that I think that they need to be able to execute on this. So I've come up with some principles and you said at the first of this you were going to go over my principles but no, you were going to go into your demo instead. Let's go over my principles right quick. All right. All right, we have 10 of them. What is continuous integration? We do have what she was talking about revision control. That is your number one. Build automation, automate deployment, self-testing build, and so on and so forth. I'll go through each of these in detail to give you an idea of what you will be doing at each of these steps and starting from the top to the bottom of this list is what you will need in kind of an order of importance. So starting at the top of the list, you must have revision control to make continuous integration work. Also remember you can use the show the answers and questions window there on your right hand side or you can tweet your questions. We're we're sent here monitoring the the hashtag so that in case you want to get yourself out there a little bit more publicly you can. All right, let's start with revision control. So revision control is basically your typical get your typical SVN, CVS if you must, but please get in revision control. Your code must be in version control so that you can start continuous integration. Otherwise you're trying to integrate using something of a method of swapping files with each other. Integration means multiple developers working together. So how else can they integrate their code unless there's a way to manage changes? Revision control does that for you and it's a base of what continuous integration is based around. Then we have the next step, build automation. A single command should have the capability of building a system. So what Michelle is going to show you here in a little bit is actually how we do this. You know, this sounds intimidating, but it's true. So usually what happens is you or some of your developers will build a system that will essentially build from a bash script. So all they have to do is type in the name of the command and then you have a site start to build. Sometimes it's a series of bash scripts, but what you want to do is is get that down where you don't have a manual process and that way people can develop faster and they can build it from a fresh checkout. That allows you to continuously integrate faster as well, which kind of goes down to automated deployment. Once you're able to build your system, why not deploy it in an automated way? Deploying it in an automated way allows you to one, just build an environment on a trigger. Let's say like GitHub. GitHub has the ability to hit up an API whenever you make a pull request. So with that said, then you could have it build on that trigger. It would be automated so it's much faster than a manual process. Have you ever had it where the project managers always sat there nagging you about, okay, can you build the dev for me? Can you build dev for me? Can you build dev for me? And this happens like multiple times during the day because you have the updated code. They want to see that. Now if it was automated, you wouldn't have to take that time out of your routine to go and build dev for your PL. It's also gives you, it's less mistakes because it's the same process over and over. There's no human interaction or error. It usually provides an automated contingency plan. So like whenever you're deploying to live, you have yourself a plan to roll back. So you build that into your script. Then you can just, it says, oh, I failed. Let's restore the database and everything. And this happens without you having to do interaction. Jenkins is a really good CI server out there. There's a couple others that we, that I've heard about here recently, like Go was a great one at DrupalCon that we got to hear about from Rob Ristroff. And there's also Travis CI, which is one we use in particular. And I'm pretty sure that Michelle is going to tell you about. Next, we would have self testing build. So Travis CI and the like, they allow you to automate your builds. In the build process, you can also take advantage of that time to test your build. So what that means is, and so we're building upon our original concept. We had our revision control, using our vision control, we can store tests now, right? And using our automated deployment and automated build, we can now run tests during that time using PHP unit. This will confirm the behavior of our build at that particular time. And so what this allows you to do is to have some insurance that, hey, everything's okay. It's a piece of mind. Now, that's not, it kind of depends on the quality of tests you write, which Michelle also talked about just a moment ago. But at the same time, some tests is better than no tests. You have that green or red lights, let me know on every build that, hey, I'm good or no, we can't accept this pull request. It's better than code review. So what's better than self testing your build? Well, let's minimize what's happening. Oh, the differences between production and our testing environment, or in this case, maybe even our local development environment. So this is kind of where chef and what we use virtual box or virtual machines come in, come in handy. Because environmental changes can lead to our failures. For instance, one here recently I found out is CVCRM does not run on anything lower than PHP 5.3. Well, I mean, we all should be up to the latest version of PHP. Yeah, I know. But there's still some servers out there running Ubuntu 10.10, which only get 5.2. So if you don't have a machine or environment that is running that particular operating system, then how do you know whenever you deploy that it's not going to blow up. So test in a clone of production. You can make a scalable version. You don't have to have it to where it's exactly the same. But test in a clone production. So another thing you would like to do, I threw Michelle off here. Another thing you'd like to do is frequent commits. You want to commit often. Frequent commits makes debugging easier. It gives you a better history of what's going on. I don't know if any of you are developers out there, but the glory of get bisect is phenomenal. If you don't have frequent commits, however, that makes it a lot harder for you to do, said, get bisect. Because with bigger commits, your problem is in a bigger area. You can't narrow it down. So it makes it easier, merge, move, add, and make changes. It makes code consolidation easier. Alan, can I ask a question really quickly? Yeah, go for it. Great. I have two. Why are you using Chef instead of Puppet or Ansible? Okay. So that's a good question as far as, you know, there's different flavors, many different ways to do things. Puppet is definitely a great server management system. Chef happens to be what we do a user group at. They're in Chicago and what are, generally, what our team knows how to use. Puppet definitely can be used. And also, Chef was supported by Vagrant before Puppet was. So that was another reason we use Vagrant. Michelle, if you have anything to add to that, go for it. Yeah, we wanted to use Vagrant, which we'll go over momentarily, to build locally, with the same way that we were configuring our production servers. So we provide our own hosting. We use Chef to configure our production servers and can use those same cookbooks to configure our local environment. And you're right, Vagrant was, it's supported by Puppet now as well. But the other thing I want to say about Chef, and I don't have enough experience with Puppet to make up anything but the scuttle butt opinion, but I really love writing cookbooks in Chef. It's Ruby-based. I find it a lot of fun to put my configuration management in code. I found it really easy to pick up. I was able to get started with it really quickly. I think a main theme, though, that we have is you should do what everyone will adapt. So if everyone feels excited about Puppet, if that's what people enjoy doing, go for it. We found that Chef was what everyone got excited about here. Chef was, you know, Chef solved the problems that we had, but I wouldn't necessarily recommend one or the other. And with that said, I've actually written Puppet, I forget what they call them there, because it's been so long to go. I wrote Manifest, I think is what they're called, but for Puppet, when I was working at allplayers.com, and it has a lot of the same functionality, and it's written actually really similarly. So, you know, they're both very powerful. They're just a little bit different flavor, in my opinion, for what that's worth. Disclaimer here. Great. All right. No problem. Is there another one you said? You have two? I think you answered the question. Okay, got you. So with Frequent Commits comes Code Consolidation. So what that means is, is everybody should be on the same page. I, you know, this is controversial, and I'll admit that. But the idea of everybody, you know, refactored, not refactored their code, rebases their code once a day would mean that they have all the newest changes for that day, even if that code breaks something that they're currently working on. Now, with that said, I know that you want to move forward, but what that does is that forces a problem to be fixed, and a problem to be, you know, for people to be aware of problems. And therefore, you also have less problems merging code whenever someone fixes the break, because at the point that everybody brought stuff in, then everybody also has all your changes, and then you're able to, you're able to see exactly where people's been making changes. So you should expect that stuff to be broken on development once a day. Additionally, you get to run tests whenever you rebase at that once a day mark. And then, you know, what's broken ahead of time. Yes, you'll know, okay, well that, if you can justify that this is broken because so-and-so is not done with their feature, that's cool. You've justified the test failing. But at the same time, what of the ones that you didn't know were going to fail? Oh, that helps you out, you know? It's like, it's an insurance, and it helps you know faster. And we're all about going faster. So you might want to also have fast builds. Now, whenever I was working at All Players, we had tests that run on Selenium. Selenium isn't always known to be the fastest in the world. So we had this suite of tests, and it's a software as a service that would run for an hour. Okay, during what we would call our code freeze, we'd have the problem of everybody's trying to get their last commits in before 2 p.m. on Friday, because that was our code freeze. That's when we couldn't commit any more changes to, to the release branch. That's where we're, that's where we're stopping bug fixes from there on out for that release. All right, everybody's trying to get their changes in. All right, we have automated, we have an automated build set up. So every five minutes or so, this thing is spinning up another environment, and it's trying to run that one hour test. So it wasn't fast. It was actually clogging up our workflow, because we had to wait for the test to be finished to be able to go and, and, and see what those, you know, to see how that particular build was fair. So the concept of fast builds is important there. At that point, you need to say, okay, there's a new build. Let's go ahead and either abort the tests, because we want, you know, we'll probably get this on this new one, or we need to make it to where we can build multiple ones simultaneously. In addition, it's helpful to get your tests running as efficiently as possible. And, and a good alternative to, to Selenium would actually be, and I'll talk about build availability here in second, would be to use something like Phantom JS, unless you specifically need the ability to test browser, you know, per browser, you know, and see the differences. Phantom JS is webkit-based, and it is extremely fast. And Michelle can attest to this. She'll, she'll show you as we use it on Travis CI. Build availability, make it where your build is available to everyone. Okay. Let people in. Okay. So this means like, I'm not saying like, okay, I got this top secret project. Let the world see it. No, let your clients see it. Let your PM see it. Let your developers see it. So at this point, you can, you can see any build that's ever been made and be able to make them on, on the fly. This gives people trust in what you're doing. So they can see it working at any point. Also, with that said, make your test results available. So this provides kind of a warranty. When we get to talking about behavioral driven development, which Michelle's done a little bit of the hat. Now I say a little bit, she's done a lot. You can actually use your test cases, your use cases as a, as requirements and as a insurance for the project. So in this case, if each of those requirements that you built into the hat test cases do not show up as green, well, then you have something wrong that you need to fix. But it also shows if you can keep those green, you have a warranty saying, okay, these have all passed. The functionality that you have asked for is being delivered to you now. You can also find your, find your problems earlier. So what we used to do at, at all players and what we've wanted to do at Promet Net, is actually send out the test results to everybody. So at this point, our project is done, our, you know, is done building. And now we have a list of test results. In fact, it kind of happens during our code review as it is with Travis CI, because we get the little green checkbox or the, or the red X that says, hey, you know, I'm a, I'm broken or I'm fixed. And that's pretty much where we want, where we, that's your 10 properties of continuous integration. That's, that's where you want to be. You don't have to be at the, through the entire list. But try to keep those in, in mind in that order, because that is like your ultimate goal, the holy grail. The, and the, we want, I'm going to show you a little bit about what we do. Please know that it's been a evolving process. So I had said earlier that I had started with Selenium and Jenkins, and I was testing sites that were about to go live. And that wasn't working because we had just caught a married, a big ball of Drupal configuration stuff. And something that, that Ellen, you didn't mention was getting your being completely database independent. We getting ourselves database independent, which means that we build from scratch as, you know, as much as possible as often as possible, which means putting configuration in code. If you're not doing that, that's your step. You have to, you absolutely have to put your configuration in code. That was our first thing, so that we could take the ball apart and put it back together over and over and over and over again, so that we didn't have a giant ball that was not replicable. So, so what, what did we do? Well, we, we used Chef, like we said, Chef, Puppet, these are configuration management tools. So we used Chef to lock down how each of us were building locally in a way that matched production. We used Chef in, in, this is our Chef cookbook here. You can check it out on GitHub, how, how we spin up virtual environments using this Drupal cookbook. But we used Chef to configure our production servers using a Drupal cookbook and then used Vagrant. Vagrant up is the, the docs online. We use Vagrant to configure virtual boxes. So now our projects include a way to build with a single command a local environment that replicates how production was made. And finally, with that said, you know, we have the first four of my principles already covered, you know, and that's a great place to be if you can just get that far. Right. And then we, we recently, maybe about four months ago, because we have a reliable build and we can, we can build that big ball of Drupal over and over and over again and have that replication and that assurance. And I can check out a git commit and build it the exact same way that it looked when that git commit was made. When we have that sort of ability to go back and forward and rebuild and build and replicate our process, then we can really do testing because we know what we're testing now. When a B-hat test fails, and this is what a B-hat test looks like. When a B-hat test fails, I know I can say exactly why it fell and I can roll back. So enough teasing about a demo. Let me actually show you what we do here. All right. This is what it might look like. I'm inside a demo here, a demo project. And this is what our route looks like. I'm not going to go over everything inside our route. You see that we have Composer. I had alluded earlier to using Packagist. So this is something that Will Milton has done. He has put almost, soon he will have all of Drupal 7 modules and Drupal 7 core on Packagist. Packagist makes projects available through Composer so that we can have a Composer file that includes all of the requirements for our projects. It's very Drupal 80. All of the modules that we would need to build this project so that our root project does not include Drupal core or Drupal contrived modules. It just includes requirements for them with Composer, which is very cool. It makes for a very lightweight site. Really good. Can we get the recipe URL again? Oh, we'll have it up at the end. While we're answering questions, we will keep all of the links. Why not Drushmake? Instead of Composer, why not use Drushmake? Well, since this is going to be publicly made available, I don't want to slam Drushmake. But Drushmake, as anyone who has worked with Drushmake knows, unreliable, slow. It does a really good job at archiving what your project looks like, keeping track of patches, that sort of thing. But in terms of reliably building your site for you and managing your dependencies and making sure that a Drupal contrived module has all of its dependencies before it runs, I wouldn't do it. So with that said, Composer is the way Drupal 8 is going. Drushmake has done some incredible work at emulating some of the capability that Composer does already have. It's fair to say that Drushmake was probably around before Composer, actually. But Composer does it so much better. I really think that it's a great way to manage your requirements. And you can run just about any script you want to, just like you're running on the show. Absolutely. Yeah, props to Will Milton, Windmill Will, for putting Drupal up on packages so that we can use Composer in Drupal 7. That's very cool. So this is our vagrant file. You should know that this is many, many, many iterations of a vagrant file. We used to run bash scripts within the vagrant file. Vagrant can run a bash script to configure your environment. Now we have pre-provision boxes. This happens to be an oracle flavor of Linux because this particular site is going to run on an oracle flavor of Linux. And it can run just shell scripts, and there's quite a number of bash scripts that are being run here. It's not important that you use vagrant the same way that we use vagrant. What's important to know is that this single file, this vagrant file, excuse me, that's in the middle of our Drupal project, this vagrant file alone allows me to run vagrant up and provision my local environment with a virtual box in the exact same way that everyone else is. So I've actually already done that. Go ahead. So notice that she checked this out and the vagrant file was already there. So she runs vagrant up now, vagrant up, builds her machine, and then now she has a working environment that's just like every other developer. That's just like our staging that's just like production. Right. So by vagrant SSH-ing into my environment, I can navigate to my root project, and our project is in dub, dub, dub, and I can bootstrap it. I should be able to bootstrap it if I can't. Errors. I can use it in the same way that everyone else is using it. I forgot that I'm using an oracle flavor. All right. So just bootstrap it. Enough about vagrants. That's single file. If you already have version control down, if you're already putting all of your configurations and features, if you're not already putting all of your configuration and features, if all of your configurations are not exported yet, that is the first place to start. Make sure that your site is with update hooks and features. You are not database dependent. If you're not there yet, you can stomp around. That's your takeaway. Make sure everybody is using features, update hooks. If you are there, I would highly recommend using vagrants. You don't necessarily have to use Chef. You can configure an environment with bash. There's already vagrant boxes available to you. Go read vagrant up.com. You can look at how your developers can start repeating their process over and over again. Once you have that, we use a build script and it's just bash. We've experimented with other things. We've used ant. We've played with thing for a while. Thing is just PHP, ant. So all the problems with ant plus all the problems with PHP. But we just decided to use bash. Bash is simple. All the developers understand it. It's just command line language. You see here that I'm passing all of my arguments here to Drush. But I'm just running a bunch of Drush commands. And you might think, why is this important? You're running a bunch of Drush commands. Why have a script for it? Because this is the order that you need to run the Drush commands in order to build this site reliably over and over and over again. And I want to know if Allen makes a change to this project, if he adds something that I need to do. If I need to Drush SQLC a table or drop a table, if I need to run another script, I have a script here that just adds text on me terms. I want to add it to the project and Allen doesn't have to worry about it. We don't have to have this oral history between the two of us where I have to ping him and oh, I forgot to tell you or I'm not keeping some read me file that you have to read every time you want to build your site. Nope, you just run the same script over and over and over again and it builds the site. So if you don't have that, get going with your build automation. Make sure that you can build your site in a single command. So I built this site with that command build Drush build.sh and it looks something like this after I build it. And I can be assured that if Allen was to pull down this site, if Allen was to pull down this site and he was to run vagrant up, that he would get this exact same thing. And once you know that Allen will have the same thing that I have, then we know that I could create that environment over and over and over again. So that's when I can start using Travis. So we're using Travis to catch our commits and build the site. So our Travis file looks like this, which looks more complicated than it actually is. What this is essentially doing is repeating the build process so that Travis can build the site the same way that Allen and I would build the site. So it can't use vagrants. Actually, I think Travis can run in vagrant, excuse me. But it can spin up the site on a get commit. And if I were to add my changes to the vagrant file. Michelle, I have a question when you're ready. Go ahead. Yeah. Why rebuild the database in CI when actual deployment is going to involve modifying an existing production database? Okay, that's great. So I think what's being asked here is what happens when you have an existing production site, right? Let's say I worked on the New York Times. The New York Times has content. They are not going to rebuild their site over and over and over again. So they have legacy, they have legacy content, they're using Drupal for what it should be. It's a content management system. It was meant to have content in the database. So at that point, it's already in production. And we call it production at the point that clients start entering their own content. There's a number of ways to do it. This is where features becomes really important and deploying features and update hooks. But the production database becomes an archival fragment of the build. So I need a snapshot of production in my build now. So we would transition this process, this vacant process with bash scripts to being what we would call database dependent or it has a reference database. So every developer would need to develop on a snapshot of production. For us, that snapshot of the production database happens right after our last release. So as soon as we release our tag onto production, we take a database dump of the production database and we all start developing against that database. So instead of where I had started from scratch, I dropped my database and I installed, I run drush si here, site install. Instead of doing that, it would install the production database and then run my commands. With that said also, we would also, it helps for people who do support because they can actually grab a reference database from live at any point and just throw it in there and then have the build happen as well. This will give us a look to see exactly how our next update process is going to happen, like our next release allows us to emulate that so that we can actually test what releasing is going to be like before it actually happens. So I have a couple of follow up questions from that if you have a minute. Do you manage any aspect of the update process and site specific modules update hooks? Can you learn, can you repeat that again, site specific module update hooks? Correct. Do you manage any of the aspects of the update process? Yeah, I think I got this one. So basically what's happening here is they're asking, are we using update hooks to update our configurations and, you know, specific portions of the site? And the answer is yes. We use features primarily for things that are configurable and things that behave nicely in features. There are some things like, especially whenever you're talking about like CVCRM installs or say some panels work or some configurations that must be done in an update hook. With that said, it's easy to do. You do an update, you take your reference database or your brand, because that's the only time you would need to do an update hook is when you're working off of a production site. So you get your reference database, you build your update hooks and then you run your build again. If you notice within the build script, there's going to be a Drush DB up in our build script. That's going to update the current snapshot we have with all the updates of any module that happens to be installed. So for instance, if we upgrade the version of rules, and it has update hooks, those are going to be run as well. So that's kind of a catch-all for us, and that's how we do that process. I was going to just show briefly what a B-HAT test looks like. But if someone has interest in that, they can let me know. These are all of the resources here for everything that we've shown you, maybe missing Travis CI stuff. But Lauren, do you get the sense that we should open it up for questions? It might be a better way to use this time. I think that we're fine for right now. Just continue and then we'll save some for the end. Okay, all right. I will go back to my build here. So what I've shown so far is what we've done with Fagrant to configure a local environment virtually so that every developer can replicate the process over and over again. Once we have that, we have a script that makes it explicit how you build the site and the steps involved that could include a reference database and developing up against a reference database. And then once we have this build that we can rely on, we know that we can build it on Travis. And we can just make sure that we can have a really naive Travis that just sees if things build. If the build fails, then it'll fail on production. So that's a valid test. But you could start injecting VHAT tests in at that point. So we have quite a lot of VHAT tests. I've started to, now that I've spent several months working on VHAT, my ideas about VHAT have changed quite a bit. This isn't a VHAT presentation, so I won't go too much into it. But if you're wondering, how do I start with VHAT? We honestly shifted our weight about it for many months. We talked about doing it a lot more than we actually did it. There is a Drupal extension. There's a Drupal class for VHAT that allows you to do some basic things, bootstrapping Drupal. We do have that link, that's the Drupal extension. I definitely recommend checking that out. Again, we're using Composer to manage our dependencies. I recommend having a Composer file just to go and get VHAT and the VHAT Drupal extension. That's how we get it onto each of our local builds. Once you have that, just keeping a suite of tests, that can be pretty simple. These tests are very naive. They just make sure that all the links that you want to see are on the home page. I felt really silly writing this test when I initially wrote it. I actually wrote this test to give to another developer as an assignment. I'm just creating a ticket in RedMind and instead of saying, hey, give me a menu, give me these links, make sure I can navigate to each of these pages, it was really great that I could just write a test that says, hey, develop until this test pass. This is the kind of ticket that developers want to get. We don't want a ticket that says, give me a content type, then create a view and a menu link to it. We would rather have or at least I would rather have the kind of requirement that was made explicit here so that I could define done and know when it was done and ensure that it stayed done. That's the biggest problem with Drupal is regression testing, making sure things stay done. I wanted to highlight this in particular because this is fairly benign. This just goes and makes sure that social links are there. In fact, this test was breaking the other day from after something I had done that I thought was completely unrelated and it turned out that I had inadvertently messed up some of the permissions here. This caught me in a totally unrelated task from breaking something that I probably wouldn't have known was broken until several weeks later. Using B-HAT to, before any development gets done, defining that task. Giving a test like this to a user, having them develop against this test and they can close it out and say that it's done and knowing that these tests are running over and over again on Travis before anyone can commit anything has really changed our process. So we were finding that the last 20% of any project took 80% of the time. So we would fly through a project, we'd get really close to done and just felt like we were doing and redoing our work over and over and over again because work kept breaking work that we thought was done. These tests has helped us to find done, helped us make sure that things stay done. That was a pretty easy example. We have quite more complicated examples that relate to commerce. What we found is the best way to do things is to really write tests that are very explicit. So instead of using the Drupal extension, which is a good way to start, you can extend the Drupal extension for B-HAT and start writing your own tests and they can get pretty complicated. I won't go through too many of them here, but for instance, I wrote a test here that it looks really simple. I could have a line like given that I am on a article type node with the title test. What that would do is it would go to that page. We use phantom.js to handle the JavaScript. It would go to that page, but it would be able to find that page by doing an entity field query to find the last time that I created a node of that type with that title. It would return that node ID so I could make sure that when I told B-HAT I wanted to go to a node type with that title, I would indeed be able to go to that type. Again, this isn't a B-HAT demonstration, so I don't want to get too bogged down in B-HAT, but if you are in the process of you feel like the time is right, you've got some build lockdown, you feel that you can start injecting some tests. Again, the Drupal extension for B-HAT is a good place to start, but then you might find that tests are more useful when you're really getting into the guts of Drupal, and I loved B-HAT when we first started doing it because I was able to really get inside Drupal really early on, really think about what it means for this thing to be done, how can I prove that it's done, and it's really made me a better developer, it's made better code for me, and it's helped the client understand what I'm doing and communicate with the client a lot better. So to reiterate what we said earlier, CI is a way to interject little bits of QA in your process. That way you know that stuff that you do gets done, stays done, and everyone knows that it's done. Take a look at the resources, we got them right up here, make sure that you pick them up, see we have everything from the Drupal Cookbook that we use all the way, that I believe Will Milton did most of the work on that, all the way down to our resources for B-HAT and PhantomJS. Check it out, see what you can do with it. And on that note, Lauren, do we have any questions? We've really extended the patience of our listeners, I hope that we haven't lost too many people with our excitement for CI. We haven't actually, it looks like we don't have any questions, just positive feedback. So people really enjoyed the conversation in the slides, just a reminder to everyone it'll be recorded, so you'll have that to look over again. And if you have any questions, please let us know right now and we'll answer them. In the meantime, I'm going to switch back and show my screen and finish up. So again, just as a reminder, we have our programs coming up. We have Drupal Global Training Days, which is a low-cost or free intro to Drupal Training. So if you are a training company or a community member that is active in your community and would like to build our leadership and growth of the project, this is something you can get involved in, you can click on the link or you can contact me directly about the program. It's really fantastic so far, I think we've had over 25 countries participate this year. So we're really looking to make this a global effort and spread the word about Drupal. We also have DrupalCon Amsterdam coming up. The website link is there, I know the early bird price is ending soon, so it might be something to consider if you find this topic interesting as well as other topics. DrupalCon is a really great place and network as well as to take educational settings and educational classes and it's a great setting to just meet the community and work together. And again our next webinar is June 24th, Metal Toad is preventing on securing Drupal and the tools to test it. You can also see at the bottom of the screen our webinars and they're more listed and more to come. So just as a reminder, our membership either from an organization or an individual membership is really what helps us as well as our supporting partners and tech partners. This is what really funds the project, lets us have the ability to give scholarships and grants and to maintain our servers. So if you have any questions, look at the link or send us a note and we'll be happy to answer anything about membership. And you know you get these really cool badges that you get to put on your site as well. Can you try to share your screen again because right now we're still seeing Michelle. So I know as you said at the bottom of the screen I wanted everybody to be able to see the links you were talking about. Good to note, just one second. Little technical difficulties. It's not a presentation unless you have some. You can see me reverting my camera to commit. Looking up how to undo it. This is hopefully better. Yes, I can see. Great. Here's the information on DrupalCon, our global training days and our webinars. Also if you just go to either Drupal.org or asos.drupal.org you can see this information as well. And here's our information on our organization and individual membership, associations.drupal.org, backslash membership. And this is where you can find out on how to become a member. And like I said, this is what helps fund us for more scholarships, grants, and of course maintaining our servers and the project. So I think let's look and see if we have any questions. Michelle and Alan, if that's okay. That's fine with me. Great. One second. Yeah, I know it says it's nine your time. It's lunchtime for me. I'm also going to look on Twitter to make sure that we didn't have any questions under the hashtag. All I've seen so far is actually somebody showing the picture of, yeah, there we go. There you are. There you go. So no questions. I just want to say thank you to everyone. Thank you to Michelle and Alan. It was a really fantastic presentation. I think everyone really enjoyed it. And if you have any follow up questions, I'm sure Michelle and Alan are happy to take those. You can find their information on Twitter earlier in the presentation. Or please follow up with me lauren at associations.drupal.org. And I'm happy to pass along any information on the webinar. That's great. Great. Thank you guys so much. Thank you very much. Thanks, Lauren. Thanks, guys. Bye.