 All right, I think we we can start So this session is regarding Drupal server configuration with the answerable at Drupal VM Let me introduce myself first. So my name is Mincenzo Gambino. I'm a senior Drupal developer from Palermo, Italy I've been working with Drupal for seven years now. I start with Drupal 6, then 7, and now 8 I'm also a Drupal teacher, a Drupal contributor, and I've been speaking twice last year once in Drupal Camp London and once in Drupal Camp New England. The moment I'm working at Back Lemon, which is the digital agencies who built websites using Drupal, we use, we build community and a large website for government education and enterprise. This is some of our clients. So as an educational, we have a University of London, Cambridge University University of the Mam, SEMA, Pearson Higher Education International, and then as government we have civil service, Department of Education as well, and then also charity like MSF and Amnesty International. So what's the agenda for today? Today we're going to have an overview about the configuration management system. We're going to have an overview of Ansible. We're going to see Ansible terminology to better understand its structure. Then we'll write a file that is going to deploy Ansible configuration to an excellent server, and after that we can go into Drupal VM. We'll see this structure and we'll use it to see how to create a Drupal development environment in your local machine using variant AVR toolbox, and then we'll see the procedure to push Drupal VM to a live server or any remote server. So let's start first. With server configuration management, what it is basically is it is a process of systematic update changes to a system using a file. It tracks and controls changes in a software. It maintains the hardware and the software of a system, and also recordings information such as software installed, network configuration, and hardware configuration. What are the benefits? The benefits are that you can fast provisioning a new server, running command with executed files. You can fast recover it from a critical events. So if something happens to a server, you can spin up a new server through that file. You can version control it because basically you're going to write text into this file, so you can version control it. You can replicate easily the environment, and then it makes it easier to maintain all the environment of the stack that you have. So Ansible. Ansible is one of the management configuration tool, and it's used YAML files to write its configuration. So basically everyone who's used Drupal 8 now knows YAML. It applies configuration over the SSH. For this, you can trigger Ansible command from any machine, and then it does not require any special software on the node. So on the external server. So we'll see the terminology. So we have the playbook, which basically is the starting point where Ansible is provisioning the machine. There you can write configuration of the machine. We can write the tasks. We can write the roles. Tasks basically is a block that defines a single procedure to be executed, like install a package or check if a package is installed. Role is just a fine way of organizing this playbook, so you can share a role. Basically, if you want to, for example, install PHP and you have configuration, which are going to be the same for all the projects you do, you can have your role and then share it across multiple instances of Ansible. Then we have facts that are just global variables, and after that we have the controller machine, so from where you launch Ansible, and then we have the inventory file, which is a file that contains information about the server you are managing. How to install Ansible? Only in this is pretty simple. You run the command to install the software properties command, then you add Ansible to the repo, and then you install Ansible. On a Mac, you can use brew, or you can use Python. So it's all easy to install pip and then Ansible. So let's have a look at the playbook. The playbook contains the first line is host, so basically is the targeted server. Then we have the parameter become, which is basically if the user, because it's SSH, the user that is going to connect to the server, it can become a pseudo, so we set this to true. Then we have the vars, which is basically the variable where we can set any configuration we want to pass through the tasks. Then we have the tasks, where the section, where the actual tasks are defined, and then we have the endlers. If you want to check if something is installed, if you want to restart Apache, or anything like this. So this is a simple task. For example, if you want to install BIM on a server, you just define the name, install BIM, then you run apt, oh sorry, you gave the name, and then the state. So we're going to install BIM and the latest release. We can see, we can pass the variable to a task, so this is a almost complete playbook. So we have the host, so it's going to be all, which is coming from the inventory file. The user that will connect to the server will have pseudo privileges. Then we have the vars, where we can pass the package that is going to be installed, so in this case it's BIM, and then we have the task. So the task name is a package, it's going to use apt. It's going to pass the name of the package through the variables, and the state is the latest release. Of course, we can pass more variables, and also we can pass array that are going to go into a loop. So this is a loop into a task, so basically we have the vars, we have the packages, and within the array we have three packages, BIM, Git, and Curd. And after that we run again the defined name of the task, install packages, we got a PT, the name, the item, which basically we get from the parameters here with items and packages. And this will run automatically a loop that will install all the packages within that array. You can also use conditional within the playbook, so for example, if you want to check if BIM is installed, you can register the conditional, so BIM is installed, it's going to run the common BIM-BIM to see the version, and then we actually have the tasks here. So basically this task, the first one, the second one, sorry, is going to check that BIM is installed, so it's going to get the result from here. So if BIM is installed, it's a success, so basically this parameter is the register here. If it's a success, it will show on the command line that it's installed. Otherwise we will output a message that BIM is not installed if the first task is failed here. Then we have the inventory file. So in the inventory file, as I said, we define the server that we're going to deploy configuration to. So we have the IP, and within the IP, I'm going to pass the Ansible user, so I'm going to pass it for the moment root, and then I'm going to pass the Ansible Python interpreter from my computer. Once I've done all those files, I will run the command Ansible playbook. I will define where the inventory is, so we will get all the server details that it needs, and then I will say where the playbook is. So let's have a look at this. So this is my inventory file, so I call it Dev. You can give any name you want. Then I get the IP, and then I get the Ansible SSH user root, and then my location for Python. So the playbook basically is going to look for the host Dev, which is getting from the inventory file. The user will have pseudo privileges. Then I will define a variable, which is doc root, because in this example I'm going to basically install Apache. Then I'm going to copy a file from my computer into that server, which will be the index HTML, and also I will copy the host file, which is going to be copied under the Apache set available files folder. So as tasks, first of all, I run an update. Then I will install Apache, so I have the task installed Apache, the name, and the latest release. After that, I will create my doc root, and the doc root parameter will be get by variables. It will be a directory, and I will set up permissions as well. After that, I will copy my index file from my folder's templates, and I will paste it and I will copy it into this nation folder in the index PHP in the server. And after that, I will copy the Vhost template into the site available folder within Apache in the target server. And after that, again, I will restart Apache. So this is my Vhost file. This is my index PHP. We can see that this is my Ansible folder, so I have my playbook in the root. I have the inventory folder in the inventory file. I have the template with the index and the Vhost file. So now I'm ready to run the command. So I run the command Ansible playbook, I will say where the inventory file is and where the playbook is, and I will see what's going to happen. So it's playing the depth inventory. The task is setting up, so it's updating. Now it's installing PIM. Again, it's updating. Now it's installing Apache. It's creating the custom document root. It's setting up the HTML file, set up Apache Vhost file. So all those names will be taken from the playbook file, so they will reflect those here. So create the custom document root, set up the HTML file. So you can actually see what's happening. So now if I go here to refresh, I have the site up and running in my API. So this is a server that I created on disillusion. So if I want to do any change, for example, let's say to the index, so I will go here, I will change it, I will save it. And now if I refresh, of course, because it's my local, it will not reflect online. So I need to provision again the machine. So I will run the command again. So it's playing depth, it's setting up the tasks, installing PIM. So now when you set up the HTML file, as you see, it's yellow because it says that it changes while the rest stays the same. And at the end, I know how many files change it as well. So if you have a long list of tasks, I can see how many changes applied and I can check against the list here. So now if I go to my Firefox, the file in the remote. Now, before I mentioned roles, so basically to write any roles, you run the command AnsibleGalax init and then you pass the role name. In this case, I'm going to create a role that will install VIM. So I will run AnsibleGalax init binChance.bim. And this will create a new role. New role as defined folder structure. So it contains default. It contains default variables for the roles. Then we have another fold bars, which basically contains also bars, but they have another higher priority against default. Then we have the handlers, the same with the end of the playbook file. Then we have the files that contains files that cannot be modified during the deployment. And then we have the meta. So basically it consists on attributes such as outer, supported platform and dependencies. Then we have again the tasks where we can write all the tasks we want. And then we have the templates where we can put again the index.hp, for example, the index.html or dvhost file. So if you want to add a role to a playbook, let's add a role to a role and then under the dynamic of the role. So if we see all my folders. So I have the role in here. And here I have all the files. So I have the folds. Files here. It's pretty simple. You can have the meta, the tasks, the templates if there is anything in the box. So basically this is my default. So I need the package version. Those are my meta. So I have the author, the description, the company, the license, the new answerable version, the platform that is going to support. We uncomment everything we need. We can tag it and also we can add dependencies on other roles or tasks. And this is the actual task. This is the playbook file. And from the playbook, I'm going to call this role from here. So basically it means that this role can be used on many other Ansible folder that I have on my machine. So now how global and Ansible come together? Gai, Jeff, created a real nice VM for the local development environment which builds with vagrant and Ansible. This project helps the developer to set up the test and development environment quickly in about 10, 15 minutes. And they introduce the developer to the local development on virtual machine rather than bump or bump. So what are the benefits of Drupal VM? So basically it creates a fully functional Drupal development environment. It has all the Drupal tools that we need to develop site. It is fully customizable and also it's getting better because of the issue and it's updated all the time. So which software is going to install? So it's going to install Apache or Nginx. It can install HP and you can choose the version as well. It's going to install MySQL on other type of SQL providers and also install Drupal 7 and 8. But what it can do more? Basically it's going to install Drupal console, Drash, Varnish, Apache Solar, Elasticsearch, Node.js, Serenium, Ruby, Manfish, Xdebug and many more. We will see the full list up there. So this selection is pretty simple. You download and install vagrant, virtualbox, then you install Ansible on your host machine. It's based on your computer. Then you install two plugins to help the development. So we have the biggest to avoid any changes from virtualbox into a network. Then you use also the plugin, the host updater. So you can create automatically the host. You can update your host file in your computer with the new be hosts. And also then you need to clone the repository from GitHub. So after you've cloned the repository, you can see the structure of the Drupal VM. So you have the default config YAML, which is basically the template for your configurations. You have the vagrant file. Then you have the playbook under provisioning folder. You have the roles that we saw before. And then you have other two files. So you can create a Drupal site from a make file or from a composer file. So to set up your machine, you copy the default config and rename it to config.yaml. And in these files basically you can set up almost everything. You can set up IP, local path, remote path, memory, CPU, which package is going to be installed if the site is going to be installed as well. And once you configure everything, you can run vagrant app. But let's go and see how it looks. So the one that I have, I've loaded in my folder in here. So I have my Drupal VM in here. So inside I have my default config. I copy it, rename it config.yaml. And if I have a look at it, you can see. You can start choosing which basically box you're going to use. So you can use CentOS as well. You can use CentOS 6, 7, Ubuntu 14, 12, or parallels. At the moment I choose 16.04. I choose the vagrant user. What type of synchronization folder it is. It's an FS. And after that I can choose the host name, the machine name, and I can give an IP to the vagrant machine. I can also choose a public IP. And then I can define the synchronized folder. So on the server, I'm going to create, I'm going to have my web folder under var www.drupal-vm. While on my machine I have it on level up of the Drupal VM folder under www. So take a look closer here. So from here the config file is going on level up and it's going under www folder. So I will have my Drupal site actually here. I can define how much memory I want to give to the machine, how many CPUs. I'm going to decide which vagrant version I need and uncivil version. I can set up the web server. I can choose between Apache or Nginx, which database system I want to use. Then this part here basically starts here. So basically we have Drupal installed site through. So it means that when I provision a new machine, I can also install a Drupal site. And I have three methods of installing a site. So I have it from a Mac file, I have it from a Composer file, or I can download it straight from Composer. For this demo I chose Mac file. So basically I copied Drupal Mac file that the example they gave me and I took it a bit and I added one more module. So you can add the same thing into Composer and then add more modules as well if you need. Or you can just run the Composer and install Drupal from Composer basically. After that I'm going to give more variables so I can set up where the Drupal core path will be. The owner of the folder. I can set up the database username, password and name, the name of the database. After that I'm going to give information about the site I'm going to install. So what's the domain name? What's the site name? What's the profile I'm going to install and which modules should be installed upon installation of the site. After that I'm going to create an account as well. So the name is admin and the password is admin as well. And also I can pass more parameters to the Drupal so to install if I need to. From here I can set up cron job in here. I can create drash aliases here. Then I can set up more configure for PHP FPM connection in here. And then I have the section where I'm going to create the host. So the first one that I'm going to create is the Drupal site. So from here it's going to get Drupal domain. It's going to create the service alias, the document root and the extra parameters I'm going to pass. It's also going to create the host for other tools that we're going to see later. Like Adminer which is a database margin system. It says proof, pimp my log and then the dashboard. Then I can choose all the Apache configuration I want. And if I'm installing NGINX I have also the NGINX section in here where I can set up basically the same thing from Apache. After that I have my SQL database configurations here so the variables that I can set depending on which database system I'm using. Then I can install the extras. So those are all the applications that you can install from Drupal VM. So I have Adminer, Blackfire, Drupal console, drash, elastic, search, java, makehook, memocation, relic, Node.js. So all these lists here can be installed as long as you're implemented then it can be installed. You can also give extra packages if you want so you can add your own. After that you can install drash so you can configure it. You can give which port is going to be open or not. You can set up more firewall configuration. Then you have dphp configuration part in here as well so memory, the display error, anything you need here. Then you have the composer part where you can set up configuration for composer. In here you can add your own roles or your own configuration as well so if you want to be installed before the machine is proficient or after so you can add it here. So for example, it says that after the machine is proficient you can configure Solar in this case. You have my SQL configuration, Node.js configuration, Ruby configuration, varnish, which has got as well a template within the system. Pimp my log, xdebug if you want, and Solar and then Selenium. And after that you have the dashboard installed directory and the known host as well. So it's pretty much configurable. So these most some tools that I'm going to show now. So we have Drupal console, drash, Pimp my log, varnish, Adminer, SQLite, xdebug. So let's have a look. So this is my site, integrated within Drupal VM. So I can log in with the potential asset there. So extend. We do the extend. I have the admin toolbar and the devil as well, which has been installed. So I mentioned that I use drashmac file for this presentation. So basically I copy the example Drupal makeYaml file and I remain in Drupal makeYaml or you can use your own file somewhere in your machine. But in this case I use this file. So it's at the core Drupal version and then devil here and the admin toolbar with the version. Now I mentioned before roles. So we're going to see which roles are included on Drupal VM. So we have a lamp, composer, solar, drash, Drupal console, Drupal and many more. Which one we're going to have a look now will be Drupal role. So we can see how it works, how it's going to install the site and how it's going to run composer as well. So within the main folder, we have the folder provision and here we have a list of roles. So the one that we're going to have a look at is Drupal one. So we start with tasks and we see what we have. We have Drupal Composer, build Composer project, build Composer, build make file. So this one is the one that the system recognize when again true to build from make file. So as you see is a list of tasks. This list of tasks is going to create copy, drash, make file into place. So it's going to get it and add it to a temporary folder. When the site is not installed, it's not installed. If the site is installed, it's not going to run this task. So again, it checks folder and generate the Drupal site with drash, make file. So it's going to run this command. So it's drash, but make minus y and then you have the Drupal make file that's going to run. When again, the Drupal site does not exist. Then it check if there is a Composer JSON inside and then it's going to run the Composer command under here. So basically it's going to say basically that it's going to go into the folder and it's going to run Composer install. Well, for example, to generate the site it's going to call the drash path and to make drash and then run this command within the Drupal core path that we passed from the config.yaml file. So this task just downloads the site from drash, make file. And now we have the site installed. So this one is responsible for installing the site. So again, it's checking that the site is installed or not. If it's not installed, then basically it's going down here checking the database and then it's running the command site installed. So again, it calls the drash path. It's running site installed and then in config.yaml we have the Drupal install profile so we can choose our own profile. The default is standard and then it's going to pass more parameter here. From the config.yaml we could pass more parameters and it's going to call it from this part here, basically. So then to restart the web server and then at the end it's going to enable the module that I gave in the list in the config.yaml file. Of course, at the end of it you can add another task. For example, if you want to import configuration you can do it. From here you can write your own tasks from within this folder or you can create your own role and add it to the Drupal VM. We can see the same. Yes, thank you. Welcome. So this install is the Composer file so as for the make file we have this time we're going to run it from Composer so again it's got the tasks and the path in here. So the run is told, the command and everything and then it's going to add the required files as well. Then we have many other roles. So here is the list of the roles within provisioning roles so you can also come here and add your own roles or modify it as per your need. You can see there is Mancache, SQL, Postfix, Postgrade, Pinfmalor, Varnish as well. After that we can, under provisioning we have the tasks here which define more tasks like Chrome, the dashboard, creation and the SSH. And then we also have the template so the template is going to be imported during the provisioning so for example this is going to add the Varnish configuration. This is the dash LSS. So as I mentioned before you can deploy your Drupal VM configuration from your local to your remote machine. At the moment I have it in my local so within my Drupal VM command I can do a background SSH and I was going to look into my box in a digital box so in my bar www I have my files in here I have the Drupal VM with Drupal and the web folder with the site. So I can also run from my virtual machine in my home computer. Clicking exit I'm coming back to my computer. So basically to deploy site to live we need to go into the example prod folder, copy the inventory file, rename it to inventory and then do the same thing with prod config inside the example prod and there we can add the extra configuration for your production environment or your remote environment. To do that you can create a droplet on digital ocean, you can set up as Ubuntu 16.04 and WebGB or RAM and then you need to make sure you can log in from your local computer doing SSH root at IP. So you can customize the config yaml for production, so you copy the same file as the previous one you copy and rename it config yaml and then you can override every parameters you want for your live environment. You need to change your inventory file so you need to set up the name and then the IP and then you need to give the SSH username. After that you again as we did before with the previous command we run unsymboled playbook we say where the inventory file is and then we give the init yaml that will create a user and then we'll set up the machine. For a remote one you will take again 10-15 minutes and then you have the same server installed on digital ocean. To have an example of the folder we have an example so you can deploy to Acre prod and you have all the script as well. So into prod I have the inventory here which I changed with my server in here and then you have the config as well. So again I kept the same name, the same configuration because I want to have my local environment the same as the production environment so I'm not going to run into any problem of PHP version or any kind of missing configuration in the packages that is not going to be on production environment so here we have the script for example for includeSolar and you can add your own script you can modify and customize this pistolbox as much as you want you can add as I say your own roles, you can add more tasks, everything for your need you need to do. Everything is under provision and here you can add other tasks so let's have a quick look at the playbook so it's going to get to the R files, so we have the main and then the fold configuration files and then we start the pre-tasks, so this pre-tasks is everything before it starts the actual task, so it's preparing the machine to host your application so it's defining the config directory including variables the host names and then here we have the roles so all the roles we saw in that folder it's been added here with the workspace too so for example it's going to install java in this case it's going to install java if the java is in the list of what is called extra in our config file or it's going to install solar or selenium or elastic search because it's a dependency it's going to install java as well so you can do the same if you have other packages it's going to create a python sim link it's going to include other tasks include the roles so at this point it's going to install drupal after installing drupal it creates a dashboard it creates a dashboard and then it runs the provision configuration if you need to the admin so this is the tool that is being installed through drupal.vm so I can log in and here I can manage a database and then you can create a database and install privileges of users really quickly if you want to do as well you want to see the dashboard so this is the dashboard for drupal.vm so it's going to tell you which host are present in this the link to it where the document root is and the alias then you go to the adminel we just saw the mylog and then it's going to give you my SQL everything so these tools give you a full functional global environment for drupal that can be then deployed and shared across all of your colleagues within the office can be customized and using ansible as we saw at the beginning and also can be deployed to live server or a dev server share dev server to show a client demo or anything but the production environment is very mental so it's nothing it's not really supported for production you can spin up an external server but then you need to take care of the security and all this thing around it so basically we saw ansible how to write playbook tasks and data terminology plus we saw drupal.vm what it can do, what it does and now basically we reached the end if you have any questions I understand because I installed several lab servers you know they're not like machine and using vibra and then also vibra and this server would be does it have for example if I want to clone website a drupal website I see the difference is ansible basically because I will only install it in a normal lab server then install it which you're a drupal and then you have your drupal website server website so the difference would be that you have I don't know the moment I've been using vor which is octopus octopus here and they have this panel that you can clone sites so does it give you, is it ansible that makes the difference and gives you the possibility of clone sites and this kind of basically ansible is just like you use it to configure the server and manage configuration between servers so you can you have a server with a certain HP version and you need to install certain packages and you need to replicate the same configuration across multiple servers because you have multiple locations where you've developed so your colleague somewhere else needs to have the same configuration you can pass it through ansible inside ansible you can clone the website but you will not find things like a year you can install it you can create your own role and install a year or depth shop and from there you can manage the site that they are in but by default it doesn't give you the chance of installing a year or depth shop you can clone the site through the Mac file or Composer file and then you can run import features or import configuration basically in that case you can clone a site and you can customize as well if you want to download the database from somewhere you have the database file you can download it and basically import it into the database you need to write the tasks for that you can investigate do you have any some tweaking performance in your podium for like a year without any playbooks anything special really just vanilla and I find that on various sites on my local machine I get vastly different speed sometimes I completely destroy the machine starting in the scratch but I think I copy the yellow files of the ones that I created and then I create a new machine and then it is fast to get so I don't know, I have one particularly right now that I'm using it's driving forever to load a page which is happening I think more than Drupal VM it may be a background issue I guess because Drupal VM just creates the server so then it's going to be basically to spin up you need to run the background up so you're using background Drupal VM is just at the beginning when you provision it so if you say that you provision it and it's fast then the provisioning so basically installing from Drupal VM is fine but then after a while your background does something your virtual server gets heavy I never had this issue Drupal VM with the 15 site in it and I have then never run to this issue not the only thing I do usually when I install a new site I just add the site in my config yaml just close the yaml file so in my config.yaml here I just add the new site within here and I also add more parameters here for the site that I'm going to install so when I pass it to other people or if they need it they can just run provision the machine and then of course you have to create your own task and your own role if you want to support it as well but no usually it's the same thing just like it can be that