 Hi everybody, welcome to the AGR presentation. Hopefully you're in the right place. Okay, so I'm Colin Schwartz. Or Colin, C-O-L-A-N on Drupal.org. This is Christopher Gervais or Ergon Logic. We've been doing AGR for a little bit, core maintainers. Christopher's been doing it a lot longer than I have, but I'm doing it for a little bit too. As far as what we do, typically we focus on big sort of infrastructure, SaaS, IS, like mass hosting projects. We don't typically do, you know, small Drupal site builds, that kind of thing. So any sort of really big projects, kind of things you work on. That's us. Do you want to quickly run through the history? Sure, so last year we actually celebrated our 10-year anniversary, I guess maybe two years ago now. Basically it started as a hosting system for a company called Bright out of Vancouver, relatively early in the development of Drupal. And so it was intended to be a mass hosting platform, coupled to Pantheon, but of course many years before they came around. Then that company kind of dissolved and its development seed took over from it. They then exited the Drupal community, Kumbit took up the banner out of Montreal, and that's where I got involved. I worked there for a number of years and was mentored by the project lead at the time, Antoine Bopré. And since then it's kind of spread out into a team. We've got actually a formal co-op where there's a number of agencies that work largely with EGIR that now maintain the core code. In terms of the community, we've got hundreds of installations, which means several thousand users that are operating on it at any given time. We have no idea how many sites are hosted on them, but in the tens of thousands at the very least, I have direct access to EGIRs that run something on the vicinity of 2,500 sites. Distributed across multiple servers. And then sort of where we're coming from is largely from a sysadmin open source standpoint. This is where we kind of live, we're open source developers. And the whole point is to largely share best practices. So as sysadmins we each have our things, the way we like to run things, but when it comes to managing Drupal sites, as you'll see, there's some pretty strict best practices that really ensure minimal downtime, really just high reliability, things that we care about as sysadmins. And so the best practices we've developed over the years are kind of what's encoded into EGIR. We try to take a tool's focus, so providing tools and not imposing policies, so everything can be overridden, or largely overridden. Unfortunately, it's not something that you can kind of point and click to override a lot of that. So if you're a developer, you can hook into it and do a lot of stuff. So that's kind of where we're coming at from that standpoint. And then as I mentioned earlier, it's a free software stack, right? The whole point is from the OS on up all the way through Drupal and whatever you add on top of that ought to be free. So Apache as a web server, MySQL and so forth, the lamp stack is kind of what we target. Cool. One thing I forgot to mention, think of it as an open source presentation, so feel free to interrupt, interject, start discussions, ask questions. That's welcome, please, if you ever want us to stop. So quickly, organizations that are using it, National Democratic Institute, their big US government agency, NASA, everyone knows who NASA is, I think CIVA CRM is the big open source CRM that a lot of folks know about here. European Commission uses AGR, so a bunch of big names there. We've got a bunch of components. So on the front end, we've got Hostmaster, which is the installation profile, which is basically that's the dashboard way to look at it. Hosting is all the modules on the front end that do things. And elder is the theme, like the Drupal theme for that. The back end is something we call provision. So that's what does the heavy lifting, and that's a bunch of custom draft commands, at least in current AGR. For installation, there's a bunch of ways to install it. Probably the simplest way is there's a Debian package we maintain, so pseudo-app to install AGR3. That's pretty simple. That's for the default, you can change some of that. Docker, there's a way to do it with Docker. John did a bunch of work on that, getting that working. And then there's the manual method. So if you want, you can go through that. That's in the documentation. All the projects, it's divided into several projects. They're all on Drupal.org, as you can see. That's one of the project pages. There's some conceptual things that go into AGR that you kind of need to understand if you're using it. Could we talk with each one? Sure. This is basically what you need to do to pull together a hosting system for Drupal sites. You start with servers. We don't actually currently provision servers, so it's just registering a server, which essentially means that we have SSH access to it. And now there's an AGR server installed. However, the AGR cloud module on Drupal does have tools to launch in the software and package some other things. Yeah, good point. There are contrib modules to do some of this. It's just not in core, but yeah, that stuff exists if you want to play with it. In future versions, that's one of the things we do intend to do. But as far as AGR 3 and what ships out of the box, it's mostly a matter of pointing out an IP address, having an AGR user account on that IP address, or on that server, and then having it sort of preconfigured. So there's actually an app package for remote servers if you want to. The majority of cases that we've seen people have one server that's running the Drupal site at the front end, so the hostmaster site, the web server, and a MySQL server all in one. That's kind of a smaller use case. That's a default. There's all kinds of cluster and capabilities and things like that that are with advanced sort of high availability and high performance functionality. So we can get into that a little bit later. But essentially servers are just a node type. They're just a content type that have things like a host name associated with them and things like that. Services are the main thing that they provide. So that'll be MySQL, Apache. We have an HTTPS service that'll generate certificates for you. It thinks of that nature. Platforms are Drupal code base. So you run Droshmake or Composer or something like that to build up the code base that you're going to run. And then sites are the actual installations. So eager from its core is kind of a multi-site thing. We'll get into that a little bit more. But sites represent largely the configuration and state. So the database, the vhost that's pointing at that particular URL and things like that. And then we've got tasks and queues, so the other sort of high level things. So we have a series of sort of cron type queues where it'll run on a regular basis and then that will trigger cron on hosted sites or run backups in the middle of the night or whatever the various queues will do for you. And then there's a sort of priority queue. The main queue is a task queue so that when you're looking at a given site, for example, if you want to run a backup, you push a button. That task gets added to the queue, picked up by the back end, and whatever heavy lifting is required for that is done. So MySQL done, GZipping with the site and so forth. Yeah, this is basically the view of the site. So as you can see, we've got a few sites here. It's got some metadata, and on the right you can see there's a block for the task queue. So those are the current tasks in the queue. They're green, they're done. Sorry, if they're green, they were successful. They're done, that kind of thing. So yeah, that's a little overview there. Deployment strategies. So there are different ways to deploy a platform, or that's the, I guess, AgreSpeak for Drupal codebase. You can use Drushmake. I think that's been in there for a while. That was sort of that historical way to do that. You can use Git, you know, you just point, when you create a platform, you specify GitURL. When you create it, it just, you know, doesn't get cloned. The manual method, you can just stick a directory somewhere and tell Agre, oh, use that as a platform. Composer's the new stuff that we've got in now. So you can use, you know, you can just specify, you can use it, so packages, right? So there's a couple of different options with the Composer one. One is that you're using, you're starting from a GitURL, but you don't have any contrib packages built in, right? So the Git model is one where you can, you commit Drupal core and all of your contrib modules into one giant repository. And then that just clones it, right? The Composer method is our preferred method, which is that you don't do any of that. You just maintain a composer.json file. And then when you deploy via this Composer deployment strategy, it will get cloned in your project, which should only have your custom stuff, your exported features, whatever that kind of thing. And then run Composer install to then compile everything else that's required for the full site to operate. The other thing that you can do there is you can initialize a project like that using Composer's create project command. So you can say create project, goal, gorilla, dash, open social, or slash, open social, whatever it is. And then that'll just pull in the project template and initialize the whole thing for you. So this is an example of the form for creating the platform. So as you can see, you need some information. Where's the doc create? You know, typically web when Composer nowadays. You can put your DURL and then, you know, what branch do you want or whatever. And so based on the name, it'll come up with a path that it's going to deploy. This is the platform too by default, but you can edit that. And then the doc root is like within the project, right? So nowadays it used to be that project at the root you'd have index.php, right? And now everything is moved into web or HTML or doc root. So you just need to specify that stuff. And then there's quite a bit of branch handling, which is something that we use with some contrib stuff to handle environments, for example, right? So we'll have a production branch, a staging branch and whatever. And we can have separate platforms that we can refresh individually to be able to track those environments. And just for more screen shots, we have a bunch of these. So this is to create site form. So once you have a platform, just to back up a bit, first thing you do is, you know, create a server, you know, tell eager about your server. Then you create a platform, which goes on to a server, then create a site, which goes on to a platform. So that's kind of the hierarchy there. So once you have the platform, you can create a site. This is the form. Yeah, domain name is, you know, like example.com. That's, you know, what you would point DNS at or whatever. There's a bit of client functionality. Don't want to spend too much time on that. But, you know, you can't allow clients to go in here and manage their own sites. That's one possibility. As you can see, you know, you pick your installation profile and the platform you want it on, your language, the database server, because you could have several. So you can have some sites using one DB and some using another, that kind of thing. So this is actually the process that's being undertaken in the back end by the drush commands that we've written, right? So to install a site, it's going to create the site directory itself. So remember, we already have a platform. So those by default go under var eager platforms. You could do other stuff, but that's kind of where we do it. So it would be var eager platform slash Drupal 8 or something like that. And then it'll create the site directory. So slash site slash example.com. It'll provision a database for you. It'll write the settings.php and drushrc files for you. Stocks a bunch of stuff in drushrc that's then used later on, like a list of all the modules that are enabled on the site. So you can compare when you're doing a platform upgrade and you know that you're not migrating to a platform that doesn't have the same code that you're using on the site. There's a lot of sort of back end stuff that's happening there. It'll write a v-host for you and it'll restart the web server to pick up that v-host, right? Or reload it in the case of nginx in order to do that. So it's all the stuff that you normally need to do to launch a new site. Except in this case, once you hit submit on that form, it does all that for you. So this is, as you can see, this is what a site looks like, site node looks like once it's created. A lot of metadata. So the metadata's on the left. And there's even a site report summary, health security, if your security's up to date, that kind of thing. In the middle, you've got all the possible tasks that you can run on the site, like things you can do with it. Well, you can only install it once, I guess, that makes sense. You can verify it. That means rewrite the configuration, like the v-host, that kind of stuff. You can take a backup. You can delete backups. You can clone it. So just produce another site that's exactly the same, give it a new name, that kind of thing. Create if you want to quickly clone something, test something, see if it works before doing it on the original. For example, you can delete, disable, reset password is neat. It'll just do basically, if you click on that, you want to run that task as soon as that completes. Where it says go to, there's a little house up here, go to the site. That'll turn it into login to the site. Then you click on that and you'll get the login screen. Click here to log in. Is that going to log in token? So, yeah, that's pretty much it here. A lot of metadata, as you can see. Whether it's HHSPS or not, client authentication, aliases, all that kind of stuff. So bear in mind, when I was saying earlier that it's largely written by sysadmins, this is what I mean. We kind of expose all the data up front. We don't kind of hide it in nicely packaged kind of views. So, this is something that we're looking at improving with future editions. But we certainly find it helpful to just have everything presented to us right off the bat. And then most of these are links to allow us to drill into this database server, this platform, whatever might be needed to dig into this. I think we're going to see in a little while the task log. But I just want to point out that for all of the tasks that we can run, you can click the run button to actually execute them. And then the view button is to get a listing of the Drush output. So Drush is what's running on the back end. We capture that and pipe it back into the front end. And you can actually watch it live a little like your Travis CI type environment where you can see it populating as it goes. And so that's incredibly helpful in terms of understanding what's going on during these tasks. It also tells you the actual command that's being run so that you can yourself go and copy paste it if say there's an error somewhere. You can replicate what eager's doing very simply. And then you can add, you know, dash dash debug and get more detail and be able to dig into it and get it to the point where it's fixed. Or just keep running it here. So we'll see that I think in a minute. Yeah, like typically if you run a task and it fails, it's red, you'd click on view. Oh, what happened? You click on view, you look at the log and you'd see where, okay, what was the output? What failed? Mike, we got a question for the view button. Where's that output displayed? So there'll be a modal pop-up? Yeah, so that's what ought to... Sorry, I don't remember if I have that. I don't think I do. Sorry. Anyway, next time. Believe me that that happens. It's incredibly helpful. That's actually where I spend most of my time because it just works, right? Unless there's a problem with the actual Drupal site, right? So if there's a problem with the Drupal site, that'll be exposed to us through this and then we can just react to it as we need to be. Go on. Okay, so the way we do updates, it's not terribly intuitive right now, although that's going to change soon. If you want to upgrade the code for a site, we do that through what's called a migration process. So what you would do typically is, a site would exist on a platform, you would create a new platform with, say, the same Git repo, but it's updated code like newer master or whatever. So you'd create that, then you would do a migration of the site from the old platform to the new platform. So that's typically how we'd upgrade stuff. Unfortunately, again, from a system standpoint, it's describing what we're actually doing. It's not describing what we're intending to do. So the intention is to run an upgrade, but it's describing the actual task as being done. So essentially that's what migration does, is move a site from one platform to another. It happens to be the way that we do upgrades because the mechanism provides for all kinds of rollback capabilities. So essentially it's going to take a backup and then redeploy the backup, run update.php on this new site that is just created. If there's no failures, then it's going to rewrite the vhost. If there are any failures, your old site remains. It hasn't been torn down at that point. And so you get this blue-green updates capability. And so that's tremendously helpful when you're running an upgrade across like 100 sites or 10 sites, or even one site, but particularly helpful when you're running it across many, many sites because then you can just kind of leave it, come back to it an hour later, and see that 98 of the sites passed. And there's a couple that are red. You can just go in, fix those, and sort of handhold those across the final migration, fix whatever little problem there might be with that. And then you're good to go, right? So there's not a lot of rework and manual stuff. And the reason that we do it this way, by the way, instead of just updating the platform underneath the sites, is that if you have a multi-site environment where you've got 100 sites, you need to run update.php on each one of those sites. And so you don't necessarily want to do that all at once because the load would potentially spike on your server. But the other thing is you don't have a rollback mechanism. So if two of those 100 sites fail, how do you get those back into a working state, right? You'd have to recreate a new platform, somehow deploy a backup to that. Oh wait, we didn't backup, we're just doing upgrades in place. That's kind of where I was saying, the best practices that we've put in place really are rigorous. However, doing them manually is incredibly time consuming and just a pain in the ass, right? It's going to be very detail oriented to be able to get this stuff done that way. And so we've just automated that. And so an upgrade that would take an hour or two, potentially, to do manually, will take two or three minutes with the system. Yeah. If I have to do mean four updates, or do you also take a span of new modules? Oh yeah. Same thing. Core in my end modules. Yeah. So the question was, you do the same for, you know, say, adding new modules, updates, or just core upgrades. It's all the same. So it's any new code, you create a new platform for it, and then migrate to it. And away you go. Yeah. Can I also pull people? Like, how many people here are responsible for security upgrades when, say, Drupal Geddin comes out, right? All right. So you're all here. That makes sense. On average, how many sites are you guys doing? Like, how many people are managing five or more sites? How many people are managing 20 or more sites? How many people are running 100 or more sites? How many people are running 1,000 or more sites? Okay, finally. All right. A million. So for those of you, if you don't mind, those of you who are running, say, 100 sites, right? How long does it take? If you're not using it here. Are you guys using it? The two of you that had 100? No? How long does it take you to upgrade all 100 sites? We do it overnight. It takes a few days, though. Okay. We'll usually three days to do an upgrade and be running a bunch of sites every night. Okay. So for NDI, I run upgrades for about 250 sites. It takes me roughly three hours. And that's me clicking a button and then walking away for three hours and coming back. Right? Like it's this really the use case for this is specifically the cases that you guys have given in that context. Well, and I think we have our own set of scripts that automate some of this stuff. And I think we're really replicating a lot of what you have here. And of course, our scripts are very specific to what we're doing, but they haven't had as much like they haven't got all kinds of rollback stuff. Like there's a lot more things where it goes wrong. We've got to mess around and fix it up again. And so I'm hoping that some of this stuff will help with that. Yeah. We like what we already have, but with a lot more rough edges and ironed out more features and all that sort of thing. Well, right. I mean, Eager's been around now for going on 12 years. And there's like many, many developers have contributed to it over those years to fix all those kinds of edge cases, right? Unfortunately, it gets a little bit stricter in terms of how it does things, right? You can hook into that and alter it, but there's a point at which your alterations are probably going to be compromising the actual reliability of it. So there is an eager way of doing things that ensures that. And it's largely a matter of just platform management, right? Doing a rigorous job of either managing Drushmake files or Composer platforms. But once you've got that in place, you're in really good shape, especially if you can maintain a smaller set of platforms because you can individually upgrade each site. Each site has a migrate button, but at the platform level, you also have a migrate button. And all it does is it queues up an individual migration for every site on it. But essentially, that just allows you to hit one button, select the target platform, and you're done. So from an administrative standpoint, it's really, really efficient. And that's what we've been optimizing for. Obviously not UI, right? That's not our forte. So we're coming to that in the next versions. But a lot of us, we're pretty back in. So if any front-end people want to help with what it looks like, please step forward. So I think one question you didn't ask, Christopher, was, is anyone using eager now? That's a few people. Two people? Yeah. Cool. Good. Great. OK, so this is the migration form. So as you can see, there's not that much to it, right? Do you want to change the name? Speaking of usability, this is how we rename sites. So say you've got adodexample.com, and you want to rename it to b.example.com. You also use the migration form. So what we do is you hit the migrate button, you get this form, and you would just change the name here, and then hit migrate, and it would just rename it. So that's another usability thing. When you first get to eager, you think, oh, where's the rename button? But you have to do a migrate, right? So that's something to improve. There's a particular reason for doing it. That way as well, because we're not just changing the v-host. We're actually going through the database, and anywhere within a field where the URL of the site exists, we're rewriting it there, too. So if you have a link to an image, for example, that's a canonical link, like a full link, we're going to rewrite that within the body text of a node, for example. So that's why we go through this migrate process, because we want to have a backup of what it was before, so that if something went wrong, you can just restore that backup, right? But that's the kind of thing that we're looking at. I've actually never had that fail, which is nice. Yeah, it doesn't fail, but it's the kind of thing that it's always good to have that backup, right? Yeah. So typically, if you do want to upgrade it, so there's new code, it'll show you the current platform that's selected where it's on now. You would select another one. In this case, there are only two platforms. You might have 20 of these. You select the one you want, hit migrate, and away you go. As you can see, there's extra stuff at the bottom. It'll tell you how many, say, upgrades to modules there are. Here, there's none. There are no errors. There's some warnings. You can hit compare, and it'll show you, side by side of, here are the module versions on the old set. Here are the module versions on the new site. Are you sure you want to do that? So you can review that, and then hit go or not, depending on what you want to do. In this case, all those warnings are because that's an older platform, right? So you would actually be downgrading stuff, right? And that's not a good thing to do, hence why it's going to flag it as a warning. So as I said, the sort of core use case, generally, is one server running a web server and a database, and people just running as many sites as they want on that. There's a lot of flexibility to distribute that and to have high performance and high availability sites. High availability, in particular, is one of the things that I've done a lot of work with. So we have these two methods of doing clustering, essentially, of the front-end components. So this is mostly the web servers being structured in different ways. The web cluster module basically allows you to have multiple web front-ends, and it'll r-sync the platforms out to each of them to keep those in sync. The challenge there is that you don't have a shared file system across them by default, and so you need to have something like S3 on the sites that are going to be running configured with S3 to have that remote data store for the files. The alternative is Webpack, which is a little bit simpler to set up in that it mounts the entire platform on NFS and just makes that available via NFS and then mounts it on each of the web servers. And so by that virtue, you have your sites files folder is also mounted on all of the servers. In the high availability mode, generally you're going to want to use the web cluster because you might be able, like one of the things that I've been able to do, for example, is have it run across two data centers and load balance across those data centers, have varnish load balancing across the web nodes and crossing over between data centers at that point. But we were able to get it to the point where an entire data center going offline resulted in one white screen and then everything just started going to the other servers that were up and running. When we brought the data center back up, everything started to flow within 30 seconds to be load balanced back across. Without any issues with regards to file availability, performance, and so forth. So good stuff, advanced usage, just bear that in mind. I just wanted to add that I think I found that it's important to point out that because Airt is a simple web stack, you can wrap anything around it. So if you are into Amazon and all their ELBs and all the things that you do, you can simply wrap that around your server and know that the AGRS single core is where your files are going to live as the primary. Same thing with all these cloud providers now provide ways to do that. And so it's important to remember that this isn't like some Docker locked into things, it's just a way of stacking. So it works within other systems, right? So the scalability is a lot of options. Absolutely. That high availability system happened to be on AWS but I've built similar things on Linode and I don't know, now we're working with Cloud A which is an open stack provider and so it's really quite flexible. And it kind of just doesn't really have to care too much, right? As long as the alarm stack is installed and there's a user with proper permissions, it can use those resources. I think on the site form you saw a little bit earlier that we have site auditing monitoring reporting. So it'll tell you, you'll see big red X if security updates need to be done, give you some other warnings, say, yeah, I mean, you're running out of file system space, that kind of stuff, so that's really useful too. And also warn you of things like the develop modules turned on on a production site. Just useful things to know about sites and keeping the health and security of those sites up to date. All right, so how do you do workflows across environments, development staging, prod, that kind of stuff? So the tool that's crucial for me at least is we have this module called remote site import, something like that. So the way that works is, say you've got three eager installations, you've got dev, staging, and prod, and you want to update the staging environment or the staging site, you can delete that, actually you don't even have to do this anymore. To remotely import, you can say pull in a site from production onto staging where I am now. And it can overwrite that site with the new, it's kind of like the drag and drop environment stuff that Pantheon and Octway have. So you can do that here the same kind of way. Though the JS isn't as nice, but same idea. So if you're, because eager knows about all its possible servers, so on dev, there's also staging and prod. So select the other one that you want to import from, pick the site, decide the platform you want to throw it on locally, and you hit go and it'll just slip in that site. So that's how you would move sites really, really useful stuff. Dev shop, I don't know if, John, you want to talk with that for 10 seconds? It's got a nice front end for all that kind of thing. The moving sites to environments and keeps track of environments more. It also does a bunch of integration where it's like CI systems. Yeah, it's a layer on top of Vegas so that it simplifies the whole like atomic platform workflow so that instead it's more like get push, get pull workflow. So it can be much more rapid in terms of development and like it'll receive requests and clone the entire platform plus site over. And we've got buttons to copy that based quickly between them. So it's just, you know, it's more useful for like one-off development sites as opposed to a thousand platform, sites on a platform. But it's very much better for developers because they can very quickly make changes and just do get push and pull. It's a good thing, but actually there are different use cases for this, right? The runner on this stuff might be interesting, development environments and quickly doing a lot of get pulls and stuff like that and then the other side is like production systems that you want to be immutable, you don't want anyone to do get pulls on them, that kind of stuff. You have to be more careful, you know, make sure using the rollback process, mire of migrations, you wouldn't be updating, you know, doing get pulls on your production platform, that kind of stuff. So they're different, you know, different things you can do with it so it's so simple just to install or create a server very quickly, like often I'll use DevShout just for the development pipeline for a client and even if they're pushing on their own like EngineX system by their server team, you know, so like the development team will hack away, hack away on branches and eventually they'll build a release, build an artifact, send that to the server team and they'll deploy it however they want whether it's, you know, so it's nice to separate, sometimes you can even have like a production server to do that to the migration type release, but use DevShout to like build the actual Chrome base until it's ready Yep, yep, there's another, so while there's DevShout, there's also another I guess distro agar system that does things in addition to that. There's BOA or Octopus which is, so if anyone's sort of a mega 8.cc it's a Drupal hosting company, it's really it's really hosted agar, like you get your own agar and you know, in your own site and that kind of stuff they it's all open source, so you can just take all that and run it yourself kind of thing so that's kind of cool, it's another use case, they did a lot of development work on agar itself so that's good, a lot of people are using it for different use cases, so yay you can also just quickly mention the agar API and you can, so just like you've got hooks into Drupal where you can alter it we have hooks into agar where you can alter it so say you can do things like just before a site gets upgraded or deleted or something, you can hook into that so you can do something now, or do something after or do something at this time, right, so any step in the process you can hook in and alter what's going on, so it's really really flexible that way and that's how all these other projects are hooking into it. We've got a web services component which is agar services which will expose all this stuff so say you've got another front end site or mobile apps, whatever, they can you can control agar with your phone or something that kind of thing, so we do expose all this with the RESTful API we've got an extension of that called agar SaaS, which is sort of for site factory setups where you want to be able to have clients your clients say go to a site decide they want to spin up a bunch of sites, they'll pay whatever, then that could send a web service call to agar back end, to the hosting service and it'll spin those up, it'll do whatever commands, it's just remote control basically and it can run some non-drupal things so the CPCRM support is really excellent, it's actually native and used by the CPCRM project for their own posting of their their own CPCRM deployments there is also WordPress support but it's kind of wonky, so the way that agar was architected kind of assumed Drupal all the way in, so it's kind of baked in and the way that we host WordPress is essentially by tricking agar into thinking WordPress is a Drupal site so we have a drush wrapper around WPCLI which is the WordPress equivalent to drush and then a bunch of provision commands that kind of wrap around that to be able to deploy WordPress code bases install sites on them, things of that nature so we can do it, it's not kind of that native functionality and so there's a subset of functionality that works well with it that is something that we are working on for the next version to kind of remove Drupal as a priority, it's going to be a type of application that we'll be able to support there'll still be a priority because it's where the communities of users is from, but we want to be able to abstract it enough to, if we're writing a v-host and provisioning a database there's lots of things that need a v-host in a database to run and to collect a code base to run it on top of so there's a lot of abstraction that we can do on that basis that we're going to be seeing in future versions I can talk quickly about the HTTPS support so we've now got fully automatic HTTPS going through let's encrypt so they have free certificates for that purpose and so we've looked into that and you can basically just say when you create a site, I remember I showed you the create site form a while ago I don't think it was on that form, it was too far down but there's an option to, you know, encryption no yes or forced it's like enabled, yeah, none enabled or required are the options so if you put on enabled HTTP and HTTPS if you put required it will redirect all HTTP requests HTTPS and as we mentioned earlier there are different queues that do things like run backups and go through things in the task queue there's also a queue for this stuff because those certificates don't last very long I think it's 90 days so everyone's not going to check is this going to expire soon if yes then get me new certificate so that's fully automated so that's pretty cool and there are tons of other control modules also just like, you know, we have control for Drupal we have control for Agar which are also on Drupal.org generally so, yeah, again you can write your own there are all kinds of other ways of doing things you know, if you have cloud services you want to use you can hook into it there's like, you know, S3 plugins and all that kind of stuff, right, so I think I mentioned this but it's just like Drupal core and that there's an Agar API that you can hook in anywhere and do things like that already custom workflows, you can tie it into your CI that kind of stuff yeah, you can inject, say, you know say you want custom stuff in your VHOS config you can throw that in there with hooks run code after site creation, etc, etc so Agar 3 is the currently stable product everyone's out there, it's using, it works it's great, we've got looking down the horizon, down the road Agar 4 is something that John's working on which will basically replace Drush with Symphony keeps the original find out of Agar 3 it's do you want to, I don't know, John do you want to spend a few seconds, nah it's good, okay cool, so yeah, so that's one thing John's working on that's a good solution for now as it might not be able to swear Drush forever so just a bit of background on that Drush 9 so we sort of rely on Drush being global for VM or server, whatever and that kind of was how everything was done with Drush 9 now there's no such thing as global Drush anymore it's a first site thing that's baked in with Composer so there's some issues around that, so yeah we can't stick with using Drush forever so hence the movement here Agar 5 though is we thought we have a good opportunity to rewrite everything and not be so Drupal specific, so do you want to talk with that Christopher here is spearheading that initiative so bear in mind that Agar was originally written in Drupal 5 on PHP 4 and there's still relics of that in the code base back then also even things like CCK Nevermind Fields and Core was not a thing, so most of our fields are custom code that will, and we manage our own database tables and all kinds of things of that nature so with Agar 5 what we're doing is starting from scratch in Drupal 8 the front end is intended to be largely a two things, so one is a default front end and then all of the entities and business logic to allow it to run as a headless Drupal as well so essentially what we're doing is keeping largely the same architecture, so there's going to be entities on the front end that represent sites and platforms and tasks and things of that nature we have replaced the queue that we have now which is largely just a database table keeping track of the tasks that we're adding with a full blown task queue called Celery it's a python application that uses RabbitMQ and things on the back end so it can be fully distributed and instead of us custom writing things in Drush, bear in mind when we started 12 years ago things like Ansible didn't exist and it was just starting we weren't aware of it at the time it certainly wasn't robust enough for us to use and so we wrote how to deploy Drupal sites and we do things like template out of e-host and create a database table and all the things that we were talking about we do those things natively in PHP PHP isn't particularly well suited to doing that kind of stuff now there are tools like Ansible that do that kind of thing much much better than we could ever hope to in context and have whole communities around doing sysadmin type things and so we're looking at leveraging that so now instead of having Drush on the back end we use Drush but we use Drush the way it's intended to be used so that site local install is fine because we can then run Drush site install and have it do its thing instead of us trying to sort of strong arm it into doing it the way that we want to do it but the architecture is largely the same the default UI that's going to come out of it is going to be comparable so that it's kind of a smooth transition for those users who are already using it but then we also have a lot more flexibility and new resources available to us new methods of doing things that will make it a lot more flexible to start re-architecting so we're going to keep those solid workflows that we talked about taking a backup and restoring the backup or deploying from the backup and then only rewriting the v-host where we have all that rollback capability we're going to keep that for now because we know that stuff works what we want to do is then provide ourselves with a bunch of new resources and tools that we can then start to innovate more more nimbly on that kind of stuff and so that's something that is currently in sort of pre-alpha I hope to get an alpha out soon I just haven't had any time to work on it I can deploy Drupal sites and install or platforms and install sites on it already I mean that's the easy part the hard part is getting all of the really rigorous stuff in and so that's kind of what we're working on now but the plan is for it to be able to host anything right, like I said if you're provisioning a database that's something you do for a lot of different things and if you have Ansible on the back end and you're basically just passing variables into it and the roles or tasks that it should be operating on then a lot of other options become available and among them is sort of what John was alluding to the ability to spin up a new server on AWS for example if you want a new EC2 instance Ansible has resources to be able to do that and so you can put a front end in place that would allow you to spin up a server configure it the way you want deploy a platform to install a site on it so from scratch get a whole new system up and running and you can be able to do that from like a local host install you can install it on your laptop and then deploy a whole cluster basically just from a local install so that's the vision for what we're looking for and for those of you that like recursion you could even use AGR to provision AGR which can on its own provision AGR I know yeah add in to an item so the other thing about that is now we started with Ansible but the back end so the heavy lifting tool that's plugable so we're hoping at some point to use Kubernetes that's an option right because we Kubernetes can do a lot of cool things so we're hoping to do that as well I just want to point out we may be moving now to get that in the current stable oh great yeah the helm system is I'm good friend with Scott Ribby from New York he's the main player of the Drupal chart for Helm which is the easiest parent at here how much Drupal or other things is this Helm command line thing and I have a client who someone is literally working on figuring out Helm Drupal Kubernetes right now and I connected and they're very excited so they may be happening quickly yeah that's largely what we're intending to do the Kubernetes back end is also something that like deploying Kubernetes can be a bit of a pain so part of what we're trying to be able to do here is have Ansible provision the infrastructure and then be able to start deploying Drupal on top of Kubernetes we want to be able to do the whole all the layers for it the point is there are a lot of things you can do with this but I think I got this guy very excited about actually not just doing it but contributing making it working with me to get this thing so they can just click and be in production beautiful that's good so we have an architecture doc that's actually a link you can talk in this there's links throughout this when you get a copy of this later I'll post it you should be able to follow the links we have documentation site, all that kind of stuff that's pretty much it so yeah I wanted to leave time for discussion questions, that kind of stuff so anyone have any thoughts, questions worried yeah I tried to think of a few years ago but it seems we just look for like if you have to modify only work for the soft domain structure so it doesn't work for for us we need like slash something? we have that now it didn't work a while ago but it does now it was a contrived module and it was kind of like a little wonky but now it's part of course you have to turn on an extra module so that it'll recognize that you can put slashes into a URL which by default it's assuming it's a domain name and not a URL so that's where but it'll now parse that out and understand that and create the right file structure do models call it sub-deer? I think it's called sub-deer so in any year there's a features page I can reprogram this too much but not all right yeah so if you have a VM of your own you can in the installation documents basically the easiest thing is to use Ubuntu or something like that and then you add our repository we maintain devian packages just install the sudo app install agr3 so yeah you have a VM go nuts the basic install instructions are on that on the main home dish yeah there's like four or five lines kind of to get it done yeah just go to agrproject.org all the right places any any further questions? questions, concerns yeah we're running on PHP 7 now I think we're debugging a couple 7.2 issues I think I just fixed the last 7.2 issue that I found but I think those are pretty minor right there were just notices but yeah I know it's we're on top of that any other questions if not? is the app working for 18 runner? yeah yeah that's how we deployed it the only thing left as of a week ago was a couple notices but I just I think I just committed that or Herman did who can be recruited to maintain an RPM? that's been a long standing thing that we've been trying to a lot of people have stepped up and said we run that we maintain an RPM for you and nobody has so by all means if anybody does want to maybe we put something on the blog let's try that contributors help welcome patches welcome all that kind of stuff that's why we have the manual I'll raise the possibility because I do have clients on Red Hat I just gotta tell them I'm just gonna focus on making this if you wanted to yeah that's why Datshop hasn't installed that sh script because it works on Red Hat as well it's still just basically installing a bunch of packages and stuff but yeah if anyone is looking to do this kind of thing doesn't know how needs help we do consulting support setups for this kind of stuff that's what we do so feel free to come and talk with us if you want on the Red Hat side and stuff one of the reasons we maintain the manual is to make it so that people can access they understand what needs to be done to get installed kind of hides all of that for me right but I also have a number of Ansible roles so that will deploy eager in a manual model for you so that actually supports if I recall correctly it supports Red Hat reasonably well but I might be wrong about that it's been a while since I've tested it yeah well Datshop is all Ansible and we actually it's 90% healing guy his roles are great he mostly handles both families and people that know the same exact and do access like through the VN so it's a lot more cooperation than you might even think and I mean we have actually run this on Mac OSX on Solaris we've had like really weird use cases over the years and I mean Solaris is something that I was really surprised to see but yeah I mean it's a lap stack manager right so if you can run Apache on it or in GeneX well I don't know if they were running in GeneX on Solaris that's probably true great well thanks everyone thanks again