 Good morning, everybody. Thank you for Getting up and awake and in here this morning. This is a great turnout. I am and welcome to Portland for those of you who haven't been here before So I'm Greg Lanche. I am the track chair for the nonprofit government and education track And so I am really excited to see this many people in here today. This is this is fantastic so I'm here gonna introduce my friend Paul Lieberman here. He is the Lead developer in Drupal architect they get that right Paul? Okay, good at Oregon State University and He's been doing web stuff Since well as long as there's been a web As a sys admin and as a developer So I'm going to turn the mic over to Paul and let him get started because he's much more interesting than I am So thank you everybody Good morning and welcome to Drupal con. Welcome to Portland I want to thank Greg for the introduction and also for his work and the Drupal association's work in setting up this new track for government nonprofit and education a lot of us in education been coming to Drupal con and attending Boffs and and There will be quite a few of those happening here. I tend to encourage you to attend But it's really nice to have this track For the first time Dedicated to those of us in the public sector To have an opportunity to share some of the things we're doing How's the volume back there? Okay, thank you so you might have People are pointing to me Okay, I'm not in slide show mode I'm not sure how that happened. I thought I said do a slide show. Okay. That's better So as you might have gathered from Greg's introduction, I've been doing this for quite a while and I'm actually I'm actually older than Dries. I Don't know if you tell But I've been around long enough to know that Dries didn't actually invent the web The web was around before him What what Dries brought to the table and Drupal brought to the table is a way for us to manage very large websites In in in a sane manner before that it really was chaos some of you that have been in education for a while might Remember what it was like it took some institutions About ten years just to convince the art department and the science department that they should use the same colors And then maybe we should use the same our school colors and use a common theme for the for the whole university that that was a struggle and fortunately, we're beyond most of that now and with Drupal we have The tools to really move Forward into a much much, you know much better way of doing it so Speaking of school colors That's our our Drupal mascot there Oregon State. We're the beavers and so we may be Violating the Drupal association branding guidelines by using our color for the Drupal drop But but there it is and I want to apologize to the other OSU if there are any buck eyes in the audience When I say OSU I'm not talking about Ohio State. I'm talking about Oregon State We are the beavers I Work for central web services at Oregon State University We provide Drupal hosting and custom application development and I have with me today share fan and Some of my other co-workers share does all of our Drupal training and user support and she's going to help with the questions That will have at the end. I'm not going to stop her questions during the presentation. I'm going to Just save those for the end Also at Oregon State University. I want to mention we host the the open source lab These open source lab have been doing wonderful stuff in the open source world for quite a few years they host the Mozilla Foundation the patchy foundation and this other little group called Drupal.org So Drupal.org servers are right down in our server room right next to the servers that run our websites kind of a nice arrangement for us so And the open source people are putting on quite a few presentations here at Drupal con Some of them are in the DevOps track One of them is right after this one So I would encourage you if you're interested in any of that to attend those sessions as well So what we're going to talk about today is how OSU has been successful in Deploying and managing Drupal a lot of our websites run on Drupal not all of them, but Certainly the majority so you know, I think we've done quite well at it and But not everything, you know has is really the best way to be doing things. We found things that work I'm going to talk about those things and then I'm going to talk about how we would like to be able to be doing some of it differently We'll talk about what we're currently walking working on and then some of the stuff we'll be looking at or that we have been development for the future It's supposed to be a technical session we're going to look at Some snapshots of some code snippets. I'm not going to get into any detail on the code But all of the code is available for download Following the links on the Resources at the end of the slides. So you'll be able to get the code take a look at it in depth today I'm just mostly going to skim over it pretty quickly because we don't have time to really dive into that kind of stuff So we've done a lot of things right that have allowed us to scale What we've done is we basically scaled I guess you could say scaled outward We have a lot of Drupal sites a lot of individual Drupal sites some of them are quite large But we've mostly scaled by being able to support a large number of sites rather than Put our emphasis on scaling a few really large sites Although we're working on that now So in a lot of ways we've been a victim of our own success in that It's been too easy just to create new Drupal sites whenever somebody wants them and eventually that catches up to you becomes more of a management nightmare and So some of the things we provide at Central web besides the training we can do the site installs and the updates automated backup database backup We provide the support for Drupal Look here are the numbers once again This this this isn't really meaningful in the sense that this is you know great We have this many Drupal sites we have this many Drupal sites We can support this many Drupal sites keep them updated keep them secure, but We really don't want this many Drupal sites We got there because we created a system that supported that and it worked really well And people wanted that everybody comes I want a Drupal site. Sure. Here's the Drupal site We now we have to start having the conversation. Well, aren't you really part of this other Drupal site over there? We're just beginning to look at that kind of thing We we collaborate within the University a Lot of the colleges host their own infrastructure, but they run our distribution of Drupal We communicate with our marketing excuse me We collaborate with our marketing and web communications team They have designers and we have developers so we build the themes in conjunction with them and we have gotten it to where we have just a couple of very Standard themes that have our branding and that we're using across the whole Drupal environment So we put together an application to manage this thing we call it web manage It was written many years ago. I wasn't involved It was written in an older version of Ruby on rails and what it does is it lets us manage basically the departments we call them the site owners and the authors and It creates the Apache v-host configuration Sets up the distribution directory sets up file permissions on the like sites default files or sites files and handles the database creation and And removable and remove sites and handles the Drupal installs and bucks and bulk site updates now we've talked This app is getting kind of old and you know we talked about should we put our development effort into Maintaining this improving it making it better having to do all the things we would like it to do and we decided against that That this was an in-house Application suited our needs there wasn't the kind of thing that was going to really work for a much larger audience And so we're we decided to shelve this thing and let's put our development effort into community project and So that's where we're looking to go with it right now So if you've set up Drupal multi-site For You're familiar with this with this picture here basically with Drupal multi-site you create a Drupal directory That's where your core files are. You'll have a sites all that's where you shared modules and themes will go And then every site has a sites directory under under the sites directory So in this picture, I've got site one site to and they all have if they have custom Modules or themes and their files. I don't show that in this picture. I'll go in those directories. That's the standard Drupal multi-site directories structure It works It works best if you have sub domains like in this example site 1.college.edu site 2.college.edu It does not work as well if you have sub directories Like college.edu slash site 1 slash site 2 you can make it work it involves creating some sim links out at the at the root of your Drupal files, so it's it's limited in how well it scales From from my experience anyhow, I'm sure some people have made it scale quite large, but We found a different way of doing it Which is we have one instance of the Drupal core files and the sites all That we then sim link to The sites directory which which can be anywhere So we're not we don't have to create the sites directory under Drupal core sites we create them anywhere in our web tree and we sim link to the Drupal core files so each Drupal site thinks it's its own Drupal site and just uses sites default for its files and its settings and The thing about this is it scales quite well and I first saw it I I kind of scoffed at it and said well, this this is wrong This isn't the way Drupal multi-site works, but over over time. This has proven to be a a You know reliable and very scalable architecture So the advantages one of the advantages is you can place these sites anywhere in the web tree You can nest them so I can create a Drupal site under a Drupal site or under a non Drupal site That's the advantage and it's also the disadvantage Because once you start doing that you've kind of got worked yourself into a little hole where you've got stuff all over the Place it's hard to keep track of where it is and it becomes difficult to manage Nevertheless that's still what we're doing and it's still working quite well for us So this is just a screenshot of the directory list where you can see that these are sim links That's what the little right arrow next to the file names are and the callout is showing that we use a Just a standard build number for our distribution So we'll have our core files in a directory that will use a the Drupal version number And then a build number and then you know the next release we just create another directory with an updated Build number we try and push out new releases every six to eight weeks Depending on whether there's security updates or you know modules update important module updates Or custom work that we've done that we want to get out there that's slowing down for the Drupal 6 It's been kind of hectic for Drupal 7 There's been a lot of version updates in the last few months that we've tried to keep keep up with so the the web manage application as I mentioned was is a ruby on Rails app, but it actually kicks off a Python script that installs Drupal and So the script does these things up here Basically creates a directory wherever it is sim links in all the Drupal files Creates the sites all and the sites default directory sets all the permissions and then runs drush so this is the first I've mentioned drush that What you're gonna be hearing a lot of through the rest of this presentation is using drush for these kinds of things because drush is the tool If you want to automate Drupal Drushes is is your friend. It's the best way to do it. I don't see how we would be doing this at all if it wasn't For drush, so if you see most whitesman today shake his hand. Thank him Drushes is really the way to go for doing Drupal back-end stuff. Here's a just a snapshot of the Install script that's Calling the drush site install command with a bunch of options if you look down on the on the white on the bottom there I'm showing you the help Drush help site install To really get these options. This is just the script is just showing how I'm passing them into the command And it doesn't have to be Python. You can do this in PHP or your language of choice It could be just a shell script for creating sites using drush site install So don't ask me why but after the Python script is done creating the Drupal site the next thing that happens is PHP script now You know, so we got people that are good in all these different languages So, you know, whatever language you want to use depends who's writing it, I guess But this one uses drush user create and drush add roll I guess you can't see the the add roll in there user add roll but basically You can use drush to create the user so we're automating that so when a new site goes out we've got the list of users from our web manage program and Whether their role should be There's a couple of different roles so we automate that so a new site goes out with the users already installed So after well before the add user after during the install It invokes our install profile and we spent a lot of time working on a install profiles I'm sure most of you have We just use you know single install profile for all of our of our Entire distribution is flexible enough that it can be used for different purposes administrative sites academic sites Student groups can all use the same install profile We were tired and making it more modular when I first started working on it. It was just a long monolithic string of you know database inserts and set commands and We modularized that quite a bit. We're also working on Now making sites go out with some default content Used to be that sites went out there were bare bones if you didn't take Sharers class and learn how to create content types and views you didn't get much our new sites now are going out with You know just kind of a default content and they can just go in there and edit that content change it to their own so they get a lot of the features Without having to know how to create them and we're doing that in the install profile. So One of the things that I learned at a Drupal con training last year trying to see if I have I Don't see the link to it, but it's in the resources Which was on how to use figure features for your configuration. So in other words Right now you might have an install profile where everything all your permissions Each module configuration is just in a long list of stuff to do But you can break that stuff out actually create modules for it using features And you can export your configuration save that as code With a feature you can change you can work on that feature turn it into a module to do even more stuff And this is an excellent way to simplify your install profile and start delivering your functionality through modules rather than just a Straight install profile. So as an example here A user role so most sites are gonna have some sort of a author or an editor role people to go in and Create content in Drupal and they need certain things in order to do that You'll have to have those text filters defined. Is it a a filtered HTML full HTML filter Maybe you have some custom stuff you put in those filters You're probably setting up one of the wissy-wig editors CK editor tiny mce so that they have a wordpresser like environment word processor like environment to work in and those have those profiles associated with them and finally the permissions so We created a feature out of that This is just a snapshot of part of the dot info file where you can list your dependencies say, you know I need those modules and I'm going to define these filters and And this role within this module and so at install time you just list that module as one of the modules to be installed And and you get that configuration So we've done this for our various roles for various Parts of the system that we wanted to be able to pull that configuration out and Snapshot it as a module rather than just something you have to go in and configure after This is just another snapshot of the Install tasks within your profile and your install profile. You can break it down What we found is that or what I found because I stumbled on this numerous times if you're configuring your permissions in your Installed profile do it after you've got everything installed if you try and configure permissions and the correct module is not instantiated yet it throws all kinds of error So basically we do the configure which gets everything installed and configured we set the permissions and then we build some default content so You install Drupal you got updated happens a lot We get security patches come out some updates to modules that you got to have and the process for doing that You know if you did it on an individual site you'd have to run update PHP take the site offline do the whole thing Dresch can do all that for you and so We can do it in batch mode We can't just automate say do them all because we've got to go through this web interface select the ones We want to update submit them to be updated. So it's basically I can select a couple of screen fools at a time Send them through and then I can monitor the the process progress and And then do that until they're all done. So the update script will back up the database put the site and maintenance mode Create so sim links and remember we're using sim links So all we got to do is break the old sim links sim link into the new files and that you know that updates our files Drupal core or contributed modules run the database update put the site back online clear the cache away we go and We do it with Dresch so I've Circled all the relevant commands here that we can use we use SQL dump to do a database backup You can set the site offline run the update DB Set it back online clear the cache and away we go all with Dresch commands Cron is something that you need to run on your sites and we run Cron with Cron Makes sense That's Unix Cron. So most of our websites run on Unix type systems Linux whatever which has this built-in Cron facility. That's the reason they call it Drupal Cron they call it Cron and Drupal because of that but There's a link up here for the best documentation I found and I'm pretty sure most Weitzman wrote it on running Cron Running Drupal Cron from within the Unix Cron The week we basically do it twice a day. There's some sites that need more than that If you're aggregating a lot of content using the aggregator you might want to run it more frequently That's a limitation of our system right now is we can pretty much only do it twice a day We'd like to get to where we can be a little more flexible with that in the future Once again, it's it's being kicked off from a Python script that loops through Finds each site that needs to be run and runs a drush to execute it So if you got a lot of sites you've got to worry about how you've your your front end is scaled out to be able to handle it and so This took a while to get this all in place, but this is what we have now We have a citrus net scaler sitting out in front of everything and that's a load balancer and It can basically hand off the requests to one of many web servers that we have out there eight actually So that not any one of them is overloaded each one of those web servers is running the APC Alternative PHP cache so that all the PHP op codes are pre-compiled ready to go on the web servers themselves We have one web server that's dedicated just for search engine requests So if you're a search engine you go over there leave us alone over here And that way search engines don't impact regular user browsing And we're using memcache. We have dedicated servers Just running the memcache to handle most of the Drupal caching. Here's a picture of it You can see off to the right there. We got the one server that just is doing the search engine seven others that are available for regular web browsing and The utilization on those seven is quite low. That's the way we like it We're prepared to you know for for spikes in traffic If they come along Most of these servers are running at pretty low utilization so we can we can handle that and Underneath are the shared resources. We've got my SQL that is not just for Drupal We got a lot of other applications hitting those my SQL servers Running in a master-slave Relationship because you've only got one my SQL server that can handle getting rights The other is just available if need be We also our shared file system is running on NFS A lot of limitations to that nevertheless if you are running a multiple web heads You have to have a shared file system So it's most commonly done with NFS. Although we're all looking for better ways of doing it. So managing our distribution for all Drupal 6 is life span and Drupal 5 before that We had it all set up using subversion SVN and we could pull from that Anytime we had a new distribution to go out We kept the entire distribution in SVM Drupal core Contributed everything When we went to Drupal 7 I looked that said, you know, that's that's pretty much a waste to keep all that stuff in our in our Git repository we're using git for Drupal 7 and say I can use I can have a drush make file that'll go out and get Drupal core and the contributed modules And libraries from their native Repositories I don't have to have copies of them in my repository. So basically then our git repository just can contain our custom modules and themes and You know the stuff that we've modified patch files if we have any patches that to go against any of the Contributed modules that kind of thing. We're using Git lab Provides a nice web front-end kind of like github a little different But that's what we use internally so drush make is a Wonderful tool that whoops that was what I was afraid of That lets you Basically list everything that's going into your distribution Drupal core each module what version of each module The library is where you want to get them from where you want them to go So one of the things that that we do in our drush make file is we version everything So I'll have a drush make file say core 7.22 and then for each module I'll list the version the specific version So I'm not necessarily getting the latest version. I'm getting the specific version I asked for and One of the things about that is that's in a snapshot if you want documentation What's in my distribution? Look at our make file it lists exactly what versions of what contributed modules themes and libraries are included and Then we can generate a new make file for each release. I put those files in with the release so, you know anybody can grab them and Link into our distribution So We're doing those things. We're doing them. Well Here's some of the things that that you know that we'd like to do better some of our opportunities One of the problems for us is our shared infrastructure. I'm going to go into a little more detail Let's just itemize disease. I'm going to jump into some of these a little more detail the system we've created Leaves us with a lot of administrative overhead. We'd like to automate a lot more stuff than we've got automated now As I mentioned earlier sites that went out with no content Was a big learning curve for our users. We want to we want to be able to Release you know create Drupal sites for people that they can just hit the ground running go right in there's content already there They just have to massage it that you know that it's tailored make it more tailored for the types of sites that they're building We still have work to do in that in that direction the whole Site life cycle process you create a site in development you get it ready to go you push it out to staging It looks all good. You've proven it. You put it in production We found that that's a fairly consistent life cycle even with Drupal as a content management system when people are in production Yes, they're adding content and they're working on their content But at some point they say you know what we want to switch to the new theme You know you guys released an updated theme. We're still on the old one and so we'll want to take that site put it back in development for them and They'll install a new theme make the changes and then they'll you know once again stage it push it to production that's a manual process for us right now and So we want to automate that process. Well, I've got some ideas. We'll show you how how to do that Updates although they're mostly automatic automated. They do take a long time and they're not hands-off and Currently we have no cleanup. We have no automated site removal. We have one guy that works for us I give them lists, you know, I run some queries and I find sites nobody's locked into for a year or so and You know, he spends like half his day out there just manually deleting old unused sites So we've got to automate that we've got to make it so we can clean up Sites people aren't using Here's a picture of the shared infrastructure what what that looks like So basically the problem with that is that the web server we've got, you know, one set of web servers They're running the home page. They're running Drupal sites WordPress sites. We have old static HTML sites some old PHP sites and PHP applications Other CMS's that are out there All running on the same web servers. Well, what that means is those web servers have to be configured To the lowest common denominator I can't put the latest versions of PHP on those servers or Any of the of the other things that that are used Our web servers are running PHP 5.2 something because of some legacy applications we have out there You know, so basically that means I can't optimize the servers for any of these things and The other thing about a shared environment as y'all know if you have one app somewhere out there that starts misbehaving It can you know, it could take down the whole thing so where we want to go with that is to start Creating more dedicated infrastructure Let's let's set up web heads that are just for Drupal and we can optimize those for Drupal And we're not going to run anything else on there, but Drupal you have static pages. You want to serve up great They go on another server we can play around with Rewrite rules and load balancer configuration in order to get the URLs to work out the way you want But we're not going to run mixed content on these servers So we'll dedicate Some infrastructure just for the home page. Everybody's got to see the home page. That's got to be up there all the time We got to make sure that's really super high availability. We'll dedicate infrastructure to that same for our other services We have a lot of custom applications. They each need to be optimized in a certain way. So We're working in that direction now. We're already doing it for the applications We're getting ready to do it for WordPress and we will be doing it for Drupal 7 very shortly So the things that we have in progress right now We're consolidating Drupal 6 sites into Drupal 7 sites So we we showed you earlier that you know, we had these big numbers We have a lot of Drupal 6 sites too many our strategy is that we're going to start consolidating sites As appropriate into Drupal 7 I've got another slide on that in just a minute The dedicated infrastructure. We just looked at we're going to be phasing out the old web manage program and putting more and more emphasis on agar and Mostly we're leveraging the agar back in which is the provision module and I'm going to show you some examples of that here in just a minute So the site consolidation You know, if you have too many Drupal sites, this is what we found at least unnecessary Administrative overhead you can't share data easily between the sites. Maybe there you have two departments within a college They really do want to share data. Maybe a directory some other things are class listening, but they can't do it They're different Drupal sites. Sure. They there's ways of doing it with feeds and whatnot, but They're not doing that now if they were one site wouldn't be a problem and then the disjointed navigation if you're going to college and you're drilling down to departments and they're all on their own websites It's very hard to have a Fluid navigation back and forth. So we're using organic groups To handle basically taking a Drupal 7 site and segmenting it off so that we maintaining permissions for users to edit their sections of a site And to be able to have some individuality for the sites that we're Consolidating here's what it looks like in Drupal 6 we've got Individual Drupal sites at the college level the school level Department level you can get to a lot of them Drupal 7 one site So with organic groups We're like we're still having to maintain and one of the main reasons people wanted their own sites is because they don't trust anybody Is going to come in and edit their content and trash their site, right? So, you know, we've got to maintain those permissions so that if I'm working in one area and somebody else is working in another I know that I can partition that off so that we're not There's no possibility of us Trouncing each other's files so we can do that with organic groups We can also give departments some distinctiveness in how the pages look You can do that with organic groups and it makes it much easier to share resources between the whole The whole site So we've been working very very steadily on this for quite some months share has been Mostly involved with getting all this organic group configuration together ask her if you've got any questions on that later And it's not I just want to point out it's not just You know this architecture fits administrative sites that well We have a lot of administrative sites that really are part of bigger parent units that can be consolidated and we're working on that So this is what the back-end looks like I built all of this in development This is the model for the infrastructure that will be Rolling out this summer Hopefully into production Basically, we still have a load balancer at the front end, but that load balancer Currently is doing a lot of the front-end caching you know Static content is going to be doing less and less of that and we're going to need to have that varnish Second layer in there which will be handling most of the front-end caching and routing if you've used varnish you know it does an excellent job for that and then Leveraging the agar what they call a server pack They used to call it a cluster. They came out with a new way of doing it. They couldn't call it cluster anymore It's really a cluster. They caught a server pack And in my example, I'm just showing two servers each, but that's completely scalable. However many servers you need So we'll isolate those off so that different domains Different segments of our of our websites are within their own server packs and There's still the shared resources of the MySQL database the file system and the memcast servers But on the top there The way we manage this now is you know agar provides a web front-end you can go in there You can create sites you can migrate your sites you can manage them pretty well But really we want to get away from that We want to be able to automate more of it and the way you do that is using grush with the provision module and If you want to automate that a step further you can use fabric to do that And I will I think I have a slide here talk about that Fabric is basically a way that you can leverage SSH to run remote commands on another server the scripts are written in Python But basically all you're doing is doing stuff on the remote server using Using stuff that's on that server in this case drush and provision provision is the agar back in I Should point out that basically agar is two parts one part. They call host master is the front end That's the web. That's the web the Drupal website That allows you to do things through through the web interface, but the back end is called provision and Provision is completely exposed to drush. You can do pretty much anything that agar can do using Drush and provision together and mig five was one of the developers for a for the agar project Has written up some incredible work on on how to do all this stuff. I've got a link here It's also in the back. He is zero touch deployment. You can You can Google that and take a look at this stuff. It's it's done very well Some of his code looks like this is my code, but it's very similar. This is basically a little snapshot of a fabric script But the the important parts in there or what's in the run statements, which are just drush commands provision save provision install Basically That is How you do it Yeah, so I've got the help on the bottom there if you filter on provision drush Dash-filter equal provision you can see all of the commands that are available to you there's not Documentation isn't the greatest but once you dive into it this stuff starts making sense and You can use this to automate your Infrastructure like this is this is the cutting edge for it We've also developed Some puppet modules for getting our servers installed that picture I showed you a few minutes ago there are quite a few servers on there and so We've Developed puppet modules that will do all these things listed up here So the main thing with the agar is that there's a lot of different roles You may have a web server database server the front-end server so the puppet Modules to define classes for each one of those different roles So you just tell it which one you want that code is available in the resources here We're looking at continuous integration if you've worked with that Normally you're talking about the development process, but in this picture I'm showing you how you could be doing this with the Drupal itself because drush is It can tell you when you're when your distribution is out of date if there are updates to core updates to modules in this case You can use that in a continue it can continuous integration Scheme to create a new release to create a new drush make file bring that down run your tests on it and Then eventually get it done into into production So looking ahead We're looking at how to do more self-service people need Drupal sites give them a console they can come get the Drupal sites How do we create a lot of training in development sites? We need to be able to auto expire and delete those We built a control panel within a Drupal site to let you move your site from development to production to staging Don't have to call us to do that It's in development not out there yet, but it looks very promising I'm going to rush through this stuff. I want to just show you an example of using drush and the provision module To be able to expire and remove sites. It's all there. You can back them up You can delete them. You can restore them if you have to delete the backups when you finally done with them This is just within Drupal what that control panel looks like it knows what your site the current environment So this site says says copy site to staging so it knows I'm a development site I can only copy it to staging if I was a staging site It could only copy it to production built into the Drupal people can do it themselves right from their own Drupal administrative pages it naturally backs everything up But there's options in here for them to manage their backups and restores Themselves time is running short here. So I'm going to skip There's not much here. Basically what that the back end of that control panel does is allows, you know using drush provision clone to clone the sites to To the new environment. It's pretty straightforward once you start working with drush provision so that was what I have to say about The things we've done at central web services at Oregon State University The thing that's really made us successful at the university is the people that we've got working on Drupal within central web and within the entire Organization so if you guys are a part anybody as part of OSU Drupal team wave your hand stand up take a bow They're mostly up front here but I see a few all over the place and And then the Drupal community all of you have contributed into our success So I want to thank you very much and I've got the resources here You can download the the slides from the link in the schedule and you can you can link to all this code here And we're ready for questions. We have a microphone Follow Greg up there. I can line up with microphone Even with the microphone. I'm probably not going to hear you, but I've got the chair up here to help with With that problem My name is John Pugh. I'm with think-trop and I guess it's a comment but we've been using agar for years and We built a new tool that lets you make a make agar platforms and sites from git repos Called dev shop. So it kind of allows you to fire up environments really quickly based on that git URL And has like kind of similar to that because tools to copy between sites things to click to pull code to different sites and integrates with Post receive callbacks with github so it keeps the dev sites up to date and Things like that and I just wanted to point it out and we're having a boff on Wednesday And it's called dev shop dev shop. Yes. Yes. Thank you for that dev shop is is a agar tool I've seen it and That's that is Some of this kind of stuff. Thank you. Thank you. Hi. Thanks for your presentation. Do you guys provide any? Editorial workflow tools for your For your site users like something like revision moderation in Drupal 6 or workbench in Drupal 7 We're just getting started since with the organic groups Set up what we've got now is when you log into the site if you're a member of the site It gets it takes you automatically to the groups dashboard And you'll see a list of the content you're working on what we haven't yet Worked in as some of those workbench features that you're talking about. We're looking into doing that. Thank you Thank you. I Want to ask if you had looked at all at agar version to it all I think they have an alpha release right now So agar right now is at version 1.9 2 oh dev is out and 3 oh dev is out 2 oh dev is still written to run on Drupal 6 3 oh dev is written to run on Drupal 7 And they're both pretty much at the same level that I think they're trying to keep them pretty well parallel I've worked with both of them They're still not complete there's work to be done and You know, I I'm hoping to meet up with some of the agar team here at Drupal con Let's see what we can do to help because we really want to get that that release out the door We need both. Thanks Hi, how how do you determine if a hit comes in from a search engine or from a real user Is that based on a match against the user agent string or anything trickier than that? Can we repeat that question, please? How do you determine if a hit comes in from a search engine or from a real user to route to Yeah, the the load balancer can look at the header and you know determine where it's coming from So, you know, it recognizes search engines search engines introduce themselves pretty well Just by user agent string I Think so I'm not a hundred percent sure but I know that yeah, excuse me That built-in magic that you'd load balancer just does It can be configured to do so. Yes Hi Tyler strike from University Warloo when you create websites Do you create them as sub-domains or are they part of your main? Domain either way and and that's one of the Flexibilities of the system we're using now we can easily create them as sub-directory sites, but if we Need to create them as domains we can spin off a virtual Apache v-host to handle that domain and And do that as well. So we have a mix of them and right now it doesn't matter We can do it either way. We're trying to move more towards the domain though Hi Alex show from the National Center for atmospheric research. I notice that you did not touch on upgrading D6 to D7. Do you provide upgrade support for your sites and I did I did go over upgrade? I'm sorry if I went through it a little quickly, but yes it the upgrades happen That there was if you look through the slides again There's a python script that's got a lot of circles around the drush commands that we use to upgrade sites Now that's I'm just talking about update like you know running update PHP. Are you talking about upgrade like major? Yeah, 67. Yeah Yeah, I didn't touch it because it's I didn't talk about it because it's it's it a tricky and in-depth Conversation, but you want to look at you want to look at the migrate module? Yeah the migrate d2d Which is an extension of the migrate module which is will allow you to write migrations to move your Drupal 6 sites into Drupal 7 So we're using that as part of the site consolidation. Yeah, I'm just wondering because we have a we have a similar setup where we have a central Service that supports multiple sites and do you just leave it to the site owner to do the upgrade or that's something you guys do for them We're doing it. Okay Thank you Hi, Julianne Chapman University of Illinois. I was curious about what your group makeup is in terms of size and skill sets Our group stand up we get most of them are right here the Drupal team is right here We have a few other people working on custom applications. So how many people do we have? Central web You know there's About eight of us I guess but just just really four or five of us working on Drupal within central web We have other developers Manuel stood up here He's in one of the colleges and we have people out in the colleges that are doing Drupal So, you know all in all at Oregon State University. I'm sure there's 20 or more people working working on Drupal Okay, so is your group mostly just developers or do you include ops and design all the way through the chain? Yeah, so yeah, we are developers. We maintain the back-end infrastructure. We maintain the distribution You know We help people build their sites. We do the training. We do user support We do everything, but you know, and well, we hold their hands too. Yeah. Thank you We pretty much do all that Hi, my name is to new my team actually automated the continuous integration process for government And one of the key requirements we had was integrating application security testing And I was curious if you had to do the same thing if you did anything in the security space security testing Yeah We we definitely rely on our our network group, you know, we're part of the bigger information services organization we have a You know other people that handle the firewalls and the Networking layer so we've been pretty much Isolated from that that doesn't mean that when we deploy applications we we You know, we still have to address the security issues within our own applications and Drupal is one of those We have to make sure that you know we're handling the site permissions accordingly and if we have any sensitive data in there to make sure that's stored in a Secured way when we're writing Forms and whatnot we have to protect against SQL injection So most of our responsibility is happening at the development end of it not the network in Hi, so it looks like you're using sites and sub sites currently for your departments and schools when you're moving to organic groups Are you doing any sort of hierarchy? Like are you having sub sites or sub departments inside? Which modules are you using to accomplish that? Well, what we discovered about organic groups is by default. It's a flat Structure they don't allow you to create hierarchal group structure So we developed around that and just created a Kind of a second layer We call it parent unit and we just created a content type called it parent unit and used a entity reference So that we could associate a bunch of groups with a parent unit And that was a pretty simple thing to do that that met our needs at least so far If it if it had to scale down to more higher more of a hierarchical structure We would have to be looking towards something else. I believe that didn't you say somebody's working on a module Yeah Yeah, so that's an issue that needs to be addressed within organic groups. Yes Right. Thank you. Do you have a solution for that? No, I'm looking for one. Okay. Thank you Hi Matt Garrett University of Kansas I noticed in your install user script that you guys created you have an insert into CAS user Are you guys using CAS first? Yes, we're using CAS. I'm sorry. I skipped over that The so what that script actually just did was insert The CAS username when people log in well well the install profile when it's creating the Users the install script when it's creating the users uses our our network identity We call it own it Oregon network ID and and so and that's the CAS username So that script is stuck that in the CAS user table So that then after that the CAS login would work for them I guess is an excellent way to do single sign-on for for large struples installation. Absolutely Are you using a specific module for that or is that something custom-created? We're using the CAS module, okay, right out of the box Okay, cool. Thanks, and it which just leverages the CAS PHP library, which you know is supposedly One of the best things around for in the CAS world, sir Hi Daniel from the you go I have a question about the theme So you have a couple of themes Do you have a lot of flexible features given to the user or do you allow even sub-teaming from other groups? That's in your team only So We work with our web communications people that have the designers and initially we started out in Our older Dupal six themes We sub-themed Zen we started with the Zen theme we sub-themed it and we were creating Individual themes for each college. We were kind of branding them So they looked a lot the same and we had some common features Moving to Drupal 7 we created a single theme Basing it on Twitter bootstrap We created the theme from scratch and we built in a process of what we call theme variants Which is simpler than sub-theming because basically all you do is add in a Style sheet and link that in and that gives you some You know some variants so we still have to create all those We haven't turned it over to the departments and the sites and say you create your own themes You know we're we're not doing that. We're we're maintaining the branding From the top down mostly coming out of web communications. Thank you Stephanie bridges, Iowa State University. Um, I was going to ask about the CAS users But I guess that already got answered my other question was how much Customization do you allow individual clients? Since you've got such a large scale and do you allow them to do their own customizations? Well, it depends, you know Like I just answered with the theming we keep that to quite the minimum if they go through web Communications and those people want to give them a little branding something then we build that in other than that They they can't do that on their own as far as developing features for their site We're working with people in the departments that do that our graduate college Went higher to Duble developers so they can create their own modules and we're working with them to You know get those modules in our repository trying to make sure that they're developed In a fashion that's going to be compatible with our distribution And so we do we do want to encourage that mostly, you know We found that people would like to do a lot of different things But they don't really have the resources to do it and they depend on us and some cases people You know ask us to write custom applications within Drupal and we do that as well in house Did that answer your question? Yeah, yeah We don't let our users do anything. Yeah Well, we didn't either but you know what that that doesn't create a good feeling You know you got to work with them and give them some feeling that you know They're building their websites that you know, it's it's somehow personal to them if you Some of our users have trouble mastering the exact we get it or so So, you know those people fine you just train them and hope they do their best I'm David Bass from Western Washington University. I was curious about regarding the lowest common denominator problem if something like