 All right, hi everyone. So this talk is about security in Drupal. Hopefully that's why you're here for it. And so out of interest, who is here for their first Drupalcon and interested in security? Wow, quite a big number. And who was here at the previous Drupalcon Barcelona eight years ago? All right, nice. OK, so today we're talking about Drupal and security pretty generally. So first of all, interest ourselves. I'm Stefan Carlosquet. I've been a longtime contributor to the Drupal project. I'm a member of the security team, as I've had an interest in security for many years. I've also contributed to the RDF module in Core and in Contrib. I wrote a couple of chapters for the definitive guide to Drupal 7. And I work for Acquia as a software engineer. So Acquia, for those of you who don't know, we run some of the largest websites on the planet. And we also happen to be hiring. Hello, everyone. My name is Klaus Purer. And I'm a software engineer from Vienna. I work for Epico. It's a small company doing job boards. My task is mostly doing backend development systems administration. On Drupal.org, I'm doing project application reviews. So I help new contributors publish their first modules. I'm also part of the Drupal security team. I developed the REST module, which is now in Drupal 8 Core. Currently, I'm working on the ROOS project, which is porting the ROOS module to Drupal 8. OK, so today we plan to cover a lot of things. Because what we want to talk about is not just Drupal. Security is more about the environment where Drupal runs as well, as well as just the Drupal site. So we cover server environments and server configuration. And also all things around Drupal, such as personal practices, how to view security, and what you should do to avoid any problems on your site. So this session course or addresses coders, but also site builders, just as much. Because we all have a role to play in security. I wanted to start this talk by a quick story. This is a picture of a bunch of hackers. Their group was called Loft. And this picture was taken in 1998 in front of a panel of senators in the US. And they were explaining that the internet, at the time, they were claiming that they could take the internet down in 30 minutes and that the internet was very insecure and not the place to be. Obviously, we're all here because we used internet. So we have to make it secure somehow. But what's interesting about this story is that even today, 17 years later, their statement is also true, not in the same ways. But the web has evolved since then, since 1998. A lot of security issues have been fixed. But at the same time, new vectors are being introduced pretty often like browsers. We have different technologies today, and they're also vulnerable if you're not careful. So security is a process. It's not just a feature. It's something you have to be aware of all the time. And all of us have a role to play here. All right, first of all, just some general tips about development and how your team works with Drupal development. You should make sure you use secure versions of the protocols, so avoid HTTPS, use HTTPS, especially if you have confidential data going over the wire. Don't use StelNet. I'm sure everyone knows that. Everyone uses SSH nowadays. And SFTP for file transfer. Don't use FTP. And have strong password policies in place. In the agile case, have also TFA to back this up. Make sure your servers, if you manage your servers yourself, make sure they are tight and secure. Nobody can penetrate them via any vector. So that means keeping up to date with security patches on the Unix stack. And also there's a trick in SSHD if you manage your servers at this low level. There's a way you can require SSH keys for logging into your servers with SSH. So you can kick out anyone trying to just use a password to get in. So that's pretty effective. And finally, when you take, make sure you take backups and snapshots of your sites just in case something bad happens. And we have an example coming up in one of the slides. But make sure you have backups in place, daily backups, or you rotate them to have weekly backups. And also verify that the backups are actually working so that the day you need the backups you don't end up with like an empty SQL file for your database. So just be aware of those. And finally, if you have to share your backups for development purposes with your team, there are ways to sanitize them. You don't want to share exactly what's on production because it includes credentials. And even if it's hash passwords, it's still better to avoid that. Even email addresses could leak somehow. So they have scripts to clean this up. Make sure your site going on the Drupal layer now and make sure your site is configured properly. So this is a big vector for being hacked is misconfiguration, roles and permissions is one example. And there are many more like text formats. If you misconfigure your text formats or if you allow untrusted users to access some of the risky text formats, they could be a vector there. PHP module is a module in triple seven core. Avoid using that PHP module. Make sure it's disabled. Even remove it from your code base if you want. It should be avoided at all costs. And any module in general that executes PHP from Contrib, any module that allows you to enter PHP code in the interface should be avoided. And back to the server, make sure your file permissions are correctly set. So make sure the web server cannot have right access to your Drupal code base. And restrict PHP execution from happening all over your, at least in the uploaded file directory. And if you can across their tricks, there's a trick in D8 that Klaus will explain in a moment to tighten this. There is a Drupal handbook for writing secure, or for securing your site, sorry. So there's a link in there you can follow. And the slides will be up after this session. If you don't want to, if you don't have the capacities, you don't have the staff to look after your servers at a low level, you can also hire companies to manage the Drupal hosting for you. So there are companies like Pantheon, Aquia Cloud and many others at Drupal.org slash host. They can take care of all that for you. Make sure your stack is as secure as it can be. And not only will it be secure, it will also be tuned for Drupal performance. So it's gonna be the right amount of caching, upcode caching and memcache and all that, varnish. So all those tools also provide other services on top like code management workflows. And there's also other services. If you don't want to have to worry about updating your Drupal core after each security releases, you can subscribe to services like remote administration or there are plenty of companies offering those kind of things. So this is a good service to have. You can also add modules, contributing modules to harden your site and secure login is one of them. It will enforce your users to use HTTPS when they log into your site. So their credentials are sent over the encrypted wire. Paranoia is another good module to consider. So it will make sure that nobody can execute PHP on your site, for example. It will also prevent any user from editing user one, which is kind of a critical user that should not be hacked. It has other features like that. For example, you cannot disable the Paranoia module because obviously that would defeat the purpose of it. Security review, so what we talked about about misconfiguration of your site, this is something that most of those traps can be called by security review. So this is a module that will audit your site for misconfigurations. Permissions lock, it's another module. It's like features for permissions. So you can put your permissions into code and you can version them and get. And it will also prevent anyone from editing permissions on your production site. So that way if you get hacked, it's harder to elevate one's permissions. Hacked is another handy module. It will scan your code base, each of your concrete module, and compare it with the code that's entrepreneurial to make sure they were not patched or hacked somehow. We talked about passwords earlier. So they are modules to enforce password strength. And so password strength, for example, will if configured properly, it can force everyone to change their password if their password is not strong enough. And combined with TFA, that's pretty much what you need to have a rock solid authentication wall on your site. So those are all modules good to have if you care about security. So we wanted to throw in some acronyms because the world of security is full of acronyms. So you should be aware, depending on the data and vertical you're working, you should be aware of regulations around them. So anyone working with PCI in a room, raise your hand if you work with PCI, okay. So that's e-commerce. There's a triple PCI compliance report you can download from here. Anyone working with HIPAA? Not as much. That's for healthcare. And there are other ones like FedRAM, FISMA for governments, US government. Anyone knows what SCADA mean? So SCADA is any site that monitors or supervises physical equipment, such as could be a heating in the building or air conditioning in the building, could be airport control. It could be a processing line in the factory, anything, any site that has a physical impact in the world. So possibly could have devastating impact if hacked and then used to trigger some disaster. So, security process. Like I said at the beginning with a story, it's a non-going maintenance. It's not something you can set and forget about. You always have to have staff aware of it and that has time and resources to dedicate to it. That means that you also have to account for budget. So if you're a person preparing the budget for projects, you should account for that. So people have heads up and they have dedicated time. If something bad happens, if a bad security release happened, has to be done on your site, they have time to do it. And there's also the option of having managed hosting so that can leverage or mitigate some of those problems. If you have managed hosting, those companies will take care of all the stack, at least to the Drupal layer. So all the OS layer will be taken care of. And if you have even the Drupal remote administration option, it can also help with the Drupal updates. And Drupal.org offers packaging infrastructure for free. So this is what we use as a community to make sure everyone and all the sites are up to date at all times as much as possible. So when there's a new security patch, a new release gets created and everyone gets started. So they have mechanism to alert sites and we'll go through that in a bit. So more about the security process. Like we said, Klaus and I are part of the security team. It's about 30 to 40 members currently. And our mission is to keep the software Drupal core and contrived as secure as possible. And we also try to educate the community by writing and keeping documentation up to date, doing sessions like this one. And again, it doesn't apply to just cores, it applies to site builders, to decision makers, everyone is involved. And we issue security advisories. So every time there's a new update for a module, whether it's core or contrib, there's a new security advisory being issued. Explaining what version you should update to and what happened, a little bit of information about the vulnerability is also included. So for those interested in the workflow, so I'm just gonna explain how we deal with security issues coming in. So we have a private issue tracker. So when someone finds a bug, when someone finds a bug, it's hard to read there. All right, so you'll have to listen to me. When someone finds a bug, they report it to us privately. We don't want security bugs to be open in the wild and exploit it. So they report it to us and we'll review the threat. We'll decide whether it's a real threat or if it's not a threat, it can push back to the public issue queue and fix there. But if it's a real threat, then we'll get the maintainer involved. So we have the tools for that. We use the same issue tracker as Drupal Log except it's private and we can invite people to issues. So there's a feedback loop that will happen here. So we'll guide the maintainer into fixing their module. We'll review the patch that they write. And once it's ready, it gets released in the wild here. So there's a new security release created on Drupal.org and everyone can download the new version. So I'm taking over a little bit, talking about the specific vulnerabilities that we see in Drupal Core and Contrib. So this is a graph of vulnerabilities in Core that we had in 2014. As you can see, the most common one is XSS, which stands for cross-site scripting. This is people using JavaScript to exploit something. But we also have a variety of other vulnerabilities in Drupal, which is access bypass or cross-site request forteries, things related to authentication, session, SQL injection. So there are many kinds of security issues, but still cross-site scripting remains. And cross-site scripting means that we are a content management system and we're dealing a lot with user-provided input. So everything that we output to HTML has to be safe in a sense that it must not contain script tags, for example, right? Because only you, as the site developer, want to specify which script should run. And if you look at Contrib, it's even more obvious that cross-site scripting is a big problem in at least in Drupal 7. There are percentages even more than half of the issues that we find that we fix with the maintainers are cross-site scripting. Of course, we also have other stuff there like SQL injection and access bypass is also a thing. Just to give you an idea of what we actually find and what we fix. So what is cross-site scripting? How does it work? So in the 90s, somebody decided that it would be a good idea to invent something called JavaScript and put it into browsers and browsers would just download it and execute it. And of course that has some implications, right? So you have to make sure that the JavaScript that is executed in a browser is actually that the developer wants it to be there. If we're dealing with a lot of user-provided stuff like in Drupal with commands, nodes, site configuration, whatever, it must be secured. So yeah, the browser downloads your stuff from your Drupal site and then executes anything it finds with the script tag. And if anyone can inject script tags somehow in a node body or whatever, then of course the JavaScript gets executed. And although JavaScript is sandbox and there's the same origin policy so it cannot do anything on your computer, it cannot break out of your browser and do anything on your computer, it can still do harm to the site we are logged in. So for example, if you are logged in as admin and you load that page where an attacker was able to place that JavaScript snippet, it will get executed as your user. So what they can do, they can edit other users, change their passwords, make themself an admin, whatever, you can make requests and read the responses with JavaScript so there's a lot that you can do. Same applies to other browser plugins that we have. There are some interesting vulnerabilities regarding Flash and Java. Yeah, it's basically the same principle. And how can you test for cross-site scripting? Well, there's this typical penetration tests that people do, so they just make an alert box in this script tags and put it everywhere in the forms. So for example, if you put this as a node title into your Drupal site, then it will get escaped, Drupal core does that for you, but if you as a developer output the title somewhere, then you might not escape it. And this is a good test because then if you load the page, it just pops up in your browser and you know, oops, something is executed here that I actually don't want. So not only script tags are a problem, but also like image tags where you can have special attributes in the tags that then trigger an alert. So that's also a cross-site scripting thing. So of course, this penetration test, this looks really boring, right? What's the exploit here? This is not something an attacker would use. An attacker would load an external script from their evil website and then execute that. And that's a bigger threat. This is just for testing, right? And with the test, you can actually catch, I don't know, up to 90% on your site when you just look at the pages that get rendered when you input such tags. So how do we fix cross-site scripting? Well, we have to filter text. Anytime we print something to HTML in the theme layer, we have to make sure it's escaped or filtered. So Drupal has the philosophy that any user input is saved as is to the database and the filtering is done on output. So anything that is in the database is considered dangerous. Keep that in mind. So whatever we take in, we don't filter it before saving. We just put it as is in the database, but when we get it out of the database and print it to HTML, we make sure that this is escaped and filtered. So we do that as late as reasonable, right? We just want to sanitize at the very end during the phase where data is passed around between the theme layer and the modules. It should stay as is, but at the very end, it should get escaped so that we don't have cross-site scripting vulnerabilities. So one example for making sure that you don't have cross-site scripting is using t, the t function to translate text and in there use the placeholders with the at sign or the percent sign. And yeah, they will do the escaping for you. So whenever you pass a variable to your translator with string, it will get automatically escaped for you. This is something you're seeing all over the place in Drupal Core, like the form API filters for XSS for you. It's not really consistent. So if you build a table, nothing gets escaped for you. So if you just build a core theme table and put in a note title there, you have a cross-site scripting problem. So not every, all places in Drupal Core get auto escaped for you. So we have to be careful as module developer and as FEMA. And this is the cross-site scripting cheat sheets. So we have different levels, how you can filter and escape stuff. So the first one, the most strict one is check URL. So it will escape any text that are in there, but it will also make sure that no evil protocols like JavaScript colon is used. That's what used for URLs. If you just want to output plain text, there should be no markup whatsoever. You can do escaping with checkplane. If you need to have some form of which text, like people using bold or, I don't know, an unordered list or something, then you would use check markup with the text format. So for example, somebody posts a comment and you would have the filtered HTML text format which allows bold, for example, and the rest is filtered out. So if they put JavaScript text in there, whenever it is outputted, the JavaScript text just gets stripped. They're not there. But you can also do filter XSS where I don't need a text format. You can specify exactly which tags should be allowed. Like I want the A tag and I want bold, but I don't want the script tag. Yeah. At the very end, there is the trusted possibility. So if you know this is not user provided input, this is something I pulled from my trusted developer code base, maybe it's just a text file or whatever, then you don't need to do any sanitizing and just print it to HTML output. Now talking a little bit about access bypass, so it's something that a user can do or a user can see which they shouldn't be able to. So the first thing when you can do something, it's called access bypass. When you can see something, it's called information disclosure. And yeah, we should make sure in Drupal that there are permissions in place or access callbacks or other API stuff that should prevent exactly that. Where and how do we enforce it? So we have in Drupal 7 hook menu with an access callback. Either you use the default one which just is user access, so you specify permission and checks against that. For example, the callback for a node just has the access content permission so anyone that has access content permission on the site can read the node. We use the user access function, as I said, for permission checking, but we also have more sophisticated APIs, right? Like the node access system, which was even updated to the entity access system with the entity module in Drupal 7. And we even have it on the field level, like if you go to the user page of the user, for example, the administrator is able to edit the password, but a normal user where it's not their own user account, they cannot see or edit the password. So that's field level on this small property level. But you also have to watch out for services course or HX APIs, so you have some kind of API, like the services module, you also need to make sure that you secure that because attackers are going to look for it and trying to exploit things there so you can just have an open API where you allow anyone to write nodes because attackers will find it and then it's no secret anymore and they can exploit it. Also in templates, we make sure that we don't have a lot of logic, there should be no business logic in the templates, you should only arrange stuff, you should do database calls and output stuff without checking for access, yep. This is just doing best practices basically in Drupal then you also avoid access bypass problems. So how do we test for access bypass? There are a couple of possibilities. So test your site regularly as anonymous user, for example, so that you make sure whatever you want to keep behind the wall for authenticated users or admins is not available to anonymous users, try to visit pages, but you can also use automated tools like penetration testers or some form of BHA testing where you regularly crawl your site to make sure nothing is exposed. How do we fix access bypass? I mentioned a couple of stuff already, so there's the user access function to check for permissions, so only if you have that permission you can access this thing. There's also node access which is a bit advanced so you can have nodes that are available to anyone, but for other nodes the access is restricted. You can also use entity access which works the same way as node access except for other entity types. And you can also do it on the query level. If you want to query for nodes then you have to add the node access tag to make sure that the node access restrictions on your site are respected so that the user on which behalf the query is performed only sees nodes that they have access to. As I said, we also have it in the menu definitions so sometimes people specify an access callback arrow true for development so they're just trying something out and forget so they don't specify any permissions at all. Make sure that you fix that before you push it to production because of course otherwise anyone can access that menu callback and do stuff there. Also yeah, writing automated test is a good idea so they make sure that stuff that is supposed to be protected is still protected. You can test for that, right? Cross-site request forgery. So this is an interesting one. Basically you have a path on your website that does not confirm the intent of the user. So if you look at this example, this is a typical snippet an attacker would use so if you see something like that and that's probably a sign that somebody tries to do some cross-site request forgery and what they are doing here is the source attribute of the image and it has a URL in there. Usually that should point to an image but in this case it just points to an arbitrary URL and I'm hinting here that there is some callback on your site which does a quick delete of nodes like deleting it immediately without confirmation and then of course an attacker can place that image somewhere. You are logged in as admin on your Drupal site, you go to the page, the browser tries to fetch the image but you're still authenticated on your site and suddenly your node one is gone. That's basically how cross-site request forgery is exploited. How can we test for that? So if you look at the code, you see that people use the get or the post variables, they're not using the form API, that's always a strong indication that there's some CSRF vulnerability and they don't use Drupal get token for example to verify that links actually have the intent of the user. So what you can do, sometimes for usability reasons it's nice to click a link and not have a confirmation form that something happens immediately but then you need to also pass along a token which you can get from Drupal get token so that the page callback that receives the link that is executed and knows this was intended by the user and the token is specific to that user and not that somebody's just sent you a link and suddenly a node is gone. And also looking for that you can look out for verbs in menu callbacks like as we had in this example, the delete. Delete always means an action, something to do and if that is not cross-site request forgery protected people can change stuff in your database. Nodes get deleted, something gets changed which might not be intentional. How do we fix cross-site request forgery? So the easiest one is just to use the form API. There's a good API to use confirmation forms so whenever you click on a link it says we really want to delete this node and the submit button is protected by the form API cross-site request forgery protection so then you're good. But sometimes as I said you want to use ability improvement that the link works immediately then you would validate tokens that are passed in the URL. There's a good webinar that Greg Nettison did for I think for Acre or for DrupalScout at that time how CSF works, what you can do about it so I recommend you watch that. So, Stefan. Right, so I talked earlier about a bad vernerby that happened to us last year so that was in October 2014. It was just a city bug in core that if exploited could let you write to database, execute SQL injection. So it was quickly exploited so I had a way to watch traffic and flag any request that would be exploiting this and anyone knows how many hours it took for hackers to try to inject SQL injections? Seven, yes, seven hours. So I've talked to people who have large sites and boring enterprise processes that have to go through approval before any release of code and sometimes I've heard cases where it takes days to have code deployed. So it doesn't work, I mean it has to be fixed if you're in the situation and if you're facing a bug like this you're in big trouble and in the case of this bug we, the security team did some analysis and we recommended people who didn't update their site within the first seven hours to assume that they had been hacked and there was really no way to know whether you had been hacked or not, it was hard, tricky to know. So that's why we recommended people to just assume their site had been hacked. So there's a link there if you're interested in the kind of exploits that hackers were using and it was kind of amazing to see how much imagination they have to use Drupal and deep Drupal APIs to plant their backdoor in your site. It was however mitigated on large hosting companies like Pantheon and Acquia, so if you were there you were immune from that. So the lesson learned for the community is to plan your security updates. So the Drupal security team started to have a schedule so every third week of the month that's the window for security updates for core. So it might not happen but it might happen. So what we recommend people to do is to plan on that Wednesday, the third Wednesday of the month, plan a couple of hours or a few hours of your, in your schedule and don't book anything during those hours in case you have to update your site. And there's a handbook in general how to recover from any breach. In this case in this particular breach we recommend people to, if they have a backup of their site within the next few hours before the announcement to use that backup. That was really the best way to recover from this. There was no way of recovering otherwise by running commands or anything. Drupal 7 had, when it was released it has some security improvements. So we have this slide here in case some of you are still in Drupal 5 or Drupal 6 and contemplating going to Drupal 7. Really you should maybe just go to Drupal 8 but here are some of the improvements that Drupal 7 saw. But what's worth noting is that Drupal 7 has a module called the update module in core and it can do a audit of your modules and tell you which ones are out of date and which ones need to be updated because of security releases. And you can also subscribe, you can enter your email in the form and it will send you an email if any of your modules is out of date. Drupal 8 is more interesting. It has a lot more security improvements. So twig is the number one. It automatically sanitizes your output for you. So you no longer have to worry about what Klaus was talking about, placeholders and text strings and all that. It's much more efficient, it's more attractive as well for designers. They don't have to know PHP when writing templates. So no PHP in templates means you're not tempted to go and call, do a call to the database or display some string straight from the database that was input by a user. So it makes it a lot more secure and it should basically kill our number one kind of vulnerabilities which is access. Another good one is the wizardwig. So if I say full HTML to the audience, does that ring a bell? Is that okay to have on the site? Like just grant full HTML to everyone and you're done. You don't have to worry about input, selecting your tags, everyone can use any tag and you're good to go. So that's actually a very bad practice because you can introduce a vector for anonymous users or untrusted users to have or to plant access scripts on your site. So what wizardwig offers now is a very coupled interface so you select what options you want in your wizardwig and then the text format will automatically add those HTML elements as allowed on your site. So you normally have to use the full HTML trick which was insecure. And finally just the PHP module is now gone from core. So there's no way of executing PHP in Drupal 8 which is great again and now it's back to Klaus. Yeah, just some other Drupal 8 improvements that we have. About the CRSRF stuff I was talking earlier, now in Drupal 8 is a little bit easier to implement that. And so if you consider this routing YAML file it's the new place where you specify your menu part in Drupal 8 and what we do at that path we set underscore CRSRF token to true and then when a link to that path is printed it would automatically append a token for you and when a user goes to that link the token will also automatically be validated. Of course you need to make sure that you use the URL generator in Drupal 8 to spit out the link but you're going to use that anyway like the URL function in Drupal 7 so this should work out of the box and really is a great improvement so you don't have to mess around with Drupal get token and Drupal validate token on your own. There's also another list. I love this list. There's really lots more hardening in Drupal 8. So one of the things we did after the SQL injection last year we limited PDO by a square driver to only execute one statement against the database. So what people were doing exploiting the SQL injection running multiple statements, right? So first it was the select statement then they did an SQL injection and suddenly an insert query was running or an update query and that's if you configure that PDO correctly then that's no longer possible. So even if there is a SQL injection vulnerability in Drupal 8 the scope will be limited to not run any second queries, any additional queries so attackers have a much harder time to exploit this. So what we also did we forbid PHP execution in subfolders in the HD access file. The HD access file is a configuration for the Apache web server and in Drupal 8 only index.php can be executed and I think a couple of scripts in core like core slash install PHP of course to install Drupal and a couple of others but since Drupal 8 ships with so many PHP libraries they contain basically arbitrary PHP script and you never want them to be directly accessible. So when you go to that URL in your browser slash core slash vendor something dot PHP with that configuration it would just return a four three access denied you're not allowed to directly access that file. I think that's a great improvement. We're also using this for Drupal 7 so you can even do this in Drupal 7 if you have many libraries and many PHP files lying around that shouldn't be directly accessed you can modify your HD access or engine X config and make sure that that is not allowed in any sub directory not only the files directly but in any sub directory. Most of the time Drupal just goes through index.php and that should be fine. We also improved click checking protection. So click checking is when somebody uses an iframe to embed your site on another site and they try to overlay the iframes and then when you click you're not really clicking on the things you want to click and if you send the X frame options response HTTP header to the browser then the browser knows oh no I cannot embed my Drupal site in another site and then that's forbidden and then the browser protects it from click checking. It might not be a good idea for all Drupal sites because they want to be embedded in iframes but yeah you have to decide that on a use case by use case basis and most Drupal sites won't need this so it's a secure by default option which is a good idea. Then we have hashed user sessions in the user session IDs in the database which means it's the same thing as with passwords when in the case that your database gets leaked it's out there in the internet and your site is still running and attacker cannot just steal the ID and use that to authenticate them because the sessions are not hashed and you cannot easily reverse that to a valid session ID. It's a great improvement. We also have the trusted host pattern now in Drupal 8 which means you can restrict on which domains your Drupal will answer. So what attackers did, sometimes there is a misconfigured Apache on a server and there are multiple Drupal sites and nobody knows which domains are actually accessible on this site. They send a spoofed request there asking for evil.com and suddenly the Drupal site will answer as evil.com and think it is responsible to surf the site evil.com although it shouldn't be, it should be some other domain example.com or whatever and in Drupal 8 you can specify regardless of the web server configuration only answer on example.com. There's also a very good blog post by Peter Wollin who is also from Drupal security team. 10 ways Drupal 8 will be more secure. So I suggest you check that out. It has more details about these things I've just been talking about. All right, so we are reaching the end of this session. So to wrap up we would give you a list of bugs and other links. This is a bug dedicated to Drupal security written by Greg Nelson from the security team. And here are a lot of links pointing to the security advisories that I talked about. More information about the security team. There's a report about Drupal security best practices that was written for government and non-profit but it really applies to any vertical. There's a chapter, chapter six of definitive value to Drupal 7 is talking about security. We have a group on GDO about security. Anyone that's subscribed to this security group already? All right, so more people need to subscribe. Just go in there and hit subscribe, join the group if you're logged in on Drupal.org. And then docs.aqua.com has a lot of information and documentation about security as well. And we just wanna invite all of you to come to the sprints on Friday. There's gonna be people available for mentoring new contributors and you can write or work on any issue whether it's security or anything else. So we welcome you there and that's the end. So we can take some questions now. Don't forget to give us some feedback on how you like this session but now we are open for questions. I have a demo as well if you wanna see it about the Drupal 8 security review. But first we can take some questions maybe. You should say that people should come here. Sorry? You should say that people should come here to the microphone. Yes, if you have questions please so that everyone can hear the questions. So my first question, I'll start with a very short story. Once I succumbed to dark side and I installed WordPress. I've only installed it, I haven't touched it yet. And from time to time I get these emails saying it auto-updated itself to the latest version. And looking at Dree's note last, well yesterday, I saw this nice list of things we wanted to fix on Drupal 7 and they managed to say we fixed all of them except one. And so when are we doing auto, core auto-updating? And why don't we? Because it's like a taboo in the community. I think it's not a taboo, I think you can actually do it right now. You just create the cron shop which runs Drush PM update for you, right? So the problem is that a lot of Drupal developers work in a continuous environment where they have the code committed to Git, they have to run tests before they can deploy new code. So it really depends on your company workflow if and how you want to adopt an automated workflow. For smaller sites, I agree. It would be cool to have something like that. The problem is I think- You wouldn't have Drupalgedon with such a system in place, there would not be Drupalgedon. It would have been limited, yeah, it would have been better. The problem is that for such projects in order that the web server can write to your files, it has to have write access to the files which contradicts the idea that the web server shouldn't have write access to the files. I think WordPress is solving it in a different way, so somebody could look into it, how WordPress is doing it, if it's secure, if it fits the model that Drupal is doing. I just have to fear that- No, but- People that are working in the Drupal community, working professionally there, they are having different workflows and are not interested in developing those sites. So I think it's hard to get resources for anyone working on that. Yeah, the issue exists, but there's nothing. And the funny thing about it is, I understand, for all the client sites I work in, I would disable such an option. But that's for people that know what the risks are in disabling that option and that have a system in place to deal with it. Whereas a small site builder, somebody that buys a Drupal site from a shop and then doesn't want to pay maintenance anymore, and they just leave it running in the ops that it's running. Yeah, so what is currently happening is that there are a couple of new services springing out of the place everywhere. So from complex solutions, by services, I mean companies that offer you auto-updating your Drupal sites. And there are complex solutions where people integrate fully with their build workflow and whatever, and there are easier solutions which just apply the co-update to your shared hosting or whatever. So I think it's a first step for anyone who's interested in auto-updates, they should just do a short Google search about that because I think there are a couple of companies and compare the pricing options. There's actually a wiki page on GDO about this, listing all the companies that offer managed Drupal code services. I'll try to find it while the next question comes up. Hi, you said that it is not advised to let the web server write to the files, to the module files and so on. But isn't Drupal 8 not working the way that the config files are written by the web server process? Yes, they are, but you should only do that on your dev sites. No, the special directories that are created there are specifically protected against that. So people have put a lot of effort thinking about it. How can we protect that configuration files that are automatically written so that the web server doesn't overwrite them? So it's called PHP storage, it's a concept in Drupal 7 and it basically has a special path so that it's hard for an attacker to discover where those files are and where this director is and how they could write stuff into that. But for people working professionally with Drupal 8, they wouldn't write the configuration to the file system anyway. So they would do that just for the devs to work from the dev sites, right? And what some people do as well, just while we wait for the next question, what I've done when I worked on the Drupal 8 site is I will export the CMI YAML file and Git but outside the doc root. So my doc root is a sub directory of my Git repository. So all those, YAML file lives outside the doc root and I can load them when I need them with Drush. Hiya, great presentation, thank you very much. Three questions, are the slides available somewhere online? Yes. Lovely, thank you. Yeah, let me just show the link. Is that the beginning? I forgot to mention it. There it is. I think we can put a link into also the session page, right? Absolutely, yeah. Brilliant, thank you very much. You touched upon how people can hack in. Have you got real life examples of how they've manifested, like what's actually happened to the sites, case studies, all that sort of thing? Do you have any information on that? What was the question again? So what does a site look like that's been hacked? Have you got case studies of... Yeah, I mean it's usually defaced, like the kind of thing that anonymous would do. Anonymous would just have to get angry with someone that just hacked a site and they'll just take everything down, put a single page, you know, black background with like a... I don't know. And make it obvious that it's been hacked in big. But there are more subtle hacks that I've seen with Drupal Ganon, with SQL injection, where actually all of the hacks that I've seen in the logs were backdoors. So their goal was to be absolutely sure that the site owner would not notice that the site had been hacked and that the backdoor had been installed on the file system. It was either on the file system. So again, if you follow what we talked about, it shouldn't have happened that way. But a lot of shared hosting sites allow the web server to write on the server. So they would simply download on a PHP file somewhere in your modules slash book or whatever. Somewhere you don't look very often. And then their plan, I suppose, was to come a few weeks later when you're not really suspecting any hack to happen and then use that backdoor to really do something bad to your site, or steal data or something. Or I think what often what we saw with Drupal Ganon was they're using it as a mail relay. So they're planting some PHP file there to send out mail over your server. That was a common pattern I saw. Right. And if... Well, the updates come out on the third week of the month. What if hackers somehow knew about this vulnerability before it was made to the public at large and compromise the site? So you're in the unfortunate position to say that it had happened. Going, you've got a recovery piece of documentation there. If you wanted to somehow analyze what the hackers did, looking at, say, Apache access logs, that sort of thing, would that be the approach to take, or if you were in that position? Yes. Or even if you were just paranoid, generally wondering whether your site has been hacked or not. And you don't know. Sorry. Yeah. Check all your logging. So there are a couple of different logging layers on your server. Check the Apache logs of the web server. Yeah. Check the MySQL log. You can see which queries have been run. Check the Drupal log. Drupal has a watchdog which locks somewhere, probably in your database. So check all your login. Otherwise, you can't really trace what happened, right? Yeah. And you'd be looking for things like perhaps admin access URLs, that sort of thing. Would that be the case? Not necessarily. So with the SQL injection, people planted something in a router table, for example, to execute arbitrary code. It's hard to tell as a general rule where you should look. We just say after Drupal get on, and if you haven't updated early on, then you should just assume that you've been hacked and should roll back to a safe backup. Okay. Yeah. There was just one final thing on the logs. There was a talk yesterday about logging and analyzing tool. Did you see it? I just want to remember. Began with E. Yeah. Do you know the one? Was it about the Elk stack? That's it. Yeah. Would that be any help? Yeah, of course that helps. Okay. So we used that as well. So we performed some logging to collect click statistics, for example, which paths have been accessed and righted to log stack. So of course that helps. Yeah. It's also a log application. So anything that you're logging, going through that and analyzing that is a good idea. Okay. Thanks very much. Cheers. And I've got the Wiki page I was talking about earlier. List of service providers who keep Drupal sites up to date. So that's to deal with security updates on your code base. There's a lot of options here. If I can add a link to that page on the slides. If I can apologize for making questions again. Sure. So getting back to the slide on Drupalgedan, there's a very nice thing there in the third point. So clearly Acquia and Pantheon knew about it. I know that every Wednesday you guys meet. I know that on the third Wednesday of the month, you should be on the lookout for new Drupal core release. I also know that when I get those emails, I am probably thinking about going to bed. At least in Europe. And I kind of understand that they're probably released in a good time for you guys and for the US, which probably is the biggest market anyway. But so first of all, can't we have some kind of early warning in place like... Like if it's not going to happen, you mean? I know that you say don't expect anything. And so should we take the fact that sometimes you don't say there's no Drupal core coming to mean that there's a Drupal core coming and we should back up our sites immediately and... No, we'll never say that there's no update coming if there's one coming. I think when there's no one... There's not an update coming that you say that there's no update coming. Sometimes we don't know in advance. We try like in the last few days and then the day before we say, okay, we're not going to do it. We will be ready. And my other question would be... Let me answer that one first. At that's when Drupal get on head, not that Wednesday, we said on Monday this time there will be a release. So I think it was a big communication problem that, first, that this is going to be big and that everybody has to pay attention and we should have used a lot more channels to push this out. We should have really created a panic so that people are ready on Wednesday. So we just used the usual communication channels like drupal.org, Twitter, where we make the announcement that something is going to happen, but it was not enough, especially because this was so severe. So there should have been really a storm of warnings. Yeah, and my other question is, there's this Acquia Pantheon Club of VIPs on security updates. Is that because they have people in the security team? So should every shop that wants that kind of advanced knowledge try to have someone in the security team? Is that solution? The security team application is open to anyone, so anyone can apply, but obviously not anyone can join due to the confidentiality aspect of the work. You need some background security. You need to have time dedicated to helping with patch review and writing patches. It happens that the big hosting companies have staff also on the team, so they have early notice. But do you think it's unfair or what? I don't know how to answer my own question. I know that if you can't open it to everybody or else me as Acquia Incorporated would join your security team and I would be very happy to have hacking sites for my clients that would want to use them as spam relays. So I don't know. There needs to be perhaps some advanced warning. In this case, we agree we should have made more noise beforehand, but I think it's also a lesson for all the community to learn that they should really review that process for updating code on their sites and they should be able to release code quickly, typically within a day or a few hours, so that if this was to happen again, they would be safe. But we've learned also as the triple security team we've learned from this and we are putting better process in place to avoid this to happen again. Yeah, I think this kind of vulnerability only happens every 10 years. I think that critical SQL injection was almost 10 years ago. So yeah, in 10 years we're going to watch out. Nine, it happened one year ago. More questions? Thank you for coming.