 My name is Calla Varis-Virta, I'm from Finland working for a company called XO, I'm here to talk to you about auditing triple sites, some things that I just want to say is that I'm going to be going in and out of triple in a technical manner as well, so some parts might be difficult to understand if you're not a technical person then sometimes might be boring if you are, so hopefully I bore all. Also some understanding of the business-involving triple should be useful while listening to me, so let's get a hands-up first for technical people and then triple business people, our first they are both, what I'm gonna talk about here is that why are triple audits done in the first place, there are several different reasons for those, how they are done and I'm gonna include some technical details as I said earlier and then I'm gonna go briefly into the business of triple audits, there are some challenges that needs to be taken into account when doing audits as a business, so let's start what's an audit, that side, I wonder if I can switch off the lights from somehow, better? Okay, good, that was for the light switching guy, thank you. Okay, so audit is a run-through of an implementation of a site, that means that an experienced developer goes to a site in and out all aspects of it, it doesn't mean that he reads every row of code but it's a very thorough thing usually and as I said audits are done for many different reasons and the process of doing an audit varies a lot, we'll get to that next but so how many of you have done an audit, okay, how many of you had your work scrutinized by an audit? Less people, we have more auditors than the people getting audited, okay, good one. Alright, why are audits done? So we have four categories of audits, acquisition audit, implementation verification audit, vendor management audit and support audit, these are not official categorizations of audits, this is just my hunch of how I would divide them into groups. Acquisition audit, that's part of a, well it's usually done before making the decision to buy a business, that means that it's part of the due diligence process and some people know what that means but that means a investigation of a company before signing a contract usually to buy it, so that's for acquisitions that that means obviously due diligence means also financial side and all the others but when the company's business involves web or the site in really key aspect of it then it's usually a really key aspect of the whole due diligence to do the audit for the site. Usually they are done obviously to smaller startups, anyone who based their whole business online and it's a very in-depth and focuses on kind of a hidden agenda, usually the company that's buying the other company, the buyer has plans for the site and they don't tell the target of the deal what those plans are but the auditor needs to know what those plans are so you need to know what they want to do with the site so you can actually audit it in a proper way. The next category is the implementation verification audit, I would say that's just to verify an implementation of a site that's more like an insurance policy, the customer just wants to pay to not have any problems, they're not expecting a lot of problems. These are usually pretty brief done in collaboration with the vendor who's working with the site and shouldn't ever be done to a system that's not finished. By the way when you see those lines that something shouldn't done, that's something that I've done and then I feel bad for doing it so it's something that I learned so don't do it for unfinished systems because that's just stupid. Then there's the vendor management audit and this is a bit complicated one that means that your customer who's buying the audit is wanting to switch their vendors not necessarily to you but to something else and they usually have some problems with the current vendor and that makes it a bit difficult because usually they don't tell the current vendor that this is happening so you'll have to do it without any help from the current vendor and that makes an audit a bit harder. We'll come to that also. These might be very brief but usually it's a bit longer audit. My longest audit that I've ever done was a vendor management audit took about more than 30 man days to do that. It wasn't a Drupal though, it was a custom code a lot of it and this time the client expects to find something that taken base on their decision to switch vendors. They might just have a personality clash with the current vendor but they still want something to base their decisions on and you shouldn't go into that trap. We can we come to that later as well. Then there's the support audit. This is when it happens when you move an existing system to your support. Unfortunately this is a very important audit to you but not very important to anyone else so you won't probably get paid a lot or paid at all doing this but you should still do it. This means that you'll be getting someone else's code into your support and you'll have to support it later on and then you should do an audit at least a brief one and this is also the type of audit where you can learn from the experience because in the long run if you take an existing site to your support you'll find out all the problems with the site. You'll also learn what you missed in the audit. So this is really a learning experience. Unfortunately it's also when it's a really good learning experience also a pretty nasty one because you have a really broken site in your support. That happens every once in a while. So going to how it's actually done. Tip one, you always need the source code. Always. First and foremost. Okay start taking notes from day one. You need those notes to back your memory up. You'll forget everything and remember stuff wrong later on. Then you'll have to go back and recheck and even though it's fun it's kind of exhilarating to get a USB stick of some code you never seen before and you want to just investigate code through everything at once. You should write notes while doing that because you'll forget to do that and you'll forget everything you saw and you'll just remember that it might have had this in this file and this directory somewhere and then you'll have to recheck and have to do the work twice or three times. Secure the source code. As I said you always need to search source code. Ask for a dump from the database. Now there's the whole concept of having private data in the database which is kind of problematic. Sometimes it's possible to obfuscate the most private parts of it. Social security numbers or something like that in the database so you can actually get the data because you need the data as well because you have to see how the system works with data. Without data it's kind of a shell. The obfuscation is done by the current vendor or your customer if they can do that. It's sometimes very hard. Sometimes even impossible. You just need it so you'll just have to figure out a way to do it somehow. Don't ever settle for partial source code. Now this is a really typical thing I keep hearing. We'll just give you the custom modules. That's not enough. You need all the modules. You need all the source code. There might be hacks in core. There might be hacks in the content modules and there might be stuff in the wrong places all around and there might be some really odd configurations in a non-hacked core as well. You need the whole Drupal. I've been hit with this once. We had a really juicy big customer and I tried to persuade them to give me everything but they didn't. They only gave us the custom modules and when we finally got the whole site into our support it had two cores running different versions side by side. You can imagine what kind of a mess that was. Drupal is, unfortunately, still is, totally impossible to update. Well, let's try and forget that. Anyway, then the next thing you should do is to install the site. Now this might sound a bit odd. They have an existing site installation somewhere but installing a site gives you a lot of insight into the whole thing. Have you ever done your custom modules and done a lot of update hooks there and forget to update the install hook? Yeah, that's why. You learn what's missing from the system and what's not documented or what tweaks need to be done that's not supposed to be there and you'll probably have to stop several times getting more data. Code, vcls, API definitions to create dummy APIs. We serve enough calendar time for this and it's still worth the time. Next thing, you must understand the architecture. Once you have installed the site, look at the architecture. Drupal sites are based on different combinations of modules. Certain combinations of conjured modules. You can do blocks with context or panels. You can search with search API, maybe Apache source search integration. Translates, translations, you can use internationalization, you can use field translations. Translate blocks. Use a multi-site, maybe. You can control access by domain access, taxonomy access. So plenty of options. You have your favorite option, obviously. You have your favorite combination. You know them by heart. Don't be biased. They're probably not the only way to do it and it might be just equally proper way to do stuff even though it's not the way you do stuff. So don't get biased by your own opinions on which contracts to use. Next on architecture, does it fit its purpose? If you see a Drupal and all the custom code in the Drupal, it's just circumventing the shortcomings or Drupal's practices because it doesn't fit the bill, then it's not the right tool for doing that. And I know this might get me killed saying Drupal conduct isn't suitable for everything, but it isn't. So in some cases, you have to see that this is not just not the right tool. It's the site using Drupal as it should. I've seen a lot of, well, several sites using Drupal in a way that it has like three pages out of Drupal and then everything else goes to a custom module to some custom backend and pushes data from there. Just an empty shell with some theming on it. That's not a proper way to use Drupal. Usually the customers are asking that why doesn't any contracts do anything to my site? That's because your site isn't on Drupal really. It's somewhere else. That happens. Are there custom parts somewhere where there's a well-working contract available? The custom parts might be built well and they might be even good for the purpose. But if there's a contract available, you have to judge by looking at the contract's possibilities and the custom code, that which one would be better, at least have an opinion about it when you write the report. And then you have to look at if it's overly complicated. Obviously, Drupal sites can be overly complicated and they can be for various different reasons overly complicated. But you'll have to report that as well. So make sure you understand the architecture. It might take you a long time and it's really easy to fall in the trap of just writing the report without understanding the whole thing. But you really, really, really need to understand how the architecture works. What were they thinking when they were doing it? Because if you write your document or your report, your audit report, without understanding the whole architecture, it's just going to be a bad report. And it might take days to go through the code and go through the modules and wonder why is this done? Why is this done? What is this? Why is it here? But when you finally realize why it was done, then you can actually write an intelligent document saying what should have been done and what should be done next. Then there's the whole concept of reading code. Remember, reading code isn't the main purpose of an audit. It's not a code review. But reading code isn't a big problem in Drupal audits. And there's usually relatively little custom code to be read. And if you have trouble finding it, for instance, if it's not in module slash custom, then you might find it with hacked if it's in totally wrong place. And when there's a lot of code, as sometimes is, remember you can't read it all. Reading code row by row doesn't benefit anyone. So when you're going to be reading code and there's a lot of it, focus on the parts that make a difference in the end result. Look at security holes. Look at beginner mistakes. Look at performance problems. These are the typical ones. You want a list on your findings report. When you're looking for security holes, well, here's just a checklist. This is totally not exhaustive list. So there's plenty of more stuff to look. But I'm just listing a couple of things here just to give you an idea what you would be doing. Check for SSL login, check for old contracts without patches, check for abstractions, unclean inputs in the UI, JavaScript, a lot of XSS available there, API calls without HTTPS to the backend systems. This is something that happens ever so often that we connect to a CRM back there somewhere and it gets left on the development settings and then data is going to HTTP without any protection. And there's personal information there. Beginner mistakes, unclean access to Drupal. That's what happens when beginners start with Drupal and they don't really know how to do it. Hook for unnecessary custom modules? Well, obviously you have to think that also the time span where the contract is available when the custom code was made, written or is it doing exactly the same thing as the contract modules? Look for wrong hooks. That's something also that happens every once in a while that people just use a hook they find first and that's probably not the hook that they should be using so they might run stuff too frequently, for instance. Performance problems, static cache is time consuming functions, check out the amount of processing in any hooks. Look for slow backend APIs. That's something that you can't do anything about but you might want to list that on your report. Check out the caching strategy. There are plenty of different caching strategies but you should definitely have one at least. Well, we don't cache anything. It's a caching strategy as long as it works. Look for unnecessary but very slow contract modules or misuse of contract modules. I've seen that as well. Site crashes every once in a while. It uses a contract module for something that it definitely shouldn't be used. Then there's the whole social engineering side of the whole audit. You'll be talking, not in all audits but most audits, you'll be talking with the original developers. It's very important. Don't miss that opportunity. If you have an opportunity to interview the original developers, please do that. They'll tell you not only how it works but why it works in a certain way. They might even tell you that this and this part is totally wrong and we had to do it like this because we were tied on time or something like that. They'll point you straight to problems. Just be polite and friendly. I wrote a blog post about the social aspects. I thought it's just a couple of weeks back. I had a couple of lines there. What you should do when talking with the original developers, they were first introduce yourself, your background and your expertise. You get their respect in that way. Explain what your task is and what is not your task. Your task is not to blame anyone or something like that. You're just trying to find kind of an understanding of how the system is right now. Don't speak any of your findings out loud when you're looking at over someone's shoulder and you see something really horrible. It's sometimes really hard to not gasp or do something like that but you just have to keep your mouth shut and smile and nod. And you use positive language always. You're not there to find out problems. You're just there to verify it's as good as the climate is. And when you're asking questions about the code, be very understanding of the situations with the original developers. They probably were busy and there were demands in all different directions and stuff like that. And if they deny you something you need, you really need, talk with your customer and don't go bossing around in someone else's office. You won't get any friends like that. Let your customer do the whole telling people stuff for you. You don't want to do that. But it's not just the code. Okay. You might have a really good site, a really professionally made site and it's installed by Total Newbie. It doesn't come up with a good result at all. So always look at the production environment. Now this is something that is usually pretty difficult to get, which is access to production. But you should try and do that because if you don't see how it's actually running production, you probably miss out a lot of what's actually happening in the site. You at least need read access to the production or at least a copy of all relevant configuration files. The problem is though that you should verify that you actually have all the configuration files that they have and you can't if you don't have access. Even if you have access supervised so that someone's looking over your shoulder when you're poking around, that's okay. You can just poke around. As I said, keep your mouth shut, but poke around. Make mental notes. Don't write them at that point. So there's a lot to check in the production environment obviously for security, for performance, for reliability. Installation and server configuration. Look at the PHP. Look at the HCTPD process management configurations. Upcode caches. PHP scary options. Apache, nginx configurations. MySQL and other databases, replication configurations. Backups, by the way. That's a wonderful thing. Ask if they do backups. They say yes, then maybe they do. You can't usually check that, but you can just hope that they were telling you the truth. Check for open ports, services running, MySQL passwords, all the extras, Swanish, VCLs, MongoDB, Redis, Solar configurations. Do check the Solar schemas as well if you're poking around in Solar. Take a look at the Drupal configuration. User roles and privileges, registration and login settings, caching settings, content modules settings. Content modules might have really scary settings, beware. Good, look at them. You might have trouble finding them as well. They might not be listed anywhere. You just have to look at the module to see if it has any many callbacks. Custom module settings, even scarier, usually. They might be just config files on the disk. Make sure you check them because they might have really, really scary stuff. SEO configurations, that's something that people tend to forget. Us developers, we don't think about SEO. Cleanup for automatic imports. An automatic feed import running some RSS feed-in might chew in a lot of data over time. It might have a really big chunk of stuff in the database. Multisite configurations, language configurations. Well, the site might not be about performance, but a quick benchmark never hurt anyone. Depending on the audit, obviously, performance is just a part of the audit or it's a main part. In acquisition audits, performance is usually important. The idea is that the customer, your customer, the buyer, wants to use the site for something bigger, better, and something that needs more juice in the system. They're going to ask you that the site is now serving 100,000 uniques a week or 20,000 registered users. Can we double that? Can we triple that? Where's the limit? It's really hard to estimate that, honestly, but you have some idea of that when you do a quick benchmark of the site. Not a quick one, but if you benchmark it a bit. Just run a couple of pages with anonymous users, logged in users with a benchmarking tool, profile the site, under load, see how it's doing, what it's doing, especially what it's doing. You'll probably see potential bottlenecks very fast and you'll be able to just write them down. Now, two really important things. Auditing is a gentleman's game now, or a woman obviously, but you have to be very polite. Usually, you write one or two written reports that are kind of the outcome of the audit. Sometimes just one, but sometimes two if you need a non-technical and a technical one. Usually, the non-technical is just for the business people to understand something. The technical one is for deciding what to do about the site. Sometimes you can add code snippets and runtime grinds from the code, but sometimes you can't. In acquisition audits, generally speaking, you can't add any code or anything in there, because usually the custom parts of the site are kind of owned by the company still running it, and the buyer doesn't have any, doesn't get any access to them. Even as you're doing the audit, you can see the code, but your customer can't see the code. They are not allowed to see that. They can just see your opinion. Your opinion is what you write down, not what's happening or what's in the code. This is what I usually do. I divide audit document into three main parts. There's the introduction, explain the system, architecture, the platform, the modules, implementation on a high level. Then there's the finding spot. I list all things that I think are worth mentioning in the document. I sometimes mention good stuff, but mostly it's very problem-focused. We tend to be as coders. Then there's the improvement suggestion spot, where you suggest improvements. Now, this is debatable if it should be a part of an audit, but I usually do it as a kind of professional courtesy always to tell what I think should be a better solution, because it's easy to blame some solution bad and not give an alternate better solution. Just being kind of, you can always say that something's bad and not tell what's better. In that sense, I would suggest that everyone who's writing these documents would write also your idea of a better way of doing that. Whatever you do, don't ever bash. Don't get personal. Don't bash. Don't bash the system. Don't bash the coders or the decisions. Just try to be as neutral as possible. Even off the SQL injection hole in the front page, you have to be neutral. You don't have to understand why it was written there, but there's no need to bash the expertise of the original vendor, even though it might be debatable in that case if they actually have any expertise. But still, don't write it down. Not only because it doesn't make any difference in the end result, but also because you'll be at the receiving end at some point, and you'll really appreciate the professional courtesy from the auditor, not bashing you for your really hard and fast decisions you had to make, that and that fall or that and that spring in that project when you were running out of time. There are different circumstances when we do triple sites, and we all know that some circumstances are harder than others, so you will have to just understand. And we're a small community. We don't need to get business by bashing other people in this community. List only real findings. Now, what if you can't find anything wrong with the site? Doesn't it feel kind of sad that you're taking 6,000 euros of your client's money and not even giving him anything in return? Isn't it kind of sad? Maybe you should come up with something. You should not. Just write down that there's nothing wrong with the site. Manage the customer expectations when you're selling. Tell them that you might not find anything, that it might be totally fine. The site might have nothing wrong with it, and never exaggerate problems, because when you go to that, you'll just get into the whole game of blaming people for trivial problems that can be easily fixed or that were done in a haste and can be well rectified in no time, or the problem isn't even that big. Unfortunately, I see some of this in the security audit business. They tend to list really odd security holes, holes just to fill up the whole report. I don't like that. We don't have to do that, so let's not. If you can't find anything, don't list anything. End of story. I just did a really big, big, big audit this spring, and I couldn't find anything wrong with the system. The only thing that was wrong with the system was it was built in 2006 and using that time's technology. I told them that it's like a really beautiful car that's a bit dated. There's nothing wrong with it, but it's old, and the customer was fine with that, even though that was a big audit. They shelled out a lot of money for that. So just real findings. Also, don't put your junior devs to do audits. It doesn't end well. In the business of an audit, the time needed for a drip audit is really, really hard to estimate. You could estimate it well after you've done it when you know what the site entails of. That's kind of redundant at that point. You could also just check the clock. Ranging from two man days, I would, like, an hour game with audits isn't something that I would opt for. I would say that from two man days, and then you can go on. Obviously, a good deal is to get a limited budget to do that. Try and do that. Probably not that easy to get, but usually the customer wants something, something, some limit, some number, something, and then something in return as well. Pricing goes by the hour or whatever, but it goes by the price of the highest consultant you have. Always, you should also use the highest price consultant you have to do that. For support audits, obviously, time is very limited, and you'll just have to do what you can in the limited time. Then who is the person who can do an audit? Now, the person needs to be a real expert. Honestly, a real expert. I would say that years of experience in not only Drupal, but Rocksolid PHP programming skills, they need to know what they're looking for. The more experienced they are, the faster they can find the problems in the system. They know where they are. They know their way around different architectures. They can understand different setups, different architectures, different systems. When doing integrations into external systems, you'd have to understand a lot about integrations. High performance security integrations, hugely beneficial. I would say that four years of experience to start, okay, so you can go to our website and see how many references do we have from audits. Not many, I can tell you. The most problematic part in selling Drupal audits is that the customer would want to buy from an experienced, well, someone who's experienced in audits, and still they won't give a public reference for doing that. So they're pissing in their own pond. They can't find anyone who's done audits publicly because they won't allow you to tell that. And it's a really subtle business because when coming to an audit, the NDAs get really heavy. And we once had to take an insurance policy just for signing the NDA because the damage causes were so steep, there were two dangers that they would tip the whole company if we would go to that. We didn't, obviously, but still we had to take an insurance policy for that. So they really get bad. Some are so bad that small Drupal shops just can't do audits. Some audits are the kind of an audit that only go to bigger Drupal shops because they have insurance policies or money enough to sign such an NDA. They might also demand a personal NDA from the person doing the audit. That would find as well, read that if you're about to sign one because you really want to know what it says in the contract. So one out of 20 audits I've done, give out a public reference, that's about the range. So do 20, you get one reference, do another 20, you get the second one. Then your experience, you have multiple references. All right, okay. When is the perfect time to sell an audit? Well, obviously when the customer is changing vendors from someone to you, you should try and sell an audit. That's the place where you should basically demand it. It's for your own security to know what you're getting into and the same goes for taking existing sites into your support. That's also a place where you should definitely demand to, well, to have some conversation for the audit. If you can't get any compensation then do it on your own money, out of your own pockets. You'll have to do it anyway before you take someone else's code into your soft care. And just a reminder, selling audits, never promise you'll find anything wrong. You don't know that. Never promise that you'll find everything that's wrong. You can't guarantee that. Nobody can. Unfortunately, you'll probably miss out a lot. I can guarantee you that. And when it's in a support audit, they'll come back and bite you in the ass. In other words, it's maybe not. You get a free pass. Then a quick recap. Get the source code. All of it. You need it. Get the configurations. Get access to production. Understand the architecture. Optimize code reading. Read only code that matters. Remember to test the performance. Be a gentleman. And list only real findings. Okay. That's it. Thank you. Well, according to schedule, I have about 15 minutes left. So if you have any questions, please use the microphones. Or I'll have to repeat your question if I can. When you're doing an audit, is it always one person that does it? Or do you sometimes work with a system manager that knows more about that field? Okay. That's a good question. Obviously, people's expertise differ a lot. So you might need an expert of a certain area to do that. So not at all. We do all audits by ourselves, but there's a person who's mainly responsible for writing the report. So small audits yourself, maybe, especially if it's just a regular triple site. You can, you know, pretty much all that's needed. But when it's a big audit or has a lot of different systems around, then you might want to use other expertise you can find in your own company. Okay. I have a question about your reports. Is it not smarter to have the findings and the solutions together instead of two separate chapters? Well, yes, this in a way. But I kind of think that they are not at all always one to one. So there might be three or four findings that can be all solved with just one solution. So there's no connection one to one, but it's one to many or many to one or many to many in the findings and solutions because they're not usually that straightforward. And do you do something like Moscow? Like the must-haves and the should-haves for recommendations? Oh, so I would prioritize the findings and the listings. Yeah, I usually at least somehow explain the severity of the finding or the priority of the fix that should be done, especially if it's a security thing. Then you usually want to say that this is something that you really need to fix. But yeah, you can do a different kind of prioritization or severity listings or severity, whatever. And you can even categorize them by that. I actually have done that. That like critical findings or something like that. Thank you. Anyone else? Okay, then I'll thank you. Take the survey.