 At this point, we've clearly identified the account that's having issues. So let's talk a little bit more about cleanup. The first method that I want to talk about is specific to workers. Now we'll see in here, this isn't quite so easy for this account because he's got a couple of different installments. He's got a Drupal one, DocuWiki known, and we can go in there and look at the files and suddenly confirm that those are actually applications. You don't really want to assume, obviously, by folder names because a folder could be named anything and it could still be malicious content inside there. But I can see from the file list there, that's a typical known install up here. This is a typical WordPress install. For the majority of this, I'm going to focus on WordPress but I'm going to show you some cleanup tools that don't care about what kind of application it is. They look more at the actual code and use signatures to figure out, you know, similar like a virus scanner, right? They're going to look at, well, that's not an issue. I don't care what application it is, they've got ejected code in there, so we should quarantine that file. So... Wasn't one of the alerts that Process Manager was showing for known? It was, yeah. Yeah, so an under known, it was in the under known one. So for that, what we'll do is we'll run a scanner on the entire account and that will pull out some of those things. Exactly, yeah. But in some of the things that I'll talk about, like first with the manual thing, I'm going to do manual based on a WordPress install but some of the same things apply with other applications. So with manual, the way that you want to approach this is to break down the content and we're only worried about the file system here. So you've got file system and database. The database rarely is ever going to be hacked. The only thing that I ever look at in a database is are there any strange additional user accounts that shouldn't exist? For the most part with your users, they'll probably have a single administrative user. It's their account. There are some cases where with the WordPress account, if I look at that user table, I'll see like wp.service as a username and that's clearly an injected admin account that they're using to take control and do some things. I see that occasionally, but typically, you know, we're mostly focused on the file system. And so with the file system, what we're trying to break down here is what's the legitimate content that the user added versus the stuff that can safely be replaced with a fresh copy. So that's how we're going to clean this up. So with the WordPress install, you've got a bunch of files in the root directory, wp config, wp comments and those kind of things. And you've essentially got three folders. All of this other stuff, those are other installs right now. So we're not going to focus on that. So focus on the things that I haven't highlighted. You've got three folders and then a bunch of files. This is an additional one. It's not related to WordPress. We can take a look at that. It's probably at site now. But you've got these. wp admin is going to be all your administrative functions. That's when you're in the dashboard WordPress. wp includes is going to be all your JavaScript files, all your CSS, all your assets and things like that. Those two folders, you can completely delete and replace with fresh copies. So let's do that now. There is no reason, and actually I don't, you know, not only should the plugins and themes not add stuff there, but they don't and they may not be able to, but they definitely shouldn't. These can be replaced every time you update WordPress. So you can safely delete these. You can delete most of these files here. So obviously not going to test this Word document. That's something that the user... What about that XML file? Yeah, let's look at the XML file. I guess it was just like that. It looks like maybe the user was trying to verify them site, but there's, themselves, their site with being closed. So, you know, it doesn't look entirely dangerous. It's an XML file, so it can't execute as code. So I'm not going to be concerned too much with this one. I'm mostly focusing on PHP scripts, things that can be processed on the server in that way. So we'll skip over that. The only other two files on here that you want to hold onto are HD access and WP config. So WP config is my connection to the database, right? That's got all the information about what my database is. That's unique to this install. You don't want to delete that one. HD access is for making nice URLs on WordPress so that people don't have to type slash index dot dot PHP when they go to the website. So we want to hold onto this. We could make sure that everything's clean in here, and this is a good example of a default version of the WordPress HD access file. And if you actually go, I just search WordPress HD access. I go here all the time. The top result is in the WordPress codex, and they give you the exact language of what a default WordPress HD access file should look like. Sorry, I have seen hacked HD access files before in that event since we can't delete them. Do you just replace it with the generic WordPress code there and move on? If you're unsure and you want to check with the user to see if they added stuff or whatever, you can always rename the HD access file, and that stops it from being addressable. And you could then put a clean one in there to make sure that the WordPress site works. But you could will often do like dot HD access dash off or something like that, but Apache stops processing it at that point. Usually when you see injected stuff in HD access, they're trying to do tricks and say, if you're a search engine, I want you to go to a different URL. And they do this so that people go into the site normally, you don't know they're hacked. But if you come through Google, they just redirect you right on to porn pages or whatever they're trying to do there. And so that's their way around it is to check and see whether the user agent, the browser that's viewing the website is a search engine. And so they'll redirect traffic based on that. HD access can see that kind of information. And so they take advantage of that. So that's usually what you'll see near the top is, you know, if they're Google or Bing or something like that, I want you to redirect a strange PHP file instead of index.php. I actually have an example of that on the HD access document. Yeah, perfect. Yeah. So that's usually what we see here. Another thing that I noticed while I pulled it up, so we'll just go ahead and address it, is that this WP config file has injected PHP code at the top of it. So normally your WP config file is going to start obviously with the PHP tag, and then it starts with a couple of comments about it and goes right into the database connection stuff. All of this, these included stuff, that's problematic. You know, this is injecting additional content in here and saying, you know, if somebody tries to go to this URL and they get a parameter or something, go ahead and let them read the content inside it and no one should ever be able to read the content inside WP config because that's been obviously interested in database passwords and information. So one thing that I can do here is just clean it up. Just remove that injecting code from the top there and then I can look over the rest of it and see if anything's crazy in there. You can compare this. You can do a default clean install and compare it to see if anything else looks weird and the rest of this is following the standard format of WP config file. Do you ever change, like, reset database passwords or anything? Like, if a site can compromise? It's not a bad idea. You know, it's worth adding to the list is something to do. I think it's unlikely that anybody's manually reading this stuff, right? They're using that injected code to take the database password and immediately start injecting stuff into the database. So once you've removed all the malicious code, it's unlikely that they have access to the database anymore. But, yes, cycling, and we'll talk some more about heartening WordPress and peer defenses after you've done a cleanup. Cycling any passwords is always a good common practice, but the database password, the login to WordPress, the FTP account, it's just good practice to make sure you're starting off on the best foot so that you don't come back a week later and you're back to the races trying to figure out an issue because it's been hacked again. So other than WP config and HD access, so we've got two main WordPress files on the top level, and then we have WP content folder. WP content stores all of the uploads, all the media uploads from the site, all the themes that have been uploaded, all the plug-ins that have been uploaded. And then it's got some various other folders depending on the upgrade folder is a temporary folder that's used by WordPress to upgrade itself. It loads a clean copy of WordPress in there and uses that to upgrade the site automatically. Plug-ins can add it like this in GG is a gallery plugin. So they can add folders to WP content. The things that we want to be concerned about with mostly are going to be plugins and themes. And this is the part that's really a little bit of a pain. The only way really to do this manual process of cleaning it up is to download each of these plugins from WordPress.org and replace it completely with the clean copy. There are scanners, and I'm going to show you how to do scanners, but here's the issue. The scanner's going to go, that's injected code, quarantine that file. And then remember when I loaded that site and that plugin was broken? That's why, because a scanner was run on the past in quarantine files and now the site's broken because it doesn't have a plugin file that it needed. Similar to the WP config file we were looking at, there's injected code at the top, but we needed the rest of the WP config file in order for the database to connect. You do need these files in there, but you need clean copies. And so there's not great ways to clean out injected code unless it follows a specific pattern where it's the same injected code in every file. And so the only failsafe way is to get clean copies of everything and replace them out. If it's a proprietary or four-fee plugin or themes, how does that work? That's the problem again. That's usually, and clearly at this point you're probably not doing this where you're going to be talking to the student about what possibilities are. Okay, are you using all these plugins? Do you have copies of them? You're going to want to go in and replace them with this kind of stuff. So it's by no means the expectation and this is a struggle that we have every client hosting as well as we'll often start the cleanup for some of them and then it's like, if I go in here and they have 25 plugins and this is one of 15 different installs that they were playing around with, then we're going to have a conversation about what do you really need in your hosting account because you've been hacked and we need to figure out what's the need to have stuff versus the largest plug with known for a week and never really did anything. Well, there's no use making that a clean and tight install again versus just removing it, right? So it's worth having conversations with the users. But yeah, when it comes to premium themes and plugins, if they have it, they should have access to download a clean copy. Otherwise, you're going to want to do the scanning method and run a virus scan to see if you can identify those files. The problems that you'll run into can sometimes be if the scanner doesn't find everything and then you've done all this other cleanup work but those still exist and then they're able to then spread that out. The thing to know about Cpanel specifically when it comes to security is that all the files in your account are owned by that Cpanel user because they all share the same permissions. You know, that known install can inject code into that WordPress install. That WordPress install can then spread and create other folders in Drupal and other things. So they're all, all those applications are owned by the same user and all these files have the same permissions for that user. So if a plugin has code that allows it to execute as your Cpanel user, there's no limit within that user's account to be able to do what it wants. So that's where... Everything's at risk at that point. Exactly, and that's where it can be tough with the cleanup because you'll clean something up but if you miss one thing, it can spread right back out again. So, yeah, it's also a good method as you're doing these manual cleanups specifically, but with any cleaned up, once you've done what you can, take a backup of it. You know, take a snapshot of it. You know, you can use install to back up. You can download those files manually. Keep a hard copy. Return to it a week later and see where we are. Did it get hacked again? It definitely is, it definitely is. And that's again where the conversation with the user is, have you made any changes recently? When was the last time you wrote on your site and restoring a backup? So, Reclaim Posting keeps 30 days of off-site backups on all of our servers. Files and database. So, if you put in a support ticket to us and say, I need you to roll back a site to two weeks ago on this date, we'll do it. No questions asked. It's easy enough for us to do. There's also a C-Panel plugin to do it under the files area. So, the feature is called R1soft. So, those are off-site backups not stored on your server and we can roll back to any particular date. You know, using it with the plugins is helpful because if the user doesn't know and they say, well I haven't written stuff in two weeks, you can go back and maybe I just take one example. Let's see, is there a test folder two weeks ago in there? Is the WP config file? Does that have injected code? So, you may have to roll back and see. You don't have to restore those backups yet but you can download an individual file and say, is this a clean copy of it? You know, try to figure out when it was hacked. Based on that. You can also look at time stamps for when stuff changes. So, if you find an injected code site when somebody injected that code the time stamps should have changed. So, we're looking at, I'm totally clear with that. Exactly. So, you might say, and he might say, I haven't written since the summer. So, at that point, you could safely just say give us the oldest backup that you have a copy of if it's in the last 30 days. Give us the 30-day version because that will hopefully be a month better than what we have at the moment to start the cleanup process and maybe even take care of most of it for you. So, the key thing with rolling to a backup versus the manual, if you're doing a manual you're going to get a clean copy of WordPress and clean copy of all the plugins and themes which means you're going to be up to date. If you roll from a backup the reason that it got hacked is because something was out of date. I can almost guarantee you this. The reason WordPress gets hacked has been of WordPress, the plugin, or the theme. Specifically, some of the more popular ones. Jetpack has meant some of these plugins and themes that are very popular. The 20 versions of themes, 2015, 16, those are getting updates all the time and it's because they're fixing security holes but those security holes are public. Hackers look at them and they also know that there are millions of people with that because that's a default plugin where Jetpack is run by millions of people so I can probably start scanning the web for that specific loophole and find sites that I can take advantage of. So, it's really important to keep your stuff up to date. So, if you do roll from a backup the first thing you want to do is run any updates that are necessary on the site. And then cycle the passwords, right? Right, right, right, yeah. Cycle the passwords too. And you'll still want to do some scanning and investigative work just to make sure that when you roll from the backup there wasn't some injected code sitting there that's going to then turn around and bite you. So obviously we can't delete the WP content folder because that's all the user's actual content, the plugins, the themes, the uploads. We can replace the plugins and themes. We can look inside the uploads folder for any PHP files. You should never see a PHP file inside the uploads folder. It should always, it should, and that's within WP content. So by going to the uploads folder plugins can add folders. Again, that gallery of plugins adding some stuff. I kind of think that's totally normal and then WordPress separates it by a folder for a year. You should only ever see images in here. The Linux commands that you can run to say show me the PHP file inside this folder and you'll go recursive down into there so you don't have to drill around and do some stuff as well. Graph and copy workers. I'm going to go in here. Now I don't need to do the WP content folder but you'll see here everything else I'm good to go. They don't include a WP config file. They do a sample version of it. So I'm okay there to just upload everything here including the WP admin and the WP foods folder. And so I just skip over that WP content one. We have clean version of WordPress. So I know there's no injecting code within the core WordPress files and meanwhile I'll work on the plugins and themes and then we're going to look at some of the scanning tools that you can use within WordPress as well to clean some stuff out. You can use scanning tools at the server. This is something that can be run on the server. You just run this command and it installs it and then you can specify when you're running it on the server a specific folder to scan. This is a scanner. It's basically a virus scanner. It's got a set of signatures specific exploits that it's looking for injecting code that it's looking for and when it finds those files and then a new specific folder for you. So it's a good thing to run on the entire account. It is free and open source. It is very basic. It's okay. It will not catch everything in my experience so it's a good first pass and stuff. There are options within this thing to do a prawn job so that it runs daily. So it never hurts to have on a server and just daily run through all accounts with it. You can have an email, your report of anything that it finds. So it's not a bad preventative tool as well just to make sure that you're up and running. It could be a good first line of defense to know if an account got hacked if it quarantined this specific file that might give you a heads up before things are really, really everything's injected all of it so that kind of thing you might be able to get an early warning with one or two files that got quarantined by this. Is this run through a web interface or something like that? Yeah, unfortunately this is all terminal with this one. It's a binary that's installed on the server itself and so it's not bad. I will show you a GUI interface for a program that I like a lot better. It's one that we're going to be rolling out. We run it on all of our shared hosting and there was a fee for doing so but more and more we decided we're going to be rolling this out to all servers just because it's really good. It's made by the same folks who make the firewall so we just skidged. So you remember there was that config server firewall so that same company and config server firewall is free that same company offers the config server exploit scanner and that's what we have here and they actually have an interface for doing the scanning. A lot of documentation and a lot of different options on this one. There's no time really to go through all of it but I will show you some of the commands that you can run in here to do a scan on the site and what a particular scan looks like. If I go here to generate commands I can choose which things I care most about in terms of what am I scanning? Is it a particular user? Is it a range of users? Or is it all Cpanel users? So you can do a scan on the entire server it'll take a while if you have a lot of accounts so this is scanning every file of a user so that will take a while you can check off to have it write a log file to a specific area on the server so that you can keep track of it you can have to email that log file to you and under the scan options they've got a whole lot of different things here and some of this looks like gibberish but you can leave it all checked if you find that it's often finding a bunch of false positives it will always tell you which option you thought it was flagging based on so you may find in the past that if I choose Windows binary or executable file there are always false positives and it's never a hacked file so I've often unchecked that one one of the really reliable ones is fingerprint match fingerprint match is absolutely going to find a hacked file it's looking for specific things that it has a database of and when it flags on that one it's something injected in there so that's a very reliable one suspected exploit file it thinks something is right so I always leave it checked but it could have a false positive here and there this is just the scanning options later on down and it can also check for versions this can be a good tool just to see whether or not there's anything floating out there that's an older version than the most current maybe wiki, wordprefs there are several different content management systems for that kind of thing installtron could be a good tool for that as well but this doesn't care about it whether something is an installtron or not if someone installed it manually it's still going to look for the versions so it's looking at the file system itself and says that's a wordprefs site and I looked in the version file and it's an older version so it gives you a heads up and we'll also search for some of the more common plugins and themes there's this version of Jetpack that's out of date and you can see how far out of date it is is it at point one or is it like a whole it's very very old so square version scan is useful virus scan is a little different than the other scanner virus scan is using a piece of software called plan and to do plan A B to do the scanning and so it's just an additional defense rather than just the signatures that it's looking for and that kind of stuff it's an additional virus scanner that's included in this you've got some options here probability scanning and that kind of stuff you can set the quarantine directory that you want to use and then quarantine options so you may decide you just want to scan but you don't want to quarantine anything and that's totally fine if you just want to record this account to see what's going on with it but it's often helpful to go ahead and quarantine things because it can save you from having to delete a whole bunch of files yourself but you can go ahead and quarantine them the thing to be careful of and why it's still useful to get that report and look it over is that if I run this on that mountain and discover account it would have quarantined the WP config file it's on injected code in there and it would have quarantined it and now the site is completely broken because it can't make that database connection so while it can clean out it probably will still break things so it's useful to be able to go back in to un-quarantine it and clean out those things or to at least know what got quarantined so you can get fresh copies of it it may be that it only finds issues in like two of those lists of plugins but that's great for me now I only need two fresh copies of different plugins rather than that longer list of 25 so that can be useful as well where you're looking at that log file and getting information about it and then there's just some final information for whether you want to throttle this so that it doesn't take too many resources on the server running it as a background process and that kind of stuff can I ask you a question about that WP config file so were you trying to tell us to replace that in the STP with the clean install or no we leave that one it's got unique information so we need to go through those two manually and the HTA access to read like go through manually that's right I just want to make sure I got those straight absolutely and I can show you I've got a good example I'm actually open here where I'm running on another server so this is what a scan looks like when it's running through every account on the server and I can see all these accounts, no issues no issues this is the exploit scanner the law from that exploit scanner that was running on all cPanel users so here it found something and it actually turned out that I thought this was bad and it wasn't so I had to un-quarantine that and just move on that one was a false positive but then it started finding stuff that absolutely was bad so these were phishing sites spammers, things like that you know you have an account so you'll often go through a lot of accounts that are clean and then you'll find one that has a bunch of stuff and then you'll have a whole bunch of clean ones and then one or two that have you know several items so it's never bad to do an all-user scan with this here's an example where it found a specific plugin that had issues and it was just this one plugin and so easy enough for me I can grab a clean copy of that one plugin and it solves all those issues because this is just one after another within that same plugin folder all I need to do is replace that one plugin with a fresh copy and I'm good to go so but this is what it looks like you'll find other things in there and you can see here it tells you what the file was it tells you that it quarantined which trigger it was on so whether it was this Bayes exploit scanner the known exploit those were all those options that I was checking in this particular scan that I'm doing I only have specific things checked because I want very low false positives but I wanted to quarantine every single thing that it finds that is not that's not okay so you'll find more in here but it'll always tell you which option that you had checked it's basing its decision off of it so that's the Convixer exploit scan we'll talk a little bit and then we'll get into some of the preventative maintenance and cleaning up afterwards so let's say you ran your scans the site's back online you've replaced all the plugins and themes core WordPress you feel like you're pretty good to go and now you just kind of want to cover your basis you've taken a backup of the site and you just want to kind of cover your basis with like I don't want that to happen again so what kind of things can I do and what kind of things does Reclane do and what kind of solution as well one thing for WordPress that we really like is a plugin called WordPress WordPress has a free version and a premium version I have found that both are worthwhile but the free version definitely gets you somewhere so it's something that I almost always recommend to use install after we've gone through this whole process of cleaning things out you're running a WordPress site and you were hacked it is a great idea to go ahead and get WordPress on there and that way WordPress can do these regular scans within the file system to see if anything has changed it's got a lot of options in there for things like checking every theme and plugin against the repository versions again it can't do the premium stuff but it can check against the open source versions plugins and themes and say hey that file that's in here is not in the main repository do you want us to go ahead and delete it and so it can be doing those regular scans and notify you of that kind of stuff it also does some really nice security things in terms of making sure that people who are making strange requests to your site that they know are trying to break into it they'll block those IP addresses so it's in some ways a firewall as well obviously we have firewalls on the server and other ways that we block this stuff but it never hurts to have it at the application layer as well so WordPress is a great WordPress for the thing to do you know CXS is a really good preventative measure in the same way that you use this for identifying and cleaning up sites you can do regular scans all things but it's also about if I go back from the generate commands option it has two different features here that I want to talk about one is called CXS watch we'll watch every account on your server for new activity and as soon as a file is placed under that account it goes huh that file is interesting let me go ahead and stand it nope and it quarantines it at the time that it gets added to the account so watch is checking for any changes across any account the thing to be careful of of course is false positives you can look at the log for your watch on here but this is a really good preventative maintenance tool to make sure that you know these people are hacking into accounts can't do anything even if they had the FTP credentials and they were adding stuff in there it's getting removed at the moment it's being added you can imagine a support scenario though when someone's saying I'm trying to add stuff to my site and it disappears right after I added could be something like this where it's throwing a false positive so those are you know everything's a double-edged sword in this game right you've got to look for those kind of stuff but it can go a long way towards that preventative maintenance it's also got an FTP scanner so specific to the FTP process if a file is uploaded and sometimes that may not be malicious sometimes it might be your user downloaded a plugin from somewhere not from WordPress somebody told them download this plugin upload it to your site and you'll be good to go and they go to upload it and it did have a virus inside of it because they were on some random blog post or whatever this can look inside of every file that's uploaded by FTP and say yeah that's got injected code go ahead and quarantine that file so these are some good preventative things that come with CXS that can keep the server secure so that you're not having to go back and clean the pack sites that it's kind of doing some of that for you on the on set the last thing I do want to talk a little bit more about WordPress hardening with almost all schools and I would probably say at this point all schools WordPress is the most popular content management system PowerZip over 26 I don't know what the number is at percent of the web at this point it's a huge target it is probably not the only reason that sites get hacked but it's a big reason in keeping those things up to date and secure steps that you can take to hard and WordPress and make sure that you don't run into issues one of them is almost every hack attempt to try and log in is going to go to a site with .wp.pabmin at the end so this is a plugin called WPS hard login it allows the user to set their own custom login URL to be something like slash secure slash login slash Tim it doesn't really matter what it is but you can set in the plugin what you want your login URL to be and that way when people go to slash wp.pabmin they get a 404 page and scripts don't know what to do with that they're like okay I guess I can't get in at that point so that is a really good step because users can't get in there WordPress has a lot of options like I mentioned with the firewall blocking IP addresses and traffic like that if you install that plugin from the myapps page is that URL still reflected that's not going to be reflected but the WPF and in myapps will still work it's not actually going to that URL the way the way installatron works is that it's dropping a PHP file obviously it has accessible file system it's dropping a PHP file into the WP content folder and then using that to authenticate the user get them in and then it deletes that PHP file you may see this file at some point depending like if you've got a user still transferring their site or something because the file name is like delete me with a bunch of like characters it's like delete me dot PHP and it looks suspicious like I've seen this before and what that is is if the file didn't get deleted because they couldn't authenticate the user we see it if somebody goes to install WordPress and get in there before their domains transferred completely or before the domain got registered there was some issue but they were trying to get in well it's adding that file and then it's going but that's how it's getting in and since it's going through the WP content folder it's while it shows that admin link it's not actually using WordPress's WP admin functions and all to get in there are these good things to do to your site anything or do you think it's overkill? I do not think it's overkill no it's just being smart yeah absolutely if they have, if you're not overwhelming the user like obviously some noobs I would not give them a lot of extra stuff to do but if they've got a prominent site no and I'm big on the low overhead ones I'm not going to go into some of the hardening where you change WordPress's folder structure because it's possible to do that it's possible to have separate folders and now you don't even have a WP content folder you're telling it to look here for this and there for that it's a really way to have a strong WordPress install because all of these hackers are relying on your install functioning like a normal install that's the only way they can get a lot of hits when they're trying to find exploits so anything that you do to breach away from the norm is going to help you fly under the radar with this stuff but that's very difficult to set up you have to do HT access rules you have to change core WordPress files to say look here for this now look there for that now change this database thing to go here there are ways to do all of that and I'm not much more fan of things like just hiding the login just making sure that your username is not admin for the longest time where WordPress's default username was admin and so a lot of scripts will hit a ton of logins with the username admin if you don't have that username you're fine, you're good to go so the last thing I know we're coming up on an hour and this is a ton of information but it's important stuff I want to talk about is a service that we use called BitNinja this is a service I don't know when we started over a year ago, a year and a half now we started using them BitNinja is very much a firewall but it's a distributed firewall we subscribe to their service with all of our servers and there are obviously other subscribers out there and it's monitoring all of the traffic across all of those servers so if somebody's trying to hack into one on shared hosting accounts and their IP address gets blocked there every single one of your servers it gets blocked on as well and that's the same for every BitNinja customer so it's monitoring traffic it's a global blacklist and greylist and where it's monitoring traffic across all of the servers that are running BitNinja and then you benefit from that larger list of IP addresses it blocks a ton of malicious traffic before it can even get to the server it can't touch your WordPress install because it can't load anything it knows all those Chinese and Russian sites exactly, yeah it's a huge list of it it comes again with false positives so they have two options there's a blacklist that we can go in and do if they haven't flagged it and I find out that there's a hacker on a specific ID I can blacklist that address there's also a greylist and when something is greylisted they get a caption page so you may occasionally see a page that says are you a real user put in this, you know, string and then they put in the string and they're good to go and that's coming from BitNinja it's just a way to verify that we found some suspicious stuff with your IP address users might run into this if they're on Starbucks Wi-Fi for example that IP address might have a history that it's not their history but they were given an IP address that got flagged at some point in the past we see this with VPNs somebody logs in through a VPN VPNs are, you know, a nice hot thing for academics to use but they also have a history of being used for not so great thing purposes and so a lot of IP addresses through VPNs have a history through BitNinja to where they will get greylisted hopefully you get a caption page if you just can't get anything to load through that we can whitelist IPs and IP ranges through BitNinja as well so question about the servers so when NYU converts over to digital ocean are those servers also like are those just like your servers now so that they have this NYU has it already it's not specific to digital ocean it's a separate service that we just install and run on our servers is it already part of the future? that's correct, yeah additional costs for this made the decision that we were just going to run it on everything it reduces the load on the server, it reduces the number of, you know, malicious hacks and things of that nature BitNinja has additional tools that we haven't quite taken advantage of things like a web application firewall where we'll actually monitor the kinds of traffic and activity that are happening and try to block that in real time we tread a little bit carefully because as I'm showing you there's a lot of layers to this you turn on all those layers you're going to get a whole lot of false positives and so, you know, we are mostly using that for the firewall stuff and I can actually show you it's kind of wild to look at but you can see the activity happening in real time with what it's blocking now we have access to this there's likely a way that I can get you all access to your individual server into it but let's go to the server and I can go into incidents and this is all the traffic that it's right now been blocking so somebody tried to get to that URL and said somebody tried to get to this one their IP address had a mystery so this means it's showing a capture page to all of these users and if it's a script it can't fill that out so it's not doing anything they also recently added something where it's called a smart capture it tries to figure out if the user would be able to fill it out and it's likely a good user it doesn't even show the capture page it does a little like that and then it passes them on through and so they're trying to get away even from requiring the user to level it is that an X or a P or something like that and just do some more real time figuring out is this a real user will sometimes still see the capture page but they're trying to move towards using some sophisticated tools to figure out whether it's likely that they probably are a real user or not this is the activity across all of our servers and everything is seen that's suspicious that's right you can see the different things it also says things like honey pots where it's trying to attract specific types of URL structures and things for users and if someone hits it they're like yeah that never existed on the server and now I know that you were looking for it and it blocks your IP address so it's called a honey pot yeah the gray list is the global gray list of course those patches so these IPs are going to be captured so we're often in here looking at IPs for folks that they've been blocked of course we check the config server firewall but we also check bit ninja and so we'll white list IP addresses in here as well and IP ranges for your canvas can be white listed completely in bit ninja so that they don't affect anybody yeah that's the activity across all of our servers this is specific for that BYU server BYU 3 server and I mean you can see I'm scrolling through here and this is just still November 3rd this is the last 30 minutes yeah so yeah yeah you can start to sound like a temp oil crazy when it comes to security but then you see stuff like this and you realize oh wow servers are getting hammered by this stuff and that's why we really love this tool because it's taking care of a lot of that before it's even getting to the application you can do a lot on the application side and even at the server side but if they can't even get there that's the best we're a residential campus even with faculty on sabbatical or study abroad or tour browser or VPNs I've run the access logs and geo reference the IP addresses periodically