 Thank you So as said today, we'll learn about DDoS But not about what DDoS is because that's something that you all should know about But we actually know DDoS from the defender side mostly we hear about it I said in the news in the media or whatever some APT guys did did some DDoS some viruses some botnets But we always hear just one side of the of the border the defenders now We'll learn today about mitigation of DDoS or more specifically how not to mitigate DDoS first of all we Doesn't walk Excuse me. So first of all, we get we get some intro into DDoS What is the but what is all about something boring just one slide about it? methodology of walk of our method of delivering this kind of service. We are actually We've been attacking now for three years Via DDoS services to our customers that want to test their systems for DDoS mitigation or correct DDoS mitigation And guess what not all of them were so correct as we figured out before Afterwards we talk about some DDoS in the wild What is exactly going on in the world just some statistics again something boring and then we've come to the 10 most common from the book strategies that we Figured that you should hear about At the end we'll stay we'll have some Q&A of course So Myself Moshe Tzioni, I do security stuff, you know I'm managing a team of security researchers at Varing to we do something cool with defense But except that mainly I'm experienced with penetration testing and Ethical hacking and for as I said from the from three years of hard work about DDoS We are attacking and providing our customers a service of DDoS attacks That's the end of the shameless promotion slide. I will never you will never hear about it again So everyone gets DDoS Not in the correct sense of the world because everyone meaning you If you are an attacker you will learn maybe I hope that you will learn one or two Tactics it that you didn't know before if you're a defender you will know what not to do or what is what are the most 10 top top 10 common strategies not not to do or to avoid from and If you are neither an attacker or defender you just kick back relax and here's some Good laughs over someone else So the method the method of delivery is somewhat complex because we want to have control over our botnet We have a legitimate botnet, of course everything that I will Mention here is a legitimate and we are doing it as as Legit as we can we are we have our own botnet a pretty vast botnet all over the world all over the globe and the living this kind of controlled mechanisms for DDoS for us for our customers have some extra edges like We view exactly what is happening. We log all the stuff This is pretty hard to do actually when we are talking about botnet with thousands of computers that we want to control each one of them and Know what exactly we are missing and on top of that we have something Something more like a red team and a blue team. So the blue team is is incorrectly named blue But you can see spoke with a blue t-shirt so the blue team is is on site with a customer and Viewing all the logs not for mitigation not for telling him what to do just for taking notes for the red team and the red team Is is on the other side of the line attacking the customer with the boat with the vast botnet and together with the two sparks We have a complete image of what exactly is happening on the network or maybe in the computers of the Of the customer and through that we can we can have a good recommendation not just a good recommendation We know the site. We know exactly what happened. We can pinpoint it We can we can analyze it we can analyze our own boat net in terms of Improvement if you want to improve to fold the next time So together we have this visualization of the analysis and we can progress with the attack Moreover Some in the world they just use Mainly maybe it's a shock for most of you, but mainly by sans and most of the attacks are less than 2 gigabit per second I'm mentioning that just to acknowledge that DDoS does not have to be so large as the media claims Claims it to be of course. There are times that network bandwidth is the main main type of things But if we hear about some DDoS in the media, it doesn't mean that automatically It's DDoS by bandwidth by the network site. We'll get to it later Other than that we have reflection amplification the other this are two words that we hear a lot reflection is the actual acts of benefiting from Third-party that we want to we want to communicate with and the third party is communicating and Reflecting the so to say to the actual site that you want to attack amplification is using some kind of asymmetric Protocol or asthmatic behavior between the attackers and the defenders or the service So the servers will have to walk much harder in order to accomplish something and that's amplification Last most attack that we hear about on the internet and on the media does not require brains you can leave your brain out and then just attack with some some kind of a tool and Because of that most of them rely heavily on bandwidth consumption My point is that you don't have to be that lame you can't you don't you don't require Most of the like 90% of your brain But you need a small fraction of it in order to to designate and amplify your attack without Actually using so much bandwidth as proposed so Just just some Headlights headlines as I mentioned there is more there front-end we'll come to get we come to it at least once in the Examples, but everyone are thinking of websites that's going down by DDoS by some kind of a login to a bank or maybe the Double-double-double you site for the bank, but it's not exactly it if you can attack the back-end and we'll get to it other than that you have You have the the back-end work for you back-end is actually doing something not just Presenting a cashed page or something so you can actually amplify your attack by the back-end Keep it stealthy. They might be listening the magic of sniffing We all heard heard about it the sock team the sock manager is online and checking your his site all the time he's there with the magic of sniffing and Think of of amplification in a general way When I'm still when I'm saying stealthy I mean that use your own tools most of the attacks that you hear about and read Or maybe even you even if you have the source code for it you can read the source code and then analyze by yourself what the attack does and Attacks are mainly DDoS attacks are mainly very simple to comprehend because you need a small fraction in the distribution of the attack in order to complete it So if you know what you're doing you can easily believe me very easily write your own scripts And by that you eliminate 90% of the signatures that are residing on ideas is an anti-DDoS machines So amplification on the general term We refer to it as four pillars. We have the network attack that the usual suspect We have the CPU which is very limited in some some cases But and CPU was actually attacked by someone on 28 C3 Which I forgot his name because it's not my native language But but some very professional guys devise an effective attack over CPU Presenting a single a get or post request to an HTTP server and then Evaluating the CPU up on to 99% of the system by using hashes So CPU is again a very prominent attack that we choose to attack in the process The other thing that we choose to attack mainly is the memory memory volatile memory Everyone uses everything uses Volatile memory and we can use it to our advantage Think of it as everything that is done on the website if you have let's say a form and the form is undergoing some kind of a multi-stage You can actually do maybe part of the stages maybe all the stages and in memory residents will be very effective in DDoS and last is the storage itself you have a Some amount of storage On disk storage and even the IO buffer of the drives that are working very hard to complete the mission Okay, so last this is a true story At the request of the survivors the names have been changed will never Do shaming to any of our customers? You know what comes next out of respect for the dead The rest have been the rest have been left unchanged Okay, so ready set the actual result is face palm so every one of the of the The stories will be presented by a face face palm ratio. We'll have a scale of face palms. You'll see it and Did I mention at the end and the number one we go ten to one Number one. I can promise you that all of you will do an epic face palm That's a promise, okay number ten Number ten was common actually. It's it's less common nowadays because Everyone knows that Mainly everyone network guys knows that that's rubbish limit the rate of incoming pockets That's something that is met magic for network people to say. Yeah. Yeah. Yeah We have a DDoS like two gigabits of bandwidth. Oh, no problem. We have one megabit megabit of bandwidth So let's use only one megabit of bandwidth to upload then this way. We can't even choke the Bend the rest of the 99 megabit per second So of course you are nodding all of you. I see the heads If you have an incoming packets coming at you And this doesn't work and that's why The customer had a DDoS actually and that's why he asked the ISP and the ISP told him please Please rate the limiting limit the incoming packets into your service this that was the ISP talking and so we did and Believe he believed that he is sufficiently Mitigating the attack if we test them and he requested a test so we delivered And it was pretty simple liver because if you have The knowledge of how the internet works So you have a get request or you have let's say a get request a get request is something pretty Easy in size like one kilobyte. Let's say a little one kilobit Or maybe one kilobyte. Let's say and you request a file from the server if the file is sufficiently large You have let's say one megabyte of a file You can go with less by the way you can go with 200 kilobyte It will be an amplification factor of 200 times more think about it So now we can use the minimal amount of data that we want we want to upload in terms of and and then get it from the download itself meaning that Effectively the servers are choking themselves So the beauty of this this tactic is that will work always not only when someone will try to mitigate you So the mitigation is a of course a fail But I'll say that's the consumption by reflection, but it's an implied phase bomb Something that you all should know and it's and say okay. That's something stupid to begin with so maybe it shouldn't be on this scale at all So does Tom Lee Jones? number nine That's that's a beauty. We have a vendors that monitoring the sites all the time or maybe not the vendors itself Maybe this sock in this case in this story as I said a true story The monitoring was done by a third party That was the only job was to monitor the site Now when you monitor anything you you come to after 20 minutes of an attention your attention drifts away And I think about puppies or about your babies or about your friends or about I don't know what what you ate for lunch And then you forget about monitoring because monitoring tends to be something very very Very boring to do so you look at a graph and if everything is okay You leave it as as this as it but in this case we didn't leave it as as as I presented it We actually we attacked the site and the site was down now when was when the site was down It was a surprise attack meaning that the the demand that requests out the test was the was the Security officer for this customer for this organization and he didn't let let know of the it and the third party that is doing this kind of test So when the site was down he sees the site is down You don't have to be an expert to see that and and then he's just waited and waited and Where it's a more now you will ask yourself wait a second maybe an email maybe a phone Maybe someone will pick up the line and say to it or say to the stock listen guys We have a problem, but no one did why so so because two things it was pretty Pretty quiet. No one called it in and we'll figure out why in a second. No one called it No one got an email about it and the monitoring vendor wasn't aware of anything going on wrong on the network So what went what went wrong exactly so first of all the vendor so as I said the logging system and the logging system and The logging system was looking a bit like that You have a pic you have something like a bot the I don't know what the colors But the most of it are are susceptible as botnets and it's it's in a really relaxation somewhere So first of all, that's the screenshots that when the Security officer called the vendor and asked say guys. Do you see the site is everything is okay? This said say to him and send him this screenshot. That's the actual screenshot from the vendor So everything looks I don't know suspicious, but let's say it's relaxed after this peak. Let's say they that way Problem is that a they didn't check the site actively they all they needed to do is just Click on the on their favorite browser apparently Internet Explorer and then go to the site and If you go to the site, you see the site is down. You don't have to be an expert or a sock team to see the site is down So they didn't do exactly that now for this for the second question Why doesn't the IT got any calls or emails? Because and that's in addition to this epic fail The actual bandwidth that's used for the banks servers this customer service was used for the HQ traffic Inside a corporate so everyone that wanted to serve at that time I'm talking about hours of of of no surfing and no week of no email for the corporate sites No one could email and no one can can use the VoIP phones on the So that was a bit of fun and this kind of face palm is have a face palm You should know about it. That's easy And it's cute, right? Oh Okay Going going to number eight Okay, so I've mentioned it before back-end servers are not important to be protected against leaders Again a very serious assumption. We have to consider it heavily Back-end servers are not important Hey, that's bullshit back-end servers are always important If you think that any servers are not important don't use those those servers. It's back up so if So if those servers are not are not needed to be protected against leaders So there's somewhere and this notion of of thinking was coming from the media I guess so everyone is reading about this kind of let's say for example Supposedly Bank of America was attacked by DDoS and and the first thing the thing that you hear about is Everyone is tweeting about the login site for it for this bank. Why is not responding? So the media is responding accordingly. She's right the login site for the site is down. Oh, man But actually what what can happen is is Something much more vast than that But you didn't you won't know it unless you are sitting at the stock and know as exactly what what won't and Maybe it is maybe most of the attacks do actually hit the front end But it doesn't mean by that that the back-end are not important So in this case we try to map the site We as attackers want to attack the back-end because of this notion We want to attack it and want to know what to do how to designate some kind of a back-end server So actually that's easy because that's I don't know about easy there actually But it's it's very common from pen testers to do when when you check a black book site You just go to a site and try to assume what is going on under under the covers and see what exactly is happening But you can't really see because you're not a developer. You're just testing the site as a hacker or a penetration tester Now when you pent as the site you try to figure out where is the database? What is going on and that's that's pretty easy if you get a query for somewhere some search Supposedly is an SQL nor SQL server or anything like that maybe a file even but it is a data set You can query it you can you can do some work on it and that's why this notion is pretty bad So the back-end servers Excuse me Some problems there. Maybe someone is doing me now. Okay so in this case we have Like to guess something and that's a pretty easy guess if you have delays if you have Inappropriate delays between searches or between forms you can assume that something is going on at the back end not in the front end The front end doesn't think hard about something. That's the whole point So if the site is thinking how about something that's the back-end So you hit gold and you profit from it when you do need us and this face palm is a kitten Because you have to be a kitten to do that That's so 90s of you Okay number seven we had a Actually a pretty good customer in terms of relationships and he really respected our work that we did with the one of our DDoS And then he called us again, but this time he bought a shiny new box and This shiny new box cost a fortune of course, but any any box does and And When you buy a very very pricey box you may be connected to all of your servers We said before that the back-end servers are as important as the front-end So protect all the domains connect all your sites to it connect all your corporate machines your I don't know Maybe your bank. I don't know everything that that he could figure out. He connected to this box Now you can set yourself. So what's the problem? So we get some extra stuff or not What is really getting is protection from DDoS against all of those domains and When we did it when we did the actual test We tried to figure out what is the box is supposed to do now reading off some brochures of this new box that we didn't know before It's pretty nameless to be to be honest and So the mitigation is pretty Pretty unique they have like the strategy of of the The strategy of it is like thinking about what is going on the box is just sitting there for 20 seconds when it's Suspects something and after 20 seconds after we suppose it's building some kind of a model for the attack And then he it tries automatically to figure out how to deflect the attack This this mechanism is usually a Proceeded by something called draining of the lines when you have all the lines that are susceptible You just drain all the lines You just drop it all and then wait for new lines and then by the model that you built The box can decide what is going what is going in and what is not going on so when we try to attack we were we were very scary because Always before the attack we before attacking the site we in we gather information about something We try to assess what exactly is happening and mostly the customer does not need to respond to anything We do it non-intrusively and try to figure out the technological Benefits and technological cold trips that we want to overcome so in this case we just read the brochures and try to figure out some statistics ways that's the Defenders could do and try to circumventant So we didn't have something really good Let's say to to be honest when we started the attack, but exactly 20 seconds 21 seconds later All the site went down From all of the the world not only that six minutes later the guys from the blue team gets a call Listen guys you have to stop the attack right now someone. It's very angry Why what is happening? You knew that this test is going on We just we just shut it off. It's okay We take about two minutes to shut off an attack and start it if we want to so two minutes Let's let's let's wait a bit it took about one and a half minutes And then the attack attack was gone, but none of the servers was responsive. So So apparently they were very very stressed about it I Guess not only that You think you thought in that's the end of line not only that is that Not only the domains that we we thought that we are attacking was down all the corporate network was down of course because all the main was was covered by the box and Because of the behavior and of this kind of shutdown the complete shutdown of Most of the computers that they are involved with the internet traffic some back very back-end Let's say second to you back end of the corporate side. It was trying to communicate to something very crucial to them without Expanding exactly what but let's say some something went very wrong on the corporate side and they lost some Some hefty money about it when they were drilling with with transactions. So That that was pretty embarrassing and you can figure out exactly what happened when the box was seeing all of it By the way, I didn't mention monitoring the monitoring itself was shut down because it was connected to the box So everything went dark in a second and everyone was stressed the phones weren't warrant voice over IP So they they were PSDN or I know dials and then they dialed each other and said okay guys Let's stop it and we stopped and and then I think it took like seven hours of actual mitigation actual try to Bring back the servers to a normal operation this phase palm Is like that okay number six That's a good one I love it it happens all the time by the way It's not it's it's pretty common to encounter nowadays when you have many vendors that provide some kind of a cloud-based T just mitigation We don't trust the vendor. That's what they're saying all the time. We don't give them certificates Shamely that's what's happened when when you rely on third parties that you don't really know It's not nothing like the big five I don't know providers that provide some kind of a box of or maybe a cloud-based mitigation But you go with something maybe a startup Maybe something in in its youth when you want to support them and maybe it gives gives you for for free doesn't matter why But it gives you some kind of Another layer of security when talking about DDoS So this kind of defense is is pretty awesome for us because if we know that this kind of operation is going on Or we can assume because of the nature of the relationship with a vendor We can say okay, so HTTPS is not covered so we go HTTPS and we when we go HTTPS it it becomes even even better because THC the hackers choice did a terrific research on renegotiation for SSL and Renegotiation SSL was proved by them to be so effective like 15 times more for the CPU of the server Harder than other than your own work when you when you try to push your CPU to the most and it can be actually Pretty pretty simple to employ with only two computers But if we're talking about a very large bank you two computers will not be enough you need like a hundred So if renegotiation is is present will talk will talk will attack with renegotiation and it will be done It's pretty hard to counter this kind of thing unless you have HTTPS and you know what exactly is going on on the line and you can read it and the second thing that not only the vendor Can protect you you can't see anything because you are not actually processing the data now You can say and you will be right that you don't want to anyone else to see your data, right? But the thing is my point is if you are not trusting a security vendor, don't work with him. That's it That's a pretty simple advice and that's dollar faithful the the first one is The walking with a security vendor that you don't trust the second one Is the visibility that you don't give yourself if HTTPS is not actually terminated by anyone, okay? We need big data. It wasn't positive. Just rank some water So we need big data, that's a big world. I don't know if you heard about it before But big data is going to be a trend I tell you So we need big data. Let's collect it all like Pokemon's we have big data We have this we have this network device. We have this network device and and we want to collect it all So that's great for you, but when you collect it all you have one big problem. That's storage space and Let's be honest some protocols like PCI Tells you to save all the data. That's something that you have to do So maybe it's not all it's not only that you are wrong with your assumption of just collect the data Don't do anything with it. Just collect it responsively and when it what in doesn't happen responsively you have logs and overcoming some storage boom and Silo needed You result in a complete lockdown. You don't have to do you can't do anything on the servers It's very hard to operate without let's say 4k for minimum if you need something from the from the disk And maybe the IO itself is breaking down But the most susceptible to those attacks are not servers themselves because servers can be Cycled through their logs and and most of the network guys knows how to do that and infrastructure guys But something that is overlooked many times is the networks network switches and ideas IPS firewall as well and maybe the anti-dios mitigation machine that you have or maybe the VPN and In this case that I want to mention it was the IPS The IPS I live the vendor of course alone But the IPS wasn't cycling through its logs and the it over come it Got a storage boom and then it just disconnected the whole site So even when they wanted to mitigate the attack They couldn't because the IPS was down and because the IPS it was down No network it couldn't can be reached to the site itself and they couldn't connect to the IPS because there was no storage room on the IPS so They couldn't fix the problem and they needed to get to some bunker and press the button. I Think the correct button and that's a face palm that is done by third party number four we are under attack and Forced the on-demand scrubbing service. First of all, there's no such thing I don't know if you heard about it, but on-demand scrubbing service scrubbing service is something that needs to learn about your traffic You need to know is what exactly to scrub unless you want to teach them and that's pretty much impossible If you are talking about a dynamically changing site, but let's leave it Let's say there is such a thing like on-demand scrubbing service in this case We have learning mode learning mode is something very beautiful But let's see just switch switch off this kind of responsibility or this kind of mitigation and just we switch it on when When you want the mitigation to be actually occurring now in this case, we had it was an Australian it was an Australian customer and it's not his fault and he used some kind of a on-demand scrubbing service so such to say and the attack was Legitimate traffic if you actually know how the how legitimate traffic looks like you can mimic it pretty easily if you use your own tools with other more robust tools you can do that in a good way and Last you have to read the manual Please do and they read the manual and and discovered that the manual itself of the vendor said that's no problem We can learn on-demand I Don't I don't really familiar. I'm not familiar with with this notion, but let's say it's possible and We'll leave it for a minute from now the vendor response was epic Now the story went like this. We had the the side of the customer We actually attacked it and then it was pretty okay they we try to to make A wrap up a ramp up of the attack so we can analyze what exactly is going on So so we wrapped up and wrapped up and then we and we reached some kind of a limit We said okay, it looks fine Do we do you want us to continue with a ramp up and he said no I want to test the on-scrub on-demand scrubbing service and in the second that he switched on the on the on-demand scrubbing service All was shut all was shut and nothing can be accessed And it was pretty impressive because not only that he does like that He didn't have any kind of control over it He can't switch it off you can you can maybe try to DNS DNS it and he tried to call the vendor now now Usually we do it overnight because we want we don't want to hurt customers that are actually using the site true This is harmful to us because we want to actually mimic a true Not off-peak, but but on peaked how was when we want to actually attack via during Christmas times, let's say But in this case it of course it was a large customer that they didn't want to actually hurt the customers So that's negligible. So he picked up the phone and called the vendor The vendor is a 24-7 hour someone that is picking up the phones someone picked up the phone and and looks and and heard about He he was going on like someone that actually woke up from his sleep We met we imagined that he were if we jammed with what if we didn't have any kind of communication with them except Voice so he said to the vendor listen, I have I have to have your help My site is down. We are six hours to to Actually the morning we six hours from morning and when the morning comes the customer will try to exercise and no one can currently no one can't but it's Again, it's it's controlled. So the vendor said wait a second I have to consult with someone he consult with to someone and enough half an hour at past and then he called back to him and said okay Okay, we'll we'll try to figure it out another hour and then the own scrubbing service was down But the epicness about this is the answer the complete answer of the vendor the vendor said when when they were asked Why didn't you? Actually told us that we want we need to calibrate the the I don't know the scrubbing service that that was the the thing that he said he said another thing next time by our service of Calibrating your site That's a good vendor and that's triple face palm Why triple because failing at protection working with a vendor and the pool vendors answer We are approaching number one number three So what CDN is not dynamic? Let's enable its CDNs is the condo the distribution network is pretty popular nowadays in protecting Against DDoS and that's pretty cool because it actually walks if you have a spread out a CDN you can actually mitigate network-based Maybe even other other based attacks to your system. That's cool, right? But CDN as a culture it's called static and dynamic and it's in usually marketing But most of the CDNs are static static means that it can pull off Some requests for your site from static data not something like searchable queries and stuff And then you can you can cash those data's those data this data on the CDN and via that other customers can benefit from the No lag at all from the CDN Now when a CDN tells you there is no not dynamic which is good Which you should know about and you use your site which is dynamic on this CDN It can be very very devastating now why? Because the thing is that is it works like that if you have a CDN and the CDN is is getting a request from an attacker or or just Just a regular user it asks for some kind of a landing page Let's say if the landing page is on is is was visited by someone else when the vicinity of the CDN It will know how to respond and then respond from its own cash If it's not was if it wasn't asked by someone at At the given time it will ask by itself Issues a request for the origin the origin is the actual domain that no one should access except the CDN It will ask the origin about this request get it back and then get back it get back to the to the customer to the user An attacker will do the same attacker does not know we'll get to it in a second Does not know where the origin is and how to ask it directly need to ask the CDN about something in the DNS The DNS request that he's issuing is giving him the CD closer CDN That he can need to think about when he's talking about the site So in this case we have an origin That is getting so many requests So that's wrong if you're using dynamic The the thing is that dynamic is pretty easy to do us because if it's not dynamic CDN Meaning that he will need to issue each and every one of the request if it's a different parameter or value for those For those parameters it will need to issue each and every one of that All we need to do is just requesting from the CDN data with parameters They didn't know the same page other parameters sometimes even parameters that doesn't exist on the site But it doesn't matter he will issue a command and issue a request for the site because it doesn't know the Correct URL for that and it doesn't have it on cash So one of our customers did exactly that use the CDN static CDN on a dynamic site and we easily did us Did us his machine another thing about CDNs Which is not exclusively for dynamic but therefore dynamics is much more devastating it is that if you monitor your origin Many of the CDNs don't doesn't give you actual UIs for monitoring like traffic like did us you have your own machine and you monitor that as as good as you can and maybe some some kind of tools that you get from the vendor of the CDN and Actually, when you try to mitigate it On your sites you actually many times can't because you can't see the actual attacker and you can't Blacklist the attacker or something you just see requests from the CDN and if you block the CDN itself That's good for us Okay, and that's deserve Distributed collage of fail spawn Number two again CDNs CDNs is pretty exciting for me because no one knows how to protect with CDNs And when someone searches the web in this case some obscure site named Google We use it for finding how to protect Protect CDN origin. That's the best phrasing that we can we could walk out and the the first one after the Akamai's one which is Advertisement You have the how to protect your CDN origin server. Let's click on that. What do you say? Okay, we clicked on that and that's magic and then we have several Recommendations how to mitigate and how to protect your CDN your CDNs origin and Just magnify it for you. This is a simple trick and it is also the best solution Create some random long set of alphabetic numeric characters and use that as a sub-domain Even more so can it be guessed? Yes, but highly unlikely. Can it be leaked? Yes, but again highly unlikely There was much rejoicing reading those lines Why so? Let's talk about it. So The tactic is like that find other sub-domains Translate it to IPs scan the hell out of it. It's a CID out slash 24 6 16 Good good chances there, but it's not bulletproof. You can you can actually miss a lot of those origins From the actual name But something that is much more probable to find out is the who is service who is service never forgets means that it is Forgetting because who is a dynamic but you have who is history is online and you can check who is history? History domains and then when you check it you figure out and we figured out in this case that the if you want to some some of our customers actually bought this kind of service and try to protect against it and We figured out how we can know what is the origin you want to take to attack the average in everything the static It's pretty hard to to attack static sites Unless we have a sub-domain or unless we have another origin with that We know that will will be hurt by it by our attack and we can't just attack some some obscure Subdomain we need to know what we are attacking to some extent of course Because of the back end Doesn't really determinist isn't really deterministic all the time So who is never forgets so we look online in this case I'm just giving an example for a malicious infected site named big.com and And this site is is covered with a who is history that you can pull off from view DNS in this case But there are many other services for who is history and then which actually thought about it when you buy a CDN You don't actually change the the last IP of your site. You're just giving it out to the actual CDN provider and Then if you look up the who is history the last one or maybe one one before that you actually hit jackpot because This is the IP of the origin and that's exactly what we did And that's exactly what happened when the site went down and the customer said to us that he did Doesn't see anything on the CDN That's a wrong number one okay that's something personal because in this case the Again the the actual the actual person that bought the service from us was a security officer of the organization a big organization in Israel in this case and This big organization was very let's say politically heavy. So the IT was very Vicious in their attempts to block the attack So As I said during this week of research before the attack that which is non-intrusive We tried to figure out how the mitigation works from low low impact attacks and stuff without actually impacting anything We tried to figure out what is going on and we saw that the mitigation that they they invented was Amazing like we have one server and we are taking from this server and that's and from this on the blacklist Is not following just on those on our servers But all of our servers worldwide or just a branch of our service in some place in For in this in this example in the United States and we try to figure out why Why is that if you are talking from one source? How come you can mitigate all the sources from this area and the and the susceptible answer and in this case the correct answer was that the mitigation work like that if you see an IP if the Seam rises up and alert and then an IP is raised Then it doesn't block only the IP Remember he wants to block us not another real attract attacker And he knows that we are limited with our with our resources because because as I said we are legitimate as possible So when he tried to make it mitigate us he said to himself Okay, you have some botnet that he least or bought somewhere and that's this will be some kind of a cluster of IPs which was correct in this case most of our servers do like that So he didn't just block us. He blocked the whole slash 16 site I'm hoping that you are not cheering for him because so in for example in Germany we you have like 116 million IPs in Israel it's much much smaller, but you can extrapolate so If you have 116 million IPs You have about roughly of course 180 1800 class B ranges, right? So If we have only 1800 IPs that we need to hit and we can let's say spoof the scene We can actually block all of the customer's nation if the nation is the probable customer in this case insurance is Proval to to come from customers from this nation in this case as well as I said is very small and So we troll them a bit so So think of a monkey just typing IPs like crazy and then it just blocks all the nation by itself and he can't do anything about it because 15 minutes after it after the the actual attack they have started it's inflicted by all the nation and Before the 15 minutes were off like like half the time of that. He blocked himself. So he couldn't see And now you can see why it's my favorite Now remember what what I told you about the mega face palm that you will give me remember So that's the one but I don't see you face bombing. So I Brought my own picture Actually, I think it's kind of a tradition. So if you may face palm yourself for a second, that's okay, right? Let's say it's okay Thanks, okay, so Corrective thoughts about it Maybe the most important one is test don't be don't be afraid to test many of our customers didn't know that such services Is it is in existence? And we are not the only one that providing this kind of service? we know so There is no magic pill you have to be an architect to understand that it implies not only the front end it applies all your network All your computers maybe even more than that. Maybe your phones and emails But but test it and and you have all the money in the world all the toys if you just deploy it when without thinking it will fail you so please Be responsible about it and one less promise If you won't do that you can be evaluated to this presentation in the future Thank you. We now have about 10 minutes for Q&A so as always please line up at the microphones or Use the internet to ask on ISC or Twitter We have a person reading out your questions here And if you leave as always, please be very very quiet because the talk is not over. It's gonna go on for 10 more minutes So please be very quiet Microphone no, no, you're not hearing he's not hearing any questions Okay, the internet You need to switch on the microphone Should I ask my ISP before I did us or not? Oh That's so if they're if anti DDoS is included in the So that's partly a legal question and partly a tactical question Let's say for your sake of evaluating your security. I won't touch the legal sections because I'm not a lawyer And if for the legal section, you have to consult your law department If you have so if you have such a department department for the other part It depends on what is your focus on the testing if you if you are focusing on testing The system as a whole if you're looking at the all mitigate all the mitigation factors that you put into place In my opinion, it's important not to notify the ISP possible But if it's not possible, of course notified Microphone number three, please Hey So there are techniques that for example cloudflare has this project called railgun where they div the websites and Sort of not request the full web. So I don't know really how they do it But does this have any impact? Can you see this when you did us does this help at all or is this just? Yeah, okay Usually we test with black box. It means like pen testing We don't really know what exactly is happening on the other side except what what the blue team is being fed also seeing by the customer Such examples as cloudflare and others are examples of of partly participating factors into the attacks and the testing because they Didn't provide us with much of the explanation that we want to actually I'm not talking about I'm not talking about cloudflare because actually when I when I think of it We've never tested something with with cloudflare in mind. Not not deny. They I know of It wasn't a factor actually And let's say other vendors similar to cloudflare has approached us and and and usually Put up some difficulties. I'm not talking about technical difficulties more political difficulties Let let me know when you are doing it the legal stuff is not correct You can't do it and you have only ten minutes of time and something like that So something like well done and similar is not employed by it haven't been employed in our testing yet Hope it It answered your question Microphone number two, please how often do you find your customers well protected when you get there the first time? Let me let me put it that way We are we when we conducted tests It is important to say that I'm not I'm not continuing these tests anymore like two months from now. I'm I'm I'm off When we conducted these tests We actually did we took a length span four to six hours from the customer and through these four to six hours we actually provided Usually One one attack per hour we we tested some some kind of attack that we have in stash that we Actually prepared before according to our research and if evaluating by overnight if we have Whatever a number of attacks that we have overnight like a relic attack I will do but on an on a very lengthy like a lengthy span We have More than 95% of success So most of the attacks unfortunately are not well protected That and that answer the question. Okay, and now the internet please You said you just scan the whole slash 24 slash 16 even would IPv6 make it better Think of a whole slash 56 and random IPs. That's quite something to scan. Yeah This is good, this is great, but I am I'm not familiar with any bank that is working with us Or any bank at all that is moving to IPv6 as a whole They're just using the IPv4 and the IPv6 in top on top of that number three please So given that a lot of things are on shared infrastructure with Amazon and Cloudflare and stuff like that How do you make sure that whatever you're D dosing doesn't cause any collateral damage with people who are just innocent bystanders? It does it does but and and it's it's probably If you're a friend to to to not not customers of the actually you're referring to actually customers or clients of the customer So, yeah, give it like someone on Amazon. You're D dosing like them for something and Like something on Amazon goes down. Okay, so shared infrastructure as a whole is a whole different ballgame We have to consult legal and the SLA with Amazon and Azure and others Are pretty different from one another Azure let you do stuff if you let let them know in advance Amazon as far as I as I know of Doesn't let you do anything. It's pretty strict in terms of testing Think about it. It's pretty massive not to test your own site But again, it's it's considerable when when you're thinking about the shared infrastructure when you have Amazon or AWS or others So it depends on the on the hosts and what what are the SLAs with them? Number one, please. So, thank you for the nice talk, but can we change the roles? So I would like to To hear you a bit elaborating about what you would do if you have to run mission critical infrastructure And how you would protect it? I'm not a genius. So so for starters I I won't presume that everything that I will say will be holy It will be holy in another term, but I know that everything that I will do I will try to architect not just design a mitigation architect and mitigation a Redundancy as much as others know Nowadays how to backup and how to hopswap something it's it's pretty much the same when you're talking about redundancy and Things that need to be standing by and how much leg time that you can you have because it is can occur It doesn't it isn't There is no magic pill as I said nothing that I will provide you with a with a complete architecture will not be fail safe But it will be epic fail safe and and and and on top of that Test your systems any time you want you can be the greatest developer you will actually pen test your web applications, right? So that's the same thing if you architect something test it put it to test And once again the internet plays Are there any particular solutions or products recommended or that are particularly bad? No number four Hi So first of all the photo was not really appreciated. I mean we're not here for your amusement or And I think it's it's a kind of bad norm and now moving on to the question Have you given any thoughts to attack the saturated link without sending any packets to? the target First of all how to like legally do that and if you've ever had to do it or it was never necessary because the site fell down for other reasons Let me put it again If I if I understood correctly your question is how can you actually do some damage without? Going on to the on on and on with the traffic. Yeah, so you can be sending traffic to sites IP is actually that Sarah link with your target so that your target does not observe any traffic and so that you do not actually Did always any of them, but So I mean they do not get tons of traffic Individually, but the whole link that they say will be saturated Which would be nice to have because then you could see how? Your path actually how resilient your path is to dose attacks. I mean it's been described in academia But I don't know if you use it in practice Okay, we didn't we didn't do something like that before personally, I It's hard to think about something that we we can deliver such an attack with Without anything unless we have an exploit for that like like like I said in 28 6 3 There was an exploit exploiting many web web application service through the their hashing mechanism But unless we have an exploited a designated exploit and we talked about Generalized stuff not exploits a pair of web application service I don't think that's possible, but maybe it is in some Situations when you have a database that is working very hard on crunching something and you can do pretty much the same Which is the equivalent of exploit as an exploit as I see it Not exactly so you're sending traffic to IPs that are not related to your target except that they're being hosted in the same data Center say oh that you saturate the link, but your target doesn't see any traffic Oh, you mean so excuse me You mean that I'm attacking another sub-domain and through that is impacting the actual one that I want another IP Yeah, it doesn't have to be related at all to your target Yeah, it that happened a lot and that happened a lot when we when we got confirmation from the from the customer Of course to attack other domains just for a second to test if the infrastructure is shared for any means You have to have some kind of a shared infrastructure, of course Maybe the same host many maybe it's networking on some sites And the one example that I said about the shiny box. It was in the UK That that's exactly exactly what what happened on the tier two back end the tier two was attacked It wasn't our target, but it was attacked So it is possible through other sub-domains by by definition in that and Unfortunately, we are out of time. So please once again, thank Dalmos