 Yeah, so with no further ado, I'll give you your speakers. Hello everybody, thank you for coming at our session. I'm Masahar Paket-Lustin, this is Olivier Bilodeau, we're both from GoSecure Research in Montreal, and we're gonna talk about behind the scenes the industry of social media manipulation driven by malware. We have a lot of content, so we're just gonna dive right away into it. So, this is a four year long investigation, and it started in 2015. At that time, Olivier was working at ESET with Thomas Dupuis, and they were both working on malware called Linux Moose that was using infected devices to proxy traffic and go to social media sites. Then I came in, he switched jobs to GoSecure, and I came in as an intern, and we were looking for a research project to do all together. And that is when ESET shared to us a second version of the malware, Linux Moose, and we partnered up with my prof at that time at the university, prof Descarietsu, the university of Montreal, and my tax, which is a non-profit organization in Canada focused on research. And we decided to study that second version of Linux Moose. First, you must be wondering what Linux Moose mean, right? Like why are we calling this buttonette this way? Well, when Olivier was studying that buttonette in 2015, they found in the binaries the string Elan, and Elan is Moose in French. And since we're Canadians, they wanted it to give that buttonette a Canadian touch, although probably these guys are not Canadian, or we don't know. And that is why they called it Linux because it's targeting Linux embedded systems and Moose for the string. Linux Moose is an IoT buttonette that conducts social media manipulation. What is social media manipulation? Where basically it's just the process of creating fake accounts online and then using them to go and like and follow other people. We have a good story of an example of social media manipulation that was done by a journalist in Quebec called Émilie Bilodeau. So she decided for her investigation to become an influencer by cheating. So what she did is she created that fake account called The Pretty Runner and started to take pictures of herself running. And then she used various techniques to increase her following online. And we met with her after she published and she told us what she did. So she did follow and follow like for like. She also joined pods of influencers that like each other to give themselves more visibility. And she said that she had to bought fake followers to give her some sort of a base of visibility. So once you have like 10,000 followers and you're more credible, and then afterwards you start and you try these different techniques I just mentioned. Today what we're gonna focus is on this technique buying fake likes and follows. So it's the process of creating follows and then selling them so influencers can get popular with fake visibility. So this presentation basically is based on, it aims at sort of mapping the ecosystem of social media manipulation. It's a four year long investigation that started in 2015. It uses various investigative techniques. So we did traffic analysis, reverse engineering. We also went on forums, some sort of to understand what was going on with these actors and what they were talking about. And we sort of create that map of all the actors that are involved. And the ground basis is Linux Moos, which is what Olivier is gonna start with. Yeah, so again, like Linux Moos has been something we talked about for a while, but it's IoT button it. Now let's map it into our ecosystem. So it fits right in the ground, you know, far away from social media because it's only proxying traffic to them. You'll see that map evolved as our talk goes on. So basically, Linux Moos, we say, routers and internet of things. What we mean by that is embedded Linux systems with a busy box user land. So it's a subset of IoT, but it's still clearly a big subset of it. It has a worm-like behavior, so it spreads through telnet credentialed brute forcing. And it's main payload, or the reason why it does social media manipulation, it's offering a proxy service, which means that it leverages the reputable IP of people who are infected. So ISP IPs are more credible to Instagram or social network than data center or enterprise IP space. So the proxy is a good way for the malware operators to kind of separate themselves from the intent is not clear in the malware itself because it's only a proxy service. You need to detonate the malware and look at what it's doing to get an understanding of what the bad guys are after classic malware technique or deception. In order to catch it, and I'm gonna really go quickly through this because we have a lot of content, but we deployed custom honeypots. They were basically like one side, an ARM virtual machine where we could detonate or run the real malware. On the other side, we had a fake front end that would look like an IoT or a router in our case. So we kind of did both so that if the operator scanned us, he would think that this is a legit infected device. We deployed them all over the world and it yielded nothing. It cost me money and it was time consuming because it didn't change anything. We did this an HTTPS man in the middle because all that traffic was HTTPS. And so we redirected the flows to a MITM proxy project which is an amazing project for that kind of work because they do good raw logs of the HTTP traffic once you break the HTTPS. And what this slide is about, we didn't break or invent a new technique to break HTTPS. The thing is, we anticipated or expected that they would ignore certificate errors which they did. So we were kind of glad about it. And so at that point we had the raw traffic. We're really nice. And so Masera, what did we do with the raw traffic? Yes, so at that time when we were studying, what we had was several infected host used by operators, HTTPS traffic and plain text, yay. CNC traffic and publicly available seller market. So that is when we presented in 2016 at Black Hat Europe in London. We're not gonna go more in the details of the analysis that we've done there because there's the documents there that are published online that describes everything. But there's one thing that we hadn't talked about at that presentation that sort of led this presentation and it's a feature that was within the infrastructure of Linux Moos. So Linux Moos had seven white listed IP and we say white listed because these were IPs that had the right to use the buttonette. So they had the right to push requests through the infected devices onto social networks. So at that time when we noticed that, we were like, what are these white listed IPs used for? So our hypothesis was to think that it was a reseller model so that one white listed IP that was just a server somewhere that had access to the buttonette could have been sellers of Twitter fake followers and when they would sell it, they would just connect to that IP and just push the request through the buttonette. So we thought that, well, we wanted to find out what these IPs were meant for. So we decided to conduct several traffic analysis. So we used JupyterLab, it's an awesome open source tool that allows you to sort of use PCAP data along with MTM proxy logs that correlate them together and just use them for data frames and graphic. And we sort of developed that methodology of various variables that could have helped us understand what these white listed IPs were meant for. So we looked at Honeypot used per white listed IP. We looked at websites targeted, TLS fingerprints, but all of these didn't yield anything. It was a lot of analysis for few findings, except from two. So two variables told us what it was. It was, so the variables were account created on social networks and accounts followed on social networks. So basically for each white listed IP, we could find that they had their own lists of fake account, but they were actually all following the same account. So the potential buyers that we call. So we sort of figured out that these IPs, these servers were meant for fake account management. So where do we stand, right, at that moment? Well, we knew that each white listed IP addresses had their own list of fake account, and that was the meant, that was why they were there. And additionally, what we did is we scanned them and we saw that they were running Windows servers and that we talked to somebody at OVH who confirmed to us that they were using a remote desktop protocol was actively used. So we wondered, right, what kind of piece of infrastructure could glue all of this together? So you have Windows servers, RDP Open, so people connecting, and we know that they're managing fake accounts. And that's when automation software appeared. And so automation software is like we started looking at it like what would fill that gap, right? And we found out about software that exists that kind of turns social media around in the fact that what you're looking at when you use such software is you have a grid layout. You have like an Excel spreadsheet where your fake accounts are there and they're mapped to a proxy. And we're going to go through this and you're going to see this. But now I just wanted you to have an image in your head and we're going to map it to our ecosystem, which we love by the way. So it's in the three near the moose because automation software enables the moose, which is basically a button of proxies that it will leverage. So we tried to find the software that would be powering Linux Moose. And we based this research on user agents. So they were both using the fake accounts and the interaction with Instagram. We're both using mobile and desktop user agents. And although Linux Moose supported many proxying mechanism, we knew or we saw that proxying was the one that was used. And so we found different features from these vendors. Do they manage thousands of accounts or a few accounts? Proxy type, user agent? Can you customize browsing patterns and all that stuff? And also, we found different business models. So they all charge money for their software. By the way, their software is heavily protected. And as you'll see, some of them sported unlimited accounts. Some of them you would pay with one-time fee, et cetera. The first one we looked at was Gram Dominator. So Gram Dominator and all the Dominator names are very common. They have for everything. And so legitimately, you can pay with a normal credit card. This is really foreign for me as a malware researcher, where everything is like bitcoins or really shady or hidden. All of the social media stuff is still PayPal accounts. They probably declare taxes and all that stuff, which makes it more interesting and at the same time sad. And we should do something about it. But it has the features that we needed here. We highlighted proxy support, in which they mentioned public, private, socket. You see they're clearly not that technical, but anyway. They also support several Instagram account. So diving into the binary, I was looking for user agent strings. And so I found that they were generating user agents on the fly. And so looking at the code, they were really combining, like Android version, DPI resolution. And this is exactly what Instagram looks like from when you use the Android app. So they mimic that portion, but with a generator so that a single fake account would get its user agent. This would be put into a database. And then for that fake account, always the same user agent would be reused. So we found they were pretty clever. And so we looked at it through PowerShell. We kind of loaded the library and asked a few names to see what it looked like. And this was really similar to what we saw from Linux most. Let's look at another one, so social dominator, which is, again, under the umbrella of the dominator trademark. And what I want to highlight here is that grid layout. This, we only have one account, but it's because it was our own and we didn't have many accounts to put in there for the screenshot. But so you see the username, proxy, and it will gather the friendship count and stuff like that. You also have import capabilities, export capabilities. So it's really meant for scaling. And you have a log of everything that's going on down. This software supports Reddit, Tumblr, Facebook, Instagram, Twitter, Pinterest, Kora, and even Google Plus. RIP Google Plus. But so you see that they're into this business for real. I think this one uses the Chrome embedded framework. A lot of them use Chrome embedded framework. We have a little table that will summarize all we've found about the different software that we analyzed. I'm going to go really quickly through it because we have a lot of stuff to cover. But you'll be able to get the slides and look at the table on your own time. The other one that is interesting and needs because it's kind of a story in its own is Follow Likr. So Follow Likr uses a weird discontinued packer called Jet. Jet compiles Java into native code, but still uses a JIT engine. So we have like 100 or 1,000 of function and then a big data block. And when you look at it, it's binary and it's loaded by the thing. So it's really hard to reverse engineer. When I loaded it in Ida Pro, it gave me 267 megabyte of native code. And I was like, oh my god, there's no way I can go through this. But the packer costs something. Do bad guys pay for packer? Of course not. And so when you try to use the software without the launcher, you get an error. It's an evaluation version and it's expired. So when I looked in the strings, I see that they packed it with an evaluation version that was valid for three months. And so I was like, OK, but why when I use the launcher, this thing works? It's because they embedded in it and the UPX packed it the run as date program. So what basically the launcher does is it's unpacking or loading run as date. And then giving it inputs to fake the date, but only for that process. So your operating system and everything is OK, right? And so this completely bypasses the jet limitations so that they can use the evaluation version. I found that really funny, but I lost a bunch of time looking at it. And so this gives us follow like or running, but it's still like the same grid layout. Nothing new, nothing learned. So yeah, too bad. So we looked at many of them. We cut them for the slides. But here's the table that I mentioned earlier. But the gist of it is we didn't find the one for Linux most. No, there was always something off. And so eventually we hope that we might be able to find it. And by the way, like all the binaries I found, we have a company account for VirusTotal. I put in the domain name of this business in VirusTotal. And I was able to download the binaries and analyze them. So VirusTotal, thank you. You're like a binary distributor, and this is really good because otherwise we couldn't have those binaries. So now, where do we stand? We found several automation software vendor which we analyzed. The reseller model clearly now is not at the bottom level through the analysis that Masera told us about earlier. So where is the reseller model? This is what we're looking for. So Masera. OK, so at that time when I was investigating the crypto traffic, I was trying to find in the traffic some accounts that would, for example, an Instagram account that would advertise follows and likes. So in that case, I would be able to say, oh, see that account was followed by our buttonette, and it's advertising selling of fake likes and follows. So that must be our buttonette providers or something like that, right? So I started to investigate the traffic, and I was looking for keywords like social media, panel, and all that. I ended up finding reseller panels, just like this one called Medianesia panel. So reseller panels are basically just panels that sell fake likes and follows in bulk. When I told this to the journalist, she was really mad because she paid a high price for the fake follows that she bought, but she could have then have them at a very lowest price if she would have gone through the reseller panels. So we sort of decided to illustrate them as spiders and webs, and where they are in the ecosystem is just a little bit above the automation software, and next to Linux Mozart IoT buttonette. So reseller panels look a little bit like that, so you can log in with a fake account easily and just create a password, and they all sell fake likes and follows in bulk, and most of them offer the services on all social networks, so Twitter, Facebook, Instagram. What's interesting is the prices are lower than other sellers that we're gonna see later, but the lower your go and the more the shadier the payment process is. So I tried to buy a million followers for myself for my fake account at $1.8 for $10,000, and the only way I could actually pay for that was Bitcoin, so it's kind of interesting because as the price goes down, the payment maintenance become more and more shadier. So what was specific about the reseller panel that was interesting is the fact that they were selling popularity in bulk, but also that they all look alike. So when I was investigating them, starting to look at them, it was like, it looks like it's the same group behind these panels because it just seems like to be coded by the same person. So we did a simple investigation. We gathered information on 343 reseller panels, and we gathered information on the fingerprint of the web application, the main registration information, HTML content, and IP address, and we started to do clustering analysis. It did yield some results, but there was just one big result that we couldn't ignore, and it was the IP address. It's the fact that 66% of our sample was hosted on the same IP address. So here we just did a simple passive DNS query, and then we found that this IP hosted in the end a thousand other domains, and all these domains are related to the resell of SMM, social media inflation. So for example, you have extra like, easy boost, grand panel. And while investigating that, we found a specific actor, PerfectPanel.com, and what does PerfectPanel does? Well, it's a panel as a service provider. So that sort of answers why all the reseller panels looks the same. It's basically that there is a provider that allow you to sort of have a panel without having any technical knowledge. So they will provide you with the panel, and you can start reselling in bulk. So that sort of added another actor that we weren't expecting, an actor that we added in our little forest behind the webs and the spiders, because they enable the reseller panel sellers. Yeah. So basically panel as a service, we found five of them. What they offer is a ready to go software. They even provide web hosting, obviously all on the same IP address, and some of them even include the domain names, so you don't even have to buy that. They have specific features like API to receive orders and API to send orders. So what you could do is just connect to a panel and then to receive orders and connect to another panel to send orders and just act in the middle. They also allow you to track your workers. So if you wouldn't want to automate everything and you were in another country where you would want to have workers, then that would be something possible too and you could do it manually. So here I just give you an example of it's not perfect panel, it's in this rabbit. So we could actually just log in through a demo and you see that you have a dashboard with all the usernames and the people that are actually, that would be your customers in your reseller panel. You also see that you can select an API and just like connect your panel to another panel so you wouldn't have to have a link to Linux Moose. You would just have to have a link to a cheaper panel and sort of play in the middle. So we looked at, so we were wondering, right? Okay, so there's so many reseller panels, but how do they connect to Linux Moose, right? How are these two actors are linked? I know Linux Moose advertises them through the accounts, but that wasn't clear to us. So what we did is we went on Black Hat World. This is the forum for social media marketing and sort of found lots of discussions that were from reseller panel owners and they were looking for the main provider. So here I give you an example of somebody saying, well, mostly all panels use the same provider. The only difference is price. If anyone knows this provider, he is welcome to contact me privately. So you see that somebody who would have an efficient reseller panel would actually wanna connect directly to Linux Moose, wouldn't wanna connect to another panel, right? But then this guy, Dalton Media Studios, sort of shows well how the game plays. He says, no one will reveal the main provider. For them, he is the hidden ghost. So it makes sense, right? When you think about it, if there's a malware provider, a malware author who's actually detonating Linux Moose, that person would probably not be interested in dealing with all these reseller panels, right? It would just deal with one or two trusted party and then these trusted parties would deal with the reseller panels, right? And SMM snob here is really well showing this. So he says, guys, unless you're spending $1,000 a day on SMM panels, you don't need to search for the original supplier. A, he wouldn't be interested in your volumes and B, you just need to find the most reliable reseller for the most cheap reseller and get on with it. That would be enough. In this market, you have to put your effort not in buying cheaper, but in selling more. So you do, like, we sort of concluded from that that potentially a Linux Moose and these providers at the bottom would be hiding behind a few trusted parties. However, it seems odd to say one main provider, right? There should be many ways to sort of provide fake likes and follows. And this guy says it. He says there cannot be one provider. There are several, and there's several methods too. And while we were thinking about that, we found a new method that was possible and that's what Olivia's gonna present. And so, where are we at right now? There's a system of resellers and a panel of service providers and of course, there is shallow people wanting to buy fake fame, you know? We have the automation software that can create and orchestrate those fake accounts. But then, when you try, when the bad guys try to manipulate Instagram, well, they're getting blocked. And so what Linux Moose, the button that provided to them was reputable IP addresses and, you know, a set of proxy services. But what if there was another way, a more legal way not breaching CFAA? And so, what we're gonna talk about here is there's this ecosystem of residential proxy providers. And so, you can work around blocks using proxies and there are two types of proxies, basically that you can find online. There's the data center proxies which are easy to block because they're all together, you know, it's clearly labeled. And Netflix or, you know, Instagram can block slash 24s, like huge amounts of IP addresses in one shot without impact. But when you try to block residential proxy, the story is different, right? You, IP addresses, they can, someone can be really legit and if you block an IP, you might block legitimate account and false positive are really a pain in that context, right? You don't wanna piss off Kim Kardashian by blocking her because her IP was lagged. And so you really need to be really careful. And so, we even have like this guy next gen leads which is a junior VIP on Black Hat World who says basically stop using data center proxies period, right? At four page, you know, comment on Black Hat World. And so basically we understand what you mean next gen mister. So when you went, actor enters the games basically. So we had this botnet before, but now we have a parallel system which would mimic the botnet. So you could do things differently. And it's the residential proxy services which we, you know, present to you as a cabin because we tried to have this thing be Canadian somehow because we were crazy about Canada. And so what we have here is a cabin sitting on the floor. So the floor is kind of proxying and whatever. And so automation software right beside it and it's on the other side of tree because it's parallel mechanism to achieve the social media manipulation. We put a lot of thoughts into this. We kind of made a bit crazy maybe. But so the ones that we looked at were high proxies, storm proxies, RSOX and the glorious Valor solution which is a game of throne reference which I don't understand, but I appreciate nonetheless. The Valor solution product that we purchased was called Comet USA Static Proxies Residential IPv6. This was our only IPv6 proxy. And Valor, I need to see this later actually. Anyway, and we looked at Luminati. Luminati is very different and it needs a bit of your attention and it's kind of sidetracked from what we're doing today but this is scary as hell. But Luminati's use case has clearly says that account management is part of what they're doing. And so here what I highlight here is like, it basically says use our service to manipulate social media, right? But what I want to highlight right now is like their business model because it's completely different than the other ones and let me have a video about it and let me show you this is completely crazy. App revenue without hurting its user experience. Grow your app revenue without hurting its user experience by joining Luminati's developer community. Embed our SDK and earn $5,000 per month for every 100,000 of your daily active users. Luminati is the world-leading peer-to-peer business proxy network. We help Fortune 500 companies collect web data through residential IPs so they're never blocked or misled. We do that by partnering with app developers and inviting their users to be peers in our network. When users download or upgrade your app, they will see this screen, allowing them to join our network and get your app for free. You can offer your users other options like watching ads or purchasing your app. Users who choose to be peers would not feel any change as their device resources will be shared only when their phone is idle, connected to internet, and has enough power, allowing you to earn money from your inactive users. Implementing our SDK is quick and easy. Grow your revenues while increasing your app usage and retention. Join Luminati's developer community today. This is completely crazy and should be illegal, right? I mean, it's true, like this is crazy, but people will opt in and since people agreed, then it's legal, right? Anyway, I just can't wrap my head around them, but it was clear that it was not our line, it was not built on a compromise infrastructure. So this is why we're ruling them out of this presentation and this research was still, I wanted people to spend a minute thinking about this and pushing against it, right? We need globally to fight these kind of business models. So Trendmicro did a very good paper. So Luminati is what powered whole VPN. I don't know if you heard about them, but anyway, I refer to that paper and I advise that you look at it if you're interested in that topic. We're not gonna talk about Luminati anymore, but I'm mad about them. So Stormproxies, we got IPs from the US. There was conflicting information in the GOIP and who is database. I think this is how they pulled this off, but it said like digital energy, Chile, but the hosting organization was host one plus. We had like Victorio Mahe, which is an African ISP and the hosting organization was Joe's data center. And even though it should have been African when you trace route it, it was US. And so I think this is kind of a, you know, abusing the proper updates of the Instagram, not Instagram, but IP databases. So Instagram is kind of out of sync with these and they cannot block it because they're unsure if it's ISPs or data centers. This is our theory anyway. But then we started scanning and looking at it from, you know, using the product as we purchased all of those services. And so I use a squid for proxying. They don't protect scans from local hosts. It's a Linux system, most likely Debian Jesse because I was able, when using the proxy to scan local hosts to fingerprint, Xim version, which is a SMTP server and squid. And Debian is very rigorous about, you know, backporting patches. They don't update, you know, Xim for whatever reason. They just backport the patch themselves. And so because of that, when you fingerprint specific versions, you can be with a high amount of confidence, know that this is a Debian system. And so this one is a Debian. So is it compromise infrastructure like what we were looking for? Cause we were looking for criminal activity? No. So it's legit. Residential proxy service, legit, abusing, GOIP, who is, you know, miss update or, you know, confusing state. And so we don't have a bad guy here or, you know, we have a gray guy. I don't know. RSOX, it was all the IPs that we got were same subnet in Russia. It was backed by the admin, LLC, ISP. Admin, if you look it up, there's nothing. So you don't know if it's a data center or if it's a residential ISP provider. There's nothing on admin as an ISP. Tracerout confirmed it was Russia. The fact that it's all the same sumnet to us was like, eh, you guys are kind of lame. But okay, you know. They expose SSH when you scan locally to 3389, which is usually RDP. We were not able to fingerprint the proxy service itself so we don't know what binary or what proxy system they were using. And again, because of the SSH that we found and the SSH banner is, gives all the information. So we know it's again a DebianJSC. So again, to our original inquiry is this compromise infrastructure most likely not. And what I failed to mention so far is that this is not single IPs, this is several IPs. We analyze five, 10 proxies and they're all the same. So it kind of gives you the sense that it's not compromise infrastructure. And now let's look at ValorSolution. Unfortunately, they don't provide a management account so the screenshot is just the glorious webpage. But the IP we received was a DigiBox IP, which is clearly not residential in France. It's like a dedicated servers. But this is the first provider where the traffic was not going out from the same IP that you connected, but there was a tunnel. So it would connect in France and it would get out in the US. It would connect IPv4 and it would get out IPv6. And so it was a bit different to analyze. The outgoing traffic was labeled from AT&T internet services. So it was probably one of the best one, like a solid AT&T, no confusion whatsoever. So good job ValorSolution, I don't know how you pull it off. We geoIPed the information and this gave us a middle of a lake in Kansas. So at the time I was like, oh, that's cute. I imagine a data center, underwater data center, but I know it's bullshit. I know geoIP is not precise and there's a radius involved, but I kind of liked it. But then that vision was destroyed when we presented that blackout two days ago. They told me like any IP in the US, not properly geoIPed or labeled, is in that lake in Kansas. And so I was like, okay, I got it wrong. There's a story about that lake that I mean would, we should do a lightning talk about it. Anyway, I'm gonna skip through because we have a lot of stuff to say. But, so Valor uses Triproxy. Triproxy is a very famous Russian proxy service that is kind of shady, but it's open source. You can use it like pentesters should use it because it's really flexible. But it's still like, I've seen it before in malware, but it's still legit, you know, but so they clearly proxy a lot of stuff on the same entry IP because they had a thousand continuous port exposing Triproxy. We had identified it both by NMAP and by triggering errors and looking at the error strings and matching it with source or code. So we're sure it was Triproxy. They protected scans from local hosts. So it was the first one who did that. And IPv6 was completely sealed. So any external scans to the IPv6 address that we got were giving absolutely no result. But we know that the proxy entry is, again, Debian 9, a lot of Debian in the SMM industry, I don't know why, but through fingerprinting of the NGNX and OpenSSH. And so wrapping this up, what we're gonna say is that there are several providers and everything we analyze, I don't feel it's powered by malware, but clearly, ValorSolution is the shadiest one and the one who protects itself the most. So if I had more time, I would definitely look into these guys because there's something weird about that service. There is a paper that was released called Resident Evil Understanding Residential IP Proxy as a Dark Service, which is very interesting. And they took a completely different approach than us and we found it while doing that research after we were accepted to talk about this stuff here. But so I just wanna give them a hat tip, because they actually, they collaborate with ISP. So we use the proxies and try to assess if it's compromised infrastructure or not, but they saw traffic go through these proxies and they know what they are used for. This is a very interesting paper. And it's a collaboration between Indiana University of Bloomington and Tsinghua University and it's more than 10 people will work on it. So I think they really need a hat tip. So where do we stand now? We have an IoT button which we started with four years ago, but now we know it could be done through Residential Proxy that you could pay for, right? We have the automation software which uses these two services in order to automate or make scale your Instagram interactions, right? We have Resetter Panels and Panelist Service Provider. So we have all of that stuff, but who buys from the Reseller panel? What's going up upwards, right? Masera? Okay, so what we did is we called them Customer Facing Sellers. So we just chill out. They just chill out on top of the tree above everybody because they're the one dealing with the celebrities or the agencies related to the celebrities. I just give you an example here of Divumi. Have you heard of Divumi? No? Okay, so this is a customer facing seller. It just, it looks really legitimate. They do lots of customer services. You can pay with credit cards, so it's less shady and it's easier for anyone who's not technical or who would believe that what they're doing is okay. And what's interesting is about Divumi is that they were in a legal procedure with some of their partners who were trying to sell to steal their customer. So they ended up having to publish their list of customers and the New York time took that as an opportunity to create a huge investigation called the follower factory where they actually sort of show names of people who bought from Divumi and names that are people that are well known in the United States. And following that investigation, the New York Atonate General did a legal proceeding against Divumi and sort of announced groundbreaking settlement saying that selling fake followers or likes is illegal deception. So Divumi was fined $50,000 for what it was doing, but according to the New York Times investigation, they made over like five to six million a year so it wasn't that of a high fine. And obviously what happened once this solution was sort of pulled out by the New York Attorney General is that Divumi closed down, obviously, but obviously also they just went down to their best alternative, socialboss.org, instagrowing.net, and youtubegrow.com. So it hasn't thought much of what's going on. And then lastly, the people that are driving this whole business, the buyers, you get it, squirrel? And where do they stand? Well, they stand at the top of the tree next to the customer-facing seller and nearby the fruits of social networks. How did we sort of end up analyzing the potential buyers? Well, it dates back to analysis in 2016. We knew that Linux Moose was proxying traffic and 86% of its traffic was Instagram, so we took only Instagram and then we sort of developed a methodology which said that if Linux Moose had followed more than five times the account, and if it wasn't a super popular person like Kim Kardashian, because we know that they're following these people to look more legitimate, and it had to have a very low engagement, so they had like 100,000 followers, but like when they post a picture, there's two or three likes and it just sort of shows you that it's probably fake following. And then we did a content analysis. So from this sample, we categorized as much as we could the people that we saw. So for example, first thing that we, 20% of our sample was the entertainment industry. So people like this guy who would have 30,000 followers, but in the end, when you clicked on his YouTube channel, it'd be like 10 to 12 views or you know, like a very low. And this person with 200,000 followers who calls himself reality TV personality actor, global jet setter, and you wouldn't find anything about him online, except from that. Then 21% of our sample was selling products of services. So example of watches being sold online. All of most of the products and services that were advertised on Instagram were like luxurious stuff. Maybe that's just a result of the social network, but here's another example of a jewelry in Paris who also we found in our buttonette. And lastly, the saddest thing is more than a quarter of our sample with personal profiles. So individuals who post pictures of their food or of them hanging out on boats, and then they have lots of followers, but they know they're false. So it sort of questions us as to, you post them and you think you have visibility, but inside of yourself, you know that it's not the case. But let's say that that's just another research for psychology or something. But yeah, and then that's how we ended up calling this market the ego market. And then there was the unexpected ones. So what happens is that I could categorize about, I'd say 60 something percent of the sample, but there was so many weird profiles that I couldn't say, oh yes, this is this or this is that. So for example, here I had an Iraqi military general who was in the special forces. And unfortunately, this is his profile and it's probably the person managing his account that has bought fake following. Or it's potential, right? Like we haven't seen these people doing it. We've just seen them in our buttonette. And we also saw some quite large amount of magicians, spiritual psychic, people involved in magic. Here I give you an example of a New York home witch as she called herself. So yeah, with 141,000 followers. And then there's the fascinating ones. Oh, okay. So here I give you a, it makes sense, right? An STD's testing account will obviously, if you were to want to have followers online, who would follow that, right? So it makes sense that you buy this. And then there's dogs with more money than you, right? How do you categorize that, right? Like, anyway. So we made an experiment, okay? If you get it, it's a duck face in a mirror. Okay. So we knew that Linux Moose was targeting Instagram, but it was also targeting Periscope, Fliipagram and Kiwi, which are unknown social networks. And we found a seller, a customer-facing seller, that was selling those fake following on these social networks. So we thought, well, let's just try to buy from that customer-facing seller and see if we'll see the account going through the honeypots, right? So what we did is we bought, I created a fake account called BeautifulBird33. We bought 6,000 followers, got 8,000 because I complained. And I wanted to increase the chances of seeing the account. And luckily, that day was one of our best day in our research is that the account went through the traffic in our honeypots. So that sort of confirmed our investigation even more, right? And if you think about it, what we did is we acted as the buyer and we went on a customer-facing seller and then we saw the account going through the botnet. We don't know what happened in the middle with the reseller panels, but we know that it happened. So in terms of the revenue, I'm gonna go a little bit quicker just because we're short on time. But in 2016, when we studied the botnet, we had the customer-facing seller's prices and we concluded that most of the profit was going to the botnet. So what we did is we calculated the amount of follows that we saw in each honeypots and then took the prices that we saw on customer-facing sellers and just added this together. And we said, well, if Linux Moose has 33,000 bots that it would make, on average, for example, $400,000. However, now we know there's much more actors involved, right? So you have the buyer who buys from the customer-facing seller who has a mid-price, on average, for $95,000 for 10,000 followers. And then the customer-facing seller will go on the reseller panel and then we'll pay a much lower price for the fake following. And then we don't know how much Linux Moose is going to make out of this, right? But what we know as the cheapest reseller panel is $1.8 for 10,000 followers, which I, honestly, I'm pretty sure that's a scam, but still. And then, meanwhile, you have all these actors that are still making money. So you have the panelist service provider who makes, on average, $47 a month per client. At Proxies, it's about $2 per proxy per month. And the automation software they're making on average between $10 to $60 a month per client, too. So let's just do an exercise, okay? You wanna be trending. You wanna buy 10,000 followers provided within a week. So you're doing something. You're going to Defcon and you're like, this is my chance, I'm gonna get popular. And then you know that each bot, we know that each bot performed, on average, 1,000 followers per month on Instagram. And how do we know that? Well, we just calculated that. We took the average, and if it's not that high, it's because the botnet has to imitate a lot of human behavior to ensure that it's not flagged. It won't go like follow, follow, follow, follow, right? And then that equals to about 280 followers a week. And for that, if you wanted to go through Proxies, well, you would need 36 Proxies at a median cost of $2 a month. So it would cost you a flat cost of $7 to $2. I mean, that's quite expensive, right? For the median price of $2 per proxy per month. So that sort of leads us to think that this is not for nothing, that Linux moves exist, right? Infected devices are cheap. And otherwise, that Valar solution or more shadier actors are involved. So Valar sells for about 50 cents a proxy for a month. And then if we just let's estimate Linux moves capacity, so if we say that Linux moves has, for example, 40,000 bots, then we could say that it could make, on average, 40 million Instagram followers per month. So that's quite a lot. And if we try to see how it sort of fits on the graph, then that's how you see that, if we take the minimum price of the reseller panel, it would make much less money than we expected. And then the first quartile is a little bit higher and the median price is even higher. But what's the main takeaway from this is mainly just to understand that we often try to estimate profitability just by adding one by the other, but there's often many actors involved and we don't calculate that. So we're always like, oh, this is millions of dollars for the malicious actors. But in the end, you see that it's much more complicated than that and there's other people making money out of this, right? So this is sort of how we concluded our investigation of the ecosystem of social media manipulation. So you see all the actors here. There's other research avenues. So obviously you'd have other ways to be able to create fake following such as ClickFarms, Compromised Accounts and TrollFarms. And just to let you know, there's some journalists that are working right now into interviewing actors involved in the ecosystem that we talked about. So, and they're gonna publish really soon. So if you have, you're interested in knowing more about that, you can come talk to me and I can put you in contact with them. And there's a prof in Germany called Patrick Vendero who's also working on interviewing reseller panels owner. So I think I'm gonna stop here because I had a what to do next, but it's the, oh yeah, okay. What to do next, okay? Really quickly, just keep your attention. Policy makers will look into the sale of social media manipulation, right? Because we saw the New York attorney general doing it and then we'd be wondering other countries could do that in other states. Law enforcement target the middleman panel as a service provider. They enable so many actors to sort of connect in between the customer facing sellers and the botnet. Social networks continue to flag a new robotic activity. We went on the Discord server of Valor solution and it was kind of interesting to see that just right now, yes, just right now people are having a hard time doing fake likes and follows. It seems that Instagram has published a new AI and thus they're really struggling to do it. So I mean, it works. And you will maybe try to focus on the content rather than the container. So if somebody tells you, look at this person. Like they have so many followers. Well, instead of saying, wow, just wonder what's the person bringing to society and what's the content being published? Is this pertinent or not? And lastly, just remember I bought 8,000 followers on my account and I only have 400s now so fake visibility doesn't last. Thank you very much and I'm sorry for rushing through the last part but it's very nice to be at Defcon and I hope you enjoy your week.