 factories. I'm going to date myself. I remember it was like it was the Congress around 1990 or 1991 or so where I was sitting together with some people who came over to the States to visit the the CCC Congress and we were kind of riffing on how great the internet is going to make the world. You know how it's going to bring world peace and truth will rule and everything like that. Boy were we naive. Boy were we totally wrong. And today I'm going to be schooled in how wrong I actually was because we have Svia, Dennis, and Phillip to tell us all about the fake-like factories around the world and with that could you please help me in welcoming them on to the stage. Svia, Dennis, and Phillip. Thank you very much. Yeah, welcome to our talk inside the fake-like factories. So my name is Phillip. I'm an internet activist against this information and I'm also a student of the University of Bamberg. Hi, thank you that you yeah that you listen to us tonight. My name is Svia. I'm an investigative journalist freelance mostly for NDR and ARD. It's a public broadcaster in Germany and I focus on tech issues and I had the pleasure to work with these two guys on for me a journalistic project and for them on a scientific project. Yeah, hi everyone. My name is Dennis. I'm a PhD student from Ruh University Bochum. I'm working as a research assistant as the for the chair for system security and my research focuses on network security topics and internet measurements. And as Svia said, Phillip and myself, we are here for the scientific part and Svia is for the journalistic part here. So here's our outline for today. So first I'm going to briefly talk about our motivation for our descent into the fake-like factories and then we are going to show you how we got our hands on 90,000 fake-like campaigns of a major crowd working platform and we are also going to show you why we think that they are 10 billion registered Facebook users today. So first I'm going to talk about the like button. The like button is the ultimate indicator for popularity on social media. It shows you how trustworthy someone is. It shows how popular someone is. It shows it is an indicator for economic success of brands and it also influences the Facebook algorithm. And as we are going to show now, these kind of likes can be easily forged and manipulated. But the problem is that many users will still prefer this bad info on Facebook about popularity of a project to no info at all. And so this is a real problem and there's no real solution to this. So first we are going to talk about the factories and the workers in the fake-like factories. That there are fake likes and that you can buy likes everywhere. It's well known. So if you Google buying fake likes or even fake comments for Instagram or for Facebook, then you will get like a hundreds of results and you can buy them very cheap and very expensive. Doesn't matter. You can buy them from every country. But when you think of this bought likes, then you may think of this. So you may think of somebody sitting in China, Pakistan or India and you think of computers and machines doing all this and that they are fake and also that they can easily be detected and that maybe they are not a big problem. But it's not always like this. It also can be like this. So I want you to meet Maria. I met her in Berlin and Harald. He lives near Mönchengladbach. So Maria, she is a she's a rent-a-ree. She was a former police officer and as money is always short, she is clicking Facebook likes for money. So she earns between 2 cent and 6 cent per like and Harald, he was a baker once. He is now getting a social aid and he's also clicking and liking and commenting the whole day. We met them during our research project and did some interviews about their likes. And one platform they are clicking and working for is Paid Likes. It's only one platform out of a universe, out of a cosmos. Paid Likes, they are sitting just a couple of minutes from here in Magdeburg and they are offering that you can earn money with liking on different platforms. And it looks like this. When you log into the platform with your Facebook account, then you get in the morning and the afternoon, in the evening, you get, we call it campaigns, but these are pages, Facebook fan pages or Instagram pages or posts or comments. You can work your way through them and click them and I blurred, you see here the blue bar, I blurred them because we don't want to get sued from all these companies, which you can see there. To take you a little bit with me on the journey, Harald, he was okay with us coming by for television and he was okay that, and we did a long interview with him and I wanted to show you a very small piece out of his daily life, sitting there, doing the household, the washing and the cleaning and clicking. Where a picture is made of his new pizza creation, it's all possible. We're going to see if it's politicians or dog breeders or karate clubs. Now I just have to press like I like and I've already earned money. Could be like that. You click and you earn so money. How did we meet him and all the others, of course, because Philip and Dennis, they have a more scientific approach, so it was also important not only to talk to one or two, but to talk to many. So we created a Facebook fan page, which we call like a line under a line, because they thought, okay, nobody would like this freely. And then we did a post, this post and we bought likes. And you won't believe it and it worked so well, 222 people or the people I paid for, like this. And then we wrote all of them. And we talked to many of them. Some of them only in writing, some of them only we just called or had a phone chat. But they gave us a lot of information about their life as a click worker, which I will sum up. So what pay likes by itself says, they say that they have 30,000 registered users. And it's really interesting because you might think that they are all registered with like 10 or 15 accounts, but most of them they are not. They are clicking with their real account, which makes it really hard to detect them. So they even scan their ID, so that the company knows that they are real, then they earn their money. And we met men, women, stay at home moms, low income earners, retirees, people who are getting social care. So basically anybody. So there was no kind of bias. And many of them are clicking for two and more platforms. It was I didn't met anybody who was only clicking for one platform. They all have a variety of platforms where they are writing comments or clicking likes. And you can make, this is what they told us, between 15 euro and 450 euro monthly if you are a so-called power clicker and you do this somewhere kind of professional. But this are only the workers and maybe you are more interested in who are the buyers, who benefits. Yeah. Let's come to step two, who benefits from the campaigns. So I think you all remember this page. This is a screen. If you lock into paid likes and you see the campaigns with, you have to click in order to get a little bit of money. And by luck we noticed that if you go over a URL, we see in the left-button side of the browser URL redirecting to the campaign you have to click. And you see that every campaign is using a unique ID. It is just a simple integer. And the good thing is it is just incremented. So now maybe some of you guys notice what can we do with it. And yeah, it is really easy with this constructed URL to implement crawler for data gathering. And our crawler simply requested all campaign IDs between zero and 90,000. Maybe some of you ask why 90,000? As I already said, we also registered as a click worker and we saw that the highest ID campaign used is about 88,000. So we thought, okay, 90,000 is a good value. And we checked for every request between these 90,000 requests if it got resolved or not. And if it got resolved, the redirected URL represents the source that should be liked or followed. And we did not save the page sources from the resolved URLs. We only saved the resolved URLs in a list of campaigns. And yeah, this list was then the basis for our further analysis. And yeah, here you see our list. Yes, this was the point when Dennis and Philip, when they came to us and said, hey, we have a list. So what can you find? And of course, we searched for AFD, was one of the first search queries. And yeah, of course, AFD is also in that list. Maybe not so surprisingly for some. And when you look, it is AFD Gelsenkirchen. And the fan page. And we asked AFD Gelsenkirchen, did you buy likes? And they said, oh, we don't know how we got on that list. But however, we do not rule out an anonymous donation. But now you would think, okay, they found AFD. This is very expectable. But now, all political parties, mostly with local and regional entities, showed up on that list. So we have CDU CSU, we have had FDP, AFD de Grün and de Linke. But not that you think Angela Merkel or some very big Facebook fan pages showed up. No, no. Very small entities with a couple of hundreds or maybe 10,000 or 15,000 followers. And I think this makes perfectly sense because somebody who has already very, very much many fans probably would not buy them there at paid likes. And we asked many of them. And mostly, they could not explain it. They would never do something like that. Yeah, they were completely over-asked. But you have to think that we only saw the campaign, the campaigns, the Facebook fan pages. We could not see who bought the likes. And as you can imagine, everybody could have done it, like the mother, the brother, the fan, you know, the dog. So this was, we would have needed, we would have needed a lot of luck to call anybody out of the blue. And then he would say, oh, yes, I did this. And there was one, there were some politicians who admitted it. And one of them, she did it also publicly and gave us an interview, is Tania Kühne. Tania Kühne, she is a regional politician from Walzrode in Niedersachsen. And she was in the, it was the case that it was after election and she was not very happy with her fan page. That is what she told us. She was very unlucky. And she wanted, you know, to push herself and to boost it a little bit and get more friends and followers and reach. And then she bought 500 followers. And then we had a nice interview with her about that. Show you a small piece. It is a means of communication. And if the falsehood does not notice, it is lacking media competence. There the issue of media competence is really asked, to really think, can that be at all? So it is, so to speak, no threat, if you buy likes. Again, so why threat? Threat can only be, if we as a society say that that is the measure or the measure for us is for popularity. Okay, so you see, answers are pretty interesting. And she, I think she was that courageous to speak out to us. Many of others did too, but only on the phone and they didn't want to go on record. But she is not the only one who answered like this, because, of course, if you call through a list of potential fake like buyers, of course they answer like, no, it is not a scam. And I also think from a jurisdictional way, it is also very hard to show that this is fraud and a scam. And it is more an ethical problem that you can see here, that is manipulative, if you buy likes. We also found a guy from FTP from the Bundestag, but yeah, he ran away and didn't want to get interviewed, so I couldn't show him. Yeah, he bought or not probably. He was like 40 times in our list for various Facebook posts and videos and also for his Instagram account. But we could not get him on record. So what did others say? We, of course, confronted Facebook, Instagram and YouTube with this small research and they said, no, we don't want fake likes on our platform. Paid likes is active since 2012, you know, so they waited seven years. But after our report, at least Facebook temporarily blocked paid likes. And of course, we asked them to and spoke to them and wrote with paid likes in Magdeburg and they said, of course, it's not a scam because they are freely clicking on pages. So, yeah, kind of nobody cares. But paid likes, this is only the tip of the iceberg. So, we also wanted to dive a little bit into this fake like universe outside of paid likes and to see what else is out there. And so we did an analysis of account creation on Facebook. So what Facebook is saying about account creation is that they are very effective against fake accounts. So they say they remove billions of accounts each year. And that most of these accounts never reach any real users and they remove them before they get reported. So what Facebook basically wants to tell you is that they have it under control. However, there are a number of reports that suggest otherwise. For example, recently, NATO, the StratCom Task Force released a report where they actually bought 54,000 likes, 54,000 social media interactions for just 300 euros. So this is a very low price. And I think you wouldn't expect such a low price if it would be hard to get that many interactions. They bought 3,500 comments, 25,000 likes, 20,000 views and 5,100 followers, everything for just 300 euros. So you know the thing they have in common, they are cheap, the fake likes and the fake interactions. So we also have there was also another report from Weiss, Germany, recently, and they reported on some interesting facts about automated fake accounts. They reported on findings that suggest that actually people use internet or hacked internet of things devices and to use them to create these fake accounts and to manage them. And so it's actually kind of interesting to think about this this way to say, okay, maybe next election, your fridge is actually going to support the other candidate on Facebook. And so we also wanted to look into this and we wanted to go a step further and to look at who these people are, who are they, and what are they doing on Facebook. And so we actually examined the profiles of purchased likes. For this, we created four comments under arbitrary posts, and then we bought likes for these comments, and then we examined the resulting profiles of the fake likes. So it was pretty cheap to buy these likes. Comment likes are always a little bit more expensive than other likes. And we found all these offerings on Google, and we paid with PayPal. So we actually used a pretty neat trick to estimate the age of these fake accounts. So as you can see here, the Facebook user ID is incremented. So Facebook started in 2009 to use incremented Facebook IDs. And they use this pattern of 10000 and then the incremented number. And as you can see in 2009, this incremented number was very close to zero. And then today it is close to 40 billion. And in this time period, you can see that you can kind of get a rather fitting line through all these points. And you can see that the likes are in fact incremented. The account IDs are in fact incremented over time. So we can use this fact in reverse to estimate the creation date of an account where we know the Facebook ID. And that's exactly what we did with these fake likes. So we estimated the account creation dates. And as you can see, we get kind of different results from different services. For example, paid likes, they had rather old accounts. So this means they use very authentic accounts. And we already know that because we talked to them. So these are very authentic accounts. Also like service A over here also uses very, very authentic accounts. But on the other hand, like service B, uses very new accounts. So they were all created in the last three years. So if you look at the accounts and also from these numbers, we think that these accounts were bots. And on service C, it's kind of not clear are these accounts bots or are these click workers. Maybe it's a mixture of both. We don't know exactly for sure. But this is an interesting metric to measure the age of the accounts to determine if some of them might be bots. And that's exactly what we did on this page. So this is actually a page for garden furniture. And we found it in our list that we got from paid likes. So they bought, obviously, they were on this list for bought likes on paid likes. And they caught our eye because they had one million likes. And that's rather unusual for a shop for garden furniture in Germany. And so we looked at this page further and we noticed other interesting things. For example, their posts all the time, they got like thousands of likes. And that's also kind of unusual for a garden furniture shop. And so we looked into the likes. And as you can see, they all look like they come from Southeast Asia. And they don't look very authentic. And we were actually able to estimate the creation dates of these accounts. And we found that most of these accounts that were used for liking these posts on this page were actually created in the last three years. So this is a page where everything from the number of people who like the page to the number of people who like the posts is complete fraud. So nothing about this is real. And it's obvious that this can happen on Facebook. And that this is a really, really big problem. I mean, this is a shop for garden furniture. Obviously, they probably don't have such huge sums of money. So it was probably very cheap to buy this amount of fake accounts. And it's really shocking to see how big the scale is of this kind of operations. And so what we have to say is, okay, when Facebook says they have it under control, we have to doubt that. So now we can look at the bigger picture. And what we are going to do here is we are going to use this same graph that we used before to estimate the creation dates, but in a different way. So we can actually see the lowest and the highest points of Facebook IDs in this graph. So we know the newest Facebook ID by creating a new account. And we know the lowest ID because it's zero. And then we know that there are 40 billion Facebook IDs. Now, in the next step, we took a sample, a random sample from these 40 billion Facebook IDs. And inside of this sample, we checked if these accounts exist, if this ID corresponds to an existing account. We do that because we obviously cannot check 40 billion accounts, 40 billion IDs, but we can check a small sample of these accounts, of these IDs and estimate then the number of existing accounts on Facebook in total. So for this, we repeatedly accessed the same sample of one million random IDs over the course of one year. And we also pulled a sample of 10 million random IDs for closer analysis this July. And now Dennis is going to tell you how we did it. Yeah. Well, pretty interesting, pretty interesting results so far, right? So we again implemented a crawler the second time for gathering public Facebook information, public Facebook account data. And yeah, this was not so easy as in the first case. Yeah, as it's not surprising that Facebook is using a lot of measures to block the automated crawling of the Facebook page, for example, with IP blocking or capture solving. But we were pretty easy, we could pretty easy solve this problem by using the Tor anonymity network. So every time our IP got blocked by crawling the data, we just made a new Tor connection and changed the IP. And this also with the captures and with this easy method, we were able to crawl all the Facebook, all the public Facebook data. And let's have a look at two examples. And the first example is facebook.com slash four. So the very, very small Facebook ID. Yeah, in this case, we are we, we are redirected and check the response and find a valid account page. And does anyone know which account this is? Number four? Yeah, Max Zuckerberg. Yeah, that's correct. Yeah, this is this is a public account for Max Zuckerberg. Number four, as we see, as we already saw, the other IDs are really high, but he got the number four. A second example was Facebook.com slash three. In this case, we are not forwarded. And this means that it is an invalid account. And that was really easy to confirm with a quick Google search. And it was a test account from the beginning of Facebook. So we did not get redirected. And it's just the login page from Facebook. And with these examples, we did, we did a lot of a lot more experiments. And at the end, we were able to, to build this tree. And yeah, this tree represents the high level approach from our scraper. So in the, what's that? Okay, it's sleeping. Yeah. We have still time, right? So wait, it's better. Okay, everyone is waking up again. So, yeah, in the first step, we call the, the domain, www.facebook.com slash FID. If we get redirected in this case, then we checked if the, if the page is an account page. If it's an account page, then it's a public account, like the example four. And we were able to save the raw data, the raw HTTP source. If we, if it's not an account page, then everything is okay. It is not, it is not a public account. And we are not able to save any data. And if we call, if we do, if we do not get redirected in the first step, then we call the second domain, facebook.com slash profile.php, question mark ID equals the FID with a mobile user agent. And if we get redirected, then, then again, it is a non-public profile and we cannot save anything. But and if we get not redirected, it is an invalid profile, and it is most often a deleted account. Yeah, and yeah, that's the high-level overview of our scraper and Philip will now give some more information on interesting results. So the most interesting result of the scraping of the sample of Facebook IDs was that one in four Facebook IDs corresponds to a valid account. And you can do the math. There are 40 billion Facebook IDs, so there must be 10 billion registered users on Facebook. And this means that there are more registered users on Facebook than there are humans on Earth. And also it means that it's it's even worse than that because not everybody on Earth can have a Facebook account because not everybody, you need a smartphone for that. And many people don't have those. So this is actually a pretty high number and it's very unexpected. So July 2019, there were more than 10 billion Facebook accounts. Also, we did another research on the time frame between October 2018 and today, or this month, and we found that in this time frame, there were two billion new registered Facebook accounts. So this is like the time frame of one year, more or less. And in a similar time frame, the monthly active user base rose by only 187 million. Facebook deleted 150 million older accounts between October 2018 and July 2019. And we know that because we pulled the same sample over a longer period of time. And then we watched for accounts that got deleted in the sample and that enables us to estimate this number of 150 million accounts that got deleted that are basically older than our sample. So I made some nice graphs for your viewing pleasure. So again, the older accounts were just 150 million were deleted since October 2018. These are accounts that are older than last year. And Facebook claims that since then about 7 billion accounts got deleted from their platform, which is vastly more than this older accounts. And that's why we think that Facebook mostly deleted these newer accounts. And if an account is older than a certain age, then it is very unlikely that it gets deleted. Also, I think you can see the scales here. So of course, registered users are not the same thing as active users, but you can still see that there are much more registrations of new users than there are active users, than there are new active users during the last year. So what does this all mean? Does it mean that Facebook gets flooded by fake accounts? We don't really know. We only know these numbers. What Facebook is telling us is that they only count and publish active users. As I already said, that there is a disconnect between the registered users and the active users. And Facebook only reports on the active users. Also, they say that users register accounts, but they don't verify them or they don't use them. And that's how this number gets so high. But I think that that's not really explaining these high numbers. Because that's just by orders of magnitude larger than anything that this could cause. Also, they say that they regularly delete fake accounts. But we have seen that these are mostly accounts that get deleted directly after their creation. And if they survive long enough, then they are getting through. So what does this all mean? Okay, so you got the full load, which I had like over two or three months. And what for me was or for us was one very big conclusion was that we have some kind of broken metric here that all the likes and all the hearts on Instagram and the followers that they can so easily be manipulated. And then it's so hard to tell. In some cases, it's so hard to tell if they are real or not real. And this opens the gate for manipulation and for untruthness and for economic losses, if you think as somebody who is investing money and or as an advertiser, for example. And in the very end, it is a case of eroding trust, which means that we cannot trust these numbers anymore. These numbers are, you know, they are so easily manipulated. And why should we trust this? And this has a severe consequence for all the social networks if you are still in them. So what can be a solution? And Philip, you thought about that. So basically, we have two problems. One is click workers and one is fake. The click workers are basically just hyperactive users. And they are selling their hyperactivity. And so what social networks could do is just make interactions scarce. So just lower the value of more interactions. If you are hyperactive users, then your interaction should count less than the interactions of a less active user. That's kind of solvable, I think. The real problem is the authenticity. So if you get stopped from posting or liking hundreds of pages a day, then maybe you just create multiple accounts and operate them simultaneously. And this can only be solved by authenticity. So this can only be solved if you know that the person who is operating the account is just one person is operating one account. And this is really hard to do because Facebook doesn't know who is clicking. Is it a bot? Is it a click worker? Is it one click worker for 10 accounts? How does this work? And so this is really hard for the social media companies to do. And you could say, okay, let's send in the passport or something like that to prove authenticity. But that's actually not a good idea because nobody wants to send their passport to Facebook. And so this is really a hard problem that has to be solved if we want to use social media in a meaningful way. And so this is what companies could do. And now what do you could do? Okay, of course, you can delete your Facebook account or your Instagram account and stop. Stay away from social media. But this maybe is not enough for all of us as a solution. So I think be aware, of course, spread the word. Tell others. And if you like, then, and you get more intelligence about that, we are really happy to dig deeper in these networks and we will go on investigating. And so last but not least, it's to say thank you to you guys. Thank you very much for listening. And we did not do this alone. We are not three people. There are many more standing behind and doing this beautiful research. And we are opening now for questions, please. Yes, please thank Svia, Phil and Dennis again. And we have microphones out here in the room, about nine of them actually. If you line up behind them to ask a question, remember that a question is a sentence with a question mark behind it. And I think I see somebody at number three. So let's start with that. Hi, I just have a little question. Wouldn't a dislike button, the concept of a dislike button, wouldn't that be a solution to all the problems? So we thought about recommending that Facebook ditches the like button all together. I think that would be a better solution than a dislike button because a dislike button could also be manipulated and it would be even worse because you could actually manipulate the network into down ranking posts or kind of not showing posts to somebody and that I think would be even worse. Imagine what dictators would do with that. And so I think the best option would be to actually not show a like like counts anymore and to this to actually make people not invest into these counts if they become meaningless. I think I see a microphone seven up there. Hello. So one question I had is you assigned creation dates to IDs. How did you do this? So we actually knew the creation date of some accounts. And then we kind of interpolated between the creation dates and the ideas. So you see this black line there. That's actually our our interpolation. And with this black line, we can then estimate the creation dates for IDs that we do not yet know because they kind of fills in the gaps. Follow up question. Do you know why there's some points outside of this graph? No. Thank you. So there was a question from the internet. Did you report your findings to Facebook and did they do anything? Because this research is very new, we we just recently approached them and showed them the research and we got an answer. But I think we also already showed the answer. It was that they I think, yeah, that they only count and publish active users they could they didn't want to tell us how many registered users they have that they say, Oh, sometimes users register accounts, but don't use them or verify them and that they regularly delete fake accounts. But we hope that we get into a closer discussion with them soon about this. Microphone two. When hunting down the buyers of the campaigns, did you dig out your own campaign line below the line? And no, because they stopped scraping in August. And I you stopped scraping in August. And then I started, you know, I the whole project started with them coming to us with the list. And then we thought, Oh, this is very interesting. And then the whole journalistic research started. And but I think if we I think if we would do it again, of course, I think we would find us. We are also found there was another magazine and they did also a test, a paid like test a couple of years ago, and we found their campaign. So we actually did another test. And for the other test, I know that we also got like this ID. I think and it worked to plug it into the this URL. And then we also got redirected to our own page. So that worked. Yeah. Thank you. Microphone three. Hi, I'm Farhan. I'm a Pakistani journalist. And first of all, I would like to say that you were right when you said that there might be people sitting in Pakistan clicking on the likes that does happen. But my question would be that Facebook does have its own ad program that it aggressively pushes. And in that ad program, there is also options whereby people can buy likes and comments and impressions and reactions. Did you would you also consider those as fake? I mean, they're not fake per se, but they're still bought likes. So what's your view on those? Thank you. So when you buy ads on Facebook, then so what you actually want to have is fans for your page that are actually interested in your page. So that's kind of the difference, I think, to the to the paid likes system, where the people themselves, they get paid for liking stuff that they wouldn't normally like. So I think that's the fundamental difference between the two programs. And that's why I think that one is unethical and one is not really that unethical. The very problem is if you buy these click workers, then you have many people in your fan page, they are not interested in you. They don't care about you. They don't look at your products. They don't look at your political party. And then often the people, they additionally, they make Facebook ads. And these ads, they are shown again, the click workers, and they don't look at them. So, you know, people, they are burning money and money and money with this whole corrupt system. So microphone to hi, thanks. Thanks for the talk. And thanks for the effort of going through all of this project. From my understanding, this whole finding basically undermines the trust in Facebook's likes per say in general. So I would expect now the price of likes to drop and the pay for click workers to drop as well. Do you have any metrics on that? The research just went public I think one week ago. So what we have seen as an effect is that Facebook, they excluded paid likes for a moment. So yes, of course, one platform is down. But I think there are so many outside. There are so many. So I think I meant the phenomenon of paid likes, not the company itself, like the value of a like as a measure of credibility. It's declining now. That's my that's my assumptions. That's why many people are buying Instagram hearts now. So yes, that's true. The like is not the fancy hot shit anymore. Yes. And we also saw in the data that the likes for the fan pages, they rapidly went down. And the likes for the posts and the comments, they went up. So I think yes, there is a shift. And what we also saw in the data was that the Facebook likes, they went down until from 2016, they are rapidly down. And what is growing and rising is YouTube and Instagram. Now everything is about today. Everything is about Instagram. Thanks. So let's go to number one. Hello, and thank you very much for this fascinating talk because I've been following this whole topic for a while. And I was wondering if you were looking also into the demographics in terms of age groups and social class, not of the people who are doing the actual liking, but actually, you know, buying these likes. Because I think that what is changing is an entire social discourse on social capital in the bold US kind of term, because it can now be quantified as a teacher, I hear of kids who buy likes to be more popular than their other schoolmates. So I'm wondering if you're looking into that, because I think that's a fascinating, fascinating area to actually come up with numbers about it. It definitely is. And we were also fascinated by this data set of 90,000 data points. And what we did was and this was very hard and was that we tried, first of all, to look who is buying likes like are this automotive, you know, to to this, yeah, what what kind of branches who is in that. And so this was this was doable. But to get more into demographics, you would have like to to crawl to click every page. And so we did not do this. What we did was, of course, that we that we were a team of three to 10 people and manually looking into it. And what we of course saw that on Instagram, and on YouTube, you have many of these very young people, some of them I actually called them and they were like, yes, I bought likes. Very bad idea. So I think yes, I think there is a demographic shift away from the companies and the automotive and industries buying Facebook fan page likes to Instagram and YouTube want to be influencers influencer culture is obviously and I have to admit, here we showed you the political side, but we have to admit that the political likes, they were like this small in the numbers. And the very, very vast majority of this data set. It's about wedding planners, follow up graffiti tattoo studios, and influences influences influences and YouTubers, of course. Yes. Thank you so much. So we have a lot of questions in the room, I'm going to get to you as soon as we can. I'd like to go to the internet first. Do you think this will get better or worse if people move to more decentralized platforms? If it get better or worse? Can you repeat that, please? Would this issue get better or worse if people move to more decentralized platform? Decentralized. So I mean, we can look at at this slide, I think, and think about whether decentralized platforms would change any of these two, any of these two points here. And I fear I don't think so, because they cannot solve the interactions problem that people can be hyperactive. Actually, that's kind of a normal thing in social media that a small portion of social media users is much more active than everybody else. That's kind of you have that without paying for it. So without even having paid likes, you will have to consider if social media is really kind of representative of the society. But and the other thing is authenticity and also in a decentralized platform, you can have multiple accounts run by the same person. So microphone seven all the way back there. Hi. Do you know if Facebook even removes the likes when they delete fake accounts? We don't know that. No, we don't. We don't know. We know they delete fake accounts, but we don't know if they also delete the likes. I know from our research that the people we approached, they did not delete the click workers. So microphone two. Yeah, hi. So I have a question with respect to this one out of four Facebook accounts are active in your in your test. Did you see any difference with respect to the age of the accounts? So is it always one out of four through the entire sample? Or does it maybe change over the over the like going from a zero idea to well 10 billion or 40 billion? So you're talking about the density of accounts in our IDs kind of so so there are changes over time. Yeah, so I think now it's less than it was before. So now they are less than for then before it was more. And so I think it was, yeah, I don't know. But you didn't see anything specific that now only in the new accounts only one out of 10 is active or or or valid and before it was one out of two or something like that. It's not that extreme. So it's less than that. It's kind of we have to say we did not check this. But there were no special cases. So but it changed over time. So before it was less and before it was more and now it is less. And so what we checked was whether an ID actually corresponds to an account. And so this metric, yeah, and it changed a little bit over time but not much. So so number three, please. Yeah, thank you for a very interesting talk. At the end, you gave some recommendations how to fix the metrics, right? And it's always nice to have some metrics because then well, we are the people who deal with numbers. So we want the metrics. But I want to raise the issue whether quantitative measure is actually the right thing to do. So would you buy your furniture from store a with 300 likes against store B with 200 likes, or would it not be better to have a more qualitative thing? And to what extent is a quantitative measure maybe also the source of a lot of bad developments we see in social media to begin with, even not with bought firms and anything, but just people who go for the quick like and say, hooray for Trump. And then you get whatever all the Trump is liking that. And the others say, fuck Trump. And you get all the non Trump is like that and you get all the polarization, right? So Instagram, I think they just don't display their like equivalent anymore in order to prevent that. So could you maybe comment on that? I think this is a good idea to to hide the likes. Yes. But I, you know, we talked to many click workers, and they do a lot of stuff. And what they also do is taking comments and doing copy paste for comment section or for Amazon reviews. So, you know, I think it's it's really hard to get them out of the system because maybe if the likes are not shown and if when the comments are counting, then you will have people who are copy pasting comments in the comment section. So I really think that the networks, they really have an issue here. So let's try to squeeze at the last three questions now. First number seven really quick. Very quick. Thank you for the nice insights. And I have a question about the location of the users. So you made your point that you can analyze by the metadata where when the account was made. But how about the location of the followers? Is there any way to analyze that as well? So we can only analyze that if the users agree as it share it publicly. And not all of them do that. I think often a name check is often a very good way to check where someone is from. For these fake likes, for example. But as I said, it's it always depends on what the user himself is willing to share. Internet. Isn't this just a Western version of the Chinese social criticism? Where do we go from here? What is the future of all this? Yeah, that's the dystopian, right? So, yeah, I don't I after this research, you know, for me, I deleted my Facebook account like one or two years ago. So this, you know, this did not matter to me so much. But I stayed on Instagram. And when I saw all the sport likes and abonance and followers and also YouTube, all this views this because the click workers, they also watch YouTube videos, they have to stay on them like 40 seconds. It's really funny because they have hearing like techno music, rap music, all 40 seconds and then they go on. But when I sat next to Harald for two or three hours, I was so the illusionated about all the social network things. And I thought, okay, don't count on anything. Just if you like the content, follow them and look at them. But don't believe anything. That was my personal take away from this research. So very last question, microphone to a couple of days ago, the independent reported that Facebook, the Facebook app was activating the camera when reading a news feed. Could this be in use in the context of detecting fake accounts? I don't know. So I think that that in this particular instance, that it was probably a bug. So I don't know. But I mean, the people that work at Facebook are not all of them are like crooks or anything that they will deliberately program this kind of stuff. So they said that it was kind of a bug from an update that they did. And the question is whether we can actually detect fake accounts with the camera. And the problem is that current, I don't think that current face recognition technology is enough to detect that you are a unique person. So there are so many people on the planet that there are probably another person who has the same face. And I think the new iPhone, they also have this much more sophisticated version of this technology. And even they say, okay, there's a chance of one in, I don't know, that there is somebody who can unlock your phone. So I think it's really hard to do that with current technology to actually prove that somebody is just one person. So with that, would you please help me thank Svia, Dennis and Phillip one more time for this fantastic presentation. Very interesting. And very, very disturbing. Thank you very much.