 talk is technology did not create misinformation and it's not gonna solve it either. Basically, yep, there we go. Try not to waste your time. There's a lot of research to be done. A lot of people are doing a lot of research. A lot of it is fairly awkward. A lot of it is useless. There's a lot of wasted money. So I'm here to talk through what we can actually do, where hackers can help, and how to stop putting good money after bad. Okay, so, who am I? As mentioned before, I'm the lead technologist at the Duke University Reporters Lab in the Sanford School of Public Policy because academics like longer titles and apparently that makes us more important. I'm a journalist and computer scientist. I've worked as a reporter and photojournalist around the world. I live in New York City. And I've been working on misinformation since about 2015. Previously, I lived and worked over here in Bosnia with a couple of the folks up front. And so I've been dealing with this from a lot of different origins and a lot of different places. Okay, first thing we're gonna do is go over some nomenclature. Words matter. I hate definitions. They get in the way. People get really pissy about them. However, a couple of them are really important, including misinformation. So what this is, is it's false information and it's spread without malice. This is your aunt. This is your uncle. This is your mother. This is you. It's me sometimes. I've definitely done it. Spreading misinformation on accident. You think it's real. You think what you're sharing is important. And then what you end up doing is pushing it out on accident. Second one is gonna be disinformation. This is the same information, but its intention is different. So instead of accidental, these are the people who are creating it who know it's false. This is the disinformation that we're discussing. And when it comes to fact checking, there's two variations. So this usually goes over with journalists, but I'm just gonna make it, just go over it real quickly for you guys so that if you hear it in a couple different contexts, it makes a little bit of sense. The first type of fact checking that really existed within journalism in general is internal fact checking. And what that is, is you do an investigation, you write a piece, you spend weeks, you spend months, you spend years on it, and then somebody else in your organization at your publication, what have you, goes through and fact by fact tries to reconfirm a lot of this, right? So you can call people, you can make sure that the stats add up, et cetera. The journalist hands over their sources and you run through it. Sometime in the mid 90s, people started considering what external fact checking is. So taking those techniques, but flipping them out and using them on people who have made claims in public, politicians, scientists, what have you. And that's the type of fact checking I'm gonna be talking about today. It's the type you've been hearing about for the last decade. It's the type that's really been increasing in size around the world. There's over 300 fact checkers these days. So, can technology solve misinformation? No, it cannot. There is no algorithm, there is no AI model, there is no anything that is going to catch all misinformation and make truth valid. And what is truth anyways, but let's not get into the universal philosophies here. But it's not. Anybody that says this is going to fix misinformation on Facebook, including Facebook by the way, is wrong. Now, is misinformation caused by technology? This is the problem, it is not. So that's a good thing, right? You can't, you don't have to take full responsibility for creating misinformation with technology because it's always been around. People have lied since the beginning of time, the person standing in this, running a stall in Athens says to the customer that the person next to him is gonna cheat him and their bread is better. However, did technology make misinformation much, much worse? Yes, there is a burden on technologists, on programmers, on people who increase this spread. Unfortunately, this is a problem. It sucks, I know. But what can we do? Let's talk about the types of misinformation. Let's talk about why technology is not gonna solve this stuff. It's subtle. Okay, first, what you can get is really obvious and really bad misinformation. This is the stuff that sometimes you can catch. This is stuff everybody pretty much ignores except the true believers in the first place. So what you have is something like this. The food run is out. New York Times is accused of normalizing cannibalism. This was created by Sputnik, the propaganda piece from Russia that is banned in Europe. I found out this morning when I tried to visit their website, I forgot about that. It has been spread by info wars within the US. It is weird, it's wild, nobody absolutely believes this crap when they read it, right? Like almost nobody. This is the easy stuff, here's another one. Joe Biden obviously did not resign from the White House. This was spread on Facebook. It was published by Politifact. Third one, public health of Scotland will no longer publish COVID-19 data because somehow it spreads AIDS? I, again, this is just random shit, right? This is coming out. And so this stuff isn't what we have to worry about when we talk about misinformation and fact checking in general. This stuff is easy. This stuff is, you know, whatever. You can dismiss this. It's all mostly spread on fringe organizations anyways. So that's not a problem. This is where we start to get in trouble. These are also fairly obvious, but they're way more dangerous, right? Turkish observers, US and EU hindering Russia-Turkey efforts to solve food crisis and mediate peace. This is also from Sputnik. This is trying to actively screw stuff up, right? This is trying to actively mess with peace processes, with food. This is going to actively hurt a shit ton of people in Africa. Sorry, I use profanity a lot. Fucking thank you. All right, this is another one. This is actively spreading LGBT hate across the US. This came from, I forget exactly what publication, but it was a fairly well-known one in the US, on the far right. And this is, you know, obviously on the face of it, you're going to look at it and be like, that's insane, sure. But it sounds like it could be real. This is another one. In the same vein, this is from full fact in the UK. And then we get to the really tough stuff. This is the place where people don't realize why misinformation is so hard for computers to understand. This is the subtle stuff. This is the stuff that sounds convincing. Politicians are professional obfuscators. That is their job. When they get up on stage, when a newspaper publishes, their job is to make everyone in the audience believe they heard what they wanted to hear, right? And so when you come to town to this sort of stuff, there's different ways of looking at it in the first place, and then people will believe it a little bit, and that's where you get into danger. So for instance, social media posts make unsupported claims about Zelensky's income and net worth. This is trying to discredit the entire Ukrainian attack, right? This is trying to subtly make people a little less sure that they should be supporting one side or the other. Same here. This was published. I actually, before I saw the fact check, I heard multiple very smart people I know discuss this exact piece of misinformation as a way to show how subtle and smart Putin is by releasing this and showing he's siding with Pakistan, right? You even see it says RT. Whoever made this is good. Of course, this isn't true. Somebody fact-checked this. The original image, if you do a basic reverse image search, is it's Putin on a massive video call with like 30 ministers or something like that. But whoever released this was trying to push a very specific agenda by making something seem convincing. It's not aliens take over the world. It's Russia's aligning itself with very specific parts of the world. And this is another one. This is from Nigeria. Every hour, 50 people die as a result of diseases related to poor toilet hygiene. The sad part is it's actually worse. It's actually 61. And so they're even underplaying it. And this is the really important thing that I want to get to. The point of misinformation in general, and this is why technology is not going to solve it, is that it is to convince you to trust nothing. It's not to convince you of a single individual fact. It's not to convince anybody in this audience that a very specific conspiracy theory is correct. It is to barrage you to inundate you over and over and over and over and over with more and more stuff until you are so sick of it you don't care anymore. And everybody in this audience is susceptible to this. It's not going to just be your parents, your grandparents, your cousins, et cetera. It is everyone. I feel it's to promote apathy and benign neglect. Like, I don't care anymore, screw it. And that's why subtle just taking down and fact checking individual things is really, it's a band-aid, but it's not going to, you know, it's not a suture. And so you really, really have to think about it this way. It's like, I can write an algorithm that'll fact check this one piece of information. Great. That is not the problem. Thanks. And it's probably only going to be for a very specific silo, a very specific type of topic that you can write an algorithm for. General intelligence is not coming along, right? So let's do something and take a look at the current structure of fact checking, this sort of whack-a-mole. I want to say first, this is important. As I said, it is a band-aid. It's going to stem some blood, but it's not going to stop it in the long run. However, I'm not even sure you can fix it, right? I'm not even sure that this problem is fixable, but maybe we can drop it down just a little, right? So here's one fact check. And this is one of the big problems that you get when you do this sort of thing. Fact checkers are pedantic. They like to be very specific, which computers are very good at, too. The problem is you can miss context. Now, these people are my really good friends. They're full fact in the UK. They're one of the best fact checkers in the world. However, they fall into this trap as well. Sure. The picture wasn't from Davos 22, as it claimed. And Davos isn't technically a climate change conference. However, the point is that they're all still, that everyone's still flying private planes all the way over, which is ridiculous, right? And Davos isn't technically a climate change conference, but it comes up. It comes up at these meetings. So is this context explored in the story? Yes, but are people going to scroll past this? Probably not. Here's another one. Ted Cruz said some stuff about James Comey that Obama appointed him, right? But that he couldn't vet anything. This is factcheck.org in the US. And again, they're missing these subtleties about what he's trying to get at. He's trying to cede a little doubt about everything, right? That James Comey in the US, if you remember the stories, can't be trusted anymore. But they're focusing on the very specific details about it and just trying to knock down this one fact while ignoring a larger narrative. And sometimes they get it right, but sometimes they don't. Here's a last one. Does Ron DeSantis oppose any background checks on guns, even for violent criminals? They rated it mostly false. The problem with this is that Ron DeSantis is the governor of Florida. Actually, it's pretty much true because he supports a constitutional amendment that would remove all background checks in the US. So yes, yes, he didn't specifically say that he doesn't support background checks or that he opposes them. But if you look at just a little bit further down the path, he actually does, right? And so you get these sorts of problems. And I'm running through my slides a little quick, but I imagine there's going to be a lot of time for Q&A. So if you guys have questions, please feel free to think of them and come up to me later. OK. So hi, everyone. I'm Mr. Happy. Fact-checking is, I've just been standing up here talking about fact-checking for a long time about how it's completely screwed over, how nothing's going to happen, how nothing can work. So what can we actually do? What can we actually do? What can technologists in this room? What can hackers in this room? What can academics in this room? What can just really smart people, which I'm assuming you all are, if you decided to sit in this camp for like six days at least and hang out with each other? What can we all do? Well, there's a couple options. But my favorite one is this. So I've been working on this sort of technology for a while. Not necessarily the tracking, but the collecting of misinformation and tracking of the misinformation itself. We have huge databases all over the world of pretty much every fact-check that's been made since about 2015. I have one. Google has another. They've actually, yeah, everybody hates on the big companies. And don't get me wrong, I will, too. But Google has been heavily supportive on this. And they mostly use it for putting a fact-checking like their search results. But it's a huge database. And it covers almost every country in the world where a fact-check's been published. So you got the Indians, you got the Brits, you got the French, et cetera. My question, though, is instead of just tracking how a single piece of misinformation spreads, how can we track who's making this? There's this really interesting statistic from the Center for Countering Digital Hate that came out in 2021. In February and early March, they tracked misinformation on Twitter and Facebook. And 65% of it, around COVID vaccines, I should say, 65% of it was created by 12 people. 12 total people created 65% of the misinformation around COVID, the lies. What's this called? What they are, their lies around COVID in the world. They examined, like, I don't know, a million-plus posts. And on Facebook, it was like 73. Twitter was another ridiculously large number. But if you average it out, it came to this. If there's only 12 people doing this, maybe it's not as big of a problem as we think, right? These are people with an agenda. These are people that are trying to specifically put out the information lies to inundate you. So they're not just random people making shit up. Like, it seems, to me, it seems sometimes, that it feels like. So what I'm curious is, are these profiles actually trackable? Not just from when they publish one thing and see how that spreads, which is important. And a lot of people are working on it, myself included. However, I want to see what sort of influence a single account, a single maybe person, has on the wider infrastructure. We need to think bigger than the individual pieces of lies. And that's something that people are struggling with. Because it's hard. This is hard stuff. Has anybody in here ever tried to just drink from the Twitter fire hose? Anybody? Yeah? No? Nobody? OK. So if you ever have, I mean, I'm talking back in like 2010, was the last time I tried it because it was so much. It's just a shot of water to your face, like a fire hose. It's literally called a fire hose. And you will be pushed up against the wall. There's so much data coming through every single second that is literally impossible for a normal researcher to manage. So you work on small data sets. That is a problem. Because we need to figure out how we can more quickly analyze these things, maybe not store them, but really start looking into stuff. So a couple of the other problems that you're going to be finding. What is even misinformation to a computer? Context is important. If a politician stands up in the middle of the day and says the horrible thing, the horrible tragedy that happened this morning, what's the tragedy? What's this morning? Who's he talking to? Where's he at? This can all be wildly different, right? There could have been a train collapse in Barcelona and also a wildfire in California. He's probably talking about the train. If he's in Europe, he's probably talking about the wildfire. If he's on the West Coast of the US. But computers are not going to be able to tell these subtle differences, right? The audience, you guys are fine. Computers can't figure this out. AI is terrible at this. What is a tragedy? I don't know. I mean, it depends. The definition is very fungible to a lot of people. To everybody in this room, you probably have a slightly different definition. And then you get to the biggest problem that I've been working on, which is matching claims to something you've seen before. So you can start comparing and tracking that, right? Because especially, well, English is my expertise. But I know, and this is true in pretty much every language in the world, is that a piece of information can be reworded and restated in a totally different set of vocabulary, right? So someone can completely switch it around, completely change the words. And now if you're doing a word embedding model, you're pretty much screwed at that point. Because maybe a couple percentages off can infer some stuff. But as soon as word defect gets a completely different sentence, it's going to start going utterly sideways. GPT-3, if anybody wants to ask about that, is not good at this. It's actually terrible at this. Because it doesn't actually reason. It just looks at huge models. And when you get to the next point, which is recognizing new claims you haven't seen, it completely fails. Pretty much everything completely fails. Because you can't tell if something's a lie automatically. It's not just statistics, right? It's people saying various restating reasons for a war 300 years ago. So that's going to be the biggest struggle that we're running into if we can even get to the content in the first place, because private groups are screwing all of this up for us. I love encryption. Encryption is awesome. It really makes it hard to try to figure out who is saying what, and where, and when. So you have small WhatsApp groups, or large WhatsApp groups that have very specific membership requirements. Still is the problem here. Telegram a little less so, because it's a little bit easier. That's billowing. But trying to get into these, where the pieces of misinformation, my theory is, are often created and sort of battle tested before they get spread to the wider community is an incredibly difficult prospect. One that the companies don't really want to work on, because that's not in their benefit. People should be able to talk privately. I agree completely. I'm in a bunch of signal groups. I mean, heck, the camp that I'm here with all started as a private signal group. We wouldn't let anybody in. But it's an ethical dilemma. Feel free to ask a question about that if you want to talk about it a little bit more, or find me later. But at what point is misinformation worth invading privacy? I don't know. A lot of people in here, I imagine, would say never. But I'd ask you to think about that for about three seconds longer, and just sort of roll it around in your head. So why do we even want to do this, though? What's the purpose? What's the point? If we're never going to change anything anyways, what is the point of all of this? What is the point of trying to write algorithms to track this? What is the point of trying to figure out who is saying what? Well, it's really standard journalism questions. It's who's making it? Who is doing this? Who's paying for it, which is often the most important, follow the money? Why are they doing this? What is their purpose? Are they trying to discredit somebody? Are they trying to just throw chaos? A lot of reports have shown that, again, I'm American, so I'm going to speak to this. I apologize. I know this is the most European audience, but I hate it when American speakers use nothing but American context. It drives me absolutely insane, but I'll just use this one. Actually, no, it works for Brexit, too. So the Chinese that it was found out, I was talking to a researcher just a couple of days ago, didn't support one candidate or another. They didn't support one person or another. In fact, they supported both sides. They ran campaigns to have them fight each other. I mean, Russia did this, too. They would set up counter-surveillance, or not, counter-protest against a protest they had also seated and set up. This is, my water bottle's annoying me. I'm sorry, guys, they would set up a protest, then set up a counter-protest, and then watch each other, then fight each other. That point was just so discord. It wasn't to do anything. It was just throw wrench in it, and that is a huge problem. When it comes to this, can we cut off this head? Can we stop these people ahead of time? Can we, censorship is another touchy subject, but can we sort of proactively censor these people after, again, after they've been doing a lot of things, but before they really build up the momentum and start huge, massive campaigns? This is going to be a problem. They start small, and then they gain credibility. They gain traction, they gain followers, and then it's so big that the major social media companies start asking questions on whether or not they should be taken down in the first place. This is a struggle. So can we cut this off? Can we stop this at the source? So my idea, and I want help with this, please help me, please talk to me if you want to work on this, is can we fingerprint these fact check, these misinformation and lie creators? Is there a way that we can come up with a system that looks at the vocabulary, the style, et cetera, and finds deeper patterns so that when we get a new piece of content or even we can look at it historically, know that at least they're probably created by the same people. We might not know who they are. We probably won't know who they are for a while, but if we could start grouping pieces of misinformation, it's important. So we want to identify the new actor early, as I mentioned, and then we can start looking back at historical records and information and see where we went wrong in the past. We can learn from our mistakes. We can better insulate ourselves. We can do a mea culpa. Everybody screws up, but let's see what they tried to do before and let's see what we can do now. Okay, this will also help us detect new trends, I hope, because new topics come up every fifth day. There's some new storyline they're trying out, and they're gonna try it out in small groups first. They're gonna try and see if it bleeds out a little bit, but if we can see that early, then we can start counteracting it even earlier, and maybe that's not important for some of the big obvious lies, but if the lie is instead around, I don't know, is Russia gonna invade Ukraine? I'm wondering if we could have detected that in November a little bit better. Are these big questions, are these big ops easier to find? So let's, I don't know, I think there might be a machine learning algorithm methodology to this that can find a deeper hidden pattern. We'll see, I wanna try, I wanna try everything, I wanna throw the kitchen, everything but the kitchen sink at this, and if anybody out there wants to help, I would very much appreciate that, okay. There we go, anyways, as I said, I talked a little bit quick at the beginning, so I sort of finished this up a little early, but thank you for hearing me out. This is a really important topic, this is a really complex topic for a lot of people to try to understand, and I'm sorry if I did not the best job at it, I think I tried. Please try to understand where we're coming from, recognize it yourself, talk to people about this, explain, it's not just that single lie, it is way bigger than that, and let's try to build some tools to actually make this better. This is my contact info, I'm not gonna put my phone number up there like I usually do because I don't want weird people calling me, but if you have any questions, I'm here to answer them, it can be about this, fact checking if you're curious in general, journalism, the computer science behind it, whatever you want, thank you guys. Thank you Christopher, so if you have questions, please line up behind these microphones, you probably know that well now. There's the first question. Thank you, hi Chris, the thing you said at the end reminds me of something called the attack framework from Mitra, which is related to exploits and APT groups, and it's both a framework for categorizing the methods they use, but also naming them and which groups use which methods, so maybe you could do like a parallel for that, and if somebody knows of a parallel for that for your uses, then please say so. Yeah, that's, yeah, absolutely. I think I've heard about that program, and it definitely was part of the inspiration for this sort of idea of like how can, in my notes, I didn't bring it up, but like can we create a CVE program basically or misinformation and spreaders of it? Yeah, thank you for that, I'd love to talk later. Some from the behind microphone. Hey Chris, nice talk. You should be swearing more, this is not US television here. Fucking sorry about that, god damn mother fuck, sorry, yeah. There was one aspect I was slightly missing, that is, I understand that some people just wanna see the world burn, I get that, but many do it for fun and profit. If I look at people from Alex Jones up to Zuckerberg, they make money with that. Isn't draining the swamp also a strategy? Absolutely, yeah, there are a lot of people funding a lot of this. However, in my experience, they're not the ones actually creating a lot of the initial push for the misinformation for the lies, as I showed earlier, there's that story that was on InfoWars, but it was originally published by Sputnik, so the Russian Propaganda Department actually had something to do with that, and then it was spread in these more local organizations. Well, hello, speaking of the Russians, I noticed from my personal experience that when the invasion of Ukraine began, the amount of disinformation that we were seeing on other topics disappeared, and we saw a new influx of different types, is that something that's modelable and trackable? You know, that's what I think it is, that's the thing. So if we can fingerprint these creators, that fingerprint will probably, I hope, spread across different content topics, right? So it's more than just, it would be fascinating to see who switched their misinformation creation from one topic to another at that exact moment. That would be absolutely amazing and really indicting against the whole organizations and such thing. Thanks. So please use both microphones. While the next person on the first microphone, please. Right, right. Hey, Chris, thanks for your talk. Yes. Really important stuff. So what do you think could be a role for legislators in this? Could you make laws or something that helps us in this quest for banning misinformation? Yeah, why did I get a stick mic all of a sudden? Okay. Can you hear me? Everyone, can you bring it? Okay, cool. You know, that's gonna, that's interesting because as an American, our standards of free speech are very different than in a lot of Europe. I mean, substantially they're quite similar, but there's very interesting specifics that are very different. And so, how do you do that? What is misinformation? There's so much, you can never say that everything's a lie, right? Like you can never, it's gonna end up filling up the court system so much. Do you, how do you punish somebody first, you know, accidentally say something that was wrong? Yeah, right, but there must be some role for legislators in here for like maybe big money, ban big money on Twitter accounts or something like that. There must be some way we can help there, right? You think? Maybe we'll talk after. Yeah, let's talk after because I need to roll that around in my head for a second. Yeah, please. The other microphone first, please. Yeah, I was first. Yeah, okay, then the first microphone first. Hello, so I find it interesting that you focus on identifying the original sources of misinformation, and especially in your response to the, when you talked about Alex Jones and how he doesn't create this misinformation, he just takes it and then amplifies it, yeah. So I find it interesting that you focus on the original creators quote unquote, instead of those who amplify the spread of misinformation because for me it is, you said that it's difficult to get into those closed groups and all sorts of things. It's difficult to identify those original creators, but on the other hand, it is quite easy to identify those who pump up and amplify, right? So why not attack the problem here and at the amplification level, right? When this actually becomes a problem, when this actually gets spread to millions of users, instead of trying to find the original creators. Sure, because that's being done. I wanna think bigger. Nobody's really looking at this question, for the most part. I want to go deeper. There's a lot of people, including at these major organizations, who by the way know who these people are. They could take them out tomorrow, but they're not gonna take Alex Jones off because he has five whatever billion followers. But yeah, I wanna think, this is a different problem that I'm trying to focus on. Not just the spreaders, but I wanna know who is behind it. You know, like investigative reporting, you wanna go as deep as you possibly can and then build it up from there. Thank you. Other microphone, please. Thank you. Yeah, fake news and propaganda has been around as long as humanity is around a bit. In your presentation, you mentioned that technology is amplifying this, but shouldn't you say surveillance capitalism is what's doing the amplification and not technology itself? Could you expand a little bit more on why how surveillance capitalism would be causing this? By optimizing all the algorithms for attention and polarization, because if you push around fake news and clickbait, people keep clicking. Sure, so yeah, so that definitely doesn't help at all. That makes it much worse, but also the fact that you can spread information instantly anywhere in the world, even before it's amplified by some crappy algorithm, shitty algorithm, sorry, I should swear more, is one of the big causes, right? Without the internet, you couldn't amplify it in the first place. The fact that Sputnik can post something and I can read it in Brooklyn, New York means that that is way more likely to spread in general. If I was living in Brooklyn, New York in 1975, I could not get a copy of Pravda, right? Okay, but the likelihood that you're reading it, that's caused by these algorithms that are pumping this around. Not disagreeing there at all. That is absolutely true. Like I said, technology made the whole thing much, much worse. That is, those are part of the technology. Is technology part of the social and economic system? Yes, but at some point I could have gone, if I wanted to do just a lecture on the economics of this, that's a whole different thing. Okay, thank you. Next question, please. So I had an immediate uncomfortable gut reaction when you started talking about building a tool to track down who said a thing. Yeah. And maybe you could reassure us a little bit about why you're not creating a scary tool that bad people will use to go after activists? Absolutely true, because I'm a good guy. Yeah, right, that's what we all tell ourselves. It's how I sleep at night. The bad guys probably have a lot of this already. They're just not letting anybody know. That's where I fall on that. It's a really good question and it's an ethical question that we would need to discuss. At some point, every technology can be good and it can be bad. There is no neutral technology and only bad users. And that's a good point. It's something that we do really need to discuss, but at what point do we just throw our hands up then and say, well, they could do whatever they want because we'll never find them? I don't know, what's the trade-off here? Okay, Baren needs to follow up on that real quickly. I guess one of the reasons I sat down and didn't ask the question the first time is I thought to myself, well, most of the people that are trying to make the world better are not particularly ashamed of it. So I guess it would be interesting to focus on what kind of activism needs, that kind of anonymity and whether you would be putting them at risk. Right. And a lot of it. But it's worth thinking about. Absolutely, and let's have a beer and talk that over a lot more. Yes, you can do probably a lot of more questions after the talk with a group of people. So going deep into the questions. Next, from the cyber guy. Oh, hi, I'm a cyber guy. Okay, two things very quickly. First, your premise is that technology will not solve the problem. But I happen to know that you've been developing some technology to help solve the problem. And I'd like you to tell us a little bit more about what exactly that is. Second thing, we know that there are fact-checking websites out there, but most people don't visit them. And so most people will never see a fact-check. And so can you talk a little bit about how, what kind of strategies we can employ to improve the general public's ability to defend themselves against misinformation to understand the context and nuances of what's going on? Thanks. Yeah, thank you. Those are great things. Thanks for asking me to promote myself a little. So the big thing, so at the Duke reporters lab, the big thing that I've worked on over a number of years, which is not a failure, but it's definitely not good enough what is automated fact-checking. So I said that you can't automate fact-checking. And I know that because I've tried a lot. The big thing that we built is a product called Squash. I did not name it, my boss did, because you squash bugs, I guess. The code base is called something very different. But basically you can listen to, it listens to speeches, by politicians, debates, et cetera, automatically transcribes it all, then runs it through another system created by a group at the University of Texas at Arlington that can detect fact-checkable claims from something which is harder than you really think. It's way harder than you think. But it's really good at it, especially for American political speech. There's another group in Argentina that's written something there called Chequeado that does something very, very similar and is excellent in Spanish, especially Argentinian Spanish, though they worked with Spain too, on the UK, full fact, did something similar too. And then so you get the full claim and then you try that claim matching. You try to figure out if somebody else has checked this before and then show that straight over the video. Sounds awesome. The problem is we're only at about 50, 60% of accuracy when it comes to claim matching and because the topics change so frequently, it's really hard to build up enough of a data set around a specific claim from a very specific race. The other thing that I'm working on right now is an archive of fact-checked media. One of the big, that's what's gonna help, I hope, with the tracing is that right now when a piece of social media is fact-checked, that video image, audio, is taken down pretty quickly, hopefully, that's sort of the goal. The problem is then we have this structured data that is completely meaningless because you don't actually have the image it was fact-checking or the meme or the video or what have you. And so we're building a entire archive that will automatically download all this the moment the fact-check is created, hopefully before the platforms take it down. And then that's searchable, that's reverse image searchable, reverse video searchable, text, OCR, et cetera. And with that, we're building up this huge data set that I'm hoping we can start looking at for this finger check. When it comes to how do we get people to read fact-checks? Good God, if you could figure that out, I have a lot of grant money for you. The problem, that is the big problem. You end up with a preaching to the choir issue. Basically, nobody would go to a fact-check website that doesn't already agree mostly with the fact that there can be truth that people lie. And so you sort of have to reach them where they are. One of the big things is Google putting fact-checks directly in search results has been really useful. Yes, they've done something good. They have the fact-check insights program where they pay fact-checkers to fact-check stuff that has been reported. And when that's done, everybody that ever shared it gets a notification with a link to that article that checks what was said. So every single person that shared it can read it. What else can we do? It's going to be challenging. We're thinking about putting stuff on petrol station TVs. It's a throw stuff at the wall. Yes, there are two more questions in the audience. First, the pink T-shirt, please. Yeah, thanks for your call. I appreciate the thought to identify the actors behind that, especially when there is very small group. But I have to admit, out of curiosity, because I can't see the effect if these actors are known, because my impression is it doesn't make a difference for anyone spreading misinformation what the source is. If I know, hey, that's Russian propaganda, they don't care. They spread it anyways. What's the benefit of knowing these actors? So it's good just to let people know. People should at least be able to know who created what they're reading. But also, this is important for policy. This is not just for the individual user. It's going back to the legislation question. This is good for policy makers to understand where this is coming from, that what they're spouting, the facts or the statistics and misinformation they're repeating often on in-parliament buildings is wrong and where it came from. If we're talking about censorship, that is somewhere where you can start. We can't start if we don't have the information about who's creating it in the first place, right? So it's really a bigger policy. It's bigger technical thing. It's bigger than just to the individual user. The other microphone, please. There's one question about outright lies evolving in somewhat plausible theories that reach mainstream or reputable media. I'm trans. This is my topic I follow. And I see that a lot with transphobic talk that started as outright obvious lies, but certainly evolved into the gender critical movement as they call themselves. So what do you do about that? Well, as you said, they started somewhere. And so I'm hoping we can find where they start before they start evolving and moving, right? It's like trying to catch an invasive species at when it's rolling off the ship at the docks rather than scouring the forest for it. What you can do after the cat's out of the bag, that's a really tough question. It's something that I've been struggling with for a long time. And there's so many topics where this has happened, right? Flat Earth, Pizza Gate, et cetera, the US. And it just becomes really, really challenging at that point, but that's why I wanna see if we can cut off the head before it grows too big so that we can stop those early. So next question, please. Hi, thank you for your talk. I was wondering who should be trying to identify the head and who should be the one trying to cut it off? Like, should it be academics, journalists doing the research on who the heads are? Should it be government? I know government is really struggling with the question on how to deal with minsofo because you also get into the freedom of speech stuff. But also, I don't know, I guess government should do something, right? So what are your thoughts on this one? I mean, government should do something. And when it comes to cutting off the head, they're probably the only ones that can, in a lot of ways, them and the big platforms, of course. So it's not the message, it's the media, right? So who has the ability to stop that spread of those specific people when they're, or organizations or what have you when they're identified? That's going to be government, that's going to be big tech companies, that sort of thing. It definitely, journalists and academics, hackers, please, for the love of God, should absolutely, I think, should be the ones doing the research when government starts researching this. Of course they haven't, right? CIA, MI6, GCHQ, NSA. They've all been doing this for years, I'm sure. But they're not making it public because it doesn't benefit them to make it public. It benefits them internally. We'll see. If anybody, hey, where's the Fed? Come talk to me. Yeah, thanks. Sorry, I wish I had better answers about the government. It's tough and every country is going to be different because every country has different laws. The freedom of speech in Thailand versus freedom of speech in Argentina, you're going to struggle on both sides. Last question, please. Yes. So I was thinking a bit that for now, all these measures and research is going into somehow treating this condition that's affecting the public. But if you continue with this analogy, maybe it's also better to look at, let's say, immunization of the population because maybe there is a fertile substrate for this kind of discord to spread because, as you said, it seems like the real end game is to actually see discourse between the public in various groups or the whole public of a nation. And so maybe it's actually due. Of course, there is no control group for this because the technology group with these new problems we're facing like climate change, economic conditions, worsening, the decline of the middle class. So maybe it's also that we should think about changing also why the people are more prone to share and fall into this kind of misinformation and mistrust towards the government, for example. And yeah, I don't know. It's more of kind of an open question. Yeah, absolutely. And that's something that a lot of people are looking into. So how do you educate the youth? How do you educate adults? How do you make them less prone to believing this stuff in the first place? My usual answer is, that's awesome. It will solve the misinformation problem in 20 years when those kids grow up. We have a problem now. Adults, it's real piecemeal. The group Politifact, who I'm quite good friends with in the US, have tried this a lot. They've had a lot of success on very limited scale by going to public libraries in these small rural towns and literally basically doing this, like ask the fact checker, and then surveying. And it turns out, yeah, people tend to trust the media and fact checks more afterwards. Trust the people that you would normally get it from less. The problem is you get like 20 at a time. So what are you going to do at that point, right? There's a lot of resources. If anybody's out there, International Fact Checking Day is the day after April Fool's Day. Get it? April 2nd. And there's a ton of education resources, including curriculum and price. I don't even know how many languages, but at least eight. We hit the main ones for sure. I'm sure Dutch is on there. And so if you're an educator or somebody looking for that sort of stuff, that's available. And there's a lot of researchers around the world working on the education standpoint. I was just hanging out with one. She works at Temple University last week who's making some great strides on it. Yes, thank you. We are running out of time. Thank you for all the great questions. And thank you, Christopher, for the talk.