 So welcome everybody. Thank you for coming to this talk. It is my pleasure. I should probably introduce myself. I'm Kevin Crosston, the Associate Dean for Research at the Syracuse University School of Information Studies, and it is my pleasure to welcome you all to this research talk, and I'm delighted to be able to introduce my colleague, Jennifer Stromer-Galley. Jennifer is a professor in our school. She currently also serves as Senior Associate Dean as well as conducting an active research program, so it's not clear when she sleeps. She was president of the Association of Internet Researchers from 2015 to 2017, and she's been studying social media before it was even called social media, studying online interaction and strategic communication in a variety of contexts, including political forums and online games. She has published over 50 journal articles, proceedings and book chapters. Her award-winning book, Presidential Campaigning in the Internet Age, gives a history of presidential campaigns as they have adopted and adapted to digital communication technologies, and today's talk follows in that pattern. During the most recent presidential campaign, Professor Stromer-Galley and her team analyzed the candidate's Facebook advertisements, their messages, the tone, who was targeted, and how much was spent in ads. It's a project called Illuminating, and you can find a lot more detail on the website. Her talk today is based on this work, the steady drumbeat of election fraud during the 2020 presidential election, the role of Facebook advertising. So I'm delighted to ask Jenny to start her talk. Thank you, Kevin, for hosting this, and thanks everybody for taking a little time out of your busy days to come and revisit the 2020 presidential election. I wasn't sure anybody would want to show up because I'm still traumatized. So today, I'm going to talk with you about some research. I have to confess this is really hot off the press, and I'll explain a bit more why that's the case. So I'm still thinking about this analysis and what it means, and so maybe together we can think a little bit about what it is that is going on with regard to Facebook advertisements by the campaigns. So the agenda today, I'll first give some context about the get out the vote efforts by political campaigns and why they matter, and the context that we came into in the 2020 presidential election in the United States. I'll unpack campaign advertising on social media, and when I say social media, what I really mean here is Facebook and Instagram, because that's what we have data for. That's a challenge and an issue in and of itself, but that's the reality. And if you hear me say Facebook, which I'll use a lot through this talk, what I really mean is Facebook and Instagram. So Facebook, the corporation that owns the platforms, Facebook and Instagram, that's what I'm talking about. Then I want to briefly give you a touch of theoretical flavor. I am an academic. I like academic terms. So I'll talk a little bit about datification. So when you leave this talk today, you can feel assured that you've got some good jargon to show off with your friends. And then I'll dive into some of what Trump's campaign was doing on Facebook around their get out the boat messaging and the extent to which it seems to have further contributed to the claims and concerns around election fraud in the United States after November 5th, whatever the election date was. And then we'll spend some time for questions. One thing is, Kevin is watching the feed. And so if you have a question while I'm presenting, don't hesitate to type in that question. And if it makes sense to interrupt, Kevin will do so. And I'll answer those kind of in the moment questions. Happy to do so. Yes, I should have mentioned that there's a Q&A button at the bottom of the screen. And if you click on that, it will open up and there's a place to type in your question. Yeah. I'm going to try to actually show you some videos of ads along the way too. And I'll check in to make sure that sounds and works okay as we go. All right. So let's go back in time, shall we, to just a few actually about a year ago to the 2020 presidential election. So I want to underscore, and maybe it seems obvious, but elections at the end of the day are about the vote campaigning all of that effort, all of that money, all of the travel and the discussion and angst and polarization that occurs around political elections these days is in service of getting people to come to a polling place on a given day and time to cast a ballot for their party for their candidates from president all the way down. And that's it's important to keep that in mind because campaigns spend a lot of their effort trying to identify who to mobilize to turn out to vote on election day. That's really what campaigning is about. Now that was complicated in the 2000 presidential election. It is actually about this time last year that the nation began to realize that we were facing a global pandemic. It also happened to coincide with the presidential primaries. The states, each state in the United States takes a turn or together over a series of days, they elect their nominees for the party to serve in a contest for the general election for who will become president. You might remember Wisconsin was one of the early states Wisconsin's primary was in April scheduled for April 6. And in the month of March, there were legal contests about whether Wisconsin would still hold an in-person primary or if they would shift to absentee or vote by mail balloting for the primary candidates or maybe postpone the vote to a later date because maybe the virus would be less virulent later. A event hindsight 2020. So Wisconsin highlighted that debate that occurred and you might remember that in Wisconsin the Republicans were able to block a democratic effort to either shift absentee or change the date of the vote. And so Wisconsin showed up on us to vote long lines leading to questions about how safe would in-person voting be. So while Wisconsin was having this very public legal fight about whether people needed to show up to vote in-person at a polling place, other states made a number of changes to their voting regulations to make it easier for people to vote. And that included shifting to mail-in balloting. And mail-in balloting took on different dimensions, everything from states sending applications to registered voters to states sending ballots to registered voters so that they could then return those ballots. Other states like New York made it easier for voters to get absentee ballots so that you didn't necessarily have to have a reason to get an absentee ballot or in some states you didn't have to have a second signatory on your absentee ballot. Some states allowed for ballot drop-off boxes. And these efforts all were trying to make it simpler, easier for voters who had health concerns and did not want to go to a physical polling place to be able to vote when it was their time to do so. Now President Trump in starting in March began to raise questions about mail-in balloting. In April, he tweeted a number of false claims about balloting and mail-in balloting. So for example, this tweet Republicans should fight very hard when it comes to statewide mail-in voting. Democrats are clamoring for it. Tremendous potential for voter fraud. And for whatever reason doesn't work out well for Republicans. For this tweet on April 12th, mail-in ballots substantially increased the risk of crime and voter fraud. In March, sorry that's not true. In May, Michigan and Nevada and California all were looking at mail-in voting and making changes to the vote, allowing people to do mail-in balloting rather than having to go to polling places. And again Trump tweeted out a number of false claims in May about the risks of voter fraud with these mail-in ballot scenarios. So this tweet on May 26th on your left, there is no way zero that mail-in ballots will be anything less than substantially fraudulent. Interestingly on television in May, Trump ran a large ad by on TV that focused not on these questions of mail-in ballots and ballot fraud, but instead on Biden's sort of ability to lead. This ad ran in a number of states and it basically questioned whether or not Biden was competent to serve as president. Interestingly, about the same time that this TV ad began to run in May, the Trump campaign bought a fairly large ad by on Facebook. This particular ad by as you can see from the text basically declares that Democrats are stealing the most important election of our lives and stating that Democrats are stuffing the ballot box with fake and fraudulent votes. So what I want to talk a little bit more about is what was happening on social media around the presidential campaign for the Trump campaign. So let me unpack social media advertising. I'm going to go fairly quickly through this. If you have any questions, this is a good place to say, wait, wait, what? But I want to give you kind of an understanding of how it is that social media advertising works and why it's so important. So if you had a pulse in 2020 and you used Facebook, Instagram, Google, you likely were exposed or YouTube, especially YouTube, you likely were exposed to social media advertisements by political campaigns, whether it was the presidential campaigns or political action committees, they were everywhere. Our analysis, so my research team and I, in looking at the social media messaging by the presidential candidates, we estimate that about $160 million was spent on Facebook and Instagram between June and November 1st. That is unprecedented amounts of money. By contrast, in 2016, it was about $40 million. Now that $160 million or whatever I just said that was spent on Facebook and Instagram advertisements was only about 40% of the total amount of all ad spending by the political campaigns. Television advertisements actually make up the bulk of ad spending by political campaigns. And that's because TV ads are expensive. When a campaign runs in television advertisements, they're not buying to a person, they're buying to a region. If you live in Boston, it's a big community and it touches not only Boston, but also parts of Massachusetts, sorry, New Hampshire. And so it's a big expensive media market. And that is part of the reason why Facebook, because it's targeted and I'll talk more about that in a second, is much less expensive because you're reaching fewer people, but it's more efficient. Our estimates suggest that there were over 200,000 ads run on Facebook by Biden and Trump. And that's just their main pages. I can unpack more details like that. But just the official Biden and Trump campaign pages on Facebook, 200,000 ads, that's a lot of advertisements. All right. So why? Why are campaigns spending all of this money on Facebook advertising? Well, it has to do with Facebook's algorithms that make predictions about who you are as a user of the platform and the kinds of interests and hobbies and network of friends and things that you say on a daily basis. All of this activity that you are doing on the platform, Facebook and Instagram ends up being fed into algorithms that predict who you are as a person. Now, not really. What it's really doing is making predictions of what the algorithm thinks you are based on how you use the platform, whether or not that's accurate. You get sort of de-individuated. That is your likes, your interests, your habits. They may not really reflect who you actually are, but the algorithm is making these predictions based on what you're doing and saying. And so you yourself, this whole rich person with family and friends, et cetera, get basically created and structured in the database as a series of activities and interests, some of which may be true, some of which may not be. But the power for Facebook and the power for the political campaigns in using, so the power for Facebook is, of course, they can sell this prediction about you to advertisers. The power for the advertisers is then they hope that they can find that needle in a haystack, as Brad Pascal said, who was Trump's digital director in 2016. So I'll take a concrete example. Here in the city of Syracuse, most Syracuse residents who are registered voters register with the Democratic Party. But there are Republicans who live in the city of Syracuse. And so the power of Facebook is that if the Trump campaign wanted to target Republican voters in Syracuse, rather than doing a television ad by where now they're advertising mostly to Democrats, which is inefficient, they can run an ad targeted at Republicans who Facebook says these people are likely Republicans, and then you can find that Republican needle in a haystack. So where does Facebook, sorry, Linda, where do the campaigns get all of this information to make a prediction about who a voter might be? So again, Facebook is making predictions, but the campaigns are making predictions as well about who you are, what your interests are, if you're likely aligned with the political party or not. So I will unpack this at length, and I need to give credit to Aitan Hirsch and hacking the electorate for unpacking where campaigns get their data. It starts with voter registration data. If you're registered to vote, campaigns want to know that because you're somebody that they can actually get to vote on election day. If you're not registered to vote, then it's harder for campaigns because now they have to figure out how to get you registered, they have to find you, get you registered, and then try to get you to the polls. It's a much harder lift, much easier to go ahead, find those people who are already registered, find who are the Republicans, who are the Democrats, target those on your party, and get them to vote on election day. Now, voter registration is key. In many states, you indicate what your party identification is, and so for campaigns, they're good, they know this, but there are states where actually party ID is not part of the voter registration form. When that happens, the parties need to gather additional information to make predictions about if you're a Democrat or a Republican or none of the above. So public records, if you are a police officer, if you are a teacher, if you are a beautician, you need to get licensed in your state. That is public records, that information is valuable to political campaigns because it helps them then identify likely Republicans based on that profession. And then I'll go all the way to the right here for the internet. So layering in then all this different sorts of data, now we have the interests and habits that people undertake as they use the internet. All that gets layered into these databases that the campaigns can use to make predictions of who it is that they want to target on Facebook. So now I want to talk a little bit about Facebook's advertising offerings, so that you can get a sense of the different ways that Facebook makes itself useful to advertisers like say Donald Trump's campaign. The most common ways that the campaigns use Facebook for advertising is the custom audience. Basically what that means is if the Trump campaign wants to target men who are licensed as police officers, again using the FOIA freedom of information requests from states, they can get data about who are licensed police officers in those states. And they can build that campaign list then of known police officers. They then pass that list to Facebook. And Facebook matches that list from a campaign to Facebook users either on name, email address or other personally identifying information. Those people who are matches then get the advertisement that the campaign wants to go to men who are licensed police officers. I should note that Facebook never ever sends back to the Trump campaign that matched with Facebook list. What the campaign gets to know is a percentage of how much of their campaign list actually matches. So over 20%, 40% of the lists are matched, but they don't actually know who the matches are. You'll notice that the campaign ads always on Facebook and Instagram have a hyperlink so that if you click and engage with the ad, you will then go back to a website. The very first thing the website is going to ask for is your email address. So campaigns create a feedback loop through their advertisements to get your email address they hope because then that further gives them confirmation of who actually on that campaign list was activated by that advertising. Then there are the interest audiences and in my talk a little bit later I'm going to talk more about these interest audiences. So basically let's say that Trump wants to target people who are interested in gun rights. So they have them basically just go to Facebook and say we want to target people who are interested in gun rights. And Facebook will actually help them identify keywords or interest names that might be good matches for people who are interested in gun rights. So for example it might be somebody who follows the NRA Facebook page or somebody who on their Facebook wall under interest indicates gun rights or conceal and carry or the second amendment. So those things that are kind of keywords almost that are matches of interest between what the campaign wants and what Facebook has predicted users interests are based on how they use the platform. Those matches then are then given the advertisement. And then there's the third which will look like ads. Now this is fascinating. So basically let's say the target is people who look like law enforcement. So again let's say the Trump campaign really is going after people in law enforcement. They've built up a list of people that they know are in law enforcement. So police officers, security, maybe a sergeant, somebody who's part of ICE, maybe somebody doing corrections. So they pass that campaign list to Facebook and they say hey Facebook we know these people all are affiliated with law enforcement. Use your creative algorithms to see if you can figure out what the sort of features or profiles are these people. And then those people who look like our list of law enforcement, we want those people to get our ads. And so you can see here this is my exemplar example. You'll notice that some of the people that might be look likes aren't actually matches. So you'll notice that one's an immigrant, one's a student, one's a professor. And so those folks aren't actually in law enforcement but for some reason some aspect of what they're doing on Facebook makes them look as if they are law enforcement. So they get targeted to the ad. This is less efficient but part of why campaigns love this is it potentially surfaces new interested voters through this mechanism. Okay so datafication back to the 10,000 or what's the $50 words, 50 cent words, I don't know whatever let me give you some. So datafication is this idea that voters are being segmented into not only party and voter turnout, that is campaigns want to know two things. Are you part of our party and do you turn out to vote? And those have been the key variables but now with all this additional data, campaigns can begin to further nuance or datify their targeting to particular segments of the public based on their interests, their affiliations and their engagement levels. So in other words, the voters are de-individuated. We are broken into these fields and databases of interests that then are used to make predictions about whether you should get an ad or not. And for the party, that means or for the part of the campaigns, it means that voters are fragmented, de-individuated, which allows the campaigns they hope to really target more effectively people and ultimately at the end of the day win the election. And that's part of the argument that's in the book, which is why the picture of the book is in. Okay so I'm going to start talking about Trump's advertisements that he ran on Facebook during the end of the primary general election. I need to give you a couple of sources so you know where my information is coming from. The first is the illuminating project. So illuminating is the love of my life and there's a big team of people who helped me with this project. You can visit the website, the URL is illuminating.ischool.syr.edu and you can play with the data yourself. We basically are accessing Facebook's ad library API. What that means is that there is a doorway between our servers at Syracuse University and Facebook as we are an approved, we have a data licensing agreement, which means that Facebook is giving us access to the Facebook ad library to then analyze the ads that ran during the general election and the primaries by the presidential campaigns. In addition, just this week we got a hold of this data set. Facebook announced in January that they were going to release to academics who have data licensing agreements a special set of data that allows us to see more detailed fine-grained micro-targeting. The Facebook ad library and API only provides information about the state, for example, that a target ad was run in and some generic, not generic, but some more broader categories like age and gender, but on a few categories in terms of targeting. This new data set of micro-targeting data allows us to see what interests the campaigns were searching for in their advertising. It also provided more fine-grained detail about location, geographical location of who was being targeted in the advertisements. So that's why this analysis is really first blush because we just started playing with this data. It's a complicated, messy data set. So bear with me. Here we go. All right, let's talk about Trump. So remember, as I mentioned to you May 2020, while Trump's advertisements were talking about Biden on television, on Facebook, this ad was running. And it was a national ad by at random in most of the states. And relatively speaking, it was a large ad by. Now you might say $20,000. That's nothing, especially when you think about how expensive a television ad is, but $20,000 in Facebook buys you a lot of impressions. So this was a very large ad. And I just provide here on the right, here's some demographic information. This is summarized actually from the ad library. This is not our analysis. But you can see it's a pretty even spread. Like they were scattering far and wide this particular advertisement to the public. Then in August, this ad ran. And I'm going to try to show you or play the audio because it matters a lot to the context that I want to share with you. So give me one second to jump over. And I'm going to hit play. And hey, Florida, I'm Laura Trump. And I want to make sure you know about absentee voting or what Floridians refer to as vote by mail. My father-in-law, President Trump has always supported absentee voting with safeguards in place. We want you to know there is a difference between the Democrat imposed vote by mail system and the vote by mail system that you have in Florida. In Florida, you and the voter must both register and request your absentee ballot. This is very different and much more secure a process than when the Democrats mail everyone in a state a ballot often resulting in ballots being mailed to abandoned addresses and individuals not registered to vote. With this in mind, if you have passed your ballot by mail in the past or if you are planning to do so for this very imperative November election, you can visit both.DonaldJTrump.com to verify that you're registered to vote and learn how to obtain your absentee ballot. All right, so I'll stop that there. So you might have caught a couple of things from that. The underscoring that Democrats would be sending in, sending ballots out to everybody in a state, including to abandoned homes and people who are not registered to vote. Now, again, not true. Let me be very clear. That claim is not true, but that was reinforced in this advertisement worth noting that this ad only aired in Florida. The buy was fairly large, $13,000 and had between 350 and 500,000 impressions in Florida. Now, I asked myself, why Florida? Why just run this ad in Florida? And I don't have all the answers. I think some of it is because Florida was a very important state to Donald Trump. He needed to win Florida. I think because Mar-a-Lago is his second home, I think he also felt some connections with Florida, wanting to really make sure he won that as in effect his home state. But also, it is a state with a lot of older voters who probably didn't really want to go stand in line to vote at a polling place would much rather vote absentee. And so because on the one hand the Trump campaign in their press statements on Twitter, Trump was reinforcing this idea that mail-in ballots were fraudulent would be deeply problematic to his election win, he had to counter his campaign really, had to counter that message by directly targeting potential Republican voters to say, yeah, I know we're talking about mail-in balloting being bad, but let's be clear here. We're not talking about absentee balloting where people have to go and register and get their absentee ballot. We're just talking about that and it's really, really important that you do that. In other words, the Trump campaign had to counter their public message with these targeted ads to Republican voters to reinforce the message that it was okay actually to vote absentee because if they didn't, there was concern that they would lose those voters. They just wouldn't turn out to vote on election day. So also in August, a similar ad ran, again, you'll notice as a slightly different tagline get your absentee ballot applications day, we need to vote to save our country absentee ballots are good. I need you to get your application and send in your absentee ballot immediately. So let me show you this. There was a question that came up and seeing August prompts me to ask it now. Have you gotten to the point where you can see differences in the campaign strategy with the change in the ad director you quoted, Brad Pascale? Or you're not at that level yet as the analysis? We are not at that level yet. It's an interesting point because Brad Pascale was fired in, I think at the end of July, there is a change in tone in the advertising, just my skim of it, but it's early days. It's a great question. Things actually bring it up. So let me share with you this ad because it's also kind of interesting. Unfortunately, I won't be able to make it to the polls to vote in the Florida primary. I have some very, very strong opinions in the primary, but even stronger on November 3rd, you know what we're going to do? Make America great again. That's what we have to do. We have no choice. We have to save our country. So I'm signing today an absentee ballot. Absentee ballots are not universal mail-ins. These are really good. You have to work together and make sure everything's perfect and you send them in and very little can go wrong. So absentee ballots are good. Universal mail-ins, when you get inundated with these things are bad and will lead to terrible things, including voter fraud, etc., but absentee is good. So get your application, send in your absentee ballots immediately. All right. So I'll stop that there. So again, underscoring that absentee ballots are good, mail-in ballots are bad. Now for most voters, that's kind of confusing. Like it's one and the same really because you're mailing it in, whether both mechanisms are being mailed. Nevertheless, this was the messaging that the Trump campaign was working on. I want to go back. I forgot to mention one thing. Hold on. I can't talk and click buttons at the same time. One sec. I just want to make sure I'm clicking the right button. So the ad that showed Laura Trump, I just want to mention that when looking at the micro-targeting data, the campaign had custom audiences for that. So they knew who they wanted to talk to in that ad by Florida. So they had predetermined lists of voters they'd already identified. They then targeted those ads. That ad, the Laura Trump ad to those voters. This particular ad by just ran for a couple of days, but it was a big buy. There were 3,200 different buys on this. In other words, different segments of the voter targets were sent, sometimes little variants of this or different populations the campaign was trying to reach out to. So each buy led to a different ad representation in the database. Also only Florida. But unlike the Laura Trump ad, this one was a mix of both custom audiences where the campaign had a predetermined list they wanted to target, but it also included some interests. So some of those interests included people who were in the military, so employed in the military, as well as people who were following different news, writing new shows like Rush Limbaugh, Sean Hannity and Culture. And there also were some, as I mentioned, the custom audiences. But this was a big buy. So also the end of August, the Trump campaign ran a large number of small buys on this new ad campaign called Promises Made Promises Kept. And there were about seven or eight variants of this. They ran for a couple of days basically doing what's called AB testing. So trying out different images, and you can see these images and videos, and different targets to see which ones resonated the most, and then running those as much bigger buys to a larger audience. So this, from our analysis, is one of Trump's biggest advice during the general election period. It was over a $2 million ad buy. So over 7,800 different variants of this ad targeted to different segments around the country. It ran from August 6, 26 to October 23. So basically the entire general election period, it definitely had a bunch of custom audience targets. We don't have access to those custom audiences. I don't know what they look like. But on the interest side, some very intriguing things showed up. So some of the targets were people who identified as employment, being in small business, being entrepreneurs, or running a small business, the similar kind of right wing media, as the prior ad that you mentioned. But then also sports, and I'm just including here a couple of examples, there are tons of them, different kind of keywords, but things like volleyball, college basketball, volleyball. Not that was kind of interesting. Anyway, and then a number of issues including Second Amendment. What's noteworthy to me is the issues not there. So pro-life issues, environmental issues, taxes, immigration, none of those things were targeted as issues, but gun rights in Second Amendment was. But there were a number of outdoor interests like hunting, fishing, and other outdoor activities were targets as well. And Florida politicians like Marco Rubio, and Trey Gowdy, who isn't a politician, but nevertheless, kind of Floridians that were known were part of this target as well. But again, you'll notice here that the underscoring messaging that you vote for President Trump is about to save America, request your ballot. So Jenny, these are, you mentioned before that your data are both Facebook and Instagram. Do you distinguish between those at all or is it? So for the moment, what's noteworthy is that these ads tend to run on both Facebook and Instagram. So before we got the micro-targeting data, if you go to the illuminating site, you'll notice that we don't actually allow you to toggle to see Facebook versus Instagram. The reason for that is almost all ads were targeted on both platforms. That's puzzling to me. I don't quite know how true that is. We are completely beholden by the accuracy of Facebook's databases. So it's a little hard to say for sure, but at the moment, my statement to you is that it seems that most of the ads that I'm showing you were on both platforms. And again, this is just the official campaign ads. Is Facebook releasing ads from third parties like PACS? So the political action committee space is very, very challenging. They are, and we can. We've started some of that work. We meant to do it during the election while it was happening, but because the political action committees aren't known in advance and they register pages that then they run ads on, those pages may or may not align with the name of the political action committee. So there's a lot of sleuthing that you need to do in order to identify who the political action committees are, what pages they're running, then you can get access to the ads if they were running ads. So there is absolutely more work to be done there. I'll be honest, we haven't even looked to see if the micro-targeting data includes political action committees. So I don't know yet. We're just looking at the official campaign pages at the moment, but there were a number of those, especially in the final days of the general election. This is just the, and you'll notice there at the top, it says Donald J. Trump are present. That's the page I'm currently looking for. There is more work to be done. So the 160 million is just the Trump and Biden campaigns themselves. That's correct. All right, so last night as I was digging into this, this large number of ad buys, one of the things I noticed is that this promises made promises kept, you'll notice there's that image down here at the bottom. The text for this promises made promises kept doesn't change. So over that 7,800 ad buys, the text is the same. However, there are many variants of images and videos that are part of this ad campaign. And so while I was digging around last night, I found a kind of noteworthy ad and I did a little bit of sleuthing. So you might recall that at the end of August in Kenosha, Wisconsin, there were multiple days of protests around the shooting of Jacob Blake by police. On the third night of protests, a young man who police mistook for being a police officer shot two protesters and killed them. So that happened right at the end of August. This ad, and I'm going to show it to you guys in a minute, aired in Wisconsin only, September 7th and September 8th, and it caught my eye. And let me show it to you. One second to push some buttons again. All right. So the visual is meet Joe Biden supporters. I've seen this before, I've skin passes, I didn't even think to click on it. But last night I was like, hmm, I wonder what that is. So I clicked on it. This is a movement, I'm telling you, they're not going to stop. Sorry, one second. The third straight night police declared a riot. The vast majority of the protests. We get 59 officers injured and 47 people arrested. City mayor Bill Blasio announcing a proposal that caught a billion dollars from the New York State Police Department. We made the complete list. Won't be safe. Joe Biden's probably should have warned you guys. That's a bit, it's a bit violent. And not, I'm not sure what to say about that. So that ad, again, aired in Wisconsin, only in Wisconsin, targeted mostly at men, and most interestingly to me, it's a custom audience. So Trump's campaign knew who they wanted to target with this ad, only aired in Wisconsin, and played on, I think, for at least the target audience, the anger and frustration and perhaps even kind of racist fears that were being felt by segments of Wisconsin, rural Wisconsin voters, probably, I'm guessing, around the process happening in Kenosha and other major cities like Milwaukee. And so this ad, again, not really so much about balloting, but is underscoring some of the challenges here with looking at these political advertisements, when the video content can actually produce a very different message, potentially from what the text is providing, and it's hard to see and hard to study. Okay, so back to balloting and absentee balloting. So then in October, there are a number of small ad buys again on this theme. And in the interest of time, I won't show these to you. But these are Trump supporters at Trump events. And again, this is October. In the heart of the app of the global pandemic, nobody's masked or anything. They're there at this rally. They identified mostly as immigrants. So the guy on the right is Ukrainian. The woman on the left is, she self identifies as Latina. There's another couple of ads that include other Latina women, and there is an ad with a black man and an ad with a black woman. And all of them are talking about how important Trump is. And they're very unfiltered, very sort of unscripted, authentic people speaking their love for Trump in these advertisements. So there are seriously small buys. And you'll notice that they continue to underscore the statement that we need to save our country that absentee ballots are good with the underlying statement from these prior ads when they talk to you about that mail in ballots are potentially bad. So these ads ran in some of the key swing states like Florida and Wisconsin and interesting in Maine's second district, which you might remember, Maine got split. And there was a lot of effort by the Trump campaign to target Maine's second. We guessed as much, but we now have this micro targeting data to see that that's true. Now, the Trump campaign did have a number of custom audiences that they were targeting, but they also were using Facebook's predicted interests, including small business and entrepreneurship and the right wing media ecosystem. Also in October, these ads began to run. These underscore the need to request your ballot and the time is running out. And like the ad on the right, it's, and this is a very large ad buy, it shows you Donald Trump's signature. And it says vote like President Trump, again, echoing this idea that Trump voted absentee, he didn't vote by mail. And he's he followed all of the processes to make sure it wasn't a fraudulent vote. The Trump campaign also kept underscoring that if you went to their website to get your ballot from the Trump campaign, it would therefore then be secure, which I find kind of an interesting argument. This was an entirely swing state ad buy. They did not run these ads in states like New York, which were not a swing state. They ran them in Pennsylvania, Ohio, etc. And again, targeting the usual suspects, but then also some interesting targets, state focused sports or in the list. Now we start to see some pro life as well as gun related issues. And interestingly, other media like PBS and MSNBC, so people who identified on Facebook or Facebook identified them as being interested in Facebook, and sorry, that's weird, interested in PBS or MSNBC, they also were targeted with this advertisement. And I think that's Trump campaign's effort to try to broaden their reach a little bit in the closing days of the election to try to hopefully activate more moderates in some of the districts they needed to secure the election. So what does this mean? What is the toilet to? I don't know. I'm still looking, but here's what I can tell you so far. So it is pretty clear in looking at the advertisements, the Trump's campaign was actively working, of course, on the national stage just so doubt about the voting process in speeches on Twitter, but in their TV, sorry, their Facebook advertisements, they had to nuance that attack because they didn't want to undermine the necessity of getting people to actually turn in a ballot for the candidate on election day. So it was more nuanced arguments and a bit challenging in some ways. The campaign's messaging, this is a problem from an analysis point of view. As I said to you guys, we can see the text very easily in these in our analysis in the databases that we've been given from Facebook, but there is no transcript, no text, no way to know what the video content or even the images are in the databases that we get from Facebook. So we need to do more work as researchers to then actually look at and analyze the video content. As I mentioned, Trump's ad by that one was that one was huge, but many same text, many different videos. And so there's some questions there about problematic content and micro targeting needs more analysis. I guess finally, this is always my concern, as voters become more datified and deindividuated. What does that mean for us as a society, as a democracy, when what we're being marketed to and our interests might speak to our lesser instincts, our base motives, our emotions, and could then lead to some of the further polarization and fractionalization that we currently see in the United States. And it's not just here. So with that, I have two last things. I have to acknowledge my amazing team. So while I get the honor and opportunity to share the love of this project with you all, there is a huge group of people behind me and I couldn't do this project without them. So there's that. And again, you can go to illuminating.ischool.syr.edu to play with some of the data, not the micro targeting data that's on the site yet, but the other analysis we've been doing so far. And with that, I'll stop and see if there are questions with the time that's remaining. Yes, we have several questions. One, maybe to get started, you focused on the Trump campaign. Have you had a chance to look at this point at the Biden spending? Yeah, so I started last night. I couldn't have fit Biden into this talk that we'd have to the second talk or something. But yes, I've started to look at Biden. A couple of things I noticed just quickly is that the Biden campaigns was actually, they were running ads responding to the Trump campaign's claims about voter fraud by underscoring that mail and validating was perfectly valid and not subject to fraud. I think that my hunch is the Biden campaign was worried that the Trump campaign's continual drumbeat of questions about mail and validating would also suppress Democrat voters from sending in their ballots or getting absentee ballots. So they had to do some counter messaging on that front. And of course, they used it to further sort of attack and demean Trump at the same time by miscalculations of the soundness to be present. So in other words, the Biden campaign is doing the same kind of micro targeting strategies. Oh, yes. And sometimes doing it in response to the, but at the time of the election, they can see the ads, but they actually can't see the targeting, right? So that must make it very challenging to know where and how you need to respond. Yeah, for sure. So they can see like we could see on the Facebook ad library's website who the targets were. But the target is only at the state level. And again, it's a couple of categories, it's five categories in age, a couple of categories on gender, and that's it. And that's intentional. Facebook said they didn't want to, their advertisers don't want that secret sauce to be shared because it is a kind of a secret sauce. But for sure, the buying campaign was an undertaking the same kind of micro targeting. It seems that there was, if you will, response happening by the buying campaign, maybe also by the Trump campaign back and forth through the micro targeting, but they're making guesses about who should get those ads. And I guess another concern about that is that you have the data about the campaigns, but presumably there's a lot going on with the supposedly independent PACs, but who may be doing something in coordination. And in particular, trying to undermine the strategies of the campaigns, the opposing campaigns. Absolutely right. So there is that whole ecosystem of the political action campaigns is the next nut that we need to crack. So it does comment from one of the listeners is it seems that the Trump campaign ads, the ones that you showed were more civil and less clickbaity than the ads posted by some of the other groups and those ads that went viral. I guess that's not a focus of the current analysis. You know, it's an interesting question, right? So it is very much the case that the campaign wants, if they, if you see an ad on your Facebook wall, the campaign really wants you to click on that ad. That's why they always have a hyperlink because that engagement data and if you can't access is huge for the campaigns. So if they're less clickbaity, it's because the campaign's not doing a good job or because you're not the target, right? And that's, I think, something to keep in mind. It might not be the target, so it doesn't speak to you the same kind of way. The other thing to mention is remember when you're on Facebook or Instagram and you're scrolling through the feed and your, as you scroll, your scroller stops on a video, that video will start to roll and all of the videos are captioned. So even if you're not listening to it, you can still see the text. And so that's something that complicates those feelings about clickbaitness because it's possible that those videos and elements of those videos are calling and engaging certain targets in compelling ways. In other words, it's hard to tell what is and is not clickbaity. And we don't, we do get some impression data. So we do have some sense of that, but that gets very challenging to look at because, again, they do these small ad buys on an ad campaign. And so you have to aggregate all of those to then get a sense of whether or not a particular ad buy was, was in fact really engaging. And we've got some analysis of that on the website you can take a look at. Does Facebook allow you to target by religion? Yes. And so evidence in the data set of that? Yes. One of the ads that I, and I got distracted, so I didn't share it, but it's the promises made promises kept that ad buy did also target religious groups. And it was everything from seventh day Adventists to the Catholic church to Methodists to some like Christian rock groups. So it was a very broad spectrum of religions. I've got to look at that more and see if different ad buys target different religious interests that still to be determined. So would you agree one of the posters asks with Ellen Weintraub about ending record targeting? How do you think that would, is that actually feasible? Is that something that Facebook's new oversight board might try to impose? And so first of all, do you think it's likely? And secondly, I guess, what kind of impact would that have? Yeah, it's a really, really important question. So, you know, Twitter decided to not run any political advertisements on the platform in 2020. And they walked away from a very large, potentially relatively large amount of money. Again, $160 million just from June to November in political advertisements. And that's just the main Trump and Biden pages. That's not the entire ecosystem of money that was spent on the election. It's hard for me to imagine Facebook as a company walking away from that revenue. Not without significant pressure from Congress and from the public to say enough. It's also worth mentioning that the political parties, the political action committees, the political organizations and interest groups, they want this micro targeting because it's effective. It helps them find that needle in a haystack, those supporters that are otherwise really hard to activate and mobilize. And so they want it too. So you have these kind of counter pressures of the market and capitalist forces like a for-profit company like Facebook that are coming up against questions about what's healthy for our democracy and what do we need to have in order to make good decisions about to whom we vote. So I am a social scientist. I don't always think about these policy questions. I am starting to think about them more. But at the moment, I don't know how's that for a terrible answer. So a few other questions. You mentioned that the text says the same, but the images change. Is there any attempt, if we leave aside the micro targeting, are there efforts about regulating the content? There should be. So I think our political establishment has shirked their duties from my perspective in really fully thoroughly considering how best to regulate the speech that occurs on these platforms. It's sticky though. You're talking about potential First Amendment complications. We have a Supreme Court that has ruled that much of this political speech is protected speech, including corporations when they run advertisements. But Facebook is not the government. Facebook is a private corporation. Yes. Well, so there's two pieces there. So the government could be and they have regulated television advertisements and what can and cannot be said, if you will, on TV ads, and put some onus on broadcasters to regulate the content of advertisements on television. There has been the same sort of conversation about whether and to what extent that needs to happen on the platforms. So there's the government discussion about regulation. Then there's also the platforms discussion. And these companies are talking about what their roles and obligations are. They are they are challenged because they are for profit organizations. They have shareholders and businesses that they need to support. But they also are currently our increasingly our public square and the comments where we learn about politics and gain the kind of information and theory that we need to make good decisions. And so will these platforms be good citizens? What role will they play? I don't know. I think these the next two years will be interesting to watch for those kinds of conversations. So maybe one or two questions just to finish. So does your data set include people other than the two main parties, third party candidates, for example? It does. Yeah, so you can get the the libertarian, the Green Party candidates, the third party hands and the down ballot candidates are there as well. Wow, that's a real trove. Yes. And maybe final question. Do you have any way of assessing what kind of impact these ads actually have on the people who get targeted? It's such a good question. It's the one that I've been mulling the most and very challenging to do post, post hoc. Interestingly, Facebook actually funded a large number of academics to do research during the general election to measure potentially impact. And so that so those would be some of the folks to watch. I was not one of those. That's a longer political story I won't get into right now. You know, there are things you can do. So for example, with the micro targeting data, although it turns out it's not as as micro as I was hoping for, but you could do things like look at turnout. So it is possible to get turnout rates and numbers. And also you can actually get you can get from secretaries of state who actually voted. Now you don't know how they voted, but you can find out if they actually went to a polling place or cast a ballot on election day. And so it is possible to do some marrying up of our data sets with turnout data, challenging, complicated, hard work, because you have to go to each secretary of state and get a convince them to give you the data. But that's potentially a route. Another is doing surveys, but the election's over. So it's hard to do surveys, almost six months to a year out and say, Hey, did that have an effect on you? There's a third route, which we'll be doing some experimentation with I think the summer, which is actually taking some of these ads and doing some experiments to see how people respond and react to the ads and how it shapes and changes attitudes. So the effect question is real and it's hard, unfortunately. And I think that's probably the way that most research presentations end with a call for further research. So we're out of time. I'm sorry that there are a few questions that we weren't able to get to. I'll just release those so that people can see what they were. I'd like to thank our speaker again for a really, really interesting talk. And I'd especially like to thank all of the members of the audience for their participation and for the great questions and comments we've gotten. So thanks again and have a good afternoon. Thanks, everybody. Thanks for coming. Appreciate it.