 Hey, we're going to get started. Welcome. Thanks for coming. I'm Brittany Seymour. I'm on the faculty at the Harvard School of Dental Medicine, where I focus very broadly on health policy, prevention, with a strong emphasis on curriculum development and how to teach our future health professionals policy and prevention, basically. I'm also a faculty associate here at the Berkman Center. I was a fellow last year. I'm a public health consultant for MIT's Media Lab. So my name's Natalie Jenis. I've been working on public health communications for a while. I'm a graduate of the Harvard School of Public Health, and I'm trying to bridge the gap between digital methods, digital education in areas of public health. I am a fellow at the Berkman Center right now and a research affiliate at MIT's Media Lab at the Center for Civic Media as well. So this is really an opportunity to share some of the research that we've been working on to start engaging in a dialogue to figure out what these next steps for the future of communication of public health will look like, and to really get your input and do kind of a short brainstorming session at the end of our small little talk. Sound good? Works. OK. Great. Thanks. So I'll start off with kind of one of my earlier, and by earlier I mean just a couple years ago, research projects around kind of misinformation in public health and how do we deal with that. Vaccines is a bit, it's a hot topic it has been for a while for those of us that work in the field of public health, especially vaccines and misinformation, reduced trust in vaccinations. It's been a global problem. So do you all recognize this study? What is this one you're nodding? Do you know which one it is? I can't quite see, but it's the study by the guy whose name begins with W. That's right, that's right. Ah, yes, I'm a W guy. Exactly. The basis of so much of the anti-vaccine, except it's all been retracted. Right, there we go. Retracted. Exactly, thank you. So that's the kind of the infamous Andrew Wakefield study that kind of sparked the current concerns around the link between vaccines, child vaccines, the measles, mumps, and rubella vaccines specifically, and autism. So he put that study out in the late 90s, caused quite a stir. Over time it was found to be actually a highly fraudulent study and he has since lost his license to practice medicine. The study was formally retracted. And this is kind of a classic case for those of us in public health communication where there was a lot of misinformation, we retracted the study, we put out a lot of corrective information, and yet this problem persists. It's the concerns about autism and vaccines are still very much alive and well, despite the vast amounts of efforts that we have put out to correct that misinformation. And not only that, the concerns around vaccines have ballooned from not just autism, but to all kinds of things. If you go to the CDC's website now, there's an entire laundry list of concerns about vaccinations with corrective information that's meant to ease the concerns of parents and people that receive those vaccinations. But despite all these efforts, we still have this persisting challenge of what we consider to be blatant misinformation or even disinformation around vaccinations. And this challenge is not unique as I'm a dentist by training and community water fluoridation faces exactly the same kinds of problems as do many other public health interventions. So we kind of took a deeper dive into this story about vaccines and really wanted to look at the role of the internet broadly. And it was a very exploratory initiative and we've learned a lot in the last couple years and we think we're kind of starting to come to some reasonable findings that might help propel us forward into developing some practical solutions and that's really what we wanted to talk about today. So we kind of dove in and we said, what's going on here? This is a very basic Google Trend search where we looked at Jenny McCarthy's kind of public appearances on TV, on Oprah, on 2020. Each spike there, it's hard to read, but each spike is when she made a public appearance somewhere, put out her new book all with kind of this underlying message of vaccines are harmful and perpetuating this myth that vaccines cause autism. And the other spike, so one spike is looking at Jenny McCarthy in autism. That's how many people are Googling it and they correlate with these public appearances for her. A lot of people saw her, went to the internet and were Googling looking up more. But also we started at the same time seeing spikes in people Googling vaccines in autism and Googling Andrew Wakefield. And this gave us the impression that something's going on, these events in life are occurring and are driving traffic to the internet and people are finding this information and these specific paired search terms are kind of perhaps perpetuating these challenges that we're seeing. And again, this was very early stage, but we started to think that what was happening in real life and what was happening on the internet both mattered and in fact seemed to be mirroring one another and we thought we wanna know more about what's going on online. So this is when I was linked up with this phenomenal team at Berkman and MIT and started to use the tool that MIT Media Lab developed called Media Cloud and Natalie's gonna talk about our tool in a bit more detail. But what I was able to do with the team is to actually map out kind of, we set a defined date range. It captured the measles outbreak in Disneyland. This date range captured the new legislation in California eliminating personal belief exemptions and religious exemptions from vaccinations for public school, meaning legislation was passed that parents can no longer say, I don't wanna vaccinate my children for religious reasons or because it goes against my personal beliefs. They have to have a documented medical reason and it has to be cleared by the Department of Public Health and it was an extremely controversial move but many people in public health felt it was absolutely necessary because infants were being hospitalized for measles in the US and we just found that unacceptable. So we kind of mapped out over this period of time who's publishing about vaccines and this is what our map looked like. And so there's a lot of dots and a lot of lines connecting these dots and the dots are basically sources publishing. So the CDC's there publishing, PubMed is there where scientific articles are coming out, the New York Times is there, they were publishing about vaccines. So basically all the voices who were putting out information on the internet, we generated this map and the lines between these dots which don't really show up here but they're basically when one source links to another source. And we found that you can see there's a green, blue, yellow, pink kind of clusters that formed in this broad information network about vaccines. We found these kind of sub-communities or sub-networks within the bigger network that were linking to each other more frequently than other sources in the network. So we basically started to see the early findings of what perhaps could be echo chambers. And in fact this pink network was primarily the public health authorities and public and medical sources that were linking to, we were linking to each other a lot more than anyone else was linking to us which is problematic but also started to answer our question of despite putting out corrective information we're not seeing that reflected back to us with public behaviors and this may be because the only people reading our public, our corrective information are one another. And so that led us to broader and broader area of research. So just to give you a couple of other case studies of the type of work we've been doing with these network analyses. So as Brittany mentioned, essentially what we do is we try to map out the infrastructure of communication in digital networks. So we look at the equivalent of academic citations online which are references between sources to one another and we follow their in-links. So similar to as when looking at Wikipedia for example there will be hyperlinks within the document that reference other documents that you can get more information about that first piece that you were looking at. So we follow those through this spidering process that was created here at the Berkman Center and we are able to essentially map out how a variety of different sources communicate with one another. Another interesting layer of analysis that I'll dive into a little bit more deeply is we also look at the type of language that's being used by these individual source communities. So what we're trying to understand is not only how do they speak to one another but what is the language that they're using when they communicate with one another. And using a force directed graph algorithm which is a fancy way of saying a way for source communities to cluster based on these two behaviors in linking and language use. We continuously see that the public health community is isolated from all the other communities which intuitively makes sense but the implications for that are huge especially for public health institutions where there's an insistence on the importance of adding more science for getting more evidence for publishing more articles. And the advice that we through all of these case studies are beginning to be able to present to these larger public health information authorities is that more science isn't necessarily the answer to effective public health communication. It's finding ways to utilize language that's been adopted by communities outside of this public health cluster so that you can start bridging gaps between the core information authorities such as the CDC and the World Health Organization and mainstream media sources which is where the majority of us get our information from. So in these three case studies first was the vaccines case study where we found that the public health community was as I'm sure you can imagine completely isolated from the anti-vaxxers community. So the step here isn't necessarily for the CDC or the World Health Organization to publish more science related articles or to publish more evidence but it's to begin bridging the gaps between the sources that are anti-vaxxer so to speak and pro-vaccines. A second case study that we looked at was looking at Ebola coverage for the epidemic in the United States. And what we found was that the majority of mainstream media was focusing on Obama's role in bringing the Ebola epidemic to the United States and the cases present in Texas. Rather than, for example, the disaster response or recommendations by the CDC and the World Health Organization on what to do. As you can see in the second case, the Ebola case study the yellow community relates to US specific topical coverage of the epidemic and the purple community that circled is the public health community. Again, completely isolated on the opposite side of the map as mainstream media that was focusing on US specific content. The third example that we're bringing up here is a look that we did on health more broadly in the United States over the course of 2016. And the great thing is that over the last year health has been put on the agenda. At least now the news is talking about health specific issues be it related to insurance or related to sexual and reproductive health and rights. It has made it to the agenda which is great. But the framing of health is now something that we need to evaluate more in depth. So here we have the third map where to the left the purple community is largely mainstream media sources as well as political specific sources. So you can see that health is becoming a part of the larger political conversation in the US. To the left that orange community is a policy focused community that we found which is where many of the foundations and the sources such as the health affairs blog that publish content related to policy lie. And in the top again completely isolated is the public health community. So we need to be doing a better job of either adopting the language used by additional sub communities within our networks or finding allies and bridge figures that can provide the information that will lead to communications between these seemingly disparate groups. So to dive into our methodology a little bit better the purpose of us trying to understand how the media talks about public health issues and deconstruct the way that digital networks talk about public health is so that we can reconstruct these narratives in a way that is productive for public health. So we use media cloud which again is a suite of tools that was developed here at the Berkman Center as well as the Center for Civic Media at MIT. And our analysis is at sort of three levels. The first is the most basic. It's looking at attention and news peaks. So how was a given issue covered over the over time? What were some ebbs and flows in the amount of coverage for a given issue? We then do this link analysis where we've developed an algorithm here that essentially spiders through again these inlinks to learn how various media sources communicate with one another. And the third is the language analysis where we try to identify common terms most frequently used by different source communities to understand how the policy community is speaking about an issue like abortion differently for example from the CDC and the World Health Organization. And just a few kind of interesting pieces of information to gather from network graphs like this. The size of the sources that you see here which Brittany touched on earlier is representative of the degree centrality. So how frequently are they being referenced by other sources in this network? And that's important because we'll start to consider these sources to be information authorities. They're the ones who are publishing the most amount of content that theoretically the most amount of people will be able to see because it's linked to the most frequently. The second piece that you'll notice is that some sources that tend to have the largest degree centrality are also closer to the middle of the graph. And sources that are closer to the middle are ones that link to sources at the greatest variety throughout the graph. So the sources that will link to the majority of the other communities will be pulled by this algorithm closer to the center of these types of graphs. So you're able to see for example that healthcare.gov that's in the bottom right of the graph is not a key information authority in the entire information network but rather they're important for this policy focused conversation. Or the CDC unfortunately you'll see it is central to the health specific conversation and it is being pulled towards the policy conversation but it is not a part of the politics conversation. And had it been closer to the middle it would mean that sources from a variety of these communities are linking to CDC content directly. The problem with having all of these mainstream media sources in the middle is that it implies that mainstream media sources are referencing one another more frequently than they are the key information authority. Which is another way that misinformation especially in the area of public health can become propagated if citations between mainstream media sources are more prevalent than citations to the original data sets. So within these studies we've had a couple of patterns four patterns exactly that sort of have been emerging. The first as we've described is the distinct ability to isolate sub communities within an understanding of the digital public health conversation. The second is that we've been able to find information authorities within each of these groups that don't necessarily contribute to the broader conversation but are key sources for that individual sub community. We've found that information authorities in different sub communities don't link to one another as frequently as they do within the network which is understandable. And unfortunately that the public health community is continuously isolated. So part of this reason and it's definitely one thing that we wanna focus on during the brainstorming session later on is how public health communications takes form or takes shape right now. And just to reflect my first months of my public health degree and this applies to other fields as well. We are presented with a number of ethical dilemmas and a number of case studies in trying to argue the best ways of prioritizing statistical lives over actual lives. So we're taught time and time again that the 75% of the population that will benefit from X intervention is who we should prioritize above the 25% of people who may be harmed by said intervention. And time and time again, especially when physicians who engage in the field of public health are brought to this conversation, their issue is that they're not, it's difficult to prioritize a statistical set of lives above a patient who's directly in front of them. But we're taught to prioritize this because public health focuses on population health. So in public health, we continue to discuss with one another about the importance of statistical lives. We publish papers on the importance of statistical lives and we continue our communications with the mainstream public based on the assumption that they agree with that prioritization and value set as well, which isn't necessarily the case. We had to learn it ourselves. So the fact that this continues to propagate has resulted in the public health community at large presenting information to the mainstream community and mainstream audiences without accounting for this shift in narrative. So, thanks Natalie. So we've done enough of these case studies and we've consistently seen these patterns throughout that we really felt comfortable taking a broad step back and saying, this is happening, why do we think this is happening? Why are we seeing these echo chambers? And if you're not familiar with the term echo chamber, it essentially is what we're seeing, the physical maps. It's these communities that kind of just echo one another. So the messages, the narrative with our vaccine study, for example, the public health narrative is, this study came out, vaccines don't cause autism. Here's another study, vaccines don't cause autism. But the more vaccine hesitant community, the narrative was very, very different. They were linking consistently to the same kinds of sources with the same message that here's the truth, they don't want you to hear. Vaccines actually do cause autism and they're covering it up and this is where conspiracy theories are born. And that is echoed within that community. So that's where the term echo chamber comes from. It relies on kind of these social values that are shared within communities and there's a built-in trust within this community because it is, in and of itself, a social network with shared values. And so the opportunity to introduce new information or conflicting information is greatly reduced because it violates these norms of these sub-communities essentially. So we took a step back and it's not just public health that's facing this kind of challenge of trust. It's, this is Gallup data that kind of looks at trust in institutions, all institutions over the last 40 years, so from 1973 to 2015. And essentially other than the military and small businesses, every major kind of institution that we rely on, the media, religion, government, Congress, medicine, have all experienced a decline in trust. And so when you see this decline in trust, you see this increase in fear. And what that leads to, these are Holland Spheres, if you've ever seen these, these more apply to journalism and the press, but we think that there's a relationship to public health communication. And so I'm gonna come back to trust in a minute, but these spheres essentially, there are three spheres. The small center circle is the sphere of consensus. So what seems to be just universally accepted, everyone seems to agree on this or whatever this might be, so much so that it would be boring to talk about in the press. It'd be boring to report on. It's nothing new about it. No one really talks about it, it's just accepted. Then there's the sphere of, then outside of that is the sphere of legitimate controversy. And this is where most media happens, where debate occurs, where new information's being presented, where discussion is possible. And then outside of that is termed the sphere of deviance. Sphere of deviance is kind of defined by things that are so outrageous or so almost impossible that they are also not reported on because that would just be ridiculous. And so to bring it back to trust, the sphere of deviance grows roots when trust breeds fear and people are starting to look for information to help ease these fears and they come on this conspiracy theory that's not reported in mainstream media or within this sphere of legitimate debate, but they find it and it resonates with this fear because of this lack of trust. And so we think, so for example, in the public health community using the vaccines and autism example, within our sphere of consensus within public health, we all agree that vaccines do not cause autism. And so there's not as much reporting on that. The sphere of legitimate debate now is about how do we manage the mistrust in vaccinations and the misinformation and how do we shape policies that parents can feel comfortable with. But outside of that in the sphere of deviance is this very alive conversation that actually vaccines do cause autism and now there's this cover up occurring and the science isn't that clear and they're hiding this information from you. And so parents who are scared, this resonates with them and this is where misinformation really grows is within the sphere of deviance. And again, across case studies, we're seeing the sphere of deviance really start to expand and kind of create its own communities almost within these networks when we map them out. And so this is where it comes, so we in public health are trained to communicate the scientific proof, the evidence, the data, that's how we speak. But beyond our community, social proof has a lot of power and what we mean by that is this is where these echo chambers thrive is not necessarily with the scientific proof but more with the social proof and who's the messenger and that's what matters. That's where the trust is gained is by whoever that messenger is, not necessarily the content that messenger is delivering. So social proof starts like suddenly a lot of people are sharing this and then suddenly someone I like shares this and then in fact it's my own close friends within my own networks are starting to share this and that inspires me to share it because it's resonating and suddenly vaccines cause autism. And so that's kind of an illustration of the power of social proof and why when we started mapping out these public health topics across various discussions and saw these sub communities forming, it's really challenging for those of us in health to recognize that the scientific evidence is not necessarily the driving factor. It's that the power of social proof, the power of these echo chambers, who's in them and who those information authorities are within those networks. So we're kind of at a crossroads I think in public health communication. We have become very good at and our data back set up and we won't talk about it today but really good at what we're calling broadcast diffusion where we create this very well vetted peer reviewed message that's backed by the data and we broadcast that message to all. So vaccines do not cause autism. Broadcast that message. But that, we're finding that's ineffective. It hits walls usually at the borders of our isolated public health communities. And in fact, the way we're seeing kind of beyond the public health bubble, the way communication actually works is more social diffusion where social proof is important, the messenger is important, not as much so the content. And so we share a message, someone picks that message up and shares it to their network, someone else picks it up and as this becomes socially diffused, the sentiment can change. So each person that shares it adds their own new language choices, new narrative around that content reshares it and it becomes like a game of telephone. So the CDC sends a message at the top of that social diffusion pyramid. By the time it reaches my next door neighbor, it's a totally different message even though it may have started out with the evidence from the information authority of vaccines, right? So how do we embrace social diffusion as a communication method when often it runs so counter to the way we're trained to communicate? And some suggestions have been made for the media more broadly, not necessarily health-focused that may work in this, and something called preference-based framing, which is basically what Natalie was hinting at. We have this tool where we can really dive in and look at how people are discussing a topic, their specific word choices, the context around that word choice and really start customizing messages to fit the preferences of all these various sub-communities and see if we can get some links from that. Other suggestions are increased public participation. So perhaps almost the reverse of peer review where we write our papers and we get peer review and we get it into this final polished cemented form where it will be forever that way, we publish it. And perhaps reversing that the way a lot of other platforms work, publishing and allowing discourse and making edits in real time and allowing for public participation. And that freaks me out. I don't know how you got that idea freaks me out, but because I feel very vulnerable, I feel that that introduces opportunities for greater misinformation. However, it seems to be that that is kind of the way the world is working. And perhaps we can get better, become more comfortable with that approach. And ultimately that approach, we think will allow us to regain the public trust because the public will constantly, consistently be part of our conversation and we will show them that we welcome that. So in order to fill a lot of the gaps that we've presented today, there are a couple of methods for analysis that we can use and a couple of theories that haven't yet been applied to the area of public health communications. And having a conversation about this afterwards would be really helpful as well. So the first kind of piece of context is that in our analysis we've sort of determined two different communication methods that happen online. The first is for static content, which is largely data driven and tends to be the type of content that like the CDC and the World Health Organization will put out. It's like the broadcast method. Exactly, the broadcast method where there's one message and it's propagated throughout the web and we hope that all sub-communities will pick up that message and embody it and understand it. And through this sort of linking behavior that we've developed these maps on, that's kind of based on how static information moves through the web. The second way is for more dynamic content. So how can we figure out in public health, for example, like what's really popular right now? What are the perspectives that are being propagated widely largely through social media? So it doesn't mean that a particular frame on HIV destigmatization, for example, is going to be one that exists long term. But it's possible that today it had two million shares. So we're looking at two different things. We're looking at information that sustains through information networks online and information that becomes really hot for a short period of time and understanding which messages you can target for the short periods of time and which ones are more important to have on the agenda long term. So a second theory that is important to build here is the parasocial contact hypothesis, which is a really interesting theory that was developed at the Contemporary Media Studies program at the MIT's Media Lab. And it essentially takes on the original contact hypothesis, which talks about how under particular and appropriate conditions, having interpersonal contact with someone that's a representative of a given subgroup that may be marginalized or stigmatized against may decrease the prejudice and stigma that you feel towards that particular group. And that's been verified, particularly through this paper on the parasocial contact hypothesis, where it's expanded to mass mediated interactions. So is it possible that interactions you would have while watching a TV show or reading a story that someone's posted on the web will have a similar effect as actually having a real conversation with someone in real life? So is it possible that the digital media world can actually mimic the real world in changing our perspectives about stereotypes and populations that are traditionally prejudiced against? So turns out it worked. Specifically, their case studies were on shows like Queer Eye for the Straight, Guy and Transparent, where prejudice and stigma was reduced based on this parasocial contact through media, which is a really interesting opportunity for public health, especially for seriously stigmatized conditions and populations, such as HIV-affected populations or the excessive stigmatization of STDs and STIs. And it's important for us to start taking lessons from other fields of digital communications and start to build the tools that we use for more effective public health communications. So here, we just provided you with a boatload of information, a number of theories, a couple of applications and case studies, and we'd really like to use your brains and develop a brainstorming session on how we can translate public health better. So with your backgrounds in internet and digital communications methodologies, we're hoping to brainstorm and talk a bit about these discussion questions and figure out how we can take some of the insights from your expertise areas back to the public health community so that we can communicate better and mitigate these echo chambers that seem to have form online. So we also welcome questions on anything that we present it to. We do welcome questions. So to start, is there anything of particular interest that you have questions about that we can dive into a little bit more deeply? Yeah. Are you sure that Vaccines.co is on paper? Sorry, sorry, sorry. That was clearly a joke. Yeah, thank you so much for this. I found it really interesting. One thing that I keep wondering and now I wonder again is, has anyone started the actual dynamics within this group of people that creates this misinformation news? So I understand that there was this one paper. I understand that some people are maybe prone to believe that government wants to do bad things to them, some conspiracy theories, et cetera. But how exactly does this belief that they think they should share and they should convince other people comes about? And maybe I think understanding that pattern would also help us to create a counter-conspiracy theory on how to talk to those people about the fact that somebody wants you to think that vaccines are bad for you, but actually, you will die if you go to the vaccine or a child or something. Counter-conspiracy theories. I love that. I love that. I think this element goes back to the topic of social proof. Isn't that just the data? Isn't that the counter-conspiracy? I mean, we win. Isn't that just the data? That's the problem, is that we imagine that just presenting evidence is sufficient in creating that kind of counter-conspiracy. And it's true that the behavior within these individual echo chambers for sure varies by content and context. So what we're looking at here isn't the misinformation that's spread by individuals, which Mediaclod actually has the capability of doing now is looking at how Twitter shares of particular content build into particular networks. And we can look at the Twitter networks themselves and see who's sharing to whom, what other type of similar or different content have they been sharing. But for this, we're just looking at the media source communities. So how misinformation spreads between media source communities also has embedded within it the agendas of these sources and the type of audience that they need and the ad revenue that they can get from these additional shares. So there's a number of dynamics that are at play when you're considering the media ecosystem, which is definitely different from kind of the biases and heuristics that exist within human echo chambers of propagating information. One challenge, especially in the vaccine context, is that the public health professionals, as you say, speak the language of statistics. And it's asymmetrical warfare because they're speaking the language of anecdote. And people understand and gravitate to stories and storytelling. And statistics don't get shared over the internet. Stories get shared over the internet. Right. Exactly. And so stories of babies who are scarred for life because their parents are dumb shits are actually what need to be. Exactly. Exactly. I don't want to talk that way as the presenter. But I've also other. And we've seen a bit of this just recently on the Affordable Care Act. Right now, they're beginning to be headline stories like the boy who can't speak, the child who has $800,000 in bills, what a lifetime. Oddly, it's odd that it took until now until we're at the precipice. But you can see those stories that are animating the reporting and the media. Exactly. Well, I just want to fully agree with you. Storytelling's powerful. There's a lot of literature to back that up. Here's the challenge is a public health success is a non-event. It's really hard to create an interesting story about something that never happened. Yeah, my daughter was born and I vaccinated and she never got measles. And I mean, that's so boring. Like, how do you create a story about an individual like that? It's got to be the inverse of when it did. It's where there's a fail and a consequence. Yeah, that's true. Yeah. But it's difficult to capture those non-events and talk about them. Because we don't know when you almost didn't get in that car accident or, you know, it's, yeah. So I'm a reporter and the question I wrote in is, should I look for stories in the sphere of deviance? It's almost the inverse of that. So you're talking, it's basically a big trust problem. I feel like if institutions push really hard for reporters and media to tell stories that propagate their message, that's not going to work. I think you need honest stories about the people, quite large communities now, maybe. I don't know. But that just won't ever listen to that story. They're just gone. They're not never going to listen to that story. But if you tell their story honestly and tie it in with the language and the themes of other institutions, then I think that maybe is a better plan. I mean, you're exactly referring to this bridging concept, which would be visualized in one of these maps as a story or a source that uses language that has been used by traditionally either marginalized or disparate communities in order to bridge gaps and provide, for example, a link, a physical link between two communities that would otherwise not discuss with one another. And in the short term, a link like that is effective just for bridging the gap and helping people who read such content. But it's also in kind of a long-term game, those will be the sources that may start to be consulted by both sides of a given argument or controversy. Fix this problem. Instead of saying, here's an example of someone who didn't vaccine and it went horribly wrong and aren't we always right. They would say, here's an example of a community that doesn't believe in vaccines or whatever. But here's their doctor who is a normal doctor, who does believe vaccines work but has to work with them and talk to her and she will give you an amazing story. Because right now media doesn't have, like you say, they don't have permission to step into the sphere of demons but if an institution says, check this out, these are people too, I think that opens it up. Yeah, that's absolutely true. It's institutionally shifting the narrative at least engaging with alternate narratives without verifying, of course. But also this idea of championing individual stories which is what makes mainstream news and we can't keep talking about this as a statistical issue from the field of public health. That's what we're most comfortable with and most excited about. But it can't be a statistical versus actual lives dichotomy. Well, just on top of that, I was wondering if you were at all worried that by stepping into the sphere of deviance, you're normalizing some kinds of deviant ideas and giving them more widespread authority. I mean, I sort of think that as a reporter, you should not be trying to normalize any ideas in some ways, like that's not really my role to normalize anyone's ideas. It's just to say what's happening out there. But yeah, definitely, I would worry. I would be, this would be a very difficult story to write if I went to, I don't know, an anti-vax community in California and spent two weeks with them and wrote a big story about it. I'd be very worried that I'm gonna be like, look, the economist wrote about anti-vax, it's legit. So, but there are ways to do that. You don't only write about things that are true. So just an interesting point from the vaccine study that would maybe cause some more questions is one thing that we're able to look at is how sources on either side of this conversation, the pro-vaccine and the anti-vaccine movement, would reference the same CDC content and just have it interpreted differently. So mentioning content or creating content isn't what necessarily normalizes something because the interpretation piece is also so essential. So of the vaccine community in our map, the broad vaccine network that we mapped out, the vaccine hesitant community, the dominant authority in that community was NCBI, which is the NIH data website and abstract aggregator. So what we saw happening within that community is there was this value of very evidence-driven conversation but a ton of cherry picking of studies where only the abstract was available publicly because they're hidden in these journals where if you don't have a subscription or you don't wanna pay $40 to read the whole article, you just get the abstract. So limitations are not present. The context of that abstract within the broader body of literature is not present. So it becomes very easy to create a narrative and use science but really manipulate that abstract to fit within your pre-existing narrative. And we saw that fairly consistently within the vaccine hesitant network, which was surprising to us, yeah. So one thing I think in terms of ways of thinking about health communication and how a person were to go is to think about trust as mistrust not being the lack of trust, but mistrust being its own thing and trust being its own thing. Because you have really different populations that you're doing that. And so the ways that mistrust works in a given community isn't necessarily gonna be the same. I mean, this is what you're pointing to that yes, they're picking up the scientific articles and interpreting them differently. And over here, you think that this is self-evident. So you have different ways of both trusting and mistrusting that are going on. They're not the inverse of each other and they're working differently in these different spaces. Right, both the very same piece of content, yeah. Yeah, that's a great point. That's a really great point. And distinguishing between those is really helpful. I didn't make it up. You didn't make it up. So what do we do? Yeah, help us. Well, I'm a YouTuber and I have an idea to share. What I've noticed is, you know, I do stuff that's very explicitly public health oriented. And there are a lot of YouTubers who are trying to do stuff they want to do good. But I think there's a real disconnect. Like you were saying, you know, it's just interesting looking at those charts. It's like, of course, you know, that's just visualize everything that I see online. But there's no, they are trying and they are not finding help. There really is this, I think there's no, there's no communication on a personal level between public health experts and social influencers who are people without any sort of information or any sort of framework of critical thinking about this stuff, making these articles or making these videos, and being viewed millions of times. And to their audiences, they are the authority. To my audience, I am the authority. Even though I am not actually. And authority, but to them I absolutely am. And that makes me very nervous, but it makes a lot of people very happy that they can just sort of say whatever they want. And then they have the validation of a lot of people. So I think it's one thing that is really important and something that I would love to see and that I'm trying to figure out in working with Twitter and YouTube and stuff, is how to get public health expert voices into that space and to bridge that gap into, you know, you were talking about inviting the public to the conversation, which also sounds scary to me, but maybe the social influencers are part of that public. You know, having those conversations because they're extremely valuable in terms of affecting public health. YouTube is a powerful voice in all our studies. And often it was in the vaccine hesitant community. We're doing a fluoride study and it's a powerful voice, but in the anti-fluoride community. So I think you're on to something. And I would also say that I have not once ever used YouTube to communicate, you know. It's been a thing. Yeah. It is a very new platform and I think a lot of people who are experts right now, they're too old, most of them are not, not, sorry, they're not too old. But the institutions within which we operate are very long standing and very rigid, right? And tend to be antiquated in their methods for disseminating information. So you're right. I just think it's new. It's new. Yeah. It's kids, they're on the platform. Yeah. So I don't know how to fix that, you know. But I do think that's something we're thinking about in creating ally ships with perhaps YouTube as a company. I know they're very open to this stuff. Twitter as a company in terms of figuring out how to facilitate those relationships. Just said to you, your thoughts are great. That's great. So when I was listening to you, it reminds me of that analogy out of political campaigning, which is the question of, is it the pizza or the box? If you're thinking about how to deliver something, is it the substance of the message or is it the packaging of the message? And that's, of course, inadequate. In this case, the question is, is it the pizza, the box, or the delivery person? And I feel like you're part way towards answering that question, but don't have it fully down. So I'm not sure how to do that. I think it would be interesting to think about how to do that. I think it would be interesting to design work to actually test that empirically, to say, can you take the same people, the same entities with different messages and see if there's a difference in uptake there versus actually changing the messengers itself? My own suspicion is that it is the messengers. There's an increasing body of information, language, and scholarship out there that says that, as you say, that knowledge is social and people really don't think that much. And they don't know what they don't know, but they do pick up on the people around them and their understanding of what they know or not is really coming out of their community there. And the last piece I just want to lob on there is there are strong parallels between this and so many other things. And one of them is the work that's emerging around hate speech online and trying to understand what counter speech and what counter narrative is effective in that way. And that was going to be my last thought, but I lied. One last thought, which is that I think in public health communication, there's a sense of being special and that having a better sense of what's right and what's wrong. Oh, it's totally patriarchal. And is that what you mean? I want to believe that, too. And I think in vaccines, it's true. It's like the evidence is so, so strong. How can you do that? But it's true of climate change as well. And I think I don't know if leaning into that special sense of having better knowledge than other people is helpful or harmful in this context. I just want to go back to the YouTube thing, I think, because it's almost like it feels like a peer. And that resonates with people versus an authority that might have ulterior motives or a financial motive. Although YouTube people can do really well, apparently. But I think it's back to the idea of peer-to-peer communication. And I'll just give a real quick example, and then I know we have a couple more. So I've been working with the American Dental Association on communication about fluoride for a while. And it's been fine. But they decided to take a risk and had me do a project with them as not as Brittany Seymour, the Harvard professor, but as Brittany, the mom. And it's amazing how successful that was. And of course, buried in there is she's also an expert and blah, blah, blah, blah, blah. But I was present, and I felt very vulnerable putting myself out there as human mom, Brittany, not like hiding behind my credentials. But it was very successful, like surprisingly successful. And it was a risk for the ADA because they typically promote information in a more traditional manner. But that's back to the messenger that I think maybe we're onto something. And the fact that it's surprisingly successful is only surprising because we're public health practitioners. It's not surprisingly effective if we are the population at large. Having someone that you can directly identify with, and there's been countless studies on this, will lead to different behaviors than receiving information from a key information authority. Which is why, absolutely, diving more deeply into how individuals communicate with one another is definitely a next step for at least testing out some of these large theories that we've been developing in grassroots interventions, which are definitely our next step. Yes, I mentioned some intervention studies. Yeah, I mentioned some intervention studies. So related to what should the message be, at one point you were talking about why people seek out this information, and you said, when they're afraid. When they're afraid and they go, I actually think that's an empirical question. I'm sure there are some instances where people are seeking out information because of fear. But I don't know that that's actually the case in a lot of things related to health communication. People get interested in diets, not because they are necessarily afraid because they want something else. Or they start diving into pro-anisites because of other kinds of issues. And so some must be fear, but I really don't think all of it is. So I think that it's an empirical question, and one of those things that actually has to be added to what should the message be and how should it be. Right, right. And that may be case specific. It's specifically for the infectious disease topics that we've been focusing on. Fear is a huge driver in trying to find a scapegoat or a target for where a particular source of information came from that led to X outcome. But you're absolutely right. It is a question that has to be asked for each individual case study, for sure. And it's whatever drives an individual priority is what leads to a change in behavior and a change in perspective or not. The things that you should be afraid of. As well. Yeah, absolutely. The focus is all on online. And I guess that's because there's data. But do you have any sense of the role that offline communication in any form plays in the same propagation that we're talking about? Because I sort of suspect that to focus online is to miss probably the bulk of the actual picture. I have no idea of it. Increasingly, it's not the bulk. Because so much of this communication and the propagation of particular perspectives and the development of echo chambers reinforces what happens in our lived realities. And I think a dichotomy between a world in person and a world online may not be as productive as it once was. I think the relationships between the two, I think they're very closely intertwined. So studying how individuals communicate in a digital realm will reflect what takes place in the physical world, so to speak. Again, for a lot of these studies, we're not looking at individuals. We're looking at sources that interact with one another. But had we looked specifically for these case studies at individuals, it would be closely intertwined with what happens in a physical world. And there have been a number of other studies done at the Berkman Center that focus on individuals and sharing content. Jonas worked on a study that focused on skepticism and climate change in Germany specifically. So if you have more questions about how the digital and kind of physical realms interact with regards to misinformation or echo chambers, there's a number of other experts here who focus on that. I confirm that because I actually worked with Brittany on this on Zika and did work with people on the ground as well as do the media cloud study. And so it's not the same thing, but it absolutely paralleled the group I work in Brazil and the people who are exchanging large group messages with Whatsapp, where they're exchanging some of these things that are also on YouTube. They're sending videos, audio. These things are circulating, and they are also circulating online. So you're not doing the same thing, but they really did very much overlap. So it gives you some good sense when you're doing this work. What's happening offline, even if large phone groups are offline. But they're very much throwing conversations before having to. That's a good thing. I mean, it's a good thing. It confirms that the tool is powerful. Yeah. I was struck by your comment that people are pointing to abstract in PubMed as part of their argument in favor of vaccines. Do you think having the scientific literature open out from behind the paywall would make any real difference, or would people just misread the full context, or just ignored at that point? That's a good question. I think there are so many reasons we need to open that up. I think our audiences are smart and sophisticated and are figuring a lot of things out, and I think they may be ready for the full versions. That's, but at the same time. I know. I know. Lots of people are. Problem is not enough people reading the core material. Yeah. So that's just. There are like 350 million Americans. Like, how many of them are going to read the core thing? And draw conclusions from that. Rational. Yeah. I wish you looked at what's in the materials. Yeah. It's a combination of that, and not to focus so much on the findings as disseminating the findings in ways that are digestible and aren't necessarily in this select number of peer-reviewed journals. So definitely, I'm not sure if access to the larger article would have led to a different interpretation of the information. But a more readable interpretation of the information may have led to a different interpretation of the information. It's possible. But all of these, again, have to be empirically tested and verified and then disseminated in a popular way for us to really be able to make conclusions about it. No. I mean, the fact that the public reads abstracts means there should be public apps for the public abstracts. Yeah. Yeah. I think, and that's where I'm thinking we need. I think we just need to rethink our whole way of dissemination. But that summary is already out there. The summary is, vaccinate your kids. And like, how much nuance needs to be added to that? I mean, if people, if they're not believing that there's something else at heart, it's not the content of the article. Agreed. Agreed. And I don't think any standalone paper is worth it. I think, you know, you can't look at one paper and say, I now know X for sure, because there's a paper. I mean, it's really within the context of the broader body of literature that gets lost with these individual papers. But you've been dying. Yes. So two major things, as Natalie just said. And first of all, thank you for the talk. What we're talking about now, basically, is the so-called knowledge deficit model. And especially, this has been a topic for years now on climate change communication, with the idea being that if people just knew enough about this topic, they'd behave differently, which, sorry, is not the case. So we can talk about abstracts and open access. But this won't change a lot or anything at all, because which would lead to the second thing. And I think vaccination opponents are not just there, but it's really more a case of identity. They're not just like they have this belief that vaccination will cause autism, but otherwise, they're totally embedded into the mainstream. But rather, this is some aspect of who they are and who they identify with. And this, then, is much differently and much harder to tackle, because it's not a question about knowledge. It's not about a question of just being corrected, but rather a question of convincing someone that the group he or she believes to be part of is wrong. And this needs to be addressed in many different ways, especially, I think, through framing, through narratives, but also, obviously, through a public health system that encourages this in a larger scale. And it already does, obviously. And the bridge figures are the ones that can start to propagate different narratives, because if you're still isolated within individual source echo chambers, you'll never get a different narrative across. So bridging gaps between the sources that convey one message and the source that conveys another is what allows for the greater nuance and diversity of narratives that may finally start to penetrate these seemingly impenetrable echo chambers, which is an issue right now that we're facing in a variety of fields outside of the area of public health. So the great thing is we have some amazing thinkers and doers and who are working on trying to tackle these issues. So if you're interested in getting involved, definitely reach out to us. We'd love to continue the conversation. Are there any other quick questions before we have to abruptly end at time? We're good. OK. Thank you so much. That was really, we appreciate the questions and the pushback. Yeah. There's a couple of paper.