 All right, so this is my title. But I should have called this Build the Memex you want to see in the world. And you'll see why in a moment. I didn't have that in mind until now. My talk today is about an extracurricular experiment I'm working on. But I'm a librarian at the Evergreen State College. And I did want to mention that I'm working with two classes at Evergreen this past quarter to integrate collaborative web annotation into those programs. So we have hypothesis going at Evergreen finally in our curricular way. But this talk is about something else that relates to my campus. How many of you were aware, before you saw the program for this conference, that there were notable protests at Evergreen a couple of years ago? All right. Well, now the rest of you know that there were notable protests at Evergreen a couple of years ago. That's notable. A big challenge in prepping this talk was how to talk about media representation of the Evergreen protests in 15 minutes when we don't share knowledge about the specifics of the protest in this room. So I'll provide a little background, but then focus mostly on one event, because I don't have time to describe two events. I tried. It doesn't work. This event that I'm going to talk about became kind of a meme with a reality distorting power. So what I'm going to try to do is think about whether web annotation could, in theory, help dememify through annotations constant insistence on staying firmly anchored to text. So here's a little bit of that background. Protests took place on campus through the year of 2016-2017, largely motivated by calls for greater equity and inclusion, and often critical of the college for not doing enough on a number of issues, but especially race. There were four days in particular that stoked very significant interest in the national media. Tuesday, May 23, exactly two years ago today, some protesters disrupted the classroom of a professor, Brett Weinstein, and the protest was videoed and versions of it went viral. Then a lot happened. Lots of video was taken and circulated, other protests, other issues. And the media took notice, Washington Times on Wednesday, Washington Post on Friday, and dozens in between. On Friday evening, two things happened simultaneously. The president of the college responded to the student demands by accepting most of them, but refusing the demand to suspend Weinstein. And at the same time, Weinstein was on Tucker Carlson's show on Fox News to talk about his experience in the preceding few days. I don't know if any of you know how many viewers an average Tucker Carlson episode has, but we had evergreen learned at this time. It's a lot. And it was about this time that I began to create a bibliography of articles, interviews, and things relating to the protest. And you can see here, barely, that on the highlighted green boxes show five publications on one day, June 30th. USA Today, Wall Street Journal, Washington Post, Inside Higher Ed, and a blog interview with Weinstein, one of about a gazillion he did at that time. So this was pretty big media exposure. Two years later, I got more than 500 citations. And the list is still growing. My tagline for the bibliography says that it covers Wonkat to Wall Street Journal and PR to Joe Rogan. So if you know who Joe Rogan is, you'll have a sense for the kind of thing that I include. And I don't know if you all know how many people watch or listen to a Joe Rogan podcast, but I believe it's even more than Tucker Carlson episode. It's huge. So today, I want to show an experiment I'm working on that is related to the bibliography in which emerged from three things that happened to me about at the same time a year ago. First, I discovered hypothesis and joined in March 2018. Second, I wanted my bibliography to do more. It was just sitting there growing but doing nothing else. And third, I reread Vannevar Bush's 1945 essay, As We May Think, which you now all know about. I got to thinking about that memex. And here are the specific parts that caught my attention. Bush envisioned, and I think this is accurate, a lonely scholar at a very large desk. The desk contains the entire corpus of some body of knowledge on microfilm, as we saw, which was a revolutionary technology at the time. And the scholar painstakingly creates links among associative blocks of text, calling them individual blocks of text, creating what Bush calls associative trails. And Bush imagined the scholar creating an indexing system for tracking and quick recall. So that's pretty much the model I'm working on. My corpus of recorded knowledge is the bibliography I maintain. And one of my goals could be described as building associative trails among the 500 text, video, and audio citations in it. To date, everything, except for the bibliography itself, is unpublished. And I think one thing Bush failed to point out about the memex is the workload of the lonely scholar who's building associative trails and creating indexes for them. So here is one portal I'm working on to lead readers into my annotations. I have unidentified 12 articles that are specially helpful for understanding the protest so that people can get to the most helpful ones. And for each one, I provide a citation and a link to the article on the web. And when a paywall is an issue, a link to the library copy. Then I have a page devoted to each of the 12 articles. So this is an article from Inside Higher Ed. The format I use is I have a critical summary on the left that I've written. And in the top gray box there, I point out that hypothesis users can see annotations on the Inside Higher Ed site and add their own. I'm an advertisement for hypothesis because most people won't know what it is, but then they'll wonder. And in the bottom gray box, I provide a link for non-hypothesis users so that they can see the text and annotations through the hypothesis via proxy. And it's very cool that there is a URL for that. The next one is Wall Street Journal, Bret Weinstein himself. And here we get an interesting issue, the Wall Street Journal paywall. It's a pretty strong paywall relative to some others. What's an annotator to do when encountering a paywall? In a project like this, I can't just ignore the paywall articles, right? This is a seminal article right here. So in this case, since I don't subscribe, I annotated the Evergreen Library digital copy. And I let Evergreen users know that they can find annotations there. That's the best that I can do under the circumstances. In this case, the Washington Post article, I have annotations on the Washington Post site because I subscribe. But for non-subscribers, I have to note that our library doesn't actually have a copy. So I provide a link to a URL that Hypothesis provides with links to a page that has all of my annotations from that page. It's a great thing that you can see all the annotations that one user has put onto a single page. So now we get to an actual annotation. In this annotation, I'm evaluating a specific claim. And for the reader who doesn't want a lot of detail, I cut to the chase with a short take. You can't quite see that there. But in the annotation layer, I write a short take. Gives a very brief description of the claim and evaluation of it. And then I add a long take, which you can see is quite long, and goes into much more detail linking to sources. There's a link to the Seattle Times where there's an article with a key piece of evidence that the kind of people who choose long takes, choose to read long takes, would be interested in. This is an old image, though. And I decided the long takes were taking up too much space. So I decided to outsource the long takes to a web page. So now I have a short take and a link to the more developed explanation, which sits on its own web page. And it resides in an environment like this. The box here is devoted to claims that the protests put Weinstein in danger. Turns out in protest that danger is a universal theme. Everybody on all sides always seems to be in a lot of danger. And this claim has been important in media representations. And it's became part of a lawsuit against the college leader. So each tab gives the long take on different aspects of the danger claim. And now different annotations and different articles that discuss the same claim can now link to here. So I don't have to reproduce this in different annotations and different articles. There's 500 articles. Some of them are going to want to link to this description, because it'll be relevant. This page also becomes sort of a self-contained information environment of its own. But one thing I want to add is links back from here, back into the hypothesis and the annotations. So this is one draft of a larger structure that my annotations will play a role in. So now with a little more info about the media and the protests, I can look at the annotation level further. Why was Professor Weinstein targeted by a protest? Again, you can't even begin to summarize it in a short time. But fortunately, this talk is about the media. So we can do what the media do and reduce the issue to a single issue. Oh. Yeah. Day of Absence, which Weinstein opposed very publicly and dramatically. And many on campus then opposed his opposition. Probably the main question debated in the press about Day of Absence was, was it in some sense mandatory that white folks be off campus on the day of this campus event? Let's go to Tucker Carlson himself and have him explain. Evergreen State College in Washington and Olympia, every year the school holds a day in absence in years past non-white students symbolically left campus and attended anti-racism events. This year, student activists demanded that all white people leave campus or else. All right. He's pretty clear about two things here. There is a demand that whites leave campus on the day of absence and a threat that bad things would happen if they didn't. So a YouTube viewer. Evergreen State College in Washington would be lucky to have an annotation. So I created one on the page. And this is a closeup of it. In the blue box, I wrote an explanation that he didn't get this. Tucker Carlson didn't get this right. He made two false statements. And you can see, so that's a basic fact check, and you can see that my annotation also links out to a source that backs up my assertion. That's in the blue box there. Day of absence changes form. Links out to the student paper, which gives a better description, a pretty good description actually, of what the day of absence was. It was published two days before the day of absence. And it did not involve that kind of coercion that direct threat and mandatoryness that Tucker Carlson indicates. So this is a second use of annotation linking out to a better source. And I like to use that a lot. With this kind of link in the annotation, I'm at least grounding my claim that Carlson got it wrong and something available in the public record. And also making sure that verification is only one click away. My annotation has another comment in it. Weinstein has always been careful to say that the day of absence was not mandatory, unlike Carlson. But he didn't correct Carlson's statement. So Weinstein took a lot of criticism for that, and he addressed it in a video interview later. That's the link in the orange box. So this is a link to a later commentary on the Carlson interview. So it's like a cited by feature that connects the reader to a text published later that comments directly on the correct text. OK, now a shift of perspective. As an annotator, I might not always want to think in terms of fact or not fact. I might want to get beyond fact checking. So I might prefer to track the impact, the calling day of absence, a show of force and an act of oppression, had in the discursive realm. So we can see the media frame shifting in the Tucker Carlson interview. Weinstein, a couple of months earlier, starts with show of force. And Tucker Carlson adapts that to a demand that whites leave or else. And as a major node in the media, Carlson then sets the stage for inflammatory formulations like these. Racial apartheid, by the way, I think comes from the college fix. I think they're the only ones who have used that particular one. For tracking this sort of stuff on the web, tagging can help. And I have tags for a number of different formulations of the day of absence. This is my no white people tag. And hypothesis allows me to show all the pages that I've tagged with that. You can see the Washington Times, the Daily Collar, Daily Mail, Infowars, and others. So right here, we have a snapshot of where this particular frame has been used. And we can dive into any of these sources with a single click to explore context. I also have a Nazi reference tag referring to protest groups as Nazis or brownshirts or stormtroopers. Has a long history. And in fact, I was once called a stormtrooper by the Wall Street Journal editorial board for a protest that I participated in college in 1991. So that frame has been around for a long time. And it crops up in every room. I'm just taller back then. But they didn't mean the Star Wars kind. They meant the- Oh, OK, sorry. I was warned that people would say that here. All right, I want to sum up now. Why is web annotation important to this project? I think the great thing about web annotation is that annotations are inwardly anchored to a very specific block of text in the document being annotated. But at the same time, using devices like tagging, annotations can be outwardly anchored to associative trails, in my view, that allow you to see very quickly something that's going on with any number of sources and puts you one or two clicks away from any of them. But every node is connected to some block of text or audio or video. Everything's anchored somewhere. So I like this because it makes the annotations directly answerable to the text that's annotating. This makes it hard to just make stuff up, not impossible, but hard, and then call it the truth. And in a post-truth era, I think that's valuable, especially when you have events where the meanings are hugely contested, like the evergreen protests. I think the sort of extreme anchoring to sources might be a valuable thing to try. The radical folk singer Phil Oaks once said that a protest song was a song that was so specific it couldn't be mistaken for bullshit. One thing I like about annotation is that annotations by their very nature are so strongly anchored to their source material that you can always ask, not only ask, but also answer the question, is that annotation specific enough that it can't be mistaken for bullshit? And as I extend my associative trails through my corpus, I do sometimes find that an earlier annotation has turned out to be wrong. The developing associative trails I'm making continually force me to revise and improve, and I think that's a good thing. And that's it. Questions or comments for Paul? Hi, Paul. I'm Marina O'English from WSU. Washington people represent. Anyway, my question is, have you since live guides actually worked within the hypothesis framework, have you thought about having annotation directly on your library guide? I haven't even thought about the integration of the two. Are you saying there is a formal integration? Yeah, I just pasted it and checked it. I'd never thought of that before about checking live guides. So I'm wondering if that's something that you might want to include in this. Yeah, of course. Yeah. Cool. Well, thank you very much for that. It was great to see a really good use of the tool. One thing that kind of bothers me is that if someone were to go out and seek that commentary, they might come across the work that you've done here. But how would it scale up in the sense of like, if you're watching Tucker Carlson, the people who are consuming most of that content, are people who are not naturally going to be that critical of it? How are you going to get, for example, the slight twist where suddenly something that was a voluntary thing supposedly becomes a mandatory thing? Obviously, that's going to be lost on the vast majority of the audience. So a question I would have is, how could you get those caveats in front of more people? How could you make more people aware that in this video that their feed that they're watching is missing important aspects of the story? So you're suggesting maybe different ways of approaching these questions in different venues, I think, that maybe this approach that I'm taking isn't going to work in certain contexts with certain readerships. And I agree, and I don't really have an answer to that. It is, in general, I think opening this up, certainly to anybody annotating, is a scary proposition. And I think that that is something that has to be considered in web annotation in general. Yeah, because if you could imagine at scale, like if you have a small group of people like yourself sort of annotating that content and someone who comes across annotations, that's fine. The moment, on the other hand, this thing would have, if you were to become a common thing at sort of a mass scale, then you're going to have to decide, well, OK, whose commentary are you going to look at? Because now you've got like a choice of news sources. And then for every choice of news sources, you have a plethora of sometimes malicious and sometimes benign commentary on top of it. Yeah, I think in terms of scaling, there are all kinds of problems with this, including how much work it takes. It does need to be collaborative to some degree. Other people would have to be involved with it. But I think the question you're raising is a good one. But I don't know, how bad would it be if there was a hypothesis annotation, Tucker Carlson's site, that everybody could see that said this, would it be negative thing? Or would it just not be achieving its goal, I guess? Cool. Thanks very much. Hey, it's Doug Shepers. Sort of building off that point, or maybe answering it, I'm not sure, or addressing it, at least, to what extent do you think that there are talking points. And these talking points get out there and they get repeated and repeated and repeated. And they get repeated not only for one issue, but they are recycled for another issue. And it builds up this whole huge cultural debt of disinformation. And if you were doing something like this and pointing out, here's the point at which do you think that an accumulation of annotations, this open commons of annotations that refute existing talking points could maybe address that issue on a whole? Sure, maybe on this particular issue, the minds are not changed or whatever. But when this comes up again and again and again, somebody could point to those annotations maybe and a body of counter-argument could come up against these. Do you think that that's valid, or do you think that it all gets lost in the shuffle somehow? I'm sure that I'm thinking along those lines when I think about whether there's a future for this kind of thing. That if you provide a careful argument and you cite your sources and then you put it into a format that's relatively straightforward and simple and easily accessible, that maybe that will actually have some impact on a substantial number of people. It's aspirational. It's a hope. But if I'm responding to what you said correctly, I'm sure that is underlying my way of thinking about this, that despite a lot of evidence to the contrary, that is not the way our society has to go. And that there are people really with all different political perspectives who can look at a careful structured argument, if it's short enough, and recognize the value in citing your sources in this kind of environment. And when you're doing it on a hot topic question, more people are likely to actually look at it. I don't know, maybe it can have some impact.