 I wanna think about the nature of literacy because you may think you know what literacy is. You may think you know what learning is all about, but things are changing. So in particular, I want to start this discussion by asking you an obvious question. Are you literate? Now that may seem obvious to you. We are all presumably librarians or librarian friends or technology people, but I walked around Google and I asked this question. Are you literate? Are you literate? And I got the best answer from one of my colleagues who said, what do you take me for? And illiterate troglodyte, which answers the question beautifully, right? Anybody who has that low frequency term in their vocabulary has a particular kind of literacy, right? Not everyone can use the word troglodyte in hand-to-hand combat or conversation. So it's an interesting question now. So I started this quest. What does it mean to be literate today? Well, the dictionary will tell you it's the ability to read and write. But if you start looking in, say, any library's catalog and you search for the term literate or literacy, you can discover that we can be literate about anything, right? There are books about financial literacy, computer literacy, media literacy, emotional literacy, and so on. So I wanna talk about literacy in a particular way. It's not only just the ability to read and write, but it's the ability to read and write in a particular symbol system. I'll tell you what that means in a second. But in particular, it's operationally defined. I can't give you a platonic definition of literacy, but I can say it's defined with respect to your culture, with respect to your social group. It also has associated with it an assumed body of knowledge. You may be able to read and write, but if you don't know who the first president of the United States is, then you're kind of in some sense in the U.S., not literate, right? You can read and write, but you don't know anything. So here's a provisional definition. Literacy is the ability to function operationally within a domain. You can read and write, answer questions, and function critically, critically in that domain. You're not just a tabular ross, you don't just accept these things blankly. And so because I work at Google, I obviously did the obvious piece of research, which is to look at the use of the word literacy over time. This surprised the heck out of me. I hope it surprises you too. Because nobody talked about literacy before 1915? What, really? Yeah, really. I mean, that's the result of scanning roughly 10 million books in English. So what's happening here? What happened? Well, one thing that's really interesting to know is in 1917, there was the Immigration Act, which I think a lot of you know about here, also known as the Asiatic Bard Zone Act passed away Congress in 1917. One of the things it did was to institute a literacy requirement. So this is basically a tested post on all immigrants entering the US who over the age of 15 had to be able to read and write at least 30 to 40 words to show that they were capable of reading. So what does this mean for literacy now? Let me tell you a story. Google has a lot of really cool cafes, and there's one there called The Pair Republic. And so this is the logo, and you think, oh, how cute. It's a take off on the California flag, right? So it's on a street called Pair Avenue. Yeah, clever. But there's a big mural in that cafe, and it starts like this. And these are all my photographs of that mural. The origin of the California state flag is a flub of historic proportions. And basically says a Captain Jedediah Bartlett, who by the way invented the Bartlett pair. In 1846, there was this big movement in the Californios, among the people here, well, Vallejo and Sonoma, who arrested Vallejo and seized the Sonoma Plaza. So Bartlett, who was a captain in that force, who's usual gay, he was the farmer, right, and developed that kind of pair, said that we want to symbolize the richness of our agricultural heritage in California. So he sends that recommendation, I wanna flag with a pair on it. And he sends it to Mary Todd Lincoln's nephew, who misread the note and painted a bear instead of a pair, okay? You can't stop history. And so the story goes that this is how we got the bear flag republic, okay? Now, this is vast. It's basically that long, as long as that wall at Google. And of course, you know, this is the real state flag, with a rather escance-looking bear on it. And so is that story true? Is that how we ended up with a bear? Does anybody know? Okay. So I asked 50 Googlers who work in that building. Is that true? 24%, I don't know. 12%, only 12% did a search to verify the story. 60% of them believe it's true. 60% of them more than half believe it's true. And of the people who did the search, two-thirds of it got it wrong. Are they literate? In what sense do they know anything about California history, right? So now here's the thing. If you go to Google and search for bear flag republic, you will find, guess what? A Snopes article saying it's true. Is my California history that wrong? What's going on here? If you go and look at the Snopes article, that's it. That was the flag that Jedediya Bartlett wanted. So is it true? So now here's the thing. So those two-thirds of the people who actually searched for that read the Snopes article and said, oh, wow, green check mark, done, must be true. If you read the Snopes article carefully, the article is actually about false authority. How carefully should you be reading Snopes articles? And buried at the bottom, it says, oh, by the way, this is all a hoax. Pay attention next time. So I would argue Snopes has failed us in this. I know they're trying to be clever but I understand their point, but there are 60% of Googlers in that building who think that's the true story of the California flag. Oh my God. And in particular, if you now go to Google and type in golden state flag, of course this is what you'll see. So let me ask you, are Googlers literate? Or in particular, remember what I said about reading and writing? Who's illiterate? Who can read and write that? Now in this audience, half of you probably, right? In other audiences, zero percent of people raise their hand. So literacy I think is an interesting concept. It's something that we aspire to, something we want in ourselves, in our students, in our children, our patrons. But it actually is more than just being able to read and write. It's also that you understand that you will have gaps in your knowledge. So this is a kind of metacognitive understanding of literacy. You understand that you don't know everything. You won't know how to look things up. You have an attitude of curiosity about the world. But you can't look up everything. So there's the skill of triage, right? We have to understand how to do these things. So what do we need to know here? We have built a computer the size of the planet, basically. And we got all these exabytes of information on it. And so if you go to your favorite search engine and you type in, say, Saint-Chapelle because you want to learn more about that place in Paris, you can probably land on a book, which will look like that. But if you go to images and do that same search, you will see the true glory and splendor of the Saint-Chapelle Chapel in Paris. Now, what you may or may not know is that you can find that in images, Google images trivially. You can look kind of Bing images. You can look at it from Baidu images. Doesn't matter. You'll find images like that. Fantastic. What you may not know is that you can actually go and there is a Street View version. And so this is a little video of me actually moving around inside the Street View of the inside of Saint-Chapelle. Did you know you could do that? Are you literate? Interesting question, right? I don't want to cast dispersions. But this whole talk is about how do we know what we know and what's possible to find. For instance, here's a key idea. The information world is not uniform, flat, and equally accessible. Surprise, surprise, right? Here's an example where Google has spent a lot of time and money to build a cultural institute in Paris, which basically highlights different topics, different museum collections. And it's part of our effort to foreground a lot of this stuff. So there's a wonderful space and collection about Angkor Wat, fabulous. 3D CAD models, history, et cetera, et cetera, et cetera. The thing is when you search for Angkor Wat, this is buried on page three. You've watched your patrons. Nobody goes there. So you kind of have to know it exists. So an interesting aspect of literacy is knowing what's possible. Or at least having a dream of what's possible. For example, do you know that Google has an immense collection of archival newspapers? This is the Afro-American ledger in Baltimore, one of the first black-owned newspapers in the United States. We've got the entire print run. Did you know that was possible? So what this is, it brings up this really interesting question about what is it possible to know and how do you know it? So for example, I want to show you this couple seconds of this video of this woman putting on a VR headset and being able to look at the world through a VR environment. You can't get to that location in Florence. You can't get to a lot of these places. But this is through your headset. This is available on your laptop. And they won't let you fly over Manhattan like that. And yet this is all easily accessible to you. Here, a person grabs the sun and changes time, literally. So you can go forward and backwards in time. So I come at this, this whole discussion from my particular perspective, which is I'm a cyber-tribal, techno-cognitive anthropologist. I study how people use information systems, not just Google, but all kinds of information systems, including your library, reference librarians, including all kinds of stuff. How do they think about that and how do they know about it? But I'm also a software engineer, an AI guy, and a research scientist. And in particular, I'm gray enough that I remember this. Do you remember life before Google? Right. You've been that person, haven't you? But now, of course, we've got the entire internet in our pocket. And in particular, about that VR thing, the availability of AR and VR on your laptop or on your phone changes the way you think about information. It changes the way you think about place. It changes the way you think about culture. It certainly changes the way you think about accessibility. So for example, here is the Street View version of Stonehenge. It's difficult to look up what's growing on the inside of Stonehenge. But with Street View, you can just toggle around inside of there and look. Go check it out. So here, this is me, Street View, Stonehenge. If I want to see what's on the inside of those stones, you can just do it. And we've been developing, and I mean, we broadly, not just Google, but also Facebook, Bing, and so on, all these different kinds of things. I want to highlight Google Translate. So Google Translate, you know about. It's had its issues. It's getting significantly better. Give us another five years, and it's going to be great. But in particular, in 2015, Jerry McCann and his colleague Shane McKay, he were driving on an ambulance in Quark. And they picked up a woman who was in the late stages of labor. And they're thinking, great, I need to give her instruction. But she only spoke Swahili, and they only speak English. What do they do? Well, Jerry pulled out his phone and said, push, and translated to Swahili. You see the little speaker icon there? He kept hitting that. Push, push, push. And now that baby is a happy baby living in Quark. So having that kind of availability of deeper kinds of interesting information, a utility like that, is game changing for us. So what does it mean? What do we need to know about finding information and learning now? Because we all grew up in a world like this. Excuse me, younger people. You did not grow up in a world like this. I grew up in a world like this, right? This is fantastic. Wait, question. How many people have used one of these? OK, this is a librarian group. Sometimes I give that, I ask that same question, nobody raises their hand. They say, what is that? And I have to explain it. It's a card catalog, which is indexed by, in this case, author's last name. Suppose you want to search by the author's first name. Well, you know what to do. You take out all the cards and sort them by first name. No, you don't. That's insane. More importantly for us, you would never think to do that because that's insane. We obviously, though, if I give that to an Excel spreadsheet, you click column B, author's first name. Half a second later, you have it. Computation qualitatively, fundamentally, changes the way we think about information. I'm also angry. But you see my point, right? These kinds of things shift the ability to think about what's possible. So what do we think about what's possible? What kinds of things can you find online? Don't say everything. I've interviewed a lot of people, primarily younger people, who when I asked them that question, they say, I can find anything. What do you mean? Of course I can find anything. Not true. And you know that better than anyone. But let me give an example because here's a photograph a friend of mine took on his cell phone. He says, ha ha, Dan, that you can't guess where that is. And it's true. It's kind of a generic beach, right? But what my friends did not know is that almost every cell phone photograph, right, has EXIF metadata attached to it. And in particular, at the bottom, is the lat long. So I take that and drop that in a Google Maps and go to Google Street View at that location. That's, you can see what I've done there. That's Street View. That's the photograph. That's Street View. Oh, yeah, it's different. It's got a log on it. But not only do I know where it was taken, I know what time and day he was there when he took the photograph, right? Did you know when you sent a cell phone photograph, you're sending your location, right? You have to, I think, understand that. And that's part of what I mean by knowing about. Literacy now means knowing what's possible. Literacy means knowing the access method, not necessarily for everything. But if you don't know what you're giving up when you do that, you're not particularly connected, not particularly informed. Now, this is interesting because as a research scientist, I do lots and lots of studies. So one study I did was I asked people, how often do you search? If I ask you, you say, oh, I did 20 times. So I did 32 or whatever, right? So here's the thing. When I ask people, how confident are you? 92% they're confident or very confident in their search skill. 92% of your patrons will say that to me, right? The others say moderately confident. 66% of them search less than one time per day. Over half do less than one search a day, right? For less than five minutes. Trust me, you don't get good at anything doing it for roughly 10 minutes a week. You just don't, right? So I wanna bring out this notion that I call informacy, which you may have heard of as information literacy. But too often, the phrase information literacy means learning how to run Excel or knowing how to open up Acrobat file. That's not what I mean. I mean knowing what information is as a first class thing. Knowing, for example, what an index is, knowing what a sort order is, knowing how information is structured. I'll use a word from Ramesh's talk. What an ontology is, you don't need to know what that word means, but it means roughly how is that stuff organized? If you don't know how information is organized, you're kind of at a disadvantage. So knowing what information is and how to use and interact with it. Now let me show you an example here. This is a short video from one of my lab studies where I asked the guy in the study, to find something interesting to do in San Francisco on the weekend. Now this is a version of Google from a couple of years ago, so it looks a little bit different, but it ignored the pink dot. That's where he's looking on the screen, but because he's in my lab and so on. But listen to what he says. I love Laura laughing nervously in the background there. Like, oh no, where's he going with this? So why is that clip funny? It's funny because he doesn't know what's on the internet. He has no concept that that's a dangerous NSFW, right? Kind of query. Totally not safe for work. I'm gonna stop the video there because the next query is adult activities San Francisco Saturday night. Can't show you that. Don't do it now, please. But you see my point, right? He really does not understand what's out there. He would not have done it with my young research associate if he'd known that, right? Now one of the things I find so fascinating about this is how much people don't know. So I assume you all know how to find text on a webpage. You go to the webpage, you type command F, edit find, control F, whatever, right? So what's interesting about this is, if you're on a Macintosh, you do it that way. If you're on a PC, you do it that way. It's the same thing. Now the thing that kills me about this is how little people seem to know that skill. Wait, isn't that fundamental? Well, I did a study where I asked a bunch of people to find on this webpage, which is just the results of a 5K, how long did it take Beth Smith to run this race? How long do you think it would take you? 10 seconds, Macs, something like that? So I ran this with a bunch of people and here's the result. That's time access across the bottom, going from that bump at roughly 15 seconds to a whole bunch of people that were more than 300 seconds. Wait a second. How long is 300 seconds? Five minutes. What are they doing? Yeah, they're visually scanning, right? So what that tells me is they don't know control F. And so it doesn't matter how I ask it. I find a lot of people like that. Surprising result is if you, more than two minutes there, 120 seconds, you're wrong 25% of the time. Excuse me. How can you be wrong 25% of the time? Because people who don't know control F don't know control C. They don't know copy. They don't know paste. And so they try to enter the number into my form by they inevitably drop a digit, okay? They're fundamentally not connected to the way the information world really works. So in particular, I've asked this question of 2,000 people, 90% of English teachers, 51%. 51% of the English teachers of your children don't know control F. Yeah, 93% of Firefox users have never used control F, okay? It goes on and on. Doesn't matter how many times I repeat it. It's the same striking result. People have no clue how to find text on a page. Let alone Acrobat, let alone any of the systems you have here, right? So we've asked questions like this endlessly. We gave a whole bunch of people little simple trivia questions like this. And if the people who know control F, the result is they're 12% faster. That is they're 12% faster total time to result. And they're also more accurate. 12% is a huge amount. A huge amount. Now, there's something we said about this. Like, where did you learn control F? Yeah, I don't remember either, right? On the other hand, when I ask people, do you know control F? They say, oh, no, no, no, no. Who was supposed to teach me? Was it you? Librarians? Wasn't the English teacher? Wasn't the math teacher? Wasn't the science teacher? So there's disconnect going on between the digerati, us, and the people who are not the digerati, everyone else, right? So one of the things that this implies is that people in the inside of the information world know a lot of stuff. You are literate, digitally literate in an interesting way. For example, I started making a list of all the different genres and kinds of things I could find. This is a fool's errand. There's a million of them, right? Kind of goes on and on. But let's talk for a second about spoof sites. Everybody here knows what a spoof site is, right? You have probably used, in your class, as the great Pacific Northwest tree octopus website. For anybody who hasn't seen it, it's a spoof? Don't believe it, right? But it gets a little bit more complicated when you look at something like this, which is a beautifully produced, high quality website advertising R.I. hospital, R.Y. hospital. And it's really nicely done. They take credit cards, there's a phone number, the street address, everything. It's not until you get to that page you start wondering about the hospital. Male pregnancy, really? Is that a thing now? Well, it turns out when you dig deeply enough, you discover that's a designer's demo site. He just made stuff up. But a lot of people get fooled by stuff like that. So you need to know that spoof exists. And in particular, videos spoofed exist. Now, here's a fun one. You might've seen this on CNN, where Pope Francis goes to this chapel and then does a miraculous thing. You're laughing. That's a miracle, right? It's a miracle of image editing, right? Now, you obviously looked at that and said eh, I don't believe it, right? It doesn't concord, it doesn't match with what you think, what you understand about the world, but there's more. People who are going to be using these systems, people who are literate now, need to understand how to think critically, how to look at media like that and understand. It's probably not accurate. And in particular, they also need to know conventions of the culture. So we all know emoticons, right? We recognize the semantics of each of those, right? You know, the terms like NSFW, ROTFL, right? You know those terms. But we have to recognize not all of our patrons do because I found this on a famous question answering site. Is it true Rosa Parks would have moved to the back of us but she was listening to her iPod? And the answer, the best answer, chose by the askers, yeah, it's true. And then the responses, which is fantastic. She gives it five stars and says, thank you, I'll put that in my report. You have the nicest avatar. So what happened here? What happened was the asker didn't understand what that symbol meant. Now you need to know that's the universal symbol for a sarcastic response. So the asker didn't understand that. And yet they based their credibility assessment on the beauty of an avatar. Trust me, that's not the right way to evaluate credibility of a source, right? Of course, the weird thing is if you define that on Google, it'll tell you it's sticking the tongue out. It's a sarcastic reply, it's not serious. They missed that part of the culture, right? One of the interesting things about the internet is it's trans-cultural, it is trans-national. And so you need to be able to look up stuff safe in other cultures like this one. I don't know about you, but that looks like kind of a fun icon or emoticon I should use in my text to my daughter, right? Okay, some of you get it, right? The rest of you should look it up. Because it's actually the love hotel emoji. And an eggplant is not an eggplant. Just so you know, right? So there are these interesting, deeply embedded cultural things, but the beauty is you can look these things up. You need not be an idiot, you can actually check. Now one of the things to understand is that a literate user understands how to use a dictionary, what a sort order, a canonical sort order for a text star, and so on. You also understand that you can do things like this and look up what that image is. So where was that photograph taken? Does everybody know about Google Reversed Image Search? Yeah, you do a reverse image search and you find it was actually taken outside of Las Vegas at Red Rocks, beautiful place. You can search for archival content. Here's an image, something you may have found in a book. You photograph it, you search for it and discover it's actually from Napoleon's scientific, shall we say, voyage to Egypt. You can find this stuff instantly. It changes the equation. I didn't know about you, but when I was growing up, that was an impossible task. So all of a sudden, impossible searches have become trivial, they've become fast. That puts an interesting burden on us as teachers to teach how to do these things in a smart way because content changes, right? We looked at this kind of list before. Let's talk about video for a second. Not only has this nature of video now changed in fundamental ways, but every minute we're here, 400 hours of content is being uploaded to YouTube. 400 hours. So let's think about Google Images. Every day, 850 million images are uploaded. We index all those things. Let's talk about scholarly papers. 1.8 million papers in 28,000 different journals, some of which have quality problems like this one. I'm not gonna read it to you, but that's an actual paper published in an actual journal that has zero quality control, what we call vampire journals. The question for us at Google is, how do we automatically detect vampire journals? This one's easy. Other papers are harder to tell. And of course, libraries are also subject to this because there are a lot of vampire books. If you find a book by the author Lambert M. Sirhone, don't buy it. They're basically reprints of Wikipedia articles for which they will charge you $87, okay? It's an interesting way of spamming the Amazon catalog because they could write a program. So this person has over 120,000 publications. I don't think so. Something's wrong here. It goes on and on, right? There are millions of things like this, but importantly, we live in this interesting adversarial time where you can also search for applications that are completely bogus. No, you cannot turn your phone into a cattle prod. You cannot do a pregnancy detection test on your cell phone, but there are apps you can buy that will sell that property to you. We need to teach this to our patrons. You can index things, but is this part of your purview? Is this part of your catalog? Do you teach your patrons how to do this? So you can do this kind of stuff. You can search for these things and different applications give you all kinds of metadata filtering, the ability to search for education apps within a particular style and genre and so on. So the search engines now index these things, right? We can search for a particular thing and you can actually go into the app itself from the browser, which is a remarkable thing. It's actually running in a cloud somewhere. How does one index these AI applications like an agent? So Google Home devices or Alexa are all powerful providers of information, but how do you search for them, right? You can go to Google Home and say, what is the ALA? They will tell you, knowing full well that there's a description of it. But think about this. There's a whole rest of the world which speaks another language than American English, American standard English. So when I was in Japan, I heard this woman singing on the radio and I thought, oh, that's great. I keep hearing her name on the radio and it's suji on it or something like that. So I go to Google and I type in my approximation of that. It spell corrects it in English. That's a cute trick. It took my English spelling and figured out what the correct Japanese spelling should be and now I have her music on my phone. So there's this possibility of doing international trans-cultural kinds of things and in particular, have you ever looked at the Italian Wikipedia? Maybe not, but it's worth looking at other Wikipedia. So here's the English and the Italian version that the table of contents for the Wikipedia articles for Leonardo da Vinci and English and Italian. Notice anything different between them? Yeah, the Italians really care about Leonardo. The bio section of the Italian Wikipedia article is longer than the entire entry in English, completely longer. So once you start looking at multiple languages, different cultures, these are four different languages for the same article. You see where this is going? If you've got computation in your phone, it allows you to do things like that. That's Google Lens running on my phone, doing a replacement in place, using the same font to trick, doing the translation. So, informacy, as I've been thinking about, is being information literate, understanding what kinds of things you can ask, what kinds of responses are possible. So for instance, my son is a big Stanford basketball fan and there's a song that they sing at Stanford. It's like that. And he says to me, dad, I want that on my phone, how do I find it? Dad, you work at Google. How do I find it? And I think about it for a second, I think, oh, let's use Soundtown or Shazam or something like that. No, it doesn't work. It's a thousand drunken Stanford fans singing, right? So these things only work with nice clean recordings, not that. So I said, son, oh, let's do lyric search, right? What are the lyrics? Oh, oh, oh, oh, oh, yeah. I say, give up. You're not gonna get it. He goes to Google being my son and types in, oh, oh, oh, oh, damn it. He now has that song on his phone. So what did I not get? Content changes. And in particular, all those QA sites, the question and answering sites I was talking about, there's a million of them now. They're super important and they're now answering questions exactly like this one. They're even more important in other cultures. Naver QA site in Korea is massive and it's really rich and interesting. And yet, you know, we're also creating new kinds of media, new kinds of genres of things. So for example, we can now write programs that will go out and find images from different times and then aggregate those into a time lapse. And this is a place you might recognize. That's Lombard Street. Notice this is over the course of two years. It is an aggregated time lapse. No one person took this. Who owns the copyright? I don't know, I have no idea, right? More importantly, how do you find such things? How do you find something like this? This is actually not a video. This is a set of images that I pulled off of a Japanese satellite web server and made into a video. Do you know how to do that? Do your patrons know how to do that? So what it's really trying to get to is, how can we shift the culture of us or our patrons and so on to think about these things? So you probably have this book in your stacks, right? 20,000 Leagues Ended In The Sea. What does the title refer to? When I ask people, they typically say, it's about a submarine, right? And that goes down 20,000 leagues. And I say, okay, what is a league? It's a unit of measure about three miles. What's 20,000 leagues in miles? 69,000 miles. What is the diameter of the earth? That's probably not the depth, right? It went to. So what it tells me is that people didn't understand the title. They didn't understand their first four words. What else are you not understanding, right? So what we need to start thinking about is how do we help our patrons become critical thinkers, literate understanders of what we have to offer? I mean, we librarians, we Google, we the world. So it's important and it's no longer optional. I mean, when I was in student, when I was in sixth grade, oh yes, critical thinking skills are important. Yeah, okay, now, let's talk about chemistry. All right, it was about like that. But of course, people find websites like that, like epa-facts.com, and believe it's about the EPA in DC. Now, what's odd about that is when you actually look at the about page for epa-facts.com, which is what most people tell you to do, if you want to assess the credibility of a site, go check the about page. Wait, who wrote the about page? The people who run the site, okay? So if you go to epa-facts.com, it says, epa-facts is Project Environmental Policy Alliance dedicated to highlighting the high-cost environmental protection agency, okay, your spidey sense should be ringing at this point, right? A, it's a dot-com site, it's not a dot-gov site. Who knows about dot-gov sites? You do, your patrons have no clue, right? So they don't have a spidey sense going at this point, right? And then there's this weird thing, Environmental Policy Alliance versus Environmental Protection Agency, whatever. If you actually look up the EPA's logo, you see that they're clearly spoofing the logo, right? So when you do this and dig in a little bit more, you say, well, let's check out, where is this place? Well, it's in DC, that makes sense. But it's sweet 800. EPA's in that sweet, really? So let's look up the EPA's address, or actually let's look up that street address, and you'll find another company there called Berman and Company. Wait, the EPA shares a sweet with Berman and Company? My spidey sense is going thermonuclear at this point, right? So let's look up the real EPA, and they are at 1200 Pennsylvania Avenue, that makes sense, right? That's constant with everything you know. Them sharing office space with a lobbying firm, which is what Berman and Company is, makes no sense. Okay, now here's the striking thing. My colleague, Sam Weinberg at Stanford, has found out that 80% of university, at a well-known West Coast university, shall we say, can't do this. 80% of university people, students, at a well-known West Coast university cannot do this. And so when you end up with people looking at things like that, they don't understand that they're all cloned. That's all made up stuff. They don't understand that you can copy an entire website. So we end up with this interesting quality quantity paradox. We have a universe of content, but little kind of speculation issues with quality. How do we teach? How do we help our patrons become discerning quality readers? What do you need to know to be literate? You need to know your tools. How many people know what that is? Okay, it's awesome. I didn't learn about it until I was 35. And you may think of me, oh Dan, you're so illiterate, you know nothing. How many men know what that is? Okay, not to cast this version, but you know, I know what it is. I use it all the time. It's a thread-puller, right? It changes the game if you're trying to throw the stupid needle, right? Okay, how many people know what that is? Doesn't matter. But with search by image, you can figure out what these things are. More importantly, if you go and know how to search for a tool that changes the way you think about content. So for example, I've given this problem to, at this point, probably 1,000 different people. But in this study, I did it for 250 Google search engineers. The people who built Google for you, right? I asked them, can you find this house? It's 900 University Avenue downtown Palo Alto. Can you find an aerial photo of this house taken before 1977? How hard can that be? None of them got it right. Zero. I asked 2,000 people to do this same task. Gave them as much time as they want. One person out of 2,000 got it. He was not a librarian. Okay, so what went wrong, right? The solution you should have known about, in particular, the Google engineers should have known about, is that we have this thing called Google Earth, which has the ability to go back in time. How many people knew that, right? There's archival aerial imagery in Google Earth. And so this is a picture of that location, 1948. Check it out. Check San Francisco. In particular, check the wartime, because the city was all blacked out, as you would expect, right? So we need to think, how can we communicate and help become literate? How can we tell our patrons ourselves how to do these things? Because not only is there unbounded new interesting opportunities, new kinds of content, but we also live in a world where fake content happens. This is Superstorms Dandy, right? New Jersey. And it was trivial to find out that that was a fake photograph. No, there were not sharks in New Jersey, right? But as Hemingway said, every person should have a built-in craft detector, although it gets hard, because this was from the previous. Yeah, I'm going to go a couple of minutes over. Actually, I have four minutes left on my timer, because I have a millisecond correct timer. This is the Grand Marft Harkin or Peter Cushing, who obviously died before that movie was made. But we can build synthespians, build a computer graphics model and light it appropriately, you get great. Now, carrying this forward, there's now, of course, this property is behavior of deepfakes. Jennifer Lawrence at Golden Globes and Stephen Buscemi on the right. You can combine those two images, creating a very kind of creepy thing. You can do this in video. Ick, right? So, now, you've probably seen the Obama one or the Trump one, there's a million of them out there, but what I want to tell you is you can search for these things. You can grab a frame from that and search for it and we'll tell you it's the deepfake of Jennifer Lawrence. So, we need to teach our students and ourselves how to be better, how to be learning for these things. And so, I'm going to honor the time here. I'm going to skip forward a little bit. This is a very interesting topic here but I'm going to skip forward to the final minute or two. So, one thing to realize is that we need to be able to teach these basic skills. Learning how to ask the right questions, knowing what tools exist, what resources exist. And this is fundamental to what we do. We do, both you as librarians, you as information, content providers, us as indexers of that content. And we need to understand what information available, the content that's available online and offline, and how to transfer these skills to our patrons, to our students, to ourselves. Because in order to become an informant, we need to continuously develop these skills going forward. So, we, and I'm talking about now us, need to design these information systems to help support that. Because it's going to be lifelong learning. We need to build explanations of why I think they're showing up the way they are. We need to teach our people how to do this. And so, I've built a online massive open course called powersearchingwithgoogle.com. And to date, we've had 4.4 million students take that. So, with my colleague Anne Fernald in the back, we just redid it, so it's brand new. So, if you're looking for online course for your patrons, there's one. I write a blog, and every week I pose a question, and then the next week I answer it and show you how to do it. That's a way to get smarter. And like Ramesh, we've all got new books. Mine launched last week, it's called The Joy of Search. And it has a lot of these examples in the book. And it's overtly an attempt to help people through the mechanism of stories, learn how to become more informed and more better searchers. Because tomorrow's illiterate, as Alvin Toffler pointed out so long ago. Tomorrow's illiterates will not be those who cannot read or write, but those who cannot learn, unlearn, and relearn. And so, I want to end there with the thought that it's important for all of us, all of us, to understand what's possible, what's out there, what we can do, and most importantly, to be an informant learner, learn how to learn the next thing. That's a kind of metacognitive trick that we all need to incorporate in our daily lives. It's not enough to have learned library of science once upon a time, because it's changing daily. Thank you very much.