 Good morning. My name is Gary Price. I wear many hats. I see many friends, including many of those who pay me sitting in this room right now. I'd like to let you know, unfortunately, my good friend for nearly 20 years and co-presenter Peter Brantley was unable to make it to the presentation into the conference this year. And I will do my best to give you a full, only 30 minutes of content. And I guess we can start by saying, some of you know I do something for ARL every day, ARL Day in Review. And also, many of you know me as the thanks to Gary Price at the bottom of Cliff's post on CNINL. Gary Price is a real person, or I think so. But really, so here's some breaking news. Mind reading AI can translate brain waves into written text. So I'm going to close my eyes. You close yours. Thank you very much. I hope you enjoy today's brainwave transmission. The presentation, as you can see, is already online, so all of those of you with devices of any sort can follow along or jump out ahead of me. As you can see, I can speak fast and I can speak loudly. While some of you probably don't know that my job in college, when I would go back from the University of Kansas, Rock Truck Jayhawk, to Chicago where I was born and raised, was working at Wrigley Field. No, I did not play second base. Did somebody say that? I can talk loudly and quickly, and if I start yelling Budweiser, my mind is off somewhere else. Here is today's presentation. I do plan on updating this moving forward because what I'm about to share with you today will very likely be different by the time you get home, either later today or tomorrow. My job, my main focus today is to share with you some actual live demos of some actual live resources. Some of those that actually share citation provenance information of where the data is coming from, because as we all know, one of the biggest issues with using many GPT tools as of today is they provide little, if any, correct provenance information. But these, as Cliff said yesterday, these are the early days, and the number one thing I am hoping to share with you today is the motivation with some of the resources to go out and play with them, use them yourselves and with your staffs. Education is key to all of this, and it's constant education. What works? What doesn't work? How can we take some of these tools that might not exactly be built for our community and use them in the best way possible? I also see, and I think this is what Cliff said yesterday, I feel like old is new again. For those of us who have been doing this for a while, I feel like this is the same period that we saw in the early days. Not so much in the early days of web search, but the early days of Google. Well, is Google going to take care of everything and are we putting ourselves, are they going to put us out of business, blah, blah, blah. I don't think those things have materialized, otherwise we probably wouldn't be sitting here. But we need to think about what is old is new again? What do we do for ourselves? What do we leave to others to do? How can we work with those others to get the results that we're looking for? And remember too, there are a lot of tools, whether they be social media or research tools that are not quote unquote library tools, but these are resources that our users are using, and I think it's just as important to be aware of those tools and how they fit into this entire ecosystem. What does lack of provenance citation, where the data is coming from mean both short term? How are you going to be able to cite this in a bibliography aside from saying it's coming from chat GPT or from open AI and long term? What does this mean for the whole scholarly record going long term? And my other question is, even if data is provided, or when a response in an AI chatbot is provided, how do we preserve that? Because the same response for me, especially if it knows a little bit about me and I share that information with him, is very likely going to be different than the response that somebody else gets. Additionally, these models are constantly being updated. So the response I get today will very likely be different in the future. So the question is not only what does this mean for history, for the scholarly record, but how do we preserve this going forward? And should we be thinking about that right now? And I would argue that we should be. Obviously, there are so many other issues to talk about. Many of them were mentioned in the close presentation tomorrow. Copyright other intellectual property issues. Another thing that is not getting as much attention, I spoke at CNI many years ago, and there have been many presentations since about digital online data privacy. What is, I don't think at this point with all, for a good reason, some of the, a lot of the hype around chat GPT and other tools. We haven't talked enough about the privacy issues, both in terms of training them, and in terms of the privacy issues, if you're registered user, what data chat GPT, open AI and other models are gathering about you. Obviously, if you're using the barred service from Google, that's just more data that Google has about you. And it's not just keywords now. It can often be actual statements, which again provide more clues if somebody is capturing that data. Again, we're not going to get into surveillance issues and all that, but that's, these are the things to think about. I also think that there's another huge opportunity with AI and before we move on, and that is with non text materials. I'll see if I can fire this up here real quick for you. One of the things that we are, one of the things I'm really excited about is how open AI, how AI GPT can be used with non text materials like video. So this is Cliff from, was it last year or so, his speech. There are many tools, if you haven't played with them, that are available as add-ons for whatever browser that you're using. But we know for a lot of people for various reasons, add-ons are things that they don't know about, can't put on their browser, blah, blah, blah. As of last week, the Microsoft Edge browser, which is their replacement for Internet Explorer, now if you use their browser with their, what used to be called Bing GPT, which is now called Copilot, they natively now have a web, some YouTube video summarizer built right into it. So you could say summarize the video, here's, I'll try to make it a little bit larger. I can't, thanks Microsoft. But you get the idea, I ask, and now not only does it give me a summary with time stamps, but now I can start questioning, did Cliff mention this, did he not mention that? So I'm now actually being able to chat with the video. And I think with the amount of video material out there, I think this, the AI, which has been around for a long time, and more specifically GPTs, can help make this much more of a reference tool and a research tool. Let me make this a little bit larger again. Other issues, challenges, this is constantly changing, blah, blah, blah. One of the things I find interesting too, and a lot of these things which you can tell are hyperlinked to learn more. Well, one of the things when you look at Microsoft Copilot, Google Bard and Chat GPT, is at the bottom of all of them, they have these disclaimers. Bard may display inaccurate info, including about people, so double check it. This is yet more ammo, if you like, more reason, again it has been discussed at the plenary yesterday and in other sessions, why information literacy, GPT literacy is so important. We know these things, we're, I think it's inherent in some of us just by who we are, to doubt what we're reading. There are users who are using, for the most part, the free version of Chat GPT, even taking the time to see the small print down at the bottom of the screen. And what does this mean, both short term and long term? Lack of user awareness, remember the word from about 10, 15 years ago, satisfying? GPT can really help with satisfying, because now not only do you get a couple words, you get an entire paragraph that you can use as you see fit. So again, information literacy, awareness of this, even though you might not use all of these tools on a day in and day out basis, is so important. I think one of the biggest opportunities for the library community, as of now, is not only to become aware of this stuff, but is in the creation of tuned and focused GPTs and the like. One of the things on the reading list for you is a link to a series that Lorcan Dempsey has been writing on his blog. I encourage you to take a look at that in his latest post. He actually goes through some of the steps he used to create his own GPT. But let me show you an example of what I'm talking about. I think we're going to start to see more and more of this going forward from all sorts of collections and uses. So last week or two weeks ago, USC and the Annenberg School at USC came out with your annual report about the state of public relations in the United States. Well, along with the reports, they also created a GPT of the reports. So now along with getting the actual report, you can end at the moment, if you have a subscription to the full version of chat GPT, which I read this morning, they're back open. That doesn't mean you're going to get one immediately, but you can actually now subscribe and be able to start doing some of the stuff yourself. But the idea is now along with the actual physical document, you can start to question the document. We're also going to see what services some of them on. It's a very important I make this point. There's a bunch of resources listed on today's presentation for every one of those probably 5,000 more that are doing the exact same thing. I can only list a few, but one of these some of these services, let me go back to the presentation. So here we go into the list of resources, a service like Doc Alice's. This service, one of many allows you to upload one off documents or collections of documents, PDFs, Word docs and the like. And chatting with each of those documents as you see fit. Let me go back actually, here we go. So here is the relevance report from USC Hanneberg. How many are any people mentioned? So you can get down to this level of the document. So the idea is libraries and librarians can curate collections both for students, teach students how to do it for themselves. Teach faculty how to do it themselves. Quality of the data going in obviously has a lot to do with quality of the data going out. It's also important to remember, let me go back, there's nothing like typing in front of a group of a couple hundred people. What's important to know is that a lot of these tools like Doc Alice's are running on top of open AI. So the question then is, should our community be looking at building our own large language models, or should we be building custom versions of them that we can do more with? Again, these are long-term questions and ones obviously that take resources, costs, money and the like. And of course, because I want to show it to you live, it's not working right now. But again, all of you have this in front of you and I encourage you to take a look. So the curation of tools, the curation of ad hoc collections going forward for specific user groups. I also think that one of the things I've heard from some of the developers is that smaller is often better. So again, at the bottom of the page, I've got a few examples of specialty topical AIs or GPT tools. Everything from Bloomberg GPT, if you're a Bloomberg subscriber, to Describe AI. Last week, Walter's Clure came out with a GPT specifically for legal research. And also we've seen, and I know Ithaca SNR, I was doing research and doing a project in AIs from specific universities. And two that are listed here are from the University of Michigan and from the University of California, San Diego. Now, with that out of the way, I want to show you another thing that I think is going to be a big thing, a possible big thing moving forward. And that is agent or automated GPT. A tool like this, this is one example, will allow you to run a query in multiple steps. Do the research. Use R to create charts for this material. Produce a PDF all with one goal in mind. You can take it more simply. Another example would be plan a trip to Chicago. I want to fly on this date, list five hotels, go through all those steps, create a dossier if you like, and then go ahead and book all the reservations for you. That's the next up with agent GPT and similar services moving forward. We're also seeing AI tools built into web browsers. So for example, this browser by the way that I'm using called Arc has some GPT built into it. So for example, and if this works right, I can hover over any of these citations in this article and eventually there you go. Without even having to click, it will go out to the URL and create a quick multiple bullet point summary for me. So that's something else we're going to see more of. And of course that one isn't working. Let's try again. You get the idea. There we go. So again, I just hovered over the citation. It went to the citation, opened it up and used the large language model from Claude to create a three or four bullet point summary. So that's another thing that we will be seeing more of. So here is a list of multiple tools. I'm hoping some of you are likely some of you are already checking it out. One tool that some of you might enjoy is chatbot arena where you can take different language, large language models, run the same query runs once and compare the results to each from one result from one service to another right next to each other. But again, if there was one tool that does all the things that I'm here to talk about today better than others is perplexity. How many of you have actually played or used perplexity? Well, I'm talking a few of you, but I'm talking to an overwhelming majority of people who haven't heard about it. One perplexity. And by the way, I have no business affiliation as a consultant. As a matter of fact, to tell you the truth, earlier this year you might have remembered there was a Charleston in between conference that was focused on AI. And I helped co-organize that and they wouldn't even get back to me. So, but I'm still a huge fan. I pay for a few of these services and one of them is perplexity. So what you're seeing is actually the $20 a month perplexity version. It allows you, it provides citations. So let me just take one of these examples. So yesterday Epic Games won a lawsuit against Google. Watch what's going to happen. It's actually going to go out to the web, both a blessing and a curse, because this is all black box, what web pages are going to select from. I will tell you the perplexity has its own web crawler and its own large language model. It's incredibly fast. The other night I was watching the end of the Buffalo, Kansas City game. And within two minutes I said, what was the final score of the Buffalo, Kansas City game? And it actually pulled the citation from a website with football scores into the result. But as you can see, it's actually going to the open web and giving me book, as you can see down at the bottom where some of you hopefully can see, you're getting actual citations to the web pages where these assertions, these facts are coming from. I think that's a big step forward in something I'm very impressed by. But along with the open web is the corpus of material. Perplexity also offers specialty. Well, let me go back. The other nice thing for those of you who, as we all are learning about this, when you get a result from perplexity, you can click on rewrites, and it will rewrite the result using one of these four large language models. So if you want to compare how one large language model provides a result versus another one, it's all built right in here. This right now is using their own pre-built large language model. But if I want to see how this same result looks with Google, with a chat GPT or open AI GPT. So it's a great educational tool as well that allow you to compare models, and this all comes with that subscription fee. Another nice feature about perplexity is it allows you to search specialty databases at the same time or with individual searches. So right now I'm searching all across the entire internet, but I can click on academic. And let me go up to an example. I'll click, since I'm not good at typing as you have now seen, I will go back up to perplexity. And I will now just paste that in. And now instead of going to the open web, a.k.a. the Google web, if you like, this is going to focus on one specific database, that being Semantic Scholar. So now I can add that. I can also use a feature here that's really nice, and you get some of them for free more if you pay, called Copilot. They will ask the question multiple ways and allow you to get a more robust result. So let's see what happens here. It even asked me to further enhance my questions. It even asked me a couple of questions to get a better result. So I'm interested in economic and cultural and military. And now we'll rerun the question, ask it multiple ways. It went to 22 papers, and now here's the actual answer. So again, instead of searching just the open web, this is focused on academic papers using Semantic Scholar. Are all of you using Semantic Scholar, I hope? It is an incredible tool. It's a vital tool for the work that I do when I'm trying to find material for Day in Review for InfoDoc. Their alert feature is really, really impressive. I should also say, and let me see if I can show this to you right now, Semantic Scholar is getting into AI as well, GPT as well. So here is a paper from 2018. What I just want to show you is that embedded in the results link is an actual opportunity to chat with the paper. So this is putting the GPT at the point of use in the results. So without having to actually even click off of the results, you can start chatting with the paper here. What is the goal of this paper? And you get the idea. So this is putting the GPT in the actual search result. Another tool that I like, a little bit different, but using AI is called Scholarcy. Let me see if I can log in here. I was logged in. Here we go. And let me just give you an example. This is a paper that Kathleen Shear and colleagues from CORE published a few weeks ago, or a week ago. What this is doing is it doesn't allow you to chat, but it's using the AI to summarize the paper. And what they call flashcards. It gives the conclusion. It extracts key concepts from the result, from the paper. It organizes. So there's the key concepts, provides an abstract. And all of this is done on the fly by uploading the actual document to the service. In some cases, it will extract images, put them into a separate section. In other cases, it will extract tables and put them right into an Excel spreadsheet for you. So that's another example. This is not a GPT in the sense of being able to chat with the document, but I think it's a very worthwhile effort. And this one is Scholarcy is coming out of the UK. And before I ask for questions, have any of you seen Sight? This is one person. This company, Sight was a startup based out of Brooklyn, New York. In recent weeks, I think it was Monday after Thanksgiving. It was purchased by the company that owns reprints desk. So again, and this is similar to what I showed you with Symantec Scholar and Perplexity. This is a little bit for free, more for free. So the two services I'm actually going to be, I pay for, one is Perplexity and one is Sight. What I really want to show you about Sight, and again, the result looks the same. It's going out to all of these scholarly papers, however you want to define that, looking at all of them and then creating an actual result with citation to the underlying paper. But what I think is really cool about this is all of the flexibility you can use when you're asking a question of Sight. Do you want to use just the references? Do you want to use abstracts or just the citation statements, which is something that Sight is capturing? Do you only want to look at papers from this date to that date? And from what I've also been told, as recently as last Friday, you can now ask for all these advanced settings in your actual question to Perplexity. You don't actually even have to go to the assistance settings to finagle all these things. You can actually ask for them in your query, in your statement that you're trying to get a result on. So this is what a Perplexity result looks like, how many rats live in New York City. You see everything, you see here's the result. You can actually see all the publications consulted. You can see all the different ways it asks the question of their database. And again, this is another tool. The underlying large language model that this is running against is OpenAI. We've got about, I hope some of this was useful for the last 25 minutes. There's a lot there, there's a lot here I should say. I do hope you come and take a look, but I would welcome questions or comments or tools that I should be using in the remaining five minutes or so. Or are you all just ready for lunch or ready for one more session and then lunch? I see somebody over there, I don't know how you see, hello. I don't think, there we go. Thank you for your presentation. Thank you. You're welcome. One of the issues that I've had with site and perplexity and others that give you the citations that they're drawing from is that they don't know whether that paper that they're citing is the original source of the information or if that paper is citing another source. And I talked to the people at site about this and they said they were going to be implementing a feature that would allow you to tell, that would trace it back to the original source rather than just a citing paper. I didn't know if you'd come across other tools that have solved that problem. And I also wanted to mention my favorite tool is CySpace, S-C-I-S-P-A-C-E. I will add that to the page. Which is free as of now. Well, it's freemium model like most of them are. And is that using quote unquote scholarly academic papers or is that the open web? What is the corpus for that? A lot of them are. That's using a semantic scholar as the main corpus. Okay, so similar to what I showed with perplexity. Right. Okay. Thank you. No, that's great. I'm glad you were able to talk to Josh Nicholson at site. And yes, I mean, as I said, this presentation could be different tomorrow. It's constantly changing. And I guarantee you that within six months, hopefully Josh and now the new ownership will solve that issue of the original source of the paper. Any other questions or comments? I can hear you, but not through the microphone. Okay. There you go. Hello. Sorry. I also am a big fan of site. And I've seen Josh talk about, they're trying to incorporate data about retractions as well because they don't think Retraction Watch should have the monopoly on that information. Do you know anything a little bit? I did not ask that question to Josh when I spoke to him last week. I do know of another database, if you like, that has incorporated some Retraction data. I think at the point of use in terms of searching an academic database, and that being Lens.org, I believe Lens has now taken all that Retraction data, open data that they've made, the data they've made open of Retractions and incorporated into all of their results. And while Lens does not have a GPT yet, I wouldn't doubt if they have one in the future. How many people here have actually used Lens? Again, just a few. I think in terms of, in my years of doing this, in terms of an open web resource, both free and fee-based, Lens is one of the most powerful tools. I don't know if it's ready for, I don't know if it will ever be ready, and there's a need for this in the, for the traditional, for the average library user. But one of my favorite features about Lens is that it not only incorporates academic papers but emerges them up with patents as well. Two separate databases or use them together. One final question. Yeah, hi. Steve Ouida from Yale University. Question for you. You mentioned the importance of preserving conversation between AI and the user. Can you explain a little bit about how you are sort of on a dicey privacy consideration there, and how that might be balanced with the need for that sort of preservation? Well, again, if you want to go back in time and see how somebody formulated that answer, it's really important. Again, and by the way, as I said, I speak about data privacy all the time. It's a give and take. And I will say, though, that there is one tool that just was released last Friday that actually makes saving, just simply saving the question and response easier, and that being this new thing that came out from a little company you might have heard of called Google, called Google LM. Excuse me. Called Google Notebook LM. It's their new notebook tool that allows with one click to save both the query, the question, and the response in one place. And that, by the way, is another tool where you can upload a document and start chatting and summarizing with and the like. But yeah, I think it's the preserver. And the other question is, who is going to preserve, who's going to validate the preservation? Who's going to validate that it was actually what was said? I mean, again, I'm here to ask, for the most part, to show you a few things but ask a lot of questions. And are individuals going to, should they be included in the bibliography or the webliography? I don't know, but I think these are things we should be talking about. I'd like to thank you very much for your time and attention. I hope you're leaving about with one or two new things for you to go out and try. Thank you very much.