 Hi, and thanks for coming along to my talk. Today I'm going to talk to you about Citation Chaser, which is an R package and shiny app for forwards and backwards citation chasing. My name's Neil Hadaway, and if you want to follow me on Twitter or ask me a question, you can find me at Neil Hadaway, and you can also find me with the same name on GitHub. So before I get started, I just wanted to introduce the team. I've been joined in the team by Matt Granger and Charles Gray who've both helped with coding on the package, and Matt Granger in particular helped to design the network visualization functionality in a shiny app. I also wanted to give a special mention to lens.org who've helped to provide a token for long-term access of the lens.org database through its API. They've been incredibly supportive, so thanks to them. So what is citation chasing? Well, if we're thinking about how to retrieve articles for our evidence synthesis or meta-analysis, we, as well as searching bibliographic databases and gray literature for information, we might want to make use of the reference lists of a set of relevant articles that might also hold other relevant information for our review and also articles that cite a set of relevant records. And that's what citation chasing is. Backward citation chasing is looking through lists of references of our articles and forward citation chasing is looking at which articles cite our relevant articles to see if there's potentially more information that we might have missed to bring into our synthesis. Other names for citation chasing include citation searching, citation tracking, snowballing, poll growing, footnote chasing, reference scanning, reference checking, bibliographic checking, citation mining, reference harvesting, and many more. Some of these names specifically refer to a particular direction of citation chasing. Some of them are more general, but for now on we'll stick with citation chasing. So when we are going to perform citation chasing, we can start from a number of different points. For example, we could wait until we've screened our articles and use the final set of included studies to then perform forwards and backwards citation chasing. We might also along the way find a list of relevant reviews that we won't include in our synthesis, but we might want to scan their references and see who has cited them for other relevant records. And then we could also start out with a list of articles that we use to test our search and use that what's known as a benchmark list of articles. To perform citation chasing. Something else worth mentioning is how citation chasing is performed at the moment in some of the best gold standard systematic reviews. Unfortunately, it's not great. We had a look at reviews conducted by the, or published by the Collaboration for Environmental Evidence. And out of 16 published in the last couple of years, we found that 63% have performed backwards citation chasing, but none have performed forward citation chasing. And in 31% of cases, it wasn't clear which articles were used as a starting point. For reviews published by the Campbell Collaboration, a recent study showed that 88% did perform backwards citation chasing, but it didn't look at whether studies had performed forward citation chasing or which studies had reported which lists were used to perform the backwards citation chasing. In Cochrane reviews, 87% of similar number had performed backwards citation chasing and only 9% had performed forward citation chasing a little bit better than the others. And in only 1.5% of cases was the list of citation chasing records not clear. So you can see that a lot more could be done. And at the moment more isn't being done because citation chasing is very challenging. And those challenges relate to the fact that there aren't clear standards or best practices in how to do citation chasing. It's also often done by hand, for example, not using digitized lists of references and citations. And when it is done digitally, it's very time consuming because you need to go one study at a time to extract its references and citations. Added to that, individual tools that we have to hand like Scopus or Web of Science don't have a particularly high comprehensiveness when it comes to the total studies that are in the reference list and which studies have cited them. So we wanted to produce a tool called Citation Chaser to do this. Our objectives were that that tool should be easy to use. It should be open source and free. It should accept a variety of starting identifiers and it should allow people to refer into the tool from other tools like review management software. We wanted to allow for forward and backward citation chasing all in one place to make things easy and efficient. And we wanted to produce interoperable outputs that could be pumped back into the deduplication and screening process. In our case, RIS files. So we use the lens.org database as a data source for our tool. Lens.org is a meta database that consists of more than 245 million records. That was as of January this year. And it's an aggregator across different sources of bibliographic data. Microsoft academic graph was in there until recently when that was retired. Crossref, PubMed and PubMed Central and Core. And Lens.org are actively looking into solutions for replacing Microsoft academic graph like OpenAlex. But we made use of the Lens scholarly API which allows us to query the database automatically. For the rest of this presentation, I'm going to focus on the Shiny app which is the primary way that we see people engaging with citation chaser. There is underlying it, the R package which is available on GitHub and CRAN. And the Shiny app makes use of the powerful functions within that package. The users are encouraged to look at the R package if they want more detail. But we see most people, particularly people without a high degree of coding experience interacting with citation chaser through the Shiny app. And this is what you see when you arrive, detailed instructions on how to use the tool. And the tool is available at estech.shinyapps.io slash citation chaser. So we've seen two use cases, or we see two use cases for Shiny for citation chaser that I'll run through. Firstly, the user would directly put the identifiers into the tool themselves. And they could either do this by manually entering lists of comma-separated identifiers into their relevant box. You can see the six types of identifiers allowable there. And they can add in multiple identifiers at the same time. Or they could upload a CSV file that contains different identifiers in a two-column CSV with IDs in one column and the type of identifier in the other. And you can click on Help to find what that CSV file needs to be formatted like and see an example that you can download and edit. And in the beta version of citation chaser, which is available at citation chaser test by this URL, you can also upload an RIS file and the tool will manually strip out the DOIs that are present within that RIS and take those as starting points. The other use case is through referral. And what we mean by referral is that a developer could build a URL based on a set of identifiers that the user could click and it would take them directly to citation chaser. And you can see here in this URL that we have an indication that there is a query following the URL by the question mark. There's then the specification of which identifiers are following the equals sign and then a list of identifiers in a comma separated list. Different types of identifiers are separated with an ampersand followed by their new identifier type and then a second list. And so by clicking this, it takes the user to a pre-populated list of articles showing which identifiers they've put in. But whichever way the user starts searching for that or inputs that articles, this is the table that you'd see in citation chaser, listing your input articles that you can then if you want download as an RIS file. But it shows you how many references and citations are available for each record that you searched for. The next thing that the user can do is to search for backward citation chasing for references within that set. And we can see once we've clicked the blue button that we have a total of 136 references across those four articles. And once it's been de-duplicated, there are 132 unique articles across that set. And you can download that RIS file using the white button there. Similarly for citation chasing, you can click the blue button and you see that there were 582 citations of those articles. And 580 of those were unique and you can download those in a RIS file there as well. If you want to, you can also visualize the network. This can be quite time consuming if you start with a large number of articles. So it's worth downloading your reference and citation chasing results before moving on. But if you click visualize, you'll see your input articles as black dots surrounded by their references in red and their citations in blue. And you can see the connectivity, the connection between your starting articles. It's quite interesting, you can move around. In future, we want to develop this more to allow people to download the visualization. What you can do already is interact with this visualization and click on any of those circles and it will take you to that record in the lens.org database. So we see people using citation chaser in a number of ways. They can integrate the tool into their systematic review or evidence synthesis workflow by starting with a set of included articles, relevant reviews or a benchmark list. They can de-duplicate their reference and citation chasing results against their initial search results from searching great literature sources and bibliographic databases, for example. And any unique results that are left can be screened to find additional articles that could be useful, that are only found by citation chasing. There's additional functionality within the R package that might be useful for some people as well. The lens.org API outputs a really rich data frame that you can access within an additional output object called DF. And that holds a lot of information about authors, for example, but a whole suite of other information too. And then we have some future developments that we hope will make it even more user-friendly. We're hoping to develop co-citation analysis or weighting or filtering so that people can dive within their forwards and backwards citation chasing results to see which records were most frequent within their network. And already in the citation chaser test beta version, you can have a basic look at that frequency analysis that we'll be developing in the future. We also want to build a tool to allow people to deduplicate their citation chaser results against a larger set of, for example, bibliographic search results to show which are unique so that they don't need to screen again articles that they've already screened within their normal evidence synthesis workflow. And we also want to build in, and we are in the process of designing functions to allow people to search on titles. So when you upload an RIS file, if a record doesn't have a DOI, you'll also be able to search for titles. Although searching for titles is not particularly efficient because of very minor changes causing a problem for a match. But that's it. Thanks very much for your time. We hope you enjoy citation chaser. You can check out the shiny app at estech.shinyapps.io slash citation chaser. You can see citation chaser on GitHub as well. And you can find citation chaser newly added to CRAN if you want to use it there.