 Welcome to CallRef, a pipeline for collaborative and git-based literature reviews. I'm Gerrit Wagner and I'm Julian Bressler. Today we're excited to present CallRef. CallRef is a standard for data and collaboration management that supports the entire literature review process. In literature reviews, we still have to waste a lot of time with data wrangling, operating complex interfaces, dealing with unrevealed reference managers, and converting proprietary data formats back and forth. We're not prepared to complete the process efficiently with updates and multiple search and review iterations. Our inspiration to address this challenge are the tidy rows and statistical data analysis packages. In these cases, we've seen the benefits of shared data structures which allow users to select and combine different packages with ease. Our work also builds on git with its transparency, reproducibility, and scalability built in. With Git as a basis, we could really scale collaboration within review teams and beyond by involving the broader research community. Git also makes it much easier to undo changes or test different extensions. We believe that the design of data structures really matters in order to create an ecosystem of review tools and forget we need to think beyond tabulated data structures. These data structures would differ from conventional ones in reproducible research. For literature reviews, we need to regularly feed updates and new records into the process, and the process itself combines manual and automated steps. That's very different from the typical reproducible research analysis where we really manipulate the raw data, and all the changes are purely computational, but simply dumping everything into Git version control would create a huge mess. If we combine shared data structures, standardized operations, and a Git workflow, everything becomes much more efficient and we can cycle through the review process with ease. We believe that just proposing a data structure and management system would not be enough. To show the benefits and to get this idea to fly, Khorov is now available as a first prototype and guaranteed to start walking you through the live demo that covers the entire process end to end. So welcome to the demo. This is the three-step workflow of Khorov, and all you need to know is Khorov status. This command shows you the current state of the project and also tells you what the next operations are. So you run Khorov status, then run the next operation, then you can also validate the changes. So you basically repeat these cycles. This will take you through the whole review project. From initializing the project to retrieving the metadata and PDFs, screening activities, as well as different forms of data analysis and synthesis. So Khorov status, which now tells us that we should initialize the Khorov repository. You can take any review type from that list. Let's just go for a scoping review as an example, and you can read up on the details in the documentation. So now everything's set up. You see the different directories and files were created. So I found Khorov init. The status now tells us that there are zero records in the process, and the next operation is Khorov retrieve. And this tells us how to add search results. So you could simply copy files on BIP, RIS, XLSX to the data search directory. We could provide PDF documents, or you could run an API space in search. So let's try that. So here we are in the documentation where things described. You see the different options for API searches, and we just go for the example. So I run it, and the example is the crossref database to additional databases from our discipline, and we search for our microsourcing as the keyword. So we run these commands, and it fetches all the results. And then we run Khorov status, where we see 33 records in the process. We should run Khorov retrieve again. Schnautrock tells us that retrieve is a high-level operation consisting of the search, load, prep, and to be. So let's run that, and I'll explain to you what happens in the background. So the search results are stored in the data search directory. The settings are in the settings, Jason. That's where the search parameters or the API calls are stored. Now the load operation brings everything into the same Biptech format, and it adds a few fields. For example, the Khorov status, which keeps track of the state of each record throughout the process. This could be MD imported, and it changes to MD prepared, perhaps pre-screen excluded, or synthesized in the end. Now the Khorov origin points to the original record in the search source. So it's important to keep track of that. Now the prep operation also uses province data, describing where each field comes from, and whether there are any quality defect. For example, the title comes from this source, and it has a quality defect because everything's in capital letters. Now the prep operation resolves all of those quality defects based on high quality datasets like Khorov curations or crossref. In this case, you see that everything's fixed, no remaining quality defects, and it's even connected to a Khorov curation. So coming back to our example, we see that everything is completed. We repeated the search with no additional records that were retrieved. The load was completed for the different search sources. In the preparation, we see several records that are quality curated, as well as two records that are excluded automatically. In the pre-screen, there are a couple of duplicates that were identified and merged. So the next step, according to Khorov's status, is pre-screen, where we provide a short explanation at the beginning, and then we check each paper whether it's relevant to our objectives. So in that case, it's microsourcing, it says relevant. Same here. So in the end, we get a short overview of our coding, and the next step is to run Khorov PDFs. So that retrieves the PDFs from our local hard drive from other projects, as well as online from the Open Access PDF collection of Unpaywall. So now there are a couple of PDFs that would need to be retrieved manually, but I'd like to continue with the process. We just skip the screen, which is pretty similar to the pre-screen, so we include all in that step. And now in the Khorov status, it suggests the immediate next operation, the manual retrieval, but we can also go for the verbose mode, which gives us more options. So here, we see more operations and additional information on versioning and collaboration. We see that in the data operation section, there's Prisma, Flowchart, an obsidian vault for data analysis, as well as a manuscript that we can find here. There's information on how to build the manuscript and how to create versions. So let's just have a look at the output. You can see the verb document with the Prisma, Flowchart, all the details, the figures added based on the pipeline. There's a to-do list of the relevant records that would need to be synthesized and the full reference section with all the details. So basically, we've completed a whole literature review in just five minutes. And we can also go online to see the example repository. So you can use Git to collaborate in small teams in private or also in larger teams in public projects. So there's no limitation here. You also see that there's full transparency of the changes. You have one commit for each operation that we completed. You can go into the individual commits, have a short record at the beginning and also see the detailed changes that were applied throughout the project. So now I'm sure you're all wondering, how can I try this and how can I get involved? But there's a variety of ways in which you can get involved and they somewhat also reflect our vision for how Coloreff will develop going forward. By the way, all the different links here on the slide will be available in the video description. First of all, Coloreff is live on GitHub and you can go today and try it out. Use it for your literature review projects and let us know what you think. The more and more people will use Coloreff, the more we will be able to refine the best practices and for example, tailor to the nuances of particular review types that you're working with. As you've seen in the demo, Coloreff is built with extensibility in mind from the start. So we are looking forward to many, many contributions and extensions that integrate Coloreff with your most favorite review tools. We have only briefly touched on the possibility of curated repositories in the demo, but we envision community level curations to play an important role for the reuse of review projects and you can read more about what we mean by that following the link to our Coloreff curations. And so ultimately, we hope that Coloreff will become the data management standard for literary use. But to make that happen, we need you. So please download Coloreff today and start using it. We are looking forward to your feedback.