 and it looks like it is 12-01, so we'll go ahead and get started. I'm going to drop the slide deck in the chat for everybody, but I want to say welcome on behalf of university libraries to tools for systematic reviews. So we are in the third of our three-part series that we've been offering here through UA Libraries on systematic reviews. So our first session covered what systematic reviews are provided an introduction. Our second session last week covered search strategies and getting started in actually finding materials for your systematic review. Our final session for this semester is going to cover specific tools that you can use to aid in helping you with your systematic review process. I'm going to turn it over to my awesome colleagues to introduce themselves before we get started. Hey, I'm Alex Boucher. I'm a research instruction librarian here at Alabama. I am the library's liaison to a number of areas, specifically the College of Human Environmental Sciences, the Department of History, and Latin American, Caribbean, and Latinx Studies. Great. Hi, everyone. My name is Carly Reason, and I am the library liaison to quite a few of the social sciences. And hi, everybody. My name is Lance Simpson. I am the library liaison for the majority of the College of Engineering and the Department of Biological Sciences. And we're excited to have you all with us today. So just a couple of quick housekeeping things before we jump in. There's been a few more people to join us. I'm going to drop the link one more time in the chat for the slide deck, and I'll share that out as we go. And we'll be sure to send that out once the presentations are over today. So Carly, if you'd be willing to skip to the next slide, I want to talk really quickly about what we're using today. So many of you, we live in this Zoom virtual world. So you know a lot of these tools. But just in case you haven't had a chance to use some of those, we do want to encourage you to feel free to participate during today's session. So we'll each be covering a different tool as we talk through. But we want you to feel comfortable in asking questions as we go along. And there will be a time for questions afterwards. So if you would like to, you can either raise your virtual hand. And to do that, you can click that participants bottom down the bottom. If you're doing this from a laptop or a desktop computer, once you do that, you'll see a list of the fine folks that are attending today, as well as an option to raise your hand. If you were doing this from a mobile device, I have a screen capture here on the screen. You can click the screen and then you will have a more option that comes up on the right hand side. You can click that. And then these are the options that will come. So you can either put something in the chat if you've got a question or you can raise your virtual hand from there. There is a Zoom poll going. I think we're about to close that out in just a moment. So if you haven't had a chance, please take a moment to take that. You'll also notice in the top left-hand corner of the screen that blinking red light, we are recording today's session. So if you would not like your video capture to be on the recording, please feel free to turn your video off at this point. And we will post this later so that you'll have access to it and then we'll also have it on our library YouTube channel from there. So I'm going to turn it over to Carly to take us on into the session. All right, great. So an outline for today, we're going to start off to see if there's any questions. Last session, when we were talking about search strategies, there was just so much to cover that we didn't have enough time as we had wanted for questions. So we'll kind of start with that. And then we're going to get into talking about using citation managers as sources are selected for your reviews. We're going to go into more detail about citation manager tools like RefWorks and EndNote. And we're also going to be talking about appraising sources using tools like Rayon. And then we're going to have more time for questions as well at the end of today's session. So just getting started, are there any questions that any of you all had after attending last session that you were thinking about or wondering in terms of your search strategies or constructing keywords or control vocabulary or anything like that that you would like to talk about for the first couple of minutes of today's session? We can always get to those at the end as well as you're kind of settling into systematic reviews. And as things come up, please feel free to let us know. You can unmute your mic or you can ask your questions in the chat box. Well, we'll just get started then. So the first thing that we wanted to talk with you all about is using citation managers as you're selecting your sources. So citation managers, there are quite a few different options that you have. We're going to talk specifically today about RefWorks and EndNote. But there are other open source citation managers like Zotero. And I know quite a few people who like to use Mendeley as well. So a citation manager is going to save you a lot of time when conducting your systematic review. So citation managers will help manage all of the citations that you collect during your search when you're working on your systematic reviews. You may be coming across 1,000, thousands of articles, maybe less hundreds, even maybe less than that. But you're going to be gathering lots and lots of data. And so the citation manager is going to be really crucial to keeping all of that information organized and keeping track of everything that you have found in your searching. Citation managers are also going to help with organizing your information. So putting things in different folders or helping you organize as it is necessary or as it is helpful for your research and for your systematic reviews. They will also help with finding and identifying duplicate records. So as you're searching and as you're in different databases and pulling in more and more resources, the likelihood that you're going to have duplicate records could be pretty high. And so part of a systematic review and part of the documentation is going through and identifying what records are duplicates. And that will help with your, as I said, your documentation. So citation managers will help you with that process. And lastly, they will automatically format your in-text citations. So it's really helpful as you're going through to when you're working on publishing, they will help create that massive citation list at the very end. So it looks like the poll closed. So I'll share the results really quickly here before we go into that. So it looks like today we're talking about RefWorks and EndNote. So it looks like we all have a few people have used. Some people have some experience with these citation management tools. Others may not have as much. So I hope that going through, we're not going to be providing an in-depth tutorial of these tools. There are a lot of resources that you can access that will be helpful going on later, that we'll provide access to later. But we want to talk specifically about these tools as they will help with systematic reviews. And in terms of Rayon, we have one person who has used it and quite a few others who have not. So that's exciting. We'll be able to take a look at that in more depth. OK, so RefWorks. So RefWorks is one option that you have for a citation management tool. So this is a web-based tool, and it's free to UA faculty, staff, and students. I have a link here to create an account. You could go to the library's website and do a search or do a search on UA's website for a RefWorks, and it should take you to one of these two links. So the first link here is to create your account. And the image on the right is shows and is zoomed in on where you find that link to create your account. And you have a couple of options with creating an account. You can create your own username and password. You do have to use your UA email address. But you also have the option to log in from your institution. And that will sync with your MyBama login. So that eliminates one extra password of having to remember. So if that works for you, that's a helpful feature, I think. And then there's also a RefWorks research guide that just has a lot more in-depth information about RefWorks, about using it, and some other tutorials there as well. So what will RefWorks do and help you with your systematic reviews? So one, once you create your RefWorks account, you can create a project for your systematic review. And you can share that project with other people on your research team who also have a RefWorks account. So instead of each person on your team having their own RefWorks page or RefWorks project, you can share a project. And so as you are conducting your searches, you all can contribute to the data and the citations that are uploaded and exported to RefWorks. You can add references to RefWorks in many different ways. You can upload content all at once by uploading those RIS files or you can export resources from databases, from Google Scholar, from websites. You can upload resources from your desktop and so on. So there's a lot of different ways that you can add content to this tool. You can organize once you start to upload and add all your data. You can organize your references and folders. Depending on your research, you can use or your preference for research, you can use RefWorks in a variety of ways. You can just use it as a tool to keep track of all your data. But you can also use it as a tool to help document the number of resources that you're finding in each database when you're searching. And so one option that you have is to create a folder for each database that you searched in and you can upload those search results from that specific database to that folder. And you can begin to document and keep track of the number of citations that you're finding that way. You can also create subfolders. So it might be helpful to create a subfolder for duplicate records. So in order to document the number of duplicate records that you're finding in that specific database with that specific search. And reporting standards like Prisma, they do require the number of duplicate records to be documented. And so RefWorks is a tool that might help you with those numbers and keeping track of that information. As I said earlier, RefWorks also helps you to create complete reference lists. And you can export your citations to tools that will help with the more analysis of the article like Rayon, which Lance is gonna talk about later on. So as we mentioned in our second session, different databases have a lot of the same tools and features and functions, but you access them in a little bit of a different way. So this is an example of exporting a citation from an EBSCO database. And so what you see here is when you pull up the record of the article, if you look to the right, there's a list of tools that are tools to help you save and document your information. And there's an export link. And so if you click that link, another box will populate with different options of where do you wanna export this information to? And there's a tab that you can select for RefWorks. So you can begin to export content to RefWorks through the databases. There's ways and most all of our databases to export these references to RefWorks. It just will be in a different place. ProQuest databases you click share, there's a little dot dot dot symbol and you can select that and that allows you to share that information and export that to RefWorks as well. So it might take a little bit of searching when you're in the database to locate how exactly you export that data. But if you do have questions, you can always get in touch with us or your liaison librarian and we can help you locate where that information is and how to export all of your data to RefWorks or another tool of your choice. Another thing that will be really helpful with systematic review research is exporting multiple citations, even exporting your entire search results list. So here is another image from an EBSCO database and it's the entire search results page. And so you can select share and then at the bottom it says export results and you'll get an email with a file that you will then download to your computer which then you will upload to RefWorks. And on this next slide here, you can see once you are in your RefWorks account, if you wanted to upload all of that data, that entire search results list, you would select the add, the plus sign add and then select import references and that would allow you to upload all of your data in bulk. So when you're in RefWorks, once you've begun to start to upload all of the content that you wanna use, you can start to organize your resources within RefWorks. So you can create a project and if you can kind of see my mouse here, it says there's ProQuest RefWorks at the top and then on the right it says systematic review and then there's this little dropdown symbol and so if you don't have a project created, there'll be a link there that you can select to create your project. If you wanna share your project that dropdown link will also give you and walk you through the steps to share your entire project with RefWorks with your research team. Then you can see in that kind of list, all references is every single reference that you have uploaded to RefWorks and you can start to break down accessing information in different ways here. The sharing section, that's to share specific folders as opposed to sharing the entire project. For systematic review, I would say that I think sharing the project is more helpful because you have all of the data as opposed to just a couple of different folders of data. But within the folder section, you can see there's a underneath my folders, there's a plus sign and then add folder and you can begin to start to add folders as it makes sense for your research. This example is if you were to structure your resources by database and subfolders by duplicates. You can also update the metadata if necessary. So if you were to upload or look at your entire reference list, you might want to update some of the information or content that is part of that record. So for example, this first resource is highlighted and once you select the resource, you can click on any citation that's in your refworks account and once you select that, this new box will pop up on the right here and you can see there's a little edit symbol and you can select that symbol to then begin to edit all of the metadata within that. And so you can see these tags here. Those represent a lot of the subject headings and the controlled vocabulary that the database used to organize and to kind of describe the source. You can edit tags, you can delete tags, you can add tags. So you may, depending on what your research team decides, you may want to go in and edit tags that make sense for your research and you can add quite a bit of content here. Also, I want to highlight the notes section. You can add notes. So that might be another helpful thing for your research team to talk about when did you import all this data or other notes that might be helpful for your team. So you can go in and edit and update content if needed. I mentioned that one of the key benefits here of using some type of citation management tool is identifying duplicate records. So if that, once you're in that step in your systematic reprocess, you can highlight either all references or if you were looking specifically for duplicate records in a specific database and that's how you've organized your information, you can highlight where you want RefWorks to be searching for duplicate records is it in all references. And the example that we're looking at, we are searching for duplicate records in database number one. So that is highlighted. Then you would go up to tools and select find duplicates and from there it will identify the duplicates in that folder and you can then either organize those duplicates in a sub folder or arrange that content as you'd like. You can also create bibliographies. And so if we have all references highlighted here, our bibliography is gonna be created with every single reference that we have in our RefWorks project. So you would select whatever content you want to be included within your bibliography. So in this case all references, you would select create bibliography, select create bibliography again and then you'd be prompted to identify which citation style are you using and so on. And then you will get an entire list in the citation style of your choice of your work cited or reference list page. There are quite a few additional tools. I'm not gonna go into too much detail but you can save references from the web. So if you were to select the tools link and then select tools again, you would be able to download a browser extension that will allow you to capture and export information from the web into your RefWorks account. You can also connect RefWorks to your Word documents or Google Docs. So you can be working all in one place if you didn't wanna have your reference browser tab opened and you can also bulk edit all of your data. So those access to that, those additional tools are gonna be on the tools, on the tools link at that top ribbon. So that is pretty much just a quick overview of RefWorks. Okay, I am going to go ahead and talk about EndNote which is pretty similar to RefWorks in a lot of ways. It is another citation management program that you can use when working on any sort of research project but also specifically a systematic review. It does a lot of the similar things that RefWorks does. Most of all, I think these programs are good for for organizing your research and then managing citations, creating bibliographies, footnotes, et cetera. There are some differences though with EndNote compared to RefWorks. One of the quick differences is that there are two versions of EndNote basically. There is EndNote desktop, which is a software program that is sort of tied to your computer actually. And then there's EndNote out online, which is a cloud program. So it's more similar to the RefWorks basically. As far as accessing EndNote, the way that you can go at accessing EndNote is free to all university, faculty, staff and guests. You can access EndNote through the Office of Information Technology. It is, like I said, free either the online version or the desktop version. So desktop. This is the more robust program. This is the more robust version of EndNote. EndNote online is sort of limited as far as features and that is another difference compared to RefWorks because RefWorks is just one program so it is all basically the same. EndNote desktop is the one with the more robust features basically. You can have unlimited references. You can edit output styles and create smart groups, which is something we're gonna come back to in a little bit. It is not cloud-based as we talked about, but it can be synced to your EndNote online account. So you can kind of use the two of them in conjunction. EndNote online is free to anyone. EndNote desktop is only free to faculty, students and staff because we have a license for it. EndNote online is free to anyone. Like we said, it can be synced to your EndNote desktop platform. So if you are not actually at your computer when doing some research, you can still manage your references, manage your citations, and then sync the two accounts at any time. It does have more limited features, but another thing that's very helpful and a reason to have an EndNote online account, it is necessary for sharing your EndNote library. So what does EndNote do for a systematic review? Some of the things that we talked about, some of the things on this list are just things it does for any sort of research project. You can share your library with colleagues, which is a vital thing in a systematic review. It easily manages large amounts of references and like RefWorks, it is very easy to import your references into EndNote, straight from any sort of database. But then we get into the more specific things it does for systematic review. I have found in the past, the most important thing Carly touched on, to me at least the most important thing is the duplication. EndNote is very good for getting rid of duplicate references, which is something that pretty much anyone doing a systematic review is going to have to do. Some other things we'll talk about that are a little bit different than RefWorks are custom fields and smart groups. Now, custom fields, that kind of gets into batch editing the metadata references, which is not that different. Carly talked about that as well. But I'll talk a bit about how that ties into smart groups and smart groups are folders that have inclusion criteria, basically. We'll get into that, what that means and how that can be useful for a systematic review. Also, it is very easy to go straight from EndNote to Rayon, which Lance will touch on in a little bit. So as far as exporting references into EndNote, you rarely have to do a manual entry. You will rarely have to type in the title of an article, the journal, et cetera. You are pretty much always doing some sort of automatic export. This is just one example. This is PubMed, but basically you would just click on the send to, click citation manager, and you would download what's called an RAIS file and you can automatically import those references into your EndNote account. There are other options for importing or exporting articles into your EndNote account. You can add large amounts of references, large amounts of PDFs straight through the import button on your main EndNote library. Now, this is the desktop version right here. And this is also the X9 version, which is the newer one. It is not especially pretty. It is fairly bare bones as far as how functional it is, or at least not bare bones as far as how functional it is, but bare bones as far as the look of it. But you have your references here and your import is at the top of the screen. And we'll go into more examples of what's going on in this EndNote library, which is what it's called. So, organizing references within EndNote. Organizing references is one of the key, one of the kind of beneficial aspects of a citation manager. And in EndNote, it's a little bit different than RefWorks. The way that EndNote is organized is that the overarching organizational system are called libraries. You can have one library or you can have hundreds of libraries. I know some faculty members create a new library every time they start on a new article. Some people have just one EndNote library. You can do it either way, really. Within EndNote's libraries, you have groups, which are basically just folders. The only thing to point out about EndNote's libraries and whether or not you want to create one library or many libraries is that you can only sync one library at a time with your EndNote online account. So that is something to consider when doing this. If you were just going to have one EndNote library, then you really have to make sure that you use the groups function, which again, with groups, when I say groups, groups are basically just EndNote's versions of folders. You can create group sets. Like if you'll see on the screen a dissertation, systematic review, my sync groups, those are all group sets. And then within those, I have some other individual groups, which in that case almost work like subfolders. And you can of course just drag and drop articles over to your groups or import them in mass. So like we were talking about, one of the biggest things about this, one of the biggest things about using a citation manager for systematic review is removing duplicate references. You can do this in different ways. I don't know if Lance is gonna talk about it all, but you can remove duplicate references right in rayon. I find it easier to remove duplicate references in EndNote and then import or export your references from EndNote to rayon, but you can do it there. As far as how you can duplicate references or de-duplicate references in note, you can de-duplicate your entire library at once, even if that's 5,000 references, or you can create individual groups for each database. Now you probably have to still go within those individual groups and de-duplicate some there, but that would limit the amount of duplicate references you have. Now this on the slides is actually just a video. So when you have the PDF version of the slides, you can come back to this video and it will go into more depth about how to de-duplicate in EndNote. I am not going to go super in depth. I will just point out the quick basics of doing it, partially because it's not excessively hard to do so. Basically, if you wanna de-duplicate references, you can just go to the top of the screen, click on references, go down to find duplicates, and then go through the entire process. They will go through each reference. You will be sort of asked if you wanna actually de-duplicate it or not, but it will just go through and de-duplicate references as you go. It looks like we have something in the chat. Okay, so it's the slides. And we have another person that I'll admit. So yes, it's not especially difficult to de-duplicate your references in EndNote. It does take a little bit of time and of course you will have a good bit of reference. You will have a good bit of duplicate references because you'll be running in searches and different databases. And in the different databases, a lot of times we'll have overlapping articles. So it is a very necessary part of the process. And luckily it's not an especially hard part. Okay, so this, again, this is not a workshop, an entire workshop on EndNote. So I am not going to granular or in-depth on any one thing, but I will point out a couple of things that I think are also helpful on EndNote for System Act Reviews. And some of that is editing the metadata in custom fields of your references. I'll get into why I think it's helpful and important. Some of it Carly already touched on, but you can mass edit your references. In fact, you could edit every single reference in your entire EndNote library at one time if you wanted to. You just go to the tools section at the top of the screen, go to change, move copy fields. And at that point you can edit any field in the EndNote reference. Now, in this screenshot, I'm editing research notes, but I could edit journal, author, title. There's a section for rating your articles, which people do actually do the like inclusion and exclusion, some people do the entire System Act Review screening process in EndNote. We're gonna talk about how to use rayon for that, but you could assign ratings to your articles in EndNote. You could do basically the entire thing. In this case, I've just added something for research notes. So for example, if I import a entire search from web of science with one's particular search string, I can add this note to every single reference all at once. That can be helpful for a couple of reasons, just knowing where you found these articles. It can be helpful also for working with smart groups, which I'm going to get into in the next slide. So smart groups. I have talked about this quite a bit, but I think it is one of the more interesting and helpful features in EndNote. Smart groups are basically, again, groups or folders. So it's like a smart folder, basically. They are groups that automatically filter references based on inclusion criteria. So what that means is you could create a smart group with some sort of inclusion criteria, and then if you import hundreds of articles into your EndNote library, it will filter some of those articles into this group based on your inclusion criteria. So in this case, there are so many examples of things you could do with smart groups. In this case, I've done a couple of things. One, database PubMed. If you do that, it will just automatically create a group for all of your articles that came from PubMed, which can be helpful for deduplicating, can be helpful for keeping track of what you found, et cetera. I've also added a research note, ILL, which for me is just short for Interlibrary Loan. The reason why I might do that is sometimes it's sort of hard to keep track of the PDFs and the status of Interlibrary Loan with systematic reviews. You find all these articles, maybe you do a quick screening, you realize you need the PDF for them, but maybe you don't immediately have it because we don't provide immediate access to that specific article or journal, so you have to go through ILL to get the actual PDF. It can become a little bit confusing working with a group, like which articles did we get through ILL? Have we gotten the PDF yet? Has it been completed, et cetera? So you could create a research note on a bunch of articles that came in through Interlibrary Loan that was like Interlibrary Loan completed or something. In this case, with this smart group, it would just filter all of those articles into one group. Again, some people do the whole process in EndNote so you could create inclusion and exclusion fields and just put yes by the included article and it would immediately filter all those articles into one folder. My last example with smart groups is just this. This, for example, I've made one that's, I'm looking in the abstract and I'm just gonna look for a keyword. Basically what this will do is it will filter all of those articles into one folder. So where this could be helpful is maybe like with your scoping search, with your first searches of a database to see if there's enough out there, if you really have something important that you wanna find, like this is a, I don't know a huge amount about it because I'm not a diabetes expert but it's a diabetes clinical indicator. So if you imported 200 articles from PubMed in your quick search at the very beginning of your whole process, let's say that you really, it's very important to you that all these articles, the ones you really wanna find all talk about this clinical indicator. If you put this in, what this will do is when you import those 200 articles into your InNote library, it will automatically filter the 18 or whatever number it is that has that word in the abstract of the article. So it's a way of sort of machine screening articles to some extent. Now you could see a case where you could almost do more advanced screening of articles. I do think it's a little bit tough to trust it completely in the more formal screening of article stage but in the beginning stages, when you're just trying to see if there's enough articles out there to do a systematic review, it could be a way of kind of quickly screening articles not having to actually go in and read the entire abstract of every single article you find. It could kind of do it for you to some extent. And that is mainly what we have for InNote. We have a lot of other resources that you can go to. If you wanna learn more about InNote, we have an InNote library guide. There's InNote tutorials. At the very bottom, there's an InNote webinar that I did. So if you want to listen to me talk about InNote for 29 minutes, you could just click on that. So there's more information out there to help you if you do not have much experience with it. And as far as which one to use, InNote or RefWorks, they do a lot of similar things. I would just basically try one of them out, try both of them out and see whichever one seems to kind of fit with you and suit you basically. And with that, I think I will pass it off to Lance. All right, thanks, Alex. I'm gonna switch over and share my screen really quickly. Let's get that set up. All right. Okay, great. Let me skip ahead here. Sorry y'all, it was gonna jump me out. So we are gonna talk about Rayon, which I am super excited about. I know that it looked like there was only one person in the audience at the time that we did the poll that had used Rayon before. So I'll be excited to kind of share that out. And if our person in the audience, please feel free to jump in as you know more and wanna share things out. But I am very excited to share this with you. So now you've had a chance to think about the citation managers and gathering the data that you have. So you've gathered it all together and now you have to do the exciting part and that is actually appraising the articles that you have gathered and appraising the research that you've gathered. So that is where the tool Rayon is gonna come in. Rayon is something that was created by the Katari Computing Research Institute. It is a free tool that is available online. It is always in beta. It's something that has been continuously in beta for quite some time. So it's continually being developed. So there's new features that are being added to it and your stability that's being added to it. It is completely web-based, but there is a mobile application that's available for Android and Apple devices that you can download. And I've got an example on the screen of the mobile application and I'll show you more from the actual web-based application as well. I will say with the mobile application, you can pretty much only review articles. There's not a lot of the other features that you can do that I'm gonna talk about here in just a few moments. Some of the great things to know about Rayon. So one of the things that, or Alex and Carly both talked about being able to import and export your article records or your references from databases. So you can actually export those records from databases. You can export out as these different file types with. You can import into Rayon using RIS, comma-separated values, XML, big text, or big text, any of these that you wanna use, which you can do from the different databases. Each of them have different ways that they export out, but you can also do these from citation managers directly. And before we even go in-depth with Rayon, I wanna say that it is often, and I'll say this from experience, it is often easier. And I think Alex has mentioned this as well in his work with some of these. It is easier to import into a citation manager first and then export all of your references once they're ready to go and once you're ready to start your review of the articles from that citation manager into RefWorks. So you can out of, I'm sorry, into Rayon. So out of RefWorks or InNote, you can export a file that can be imported into Rayon for you to be able to use review. One of the great things about Rayon is that you can easily set up a blind review process. So let's say that you have a team of five to six people, you're all gonna be reviewing articles together. Whoever the person that is that will be the owner of the project will import everything in and then invite others to be collaborators. That person can then turn a blind on so that you all can make decisions on whether to include or exclude the article using that protocol that we talked about in our last session using the protocol that you've developed and not know what the other folks that are reviewing the article not know what decision they've made. So you can make that decision without adding a layer of bias into that. It's free to create an account. Again, it's pretty easy to invite collaborators on. So let's talk about some of the sorting and evaluating options that you have with Rayon. So with Rayon, and I've got a screen up here. So I have the upload references screen up and part of the reason that I included that is it has a set of guides for you to be able to import from different places. So if you do want to export out of InNote to be able to import into Rayon or upload into Rayon, there's a download example there, a guide that you can use. If you wanna do it straight from RefWorks, there's one there. If BidTex is something that you use regularly, you can do that as well. I do a lot of work with CSVs and also from PubMed. So there's different ways and guides that are on this page that can set you up to be able to do that. Things that are great about kind of getting stuff set up with Rayon is you can add inclusion, exclusion criteria as you are going through your articles. You can include PDFs with the references. So once you have all the references in, you can add the PDFs, double check the chat. Oh yeah, we can definitely get those sent out. So you can include the PDFs like you can upload those into Rayon so that you have them readily available. You can also export out. And I'll talk a little further about that later, but there's lots of different ways that once you have done your review or kind of walked through your data, there's lots of different ways that you can set it up so that you can export out based on how you've sorted your data out. So like you can sort the articles that you've chosen, there may be additional data that you gathered while you were there that you can export specifically out. And you can have that set up by tags and reasons as well. So I'm gonna skip to the next so we can talk about what that looks like. So with Rayon, a nice setup, and one of the things that I really like or a feature that I use a lot is you can set up your keywords for include and your keywords for excluding. So as you have developed your protocol, you and your team have worked, you've developed the protocol, you've decided how it is that you want to go about selecting articles, you will have a set of terms that you're looking for, a set of conditions that you're looking for for inclusion. You'll also have a criteria set that are definite exclusion criteria. If there are keywords that you have that are associated with those, you can add those into Rayon and it will look through every reference that you have listed and it will mark them accordingly. So every time it sees that keyword, it will mark it. So for example, I have the keywords for include listed here. I've got an example review on computational thinking and education in computer science and engineering. So I wanted to make sure that any article that had the phrase computational thinking that that showed up and I could easily see that. I have a couple of examples, some of these come pre-loaded into Rayon and I would normally take a lot of these out but I wanted to show you what they looked like. So the exclude criteria show up in a different color. Now we'll say I'm not a fan of the fact that they have used red and green for these because these do not work well with color blindness but the setup is currently the keywords for include are listed in green and the keywords for exclude are highlighted in red. Now what this does, let's say that you have five or 600 articles that you're reviewing and maybe you've divided it out and each of you have 100 or 200 articles that you're gonna be reading over. This gives you a chance to more quickly be able to assess whether that article actually meets your inclusion or exclusion criteria. So and you can see on the screen too and I'll point this out in our next setup. You can see I've got sort of a list here of what it looks like when you're in Rayon. So if you look on the left-hand side of the screen, you can see how many articles up at the top, the inclusion decisions. So you can see how many articles have had decisions made on them that you've made. You can see how many maybes that you add, how many included, how many excluded. You can see where the references came from. So that uploaded references there under search methods, you can add each one of those. And one of the reasons why I recommend coming out of a citation manager with all of your results in one go is because different databases as you're gonna be searching several different databases format things differently. So if you take them all into a citation manager to begin with when you import them into refworks, they will all be formatted in the same way. So then you've got the list of the inclusion criteria that I've added, the exclusion criteria that I've added. Just underneath that, there's a word cloud that it'll develop based on topics. Topics, this is not a foolproof setup. So topics is a setup that looks at keywords that were associated with any references that were there. And that's gonna be listed differently depending on which databases you've come out of. But you can also sort out by year if you want to, you can look at if there's multiple authors or the same author for multiple articles, you'll see that show up as well. So I wanna show you a couple of things just to kind of point out on this screen that I've labeled out here. One, so this is, you saw before any reviews had been made. So I made a couple of reviews in here. So we could see I made a few decisions on some of the articles, whether or not to include them. So as a part of my protocol for this example review, I decided that I did not wanna include systematic reviews. So this particular article that is displayed, I chose not to include it. So I clicked exclude, and once you click exclude or include your name or maybe, so your name gets added to that, you can see once you unblind the review, everyone who has made a decision on an article, their name will be listed there and what their decision was. It will list out with white, with a red text. So there is some contrast there. White with a red text if you add a reason. So a reason's gonna be different than a label. A reason's gonna be an actual exclusion reason. So if you have an exclusion reason, and rayon comes with some pre-loaded, but you can add your own as you go along. So it will add that on. Now, if you look below that, I've got another article there that I have selected. So those that were, it's got my name and in green there, those are articles that were selected for include. And with each one of those, you can add labels on. And I'll tell you what the importance of labels is. So this is something that you can also do in RefWorks and you can do in EndNote. But as you, if you are the only person reviewing to begin with, if you want to have other folks whenever they are reviewing, if you want to use this data set, you wanna think, you're gonna spend a lot of time with these articles and reviewing these articles. So if there are multiple tracks that you might could take with this, this is a good time to mark these. So you may add labels on as you go. So that whenever you export this back out, you can look and see, like maybe there were a lot of articles that you could not use, but there was a specific label you added on that you may be able to use them for another project that may help with other parts of your research. So those labels, I have them marked here. They show up in blue with white writing whenever they're added on. And you can't see what anyone else has added until the review or until the blind has been lifted on the review. So once the blind has been lifted on review, all of these labels get merged together. You can see all the labels, all the exclusion reasons all get added together on an article. And you can see which person reviewed which article and what their decision was on that article. So exporting out of rayon. And this is where I have a lot of fun with this and enjoy doing this part. So there's lots of different ways that you can work with the data that you've gathered. So let's say that you've done your review, you and your team, you developed your protocol, you invited everybody, you uploaded all of your articles, you invited everybody, you've done your review now. You've all made your decisions, you've resolved any issues that came forward. So now you want to use the data that you've gathered. So you know what your list of articles is. So if you wanna go back and add this back in in note so you can actually review those articles from in note, you can export this back out. You can add so any of the filters that you use that are on the left-hand side of the screen or we're on the left-hand side of the screen in the previous slides here. So any of these filters that you've added in, you can have those selected and export based on those filters. So if you want just the articles that were accepted for inclusion, you can export that. If you want just the articles that were not accepted or you want just the articles that are based on a certain criteria, you can export them filtered that way. You can also make sure that the abstracts are included, any decisions that are there, any labels that were added in. So whenever you're going forward, you'll have those and exclusion reasons as well. And you can decide you can have that either as the bibtex, as an in-note file, which is gonna be listed as RS, you know, the ref main or you can have it as a comma separated values file. And Alex mentioned earlier, so Alex and Carly both talked about deduplication in the citation managers they were using. So deduplication, I'm gonna jump back a screen here. At the top where it says inclusion decisions, if you do have multiples or you do have duplicates of your article, and this is something as you're searching databases, especially when you get out, I call them the corner scraping databases, if you run back through Web of Science, you run back through Scopus, you may see some overlap or if you're using PubMed and Synol, you may end up seeing some overlap where you have some duplicate articles. So that deduplication, you can do it in Rayon as well. And Rayon will actually walk you through and ask you to review each one to decide whether or not it is actually a duplicate or not. And it will give you a percentage based on how much of the article was duplicated or how much of the record was duplicated in between the two to decide if this is an actual duplicate or not. So last but not least, kind of thinking about what you're gonna do with your data as you go forward, you can export this however you want to, you can use whatever filters you want to to be able to see. It's one of the things that I really like about Rayon is being able to kind of move things back and forth between the citation manager and Rayon. I do wanna show you what the screen looks like as you're kind of working through a review. So let's say that you are the person that is the lead on this review and you're watching to see how the progress is going. I have a list here of what I did in this review and just the beginning, the setup. So you can see each of the users that is involved in the study. You can see how much progress they've made on the decisions that they're working on. You can see how many minutes that they have been in the review. You can see how many sessions that they've been in. You can see what the percentage of included and excluded is total for the articles as you go along. And if you have any discrepancies, it won't show you what the discrepancies are until you pull the blind off, but it will show you that there are discrepancies. So you'll know that that's something that you'll need to resolve to decide how it is that you wanna resolve discrepancies. And that's a part that you would build into your protocol as well. But I'm gonna jump back really quickly because I wanna show you the, just show you this screen one last time before we open it up for questions. So Rayon is, it's a great tool to use. I will say, so it is a free tool and it is in beta. So there are occasional hiccups and issues that I've run across. And mostly that's gonna be, it might pause for just a moment as it's uploading things, no data loss or anything like that. There is a mobile setup, but it is not quite as sophisticated as the browser setup that you would use on a desktop or a laptop computer. So the mobile setup is gonna give you the option if you're kind of quickly running through making decisions on articles, not necessarily trying to sort in and add data to those. But you can easily come from RefWorks or InNote, take that data, export it out of those into one of the file types, import it back into Rayon or upload it into Rayon to be able to use. And then once you've completed your project, you can export everything that you've worked on back out to be able to start your review of the literature once it's set. So I am going to jump. So we do have a list of references here that you can follow up. So it's a different, there's a couple of books. So there's one, the assembling of pieces of a systematic review of God for Librarians. We've got that set up that's something that is on a book through UA Libraries that is digital that you can download easily. And I would definitely recommend checking that out. The Cochran Handbook for Systematic Reviews of Interventions is on there as well. I'm a fantastic guide that Carly's put together for systematic reviews that includes a lot of our resources. And then I've got links here they're also at the front part of the slides there but I've got links for each of the citation managers and for Ray on here that you can click on to be able to follow up to either download those or create an account. So from here, we've got time for questions and I'm gonna open it up from there. So what questions are out there? Just as a quick note, I put both the links to the YouTube links to both of the webinars in the chat. Where can we go to find these tutorials on EndNote? There's some, I can link them here. There's some Lance in the PowerPoint. I think if we went back, there's a slide for EndNote resources. So we actually don't, on the library's website we don't have a ton of tutorials on EndNote. What I have linked in here are a couple of things. One, EndNote has some of its own tutorials and they have a lot of things. They have sort of short courses and they have bite-sized tutorials talking about just one little part of EndNote and then they have the short courses to kind of talk about the overall thing. We also have a webinar that I did based, I think like last April, like not long after COVID hit. EndNote webinar that's about 30 minutes long that is basically just kind of goes through from the basics to more advanced stuff. So we have links on these slides and the EndNote resources page on these slides for both of those. But we don't have a ton of EndNote tutorials partly because EndNote's tutorials are pretty good. So we haven't always felt the need to do a ton of our own tutorials. So there are links to both of those on here on the slides. There's a question about RefWorks. Carly, do you remember that? I'm pretty sure you can. Can you share with collaborators outside of UA? Do they have to have EndNote a RefWorks account? Yeah, you can share with anyone who has a RefWorks account. So it doesn't have to be specific to UA but as long as they have a RefWorks account, you should be able to share projects with them, share folders with them and so on. Any other questions out there? All right, everybody. If there's not any more questions, then I wanna make sure you've got our contact information. So we've got the videos that were dropped in the chat and you got access to the slide deck and we'll be sending out the video for this session today as well as the slide decks. You'll have access to that but you can reach Carly, Alex and I at our email addresses here and I've got those linked in the slide deck as well. Please feel free to reach out to us. We'll be more than glad to help you. However you need us. And I always like to say we're here for you. Our whole job is to be able to work and help you as you're getting started with your research process as you have questions about any of the tools, any of the resources that we have, please, please, please feel free to contact us. We'll be glad to get you set up. And next semester is coming. More workshops are on the horizon so keep a watch out. There will be some news stories and articles that will be coming up that we'll send out and let you know when the next setup for workshops for the spring is gonna be and for fall next year as well. But I wanna thank everybody for coming. Thanks for showing up today and for coming. If you were able to come into any of the other workshops, please feel free to check them out. Alex has dropped some great links in the chat here for endo tutorials and endo webinars as well as our previous sessions. And otherwise, I hope that you all have a wonderful rest of your day and a great rest of your week. And we'll see you soon. Yeah, thanks for attending. Thanks everyone. I guess that is it for the systematic review workshop this semester. What? Did you all want to do a meet in another room for men? Or do you? Yeah. Yeah, if that works, I'll send the link out. Okay, cool. All right, I'll see y'all then.