 So what are the steps in the project? Here I'm borrowing a graph from Patricia Ayala. Patricia is one of our Gerstein specialists. So please don't forget that we have a huge team at Gerstein. What we see here is a topic selection and conceptualization creation of protocol are kind of, there's a lot of iteration between those steps, right? And we see that there's a lot of preliminary researching happening there. So preliminary searching is an important concept. You're not just going to be searching on the green step there, searching for studies. No, you'll be doing a lot of searching ahead of your main search to scope out the territory before you do the scoping review. And that's so that you decide and realize what has already been done, right? So here I'm comparing the steps that Patricia listed out to CAHR steps. This is something that you can take a look at later as well. And here's another great resource, Oliver and Thomas, the systematic review for policy decisions quick guide which is available freely as well. And that one's listed as one of the resources. So we're really looking at the first three steps today. And here are some more steps that are described in a bit more detail, okay? So the big thing that you need to do even before you jump into doing anything with protocols and you'll see how this actually relates to your protocols is establishing the need and what has been done. This is so hugely important. You need to search for other systematic reviews, knowledge, synthesis, scoping reviews on the topic that corresponds to your proposed research question so that you know what's been done and you don't reproduce or replicate needlessly what's been done or that you build on what has already been done and reuse some protocols. So you can be more efficient and you can be more productive while extending knowledge, right? So we have a number of these databases that include knowledge, synthesis. So from the applied health and this would also apply to our biomedical engineers. We have the Cochrane Library Epistemonicus which is a South American grassroots initiative, fantastic resource. The JBI from Australia, this one's found through the nursing databases, fantastic. And the TRIP, the TRIP Pro is a fantastic open source medical search engine and it provides evidence that is classed according to the pyramid I showed you earlier, right? So these are really good places to look at what's already been published. So let's take a look at the TRIP demo. So let's look for, for example, something that's really hot in the news and not necessarily in a good way, ivermectin and COVID-19. So ivermectin as a treatment or a prophylactic for COVID-19, just going to type in COVID, that's going to pop out what we need. Ah, look at this. So last time I did the search was a couple of months ago. I think there were 20 something studies. So now we've got 49 results and look at what TRIP does. TRIP immediately classifies these resources into systematic reviews, synopsis, guidelines. And we see that there is, and they're actually two from Australia. Wow, guidance, regulatory guidance, there's nothing there. Primary research, there are only four. While we look at all the evidence-based synthesis and the reviews in comparison to primary research, I actually know of more than four. So it's interesting that they're not indexing all of them, but that's an interesting issue. Okay, yeah. So there's more primary research here. They're at least 10, as far as I know. So seven is good, plus four, yeah. So they've got at least 10, 11 of those. But look at this. They're ongoing systematic reviews. They're 22 ongoing systematic reviews. That's fantastic. So if you're interested in this particular field, you know that it's hot and you know that there'll be at least 22 more systematic reviews coming out. So it's in your best interest at that point to find out what aspect of this field are being studied by those other teams and try not to duplicate the effort, right? So this is really important. Excellent. So they're mostly 2021. There weren't many in 2020. There's one here, but that one was, yeah, that's an evidence-based synopsis as opposed to a systematic review. So you can always jump just to the specific kind of evidence that you're interested in, right? Let's be slow. Okay. There you go. It's showing our primary research right now. So this is one way that you would do your preliminary search. Your preliminary search is really important because it gets you a sense of what is happening in the field and what others are doing. And you can do that in a variety of disciplines, right? So I showed you the four basic healthcare and applied health sciences databases, but we also have Campbell. Campbell collaboration looks at social welfare, social justice, crime, education. So it has a very broad base. We have the epicenter, which I mentioned earlier, which looks at a lot of the historical issues having to do with the rise of systematic reviews. And that looks also at political decision-making. So this is how you would get a sense of what has happened already, right? And there are a few others, I'm not going to go through all of them and there are disciplines like certain kinds of engineering that are not very well supported. So what would you do in that case if you're in one of those fields? Of course, what you would do is you'd go into one of the databases that you're going to use primarily for searching that we're going to be discussing tomorrow. For example, GeoBase or InSpec or the IEEE Explorer, any of those databases and do a quick search for some of your keywords. Does this search have to be as detailed as the search that you'll be running in your knowledge synthesis? Not at all. But it needs to foreground any work that's going on. That's very important. So now we will be looking at the step two, which is working on your research question. And this is where you ask yourself, what will you review and accomplish and what are your underlying assumptions? Okay, so everybody's probably heard of Pico and Pico is a question framework that comes from Dentistry, actually. This is the original article from which Pico comes. And it's part of evidence-based decision-making. So asking good questions. I'm just going to go straight into offering you an example with Pico and we'll work through it quickly. If you'd like to play around with Pico and other framework, I have fillable Pico organizers and other organizers that are available to you. So if you're interested in these, please send me an email after today's session and I will send you the information. So let's take a quick look at an example. Let's say this is a research question that we have. Are there all facial injury patterns that have been used or could be used as markers of future violence in the context of family violence homicide? So this is a forensic science question and something that I've worked with one of my faculty. So when you're using Pico, you're looking at the population. So how would you describe who you're studying? The issue you're studying or the intervention. So what are the treatment exposures or what can you say about the issue that you are studying for your cohort? Often in healthcare, you've got a comparison or control. So what are the alternatives to what you're doing to the intervention? And then you have outcomes. Often there is also a T or S term in Pico. So it's a Pico or Pico. And that can look at duration type of study or how often you'll be redoing your study. So that's another consideration. So here for this particular question, when we break it down, the population is easy. Victims of family violence homicides with or facial injuries. So unfortunately they're no longer with us, but we have a control. We have a comparison that's not obvious and that would be victims of family violence homicide without or facial injuries or accident victims or other homicide victims that are not coming out of the family violence pool with oral facial injuries. Because we want to compare the kinds of injuries sustained. And of course here we're looking at those who are deceased. Now the issue is assessing whether patterns of facial injuries that are observed on family violence victims can be used as any sort of a prognostication or diagnostics or advanced warning for family violence prior to homicide. So again, we want something among the lines of early warning and it's not easy to take these questions apart to fit them into PICO, right? Now PICO is great for healthcare applied healthcare. It could be great for biomedical engineering, but in education it might not be the best. So for your context, there are many others. And this part I'm going to skip through very quickly because I want to spend a few minutes on protocols. So if you're interested in these slides, you'll be able to see eclipse. You'll be able to see how it's applied to a very quick question here. You'll be able to see spider, which looks at phenomena of interest from the point of view of design of interventions. And it's really good with qualitative studies that spider. You'll be looking at SEMO. SEMO is great and a lot of psychologists, for example, love SEMO because they look at mechanisms that explain relationships between the interventions and the outcome, right? So SEMO is very interested in the mechanism. And sometimes when you don't have the interventions, the PCC is great. That comes from the Joanna Bridges Institute where you look at what's the population, what's the concept you're studying, and what's the context, right? So PCC is short and sweet, could work for you. Biner is great for almost any research question because it wants your research question to be feasible, interesting, novel. And that's where of course looking at prior work is important, ethical and relevant. And there are so many more. So I'll quickly flip through these slides because they have examples. And this is something that previous attendees really wanted to see. They wanted to see more organizers and more examples. So almost every page has an example. And here I have two real research questions that I've been working with. One is in industrial engineering and one is in psychology. And I show how I fit these into SPICE, SEMO, Eclipse and SEMO just to get you a sense of how they can be used for refining your research question. Okay, so we've got about six minutes and now what we're going to do is jump into protocols, right? So we've outlined why we need protocols because protocols help us to keep bias in check and to set out the steps of our process but they also help us to refine our research question. And once we have a protocol, more than half of the work is actually done in terms of conceptual work, right? So let's take a look at what happens with the protocol. We have a lot of help and guidance with writing protocols. There are three sets of resources. There are Prisma resources that help us with protocols for systematic reviews, like analysis and scoping reviews. There are Joanna Briggs Institute's protocols and there are actually a few others as well but I would say the Prisma and the JBIs are the primary protocols that outline step by step. I said three and then I realized that, yeah, some of the others also list protocols but very often the protocols that you'll find in let's say OSF or epistemology because are derived from either JBI or the Prisma protocols, right? Cochrane has a protocol as well, so that's really the third one but Prisma is one of the very first that you need to go to. So what is a protocol? A systematic review protocol describes your rationale, your hypothesis and the planned methods of the review but along their way as you fill out the protocol you answer a ton of questions for yourself. So that goes back to our initial question. Do I have to talk about how I decided which kind of study to make? Yes, you do and that comes in the protocol, okay? So here's the entire Prisma 2020 protocol checklist. This is the brief set of items, only 15 items for a systematic review, okay? And I'll show you the detailed one in a moment. So that's the detailed Prisma 2020 checklist that gives you a lot of elaboration. And here I've outlined two areas that we need to look at under rationale, objectives, eligibility, pardon me, criteria. Under the rationale, the protocol tells you exactly why we do pre-searches. So why we go into databases like Trip to see what has been published because the protocol asks you to describe the current state of knowledge and the uncertainties and articulate why you need to do the review. So by doing the protocol, you save yourself more. And of course here and their objectives and eligibility we have why we do PICO. We do PICO because it helps us to determine the objectives and the criteria, the inclusion, exclusion criteria and the eligibility criteria. So PICO or all the other organizers are really great at foregrounding those parts of our upcoming research. And here's a sample scoping review protocol. This is the Prisma SCR checklist. This was published in 2018. So again, it's very recent. And you'll see Andrew at Trico's name right there. So what do we do here as well? The rationale, the objectives, the eligibility criteria show us again that whether we do a systematic or scoping review we need to do pre-searches. We need to interrogate our question. And again, when we do eligibility we don't do convenient samples. We think about everything, including how far back do we want our searching to go? So do we stop at a certain year? Do we assess the entire literature? Because that's really important, okay? So we're basically at the end of our time and we've covered the three main things. First of all, definitions and typologies of knowledge synthesis. Then we've covered tools that you have to help you ask better questions. We've also covered places to go to do your pre-search so that you can fill out your protocol. And you see how every step that we've done so far feeds into a better protocol. So the rest of this just shows you more protocol examples and places where you would get protocol checklists and more support. And this is an example of Prospero where you can find protocols for systematic, not scoping reviews but systematic reviews and scoping review protocols. As examples, you can find an OSF or in fig share. So next time we're going to talk about search, that's tomorrow. And we do have a number of others who support knowledge synthesis, specifically Gerstein, but also Oisy and Roberts. And there are colleagues at UTSC who do it as well. Now, please complete the feedback form that I have included in chat. Thank you so much for any feedback you can provide me. And please feel free to join the session tomorrow.