 Welcome, everybody. There we go. It started recording. Welcome, everyone, to Esmar Conf 2022. And this is our workshop on searching for studies in meta-analysis and evidence synthesis. This workshop is being livestreamed to YouTube and has a group of participants in the Zoom session. So welcome to all of you. If you're watching via YouTube and you have any questions for our presenter, please do reply to their tweet. We have a special treat for this session. And that's at our tweet site, which is S-H-E-S hackathon in the feed. And we'll try to reply to those as soon as possible. Participants, you're welcome to send your questions through within Zoom. We would like to draw your attention to our code of contact available on Esmar Conf website, which is esmarconf.github.io. Our workshop presenter today is Alison Bethel from University of Exeter. And I'll now hand over to you, Alison. OK, thank you. I'm now going to turn through my screen. OK, is that working OK? It's working great. Brilliant. OK, so my name is Alison Bethel. I work for the University of Exeter Medical School for NARC, which is the Applied Research Collaboration for the South West Peninsula. And this is my first time doing a live stream like this. So if anything goes wrong, just keep our fingers crossed that it doesn't. OK, so I'm part of the Evidence Synthesis team who are up here on the top right. So that's the team. And we've got Joel, Becca, Noreen, Becky and Moena. And a special shout out to Moena because these slides have been developed over the years. Moena and I have done quite a lot of teaching. She's my co-IS in the team. So we've done quite a lot of teaching. So these slides are kind of amalgamation of our sort of normal teaching. If you want to follow us, you'll find us at Evidence Synthesis team on Twitter. And we're also part of an IS team as well. So you'll find we've got an IS Twitter handle as well. So in the Evidence Synthesis team, we do systematic reviews. So what Moena and I do is we search the evidence for systematic reviews. And it's not just systematic reviews. We also do different types of evidence synthesis. So I've been involved in readers reviews, rapid reviews, scoping reviews, evidence gap maps. So there's been, you know, so there's quite a lot of different types of evidence synthesis which we get involved in. And what we do is searching. So this is a kind of overview of what we're going to cover today. And I think it's going to be about two hours. So it's quite a long session. So I'm going to get you guys to do stuff during it as well. So again, hopefully that's going to work. So we're going to start with some questions. So we have our mental quiz. So if you could go on there, that'd be great. And that's the code 59050637. I think it's in the chat as well. Click on that. Excellent. And I'm going to go in as well and see what's coming up. So gosh, it looks like there's a lot of people who have been involved in an evidence synthesis project. There is a second question. So if you want to move on to second question as well. So those of you which there are quite a lot of, I'm interested to find out if you've done the searching. So yeah, there's not many people who haven't been involved in an evidence synthesis project. And then the second one is if you have, did you undertake the searching? So it looks like most of you have done the searching for some kind of evidence synthesis. So I may be preaching to the converted in this session. But even when I go to searching sessions, even with 10 years experience, I always come away with something. I think, oh, well, I didn't think of that. So hopefully today it isn't going to be just repeating stuff that you already know. Brilliant. That's, I think probably it. Brilliant. Thank you very much. So that's really helpful to know that. So I asked about evidence synthesis. You know, in evidence synthesis, there's a lot, you know, I mentioned them already. There's lots of different types of evidence synthesis. You know, the systematic reviews, spoken reviews, living reviews. And I think I've probably been involved in most types apart from a living review. But the thing about searching is that in a way it doesn't really matter what type of evidence synthesis you're doing. The search methods are very similar. So hopefully whatever type of evidence synthesis you've done, you will be able to use some of these little tips, tricks that we're going to go through today. But I'm going to talk, when I talk about searching, I'm going to talk about it in terms of systematic reviews. So if you're interested in finding out more about the types of evidence synthesis or typology, here's a few references. The brand from 2001, 2009, I think was the first one. They did an update in 2019. And then there was one by Neil. And some co-authors in 2020. And I think he's just published another one. To do with a typology in public health. So I'm sure Neil can provide that reference for us. So we're coming back again. So I'm interested in the words that you associate with systematic reviews. So I'm going to try and share that again. Okay. So this is our work slide. So I'm interested in what people associate with systematic reviews. So it's great that you're all putting words in there. It's great. So we've got long, search, string, view, visibility. It's great to see vertical right in the middle there. Brilliant. Prisma. Summary to a point. Net analysis, SCAPs. Search, string, tags, transparency, precision. With the points. High impact, transparency, reproducibility, screening, write up, strong evidence, specific guidelines. Lack of transparency. Search, strategy, complete. Still going, that's great. So robust equation, decisions, evidence-based medicine there. Okay. That's really good. It's probably no surprise because you've all been involved in, or most of you have actually been involved in the evidence synthesis. So I'm going to go back to my slides. So I'm going to put up some information from Akron. So this is what they say about systematic reviews. I'm sure most of you will know what Cochrane is, a Cochrane collaboration, and that's taken from their handbook. So they talk about appraising, synthesizing, explicit methods, minimizing bias. Just a few of the words that they pop out for me. Campbell, again, it's best available evidence, robust methods, transparent procedures, and environmental evidence. That's a collaboration from environmental evidence. And again, they talk about explicit methods, appraising, and analyzing the data. Alison, can I just interrupt you? Sorry about that. I think you've got your mentee screen still up. Oh, sorry. No, you're right. I might just take the opportunity that we won't be, just to mention that we won't be taking raised hands just during the presentation part of the workshop. But if those of you that have any questions, please do enter them either in the chat or in the question and A section. So thanks very much. I'll let you, I'll leave the floor back to you. Thank you. I normally use Teams, so Zoom has a nice pause here, which I've been using and getting to put back on. So yeah, as I said, there's, you know, Cochran Campbell and the Collaboration for Environmental Evidence. There are three, you know, brilliant places to go to find out about evidence and systematic reviews. And they sort of explain, you know, why, you know, the things that you need to do to be, you know, to do a good systematic review. And we're looking, so that's the systematic review. And if we take that into the searching, we have to think about the searching as well. So the search thing, this is from CE, but I just chose this one because I thought it was particularly good. So it says here that, so the search should be transparent, reproducible and minimized biases. So these are things we need to think about when we're actually carrying out searches. So for example, people might say, you know, they might publish their search strategy, but they might say research for this and this and this, but it doesn't mean it's reproducible. They don't see where they searched or where that search string was entered, you know, which database they used. So it's, these things are really useful to think about from where we're carrying out the searches. And if you don't do this in the searching, then, you know, you're introducing bias right at the beginning of your systematic review. So this is a little kind of flow diagram. And as you all know, that's how systematic review works, isn't it? It goes all round and a nice little flow. But as you know, in reality, that is not what happens. You know, there's a lot of backwards and forwards and things. But what I wanted to do here was just to give a simple sort of diagram of how, you know, the steps within a systematic review. So in the four places that we've put the searching. So we've got scoping searches, fill search, supplementary searching and update searches. And the supplementary searching is down in the right hand corner, but it doesn't always happen there. But it's just to put it within the flow. Okay. So we're going to start with scoping searches. Why do we do scoping searches? You know, I'm sure you all know this. We need to find out, has it been done before? I'm involved in a project within our team, looking at systematic reviews within a particular field within the COVID literature. And we have found that there are more systematic reviews in that particular field than primary research that's been done. So a lot of people are doing systematic reviews on this particular subject. And they haven't even looked to see if it's already been done. So for this one that we are looking at, there's a living systematic review by Cochrane, but still people are continuing to do other systematic reviews. So it's really, really good to find out. Is it happening already? Is it underway or has it been published? You know, before you thought research waste, you don't go and do the same thing again. For related work, so any background being that you need to do, and to get an idea of the size of the evidence base. So that's really quite important. So often scoping searches happen at the point where you're putting a grant application in. So if we're doing that, I would do quite a lot of scoping searches to find out how much evidence is out there. Firstly, is there enough evidence to do an evidence synthesis? Or is there too much evidence? Do we need to restrict our question a little bit? Sometimes this is done. If you're not applying for a grant, if it's a question within a team, scoping sometimes happens in conjunction with developing the research question. Okay. And when you start scoping, you get an idea of the search terms. And also what you might find, or hopefully what you'll find, are key papers that come up in your search. I've got search terms in there, because it's really important. So how do you do it? These are just some places that you might want to search. So a protocol for systematic reviews, Prosper obviously is a really good place to look. At the moment, I don't think there's anywhere that just has like scoping reviews or evidence gap maps, but I might be wrong in that. There are other places you can look. We've registered protocols on open science framework, but it's not the easiest one to then go and search to see if there is something ongoing. Third line. This is in the medical field. It's been nice evidence. Systematic reviews. Again, these are all kind of more medical, cognitive databases, systematic reviews, trip database and epistemonicus, which is a fantastic resource for systematic reviews. And then you've got PubMed. These ones that I've mentioned are all freely available to many people. I'm not sure they're available freely available to everyone. And then there's ones at the bottom there, which I've mentioned, which do require a subscription. So if your university or institute has access to these databases, they're quite a good place to do some scoping searches. Some common problems with searching. I've been doing this job for just over 10 years. Prior to that, I worked for the Environment Agency and before that, I worked for a government research lab, both of them as a librarian. But still, you know, sometimes you've got, where am I going to look? There's too many resources, which ones am I going to focus on? And then once you start searching, often you get too many results, or there's not enough results. So it's, you know, these are common every time you start searching. You come across this. And then when you stop, because in a systematic review, I meant you're trying to find all of the evidence. So at what point do you say that's enough? I'm going to stop now. So it can be quite tricky, which is kind of why we have a protocol where you lay out your search methods and that in a way that stops you in terms of where you're going to search, but how you search within those databases, you can kind of go on and on a bit sometimes. So what we're trying to balance is to find as much relevant information as possible and minimise the amount of irrelevant information. Again, this is a balancing act that we are continually trying, you know, continually trying to balance these two things. But before you start searching, you know, your full search, you want a protocol. And I mean, that came up in the mentee quiz we did there. Yeah, everyone, you know, you're going to use a protocol. And for searching, the inclusion and exclusion criteria can be really helpful. I've been involved in a couple of projects recently where they had a very sketchy protocol and they went straight into finding the search terms or defining them and putting them down. And I was thinking, well, we're not at that stage yet, but actually what I think they were doing was trying to come up with their inclusion and exclusion criteria. So that might be a good way you might find that useful to just putting search terms down to help you, you know, boundary your inclusion and exclusion criteria. And then the second thing to do is look for any relevant systematic reviews. And I don't mean for your specific question, but say for elements of your question, if you're looking, say, for children with asthma, that's your population. You might want to go into Cochrane and have a look. Are there any systematic reviews that they've published on the population children with asthma? And then go in and have a look at their search strategies. Don't just look at them and use them. If you are going to use them, please reference them. But the other thing that you really need to do is appraise the search methods or the search strategies. Go and look at them because, you know, I often do this, I go in and say, well, actually, I don't need that. And so, you know, Cochrane is a great place to start because, you know, they're sort of gold standard systematic reviews. Also have a look at any, if you're doing an environmental one or social sciences, look at the Campbell ones or the environmental evidence journal. They have some excellent sort of gap maps and systematic reviews and they publish protocols as well. So, yeah, definitely appraise them and don't just think because somebody else has used it that you're going to get, that that's the best search strategy for your question. And find key papers. And you pick those up from scoping searches but also from topic experts. So in your project team, you might have some topic experts so ask them for any key papers. And then just following leads, you know, on the internet. So you can come up with a body of key papers that's going to help you validate your search. Okay, another mental quiz now. I'll play this one again. So this is about what resources you have already used. So hopefully you can see that where the science is coming up high. Google Scholar, PubMed, Identity Databases, Twitter, Psychinfo, Parmalink Research. I'd say I'm not sure what that means so any explanation on that would be super helpful. Central ProQuest, Lens, Reference Lists, Relit Databases, Medline, Mendeley, or SSB. So yeah. Huge amounts. Previous metanalysis. Lots of different preprints. We've got a message, OATD, Open Access Thesis, and Dissertations. Was that the one you asked, Alison? Yeah, it was. I can't remember what it was. They're going turning around. Brilliant because there's stuff coming up here that I never use. I've never heard of, you know, the cyber linux. I don't know what that is. It's brilliant because I've been doing this for 10 years but you're constantly learning. I should really say this is what I'm presenting here is just a way in which I do and my co-information specialist here at the university go about it. It's not the only way. In any searches that we do, we do know that there's an art to it and this is the way I do it. Someone else can do something completely different and we still get the included results. It's not set hard and fast, yes, right and wrong. That's brilliant. I'm taking a screenshot just to look out some of those. Thank you. Great. I'm going to pause that presentation again. Hopefully that's coming up again. That was really helpful. Thanks, everyone, for engaging with it. I just want to mention this one but actually from what you all put in, I'm not sure it's that relevant now because I think you all pretty much know what a database is as opposed to a platform or a publisher. There's a couple of things I just wanted to point out here. Through Ovid, you can get, say, CAB abstracts, local health and Medline but you can also access it through CABB and here, CAB abstracts and local health. Medline, you can access it in loads of different ways through Ovid, EBSCO, ProQuest or PubMed. There are things to think about. Web of Science I know that that has lots of databases on it so it has sign citation index it also has Medline, you can access through Web of Science and it also has things like conference abstracts so that's not Web of Science it's a database, it's a platform that holds databases so you should really say in your search methods which databases you accessed and not that long ago I was probably just putting Web of Science and we're all learning as things go on, as things develop and change so it's great to see that that's now yes, we do have to say that, which is brilliant and there is also some evidence to show that it depends which platform that you use will depend on the search results you get, they can be different depending on the platform so you can search, say, Medline through Ovid, you'll get different results to Medline through ProQuest it could be partly to do with the interface the platform interface because they're all different or it could be to do with when the database is updated on that platform which is why we generally say which platform it's through but you will see people saying I searched EBSCO, you know that's really like here at the University we have access to 25 databases through EBSCO, did they search all of them you need to be a bit more specific to make it reproducible so when it comes to database searching there is both free text searching and control vocabulary searching, so in the free text searching we search for specific words or phrases generally in a specific field and then there's control go-cab searching which might know for those of you who are involved in some medical research there's MESH subject headings which you'll find in PubMed and Medline but most databases will have controlled vocab and it might be called a thesaurus or it might be called subject terms and it's good to search both of those so you're trying to capture as much relevant information as possible when we start thinking about the free text searching I don't think this would be new to most people we need to think about synonyms so whether that's synonyms, variations previous terminology, abbreviations you need to try and as we're trying to capture everything you need to put all of this information in or at least start off with it whether it ends up in your search strategy you don't really know so we're going to look at a particular question what was the impact of robotic pets on the health and wellbeing of older adults this is quite similar to a systematic review I was involved in and we're going to use Pico to come up with some synonyms for the population and the intervention again I don't think I need to say what Pico means because I suspect you already know it stands for population intervention and it's usually used or was developed to try and help people come up with a research question but in terms of searching we use it to try and combine our terms together so when you think of population we've got older adults so it's not a phrase it's just words that you're going to search in a database it's going to find relevant information for you and mostly in searching you'll search for just the population and the intervention in this kind of question outcomes are often not in the title and abstract which is what you're searching so databases don't have access to full text and some people have had that in the past they think they're searching the full text of papers you're not you're searching just in the bibliographic details which will be author, title, abstract the author affiliation the words that you come up with are going to be within that very small bibliographic details okay so we're now going to go I'm going to now try and share a jam board which I'd like to put in some terms the population and the intervention I'll just let folks know that I've popped that link in the chat to make it easier and if you use the sticky notes to do that geriatric elderly robotic pets over on the right for the intervention brilliant senior citizens aged over you and got some truncation and they're already you're a step ahead of me it's always difficult to know where to pitch this sort of thing because you don't know people coming having done very little but I think you all do that you've done lots so that's frail population centurion what happened there can we undo we've lost all our oh dear oh we've got pages it looks like we've gone I've got three pages here but there's nothing in them oh I see that's the same there were some brilliant terms in there particularly how there is one about frail senior citizens so these are terms which I'm going to show you a bit of a search strategy I've got I don't have those in there so that's brilliant thank you this is an alright moment to take a question with a raised hand so I think there was one from Anna if she'd like to ask you'll need to unmute yourself Anna in an inverted hand but I might just take the opportunity there was an earlier question is that a right Alison to ask so this is from Concepcion I have certain doubts about how to inform about the databases and platforms specifically when the search for a systematic review is performed in PubMed how should I inform this the first option literature search was conducted including databases such as and she lists them or literature search was conducted including databases such as sorry the first option was the list included PubMed, Mbase, Mbase.com Cinehill in brackets EBSCO as an example and then the second version is Mbase.com in brackets Cinehill EBSCO in brackets she says she thinks the first is preferred but she'd love to hear your thoughts yeah well first of all I wouldn't say included I would say was you know because included I mean it kind of indicates the resultors to research these databases so that would be the first thing and I think what you were saying was to list the databases and then put the platform at the end is that right yes that's right so the first version one of them would be the web yes that's right you probably have the suggestions straight up normally I would put in the databases and then putting brackets afterwards via Ovid so I would normally start with medline, Mbase, like info and then in brackets via Ovid and then Cinehill via EBSCO host so that's the way I would do it rather than individually but it's um I mean it's up to you how you want to do it but that's normally how I would do it and at the end of the presentation there is an example of some search methods how I've written it up I'm not saying it's correct but it's one that I've already done fantastic it helps for me thank you very much I hope and I don't believe we have any further questions so I'll let you continue okay thanks these were just some terms that I came up with so older people older women elderly aging and I think we had older people older adults they were all in there and the intervention it disappeared before I got to see many of the interventions so robotic animals robotic pets robo pets and paro you wouldn't know this but there is a robotic field called paro and there's quite a lot of research there's been research on that and I think there's a robotic dog or cat called Ibo as well so you'd probably put those you would pick this up from your school ping searches or from some subject specialists so how are we going to structure this then we thought of this in limbs and variations so this is a little table I put like this you know it might be too simple for other people but I quite like putting the population and then the intervention just in a table and then I can add all the terms when I start to search I can start crossing them off or adding things in and when you go down it's on across it's and okay it's fairly straightforward if your research question doesn't fit into Pico you might want to think about it in terms of concepts so concept one or concept two that might be more helpful to you so the next thing we're going to think about is truncation and well and free searching and we saw that you put somebody on the jam board you put some truncation and so people already know what they're doing so a phrase searching is means it's just going to search for older person as a phrase for older person it's an asterisk will be older person older person so I think people probably know that and so I'm going to put that into my search here in the table so I'm going to put old asterisk I'm going to leave older women and robotic animals I'll put an asterisk will be pet and robotic pet so that's how you put your quotes and ask world card then and then the last thing we need to think about are fields so which fields are you going to search in and proximity searching has someone raised a hand yep it's Anna again let's try again Anna would you be able to unmute yourself I'm unable to unmute you hi I didn't raise my hand so I don't know what I feared so before but I didn't do anything on my screen I still raised my hand so I do think that my hand is not raised at least in my program but I don't know it's lovely to hear you anyway not a problem I'm not sure what's happening that's fine thanks very much it is lovely to hear a voice because I feel like I'm just talking into my window into the street here so yeah we're talking about field so you then want to think about the fields that you're going to search in the unfortunate thing about proximity searching is that it's different in the different platforms so there's ADJ is in Ovid near X I have to write these down is in Web of Science ProQuest and Coffin all use that but EBSCO uses N so if you're going to use proximity searching make sure you have a look in the platform that you're using to find out what syntax to use for and it means different things in different databases so again do double check what it is but I think the proximity searching is a really neat tool for systematic reviews it means it doesn't it's not as specific as phrase searching but it's not as broad as using an and looking for older people in care homes you probably wouldn't do this but you might do older people so ADJ within three words of care home rather than looking for older people and care home anywhere so that's just what it means I'm going to show you how to do that on Ovid shortly so this might be how your search would end up I've kept older women with an asterisk as a phrase older people I've put an ADJ2 and robotic pet I've put ADJ1 but you don't know what the proximity is going to be until you actually start searching and you do have to do a bit of sort of playing around doing different ones and seeing which one's going to give you information you don't want to lose stuff and it might depend on the database so I sometimes will use adjacency searching in large databases like Web of Science and Medline but when it comes to a smaller databases say like global health I might use an and because that makes it slightly broader so that you might want to change it depending on the database Alice I have a question from Gloria about what you mean regarding proximity okay it means you're searching so if you have older people if you use it as a phrase it looks for just that phrase within the title and abstract if you use proximity searching so you put ADJ2 say in there it will look for those two words within two words and they can be reversed so it could be people who are older or it could be older adult people so we'll pick all of those up without having to put all the phrases in so you do sometimes these searches where they just put lots and lots of phrases in but actually you would have been able to capture it by using a proximity but I'm going to do it in the demo so hopefully it will make sense and I'll show you one not the other so hopefully you'll be able to see on the record what I mean when I'm saying that does that help at all? Thanks very much I have one more question would you turn the whole list into one long string A or B or C etc or run multiple shorter strings normally I would do it line by line because I think in terms of reproducibility and transparency it's much better you can see what someone has done but also when you're developing the search you can see where some of the big numbers are coming so you might put something in like trying to think now if you put it in a string you might miss that so that's something to think about and I know not all databases will allow you to do this which is why for me start developing a search we'll often use Medline in Ovid because it does allow you to do it line by line you can see much better what you're actually doing whereas if you do it in say ebsco when you start combining searches it puts it all into this huge block of text and then it gets really complicated if you've got quite a long search or quite a complicated search it's really difficult to figure out if you go back to a year later it's really hard to figure out what you've done and if you publish that it's really difficult for other people to figure out so that's not ideal yeah does that help yes thanks so much Alison so hopefully if you search I think hopefully the link for Ovid will be in the chat and you can use the username and password so I'm hoping that's going to work for everyone I'm not sure what we'll do but it doesn't, you might have to go on the pod map or something so I'm going to pause and then I'm going to bring up my Ovid and we're going to use global health so again that's a thank you to Ovid and Calbee for allowing us these username and passwords to have a link just in case folks haven't seen it in the chat that link is available there with the username and password thanks everyone you can either just watch what I do or you can follow with me in what I do just to have a practice because after I give you the demonstration I'm going to allow you guys I'm going to suggest that you go off I'm going to give you a different question and you go off and try it yourself okay just click here to make sure you can see all the search I don't know if that's coming up so I'm going to use the search terms what's happening here okay so I'm going to try and remember them so if you put all the people up an asterisk and we're going to use that as a phrase we're going to put quotes so in Ovid you don't need to but you do have to another database so I'm going to do it here as well just for some good practice and in Ovid you do a full stop then you put PI which means it's going to search for in the title then a comma and AB which means it's going to look for in the abstract so I'm showing you the advanced search so there are lots of different ways of doing this and I'm just showing you the way I would do it but you may find a way that's better for you and then I'm just going to click search so I've got 6,846 okay so now I'm going to put in all the ADJ2 people again with an asterisk full stop PI comma AB and I'm going to search for that you'll see there's over 7000 now so what I would do is I would look to see what additional papers it's coming up with so to do that I would do two not one you're looking for all the ADJ2 people but not all the people as a phrase so these are the additional references that you're going to find you've got 500 in there and that's when we'll go in and have a look so you'll see here older adults people so you may say okay I'll put older adults as a phrase older adults people get older it might be helpful older people's age people with older age so hopefully that shows you what you're what the adjacency is now doing whereas if you go into so if you used older people asterisk just the title and abstract these are the ones that you're going to miss okay and for a systematic review they might be really relevant so what I tend to do when I'm developing a third is I'll end up trying different things so older ADJ3 and I'm going to put sort of PI on that baby so I'll come up with slightly more again so then I'd do four not two okay so we're looking for the ADJ3 the ones that are using ADJ3 and not ADJ2 we'll come up with 219 there and then I would go and look older individuals and people older individuals might be a phrase or a term that you might want to consider people who are older older or disabled people older adults older people you can see these are less relevant than the previous one but it might be useful and then when you go into ADJ4 or 5 you're probably going to find there's too many words in between them that's going to give you relevant stuff so it's a really powerful tool to use when you're doing systematic reviews so I'm just going to use ADJ3 I think I'm just going to delete these that says we've got older people we had older women PI, AB I'm going to put elderly and ageing I think I was going to if you put aged in I only know this because I did this recently aged.ti, age you do a lot of things because you'll get school aged so you think of it as a subject in the text your patients aged more than 30 years among children aged so that's the only one you don't want to use and that's why you go in and have a look to see what results are coming up and that's also why the question that was asked earlier you can see this one you get 200,000 takes anything else I'm not going to put that one in or you might want to put it in as an adjacency with adults so you might want to sort of find can I ask a question Alison there's a little back and forward in the chat and folks are wondering can I ask a question it's a really good question because in global health when you put CW it doesn't insert the types on the I'll show you I don't often use global health but I did I'll show you so what I was doing was you've got all of these subject headings organism descriptors broad terms it was coming up with a lot of stuff in here which I wasn't interested in but it's really good because it's very hierarchical and it's very clever but I didn't want humans would be somewhere and then underneath you'd have other terms I'm going to show you this in a moment but when you put TW it was coming up with a lot in here I wasn't interested in let's see if I do let's see can you give me six you've got here as far as total food intake of older people and that's not I just wanted to look for the stuff on the types on the abstract it's just a quick question referring to what TW specifically means you might have that question so I'll just ask TW stands for text word and I think it varies depending on which database you're in it's title abstracts and sometimes the keywords when you're searching Medline it seems to make no difference between TIAB and TW which is what I normally use but I think global health is slightly different from the other TIAB one last sneaky question sorry I didn't mean to interrupt I think these are the truncators instead of the asterix would you perhaps use a question mark like WOM question mark N yeah you can do that and then it will have women so yeah you can definitely do that unfortunately I think of it as the only one that allows that I might be wrong but of course you can definitely do that good point thank you now I'm thinking Fred it happens every time you search oh I wonder if I'm going to stop doing I wonder if I'm going to be infided TIAB and not TW um but yeah I'm going to remove these ones because obviously the other thing you can do is in here you can actually edit that I might edit that to be hold click ok and then run the function ok so that's in terms I mean it's certainly not on exhaustive list I want to show you the thesaurus in global health I'm going to click this map I don't know if other people have that on theirs and I'm going to put older people in ok so we've got some terms there we've got elderly I'm going to click on that and this is your hiring article list so the control terms are it's a set of terms which when a paper is indexed on a database they will assign index terms to it and it's the idea of it is that something which doesn't have the subject and the title it will be indexed under this and you should be able to find it but all the research kind of shows that actually you have to do both just using free text terms won't capture everything and just using control of vocab won't capture everything but you have to be careful with using it say in medline you need to be really specific because there are thousands of records on there so don't go broad in your choice of Thesaurus term there but in something like global health much smaller database you might want to go a bit wider and again play around with it and see what's coming up so it shows that it's used for elderly so I'm going to click elderly and I'm going to click old age so there are two ways of searching you can ask the term it's here or you can explode it with this here and what that means is you've got elderly and underneath in the hierarchy you've got elderly patients and what it's doing when you explode is searching for elderly and elderly patients or elderly patients so you're searching for both of those subject terms not just one I hope that makes sense but I'm going to be focused from now I'm not going to what you call it explode them and I'm going to continue so now we have five lines of search and I'm just going to order these together so on your screen remember to click here you might need to expand if you haven't already so you can see all your searches and what I like this interface it's very simple it's got the searches here you've got your results here and then you click on them all and click on okay so we now have 99,811 but it's actually isn't that many and you're thinking of all the people so this is you know it's a very small database so that's something to be aware of when we start putting the next subject terms in so what we were looking for robotic animals or robotic pets so I think we're now going to look for those so robotic I'm going to put ADD three animal and I'm going to do .ti,ab one hit not very many so I'm going to do I'm going to say I'm going to say I'm going to put robot animal and the six so I'm interested really in what they would be normally if you're looking at big numbers you'd do one not the other but as there's only there's not many hits and animal robot yeah I'm not sure why that's interesting robotics animal like robots okay so maybe you put robot ADD three animal but I did wrong so I put robotic so that's really easy to see that you've done something wrong there so I'm going to take this one and I'm going to leave it with robot animal and then we have so I'm going to do the same for pets normally you probably wouldn't use pet asterisks because we come up with so many hits so you need to remember where you're putting your asterisks but because it's near robot I think it's probably going to be fine so that's the I there's only three paro there's numbers and there aren't many so in this instance I would be tempted just to put robot in that comes out with over a thousand okay that might be that you might want that you might not okay or in this case you might put an ant rather than a different thing you might put animal or a pet and combine that with a robot with an ant so there are different ways of doing it so I haven't I was practicing this and I didn't do this so I'm going to try it and I'll never speak to this group so if we did that and then combined it with a robot with an ant we've got 90 so if we just did these we've got 9 and we've got 90 with pets or animal combined with robot that looks alright to me so I'm going to do the next thing we need to do then is search for a subject heading within this assortment for this so I'm going to put robot again a robot I think so we have robot here but if we look at the slope for that it doesn't look like it's going to be relevant and if we click on it the plodder term is equipment so if we put that in at all I would just leave that so I now have peril number 9 or number 14 which is 98 and I'm going to combine that with my elderly term so my population my intervention combines and so I have 6 hits so smaller to base you're not getting a lot of hits if you rerun this in midline you'd probably get a lot more but it's useful just to look at and see what's coming up look at the complete reference and have a look and see what these subject headings are here what they're using to have useful thoughts there they've also got nursing homes might be interesting to use that as well as older people aged elderly people older adults in here so hopefully that has made some sense it's really simple and every day to base allows you to do it in this fashion sometimes you have to do it slightly differently I'm going to pause now and I'm going to go back to my slides I'll just let you know Alison that there's quite a bit of activity in the chat and folks are helping each other a bit of back and forward so it's just been wonderful and personally I just thought your run through was just fantastic I did practice this and then obviously you go oh what about this and you know that constantly what you're doing is you're going oh what about this what the results are and go back the notes really help I remember showing that someone recently well obviously that's what you're going to do see what you're going to miss if you use one term and not another so yeah hopefully you know it's been useful for people are there any questions in the chat that I can help with I don't think there's any open questions at the moment I'll let you keep going and I'll just check just in case we've got something on Twitter that will look actually so I'll let you proceed so the next bit is yeah any questions so in the Q&A I don't know if anyone wants to ask any questions or unmute themselves to ask any questions at this stage or you know does anyone have any comments you know a different way they might do it Neil I don't know if you have any questions because I know you do you know you do a lot of searching so I don't know if you have any any pearls of wisdom to share about it Hi Alison thanks very much really interesting workshop so far just following what was going on in the chat about a number of databases and duplicates I guess my only question which isn't a question it's common is that yeah it's really important to have multiple databases isn't it in a systematic review because duplicates are relatively easy to remove and you don't need to manually screen them all in most cases they're quite easy to remove automatically but making sure that you've got a comprehensive set of databases and I'd as well say for my subject in environmental science we want to make sure that we cover different geographies different disciplines or biomes or you know different databases cover different topic areas different languages different regions and it's just nice to well it's good practice to throw in a lot of diversity there to make sure that you've covered things but particularly the language thing which is really challenging isn't it Yeah yeah I've been involved in a project it's still ongoing to do in Argentina so we had a librarian from Argentina I did the search in English on I can't remember how many databases and we sent them to her and she re-ran them on Spanish and Portuguese language databases so you know it was a lot of work I have to say to get to that stage but it was it was great to be able to do that you know because we had a partner out there it was able to help us with it in the end I don't think there were any included papers from it I need to go back and check from the projects finished but it's certainly a really really useful thing to be able to do particularly I think there's a lot of evidence for using Chinese language databases I've never used any but I think in particularly in the medical field there is there are geographic databases you know if you go to your library you can usually find them yeah I think the last thing I would say for me this highlights that one of the great things about doing systematic reviews and systematic approaches to evidence synthesis is that it sort of puts your mind at ease because you never know oh I didn't think of that or I might have missed that and if you set it out carefully with constructive feedback from experts in advance you then know that you're not going to wake up in the middle of the night in a sweat because you realize you missed a whole body of evidence I don't know if everybody does that but that's the kind of thing yeah no still you know you still think that and we now we peer review our search strategies so we have an information specialist team and there are five of us so we now peer review each other's search strategies and it's great because you go oh okay I'm okay I'm alright because I was doing a project recently with peer support and my colleague she had misspelled something and it was line by line she had misspelled something and it came up with results so she didn't really notice it because it was a really long search and because I had gone down in detail we picked it up so yeah it's it's definitely worth either having a colleague or peer review it or if your search you know particularly in the environmental field the searches are in the protocol so they will be reviewed at that stage but then you've probably already done the searches by then so and then I can hand over to Yanis if you have questions again but there is one question about grey literature yeah okay yeah we're going to come on to grey this is just the database searching we're going to come on to other types of searching and that's in the little box at the bottom right the supplementary searching I'll mention grey lit there another one's come through could you download all the results from your search in OATD oh that sounds like it's a specific question to the person familiar with OATD and then a few more comments about grey literature so I think we'll let you continue okay and I did mean to show yeah in a systematic review as Neil rightly said you search you'll have to search across several databases I think in Cochrane it says minimum of two plus subject specific databases I think but I've never done this even a rapid review but I haven't searched at least four databases and as Neil said you can easily remove duplicates we use EndNote I'm a big fan of EndNote desktop but I know you have to pay for it so removing duplicates there is pretty easy and there are ways that you can you can structure the database to be able to collect different it will look in different fields you can choose the fields that are looking for the duplicates so that's really helpful but there's lots online about how to do that using EndNote I just wanted to do a little plug one of the hackathon projects is precisely on that point about having an open source tool for deduplication so just encourage folks to keep an eye on our twitter to see the results of that hackathon project I mean there's some great stuff that's already come out of a hackathon so yeah they're brilliant yeah sorry Yannick can I just ask what the time is I don't have a watch it's 43 past I'm not sure how to say this because I'm an Australian sorry go on Neil it's 11.43 GMT you've got 45 minutes left thank you Neil I'm now going to give you 15 minutes to do your own searching on global health via Ovid so I've changed the question slightly so what are the effects of animal therapy on the health and wellbeing of older people so if you've followed along with the search and put some of the older person terms in can I ask you to put in two additional free text terms for the population and then put three free text terms in for the intervention and use an adjacency term at least once in that and then one disorders term for the intervention and I've given you two key papers so this is what we would normally find people would give is often it's more so there's two papers there one from 1996 and one from 2020 and the second one animal assisted therapy improves communication and mobility among institutionalised people with cognitive impairment so those are the two papers that you're going to use to try and develop your search strategy so I'm going to say 15 minutes for that how does that sound Neil and Jana is that okay yeah let's check in and 15 minutes time if folks need a couple of more minutes we could possibly extend but that sounds great okay thank you so if you're watching this video in catch up you can scan forwards 15 minutes and we'll put up a slide to indicate this is a pause, a deliberate one I'm going to step away from my computer for the moment just for about five minutes you're on half term here and I'm going to see what my son is doing sounds good thanks very much Alison for everyone else we'll check back in in 15 minutes that's 12 GMT or exactly on the hour wherever you are and we won't change the slide actually because there's important information in that slide but we'll make a note on the YouTube channel and we're going to say that one hour 15 in there is a 15 minute pause so we'll see you again in 15 minutes thanks very much all right just checking in to see how everyone's getting on does anyone have any questions maybe I can try an answer I've had a look at the chat and there's some great questions and some great answers so thank you to all those people thanks Alison I thought it might be just a nice idea to ask folks to pop in the chat where they're joining us from just because we've had a few people mention they're from pretty far away or far away from me I'm in Australia so I thought that might be nice if folks might like to add just to pop there the country where they're joining from just in the chat and I recognise a few names as well just fantastic thanks guys Scotland I'm heading out tomorrow where are you headed if it's alright to ask sorry that's just my natural chattiness coming out the other coast Glasgow area that's where I'm originally from so we're on holiday now for a week here my son is so we'll drive up and see my folks and family that just sounds fantastic the weather is not shockingly bad thanks everyone that was really lovely to see everyone's place from where they're joining if anyone has any burning questions now could be a good moment to include them in the chat especially with regards to the search we'll see at the end but there are lots of others out there if you're using a specific space or a specific platform you do lots of online tutorials if you ever need to do anything and the other excellent place to go obviously is your library if you're not in an institution or your institution doesn't have links to good resources I think all university libraries their stuff is available freely so look around and see what's out there there's some great stuff out there I'm going to start again but what I'm going to do is hopefully I've done this search and I've given you two key papers now I'm going to give you another two and with the search that you've done see if you've found these two as well it's just a little bit of extra I picked this one up from I did a course with the people out there at the university which was great and needed this which was kind of interesting it would be fantastic to hear from you guys you might like to put in the chat if you picked up these two papers with the search you constructed yeah we do also have a Google forum free to fill in at the end so interesting how many hits that you got and if you used any control vocab what it was and did your search find the first two references and did your search find the second two references Niels just popped that in the chat I'm going to try and find it now so someone's asked how to find the source term so I can go back in and solve that with a problem can you show the test references again Alice? oh yeah sure sorry I will sorry I admit to quickly yeah sorry about that there you go I'll stop playing around with it I can ask if we come to an end on that Niels are you able to share the form I couldn't figure out how to do that would you be able to do that the Google form is that possible yeah no worries I'll just give that up on another screen we've only had one response so far alright maybe I'll let people fill that out and what I'll do is I'll go back into the Google form and show you the search that I did just to see how it compared it's not a search for a systematic review it was a search but I didn't do it for stuff sounds like a good idea people can have a look at the results coming in via that Google form once you filled in your results you should be able to see the analysis of everybody's responses so hopefully you can all see my all of it again I'm going to remove all of this which is the one that I was I'm going to go into my search I'm going to run this search if you sign up you can save your searches and then re-run them in a systematic review so this was a search that I came up with so cats and dogs and I used X or domestic animals I think someone asked about how to find that so in the advanced search there's a map of subject headings and I'm going to put in animal and if I click on animals here that brings up the hierarchical list so animals probably not but there's pets there I'm going to click on pets and then you've got related terms when I was doing this the other day I initially I exploded it but then I realised that actually mice that's a lot of experimental stuff but you wouldn't choose to use mice in your search so this is a case where you would look for the narrow term of pets you might choose cats, dogs but domestic animals is probably the one that you've been most interested in which is what I chose to do in the search and I've got 3,746 which seems not too many since I uploaded a pile of stuff references to screen for someone the other day and there were over 14,000 so it doesn't seem that many at the moment so some of the things that you also I didn't mention earlier is you need to it's a good idea to put in your dark crafty and there are various ways to do that you can click on the first one and go to export you can export to Microsoft Word you can email it to yourself so there are lots of ways of doing it find out which is going to be best for you and the other thing you can do in a systematic review is download all of the hits you don't go through them at this point and because Neil mentioned you're having search in several different databases you can download them all and download all the results and then you remove the duplicates before you screen so different databases have different rules for downloading results so here at the University of Exeter I can download about 500 at any one time I think it's up to 1,000 but it tends to go for a study of 500 that's up to 500 and your export format is generally in risks then it's compatible with a reference management tools that you're going to use I use EndNote so I just upload them into EndNote okay so this is a search so I've also got the save I've also saved the search of the key references I would do this in a different way and then I'd run that another good thing about it is that it adds your save searches on to the bottom if you have a science you can do that it overwrites what you've already done it's not much use and then you just do one for my key references okay I know key references are really important but they're good to validate your search but they're also good to look at the detail of the reference so it's good if you've got quite a lot and it's quite an unusual subject area it's good to see which databases they're in which lines, are they in environment complete are they in midline and that will determine which databases you're going to search in but also if you find all of your key papers don't try and develop your search just to pick up your key papers I think that's what I want to say because you already have them so I think it's good to know that you can use it as a fail-safe in a project recently we were doing on peer support we searched for systematic reviews first because it was a map and then we looked at primary studies in the included systematic reviews and say there were maybe about 60 and there were two ISs working on it we then developed a search for primary studies using that development set that we had and then at the end we used the test set so that the other person's development set to see what it picked up so kind of what we did there so that's a if you have enough key papers it's unlikely I suppose but that's quite a good way of helping you develop your search okay so I think that might be the full search so hopefully I'll be able to go onto the Google Forms at the end and see what the results were so thank you so much for engaging with that and completing the Google Forms that's brilliant and this is the sample search the one that I just showed you so we've got third Vogue rather than Vogue Asterix because it might pick up a lot of stuff and it's irrelevant I put all 80 day people, person, adult or resident and I've also put some nursing in care home terms in there as well okay so any questions at this stage I have to confess I just put in some silly answers into the form because I was curious and we've got a really nice little comment really sorry I went to get a drink and a Lou break rather than the search but I'm loving the workshop and I suspect others might have done the same but do you know what Alison I think it was such a good exercise I'm actually waiting to follow the recording and do it myself in my own time because I think it's worth trying and also to check whether you pick up those references that you said and obviously not cheating and seeing your final strategy which you've just shown having a real go and trying to pick up those other references so thank you very much okay so I'm going to talk about search filters which are pre-tested and published search strategies and they test them on large numbers of papers and it may filter out relevant studies so you need to be careful of them and they often perform inconsistently across subject areas this is a place to go for search filters the ISSG search filters page because they don't just link to papers so there's some other good sites out there but you should find them in the ISSG search filter page so this is the Cochran RCT search filter which is one that I do use the other one that I've used is the geographic search filter developed by Knife that's for UK I think I've probably used the whole thing but I've adapted it for what I need as well like you would with any search strategy this one I'd probably use as is and this is I think it's a balance of precision and sensitive I think they've got another sensitive Cochran RCT search filter which will capture more but it depends on what you're looking for so do use them but use them with Cochran is all I'd say check your results when you're using them so when you're doing your field search to a particular database to develop it in use your key references look at other systematic reviews and what's really important is to give yourself time particularly when you're having to use Ovid I like it but when it comes to searching in things like ProQuest and EBSco you can start pulling your hair out it can be extremely frustrating because often it's not doing what you want it to do and you'll combine two sets of concepts with an and it comes out with more hits than you started off with so it can be quite frustrating and often you'll find you may need to do a shortened version of your search strategy unfortunately because they don't all respond to complex searches in a great way use free text control the vocab play around with the JSON say use the not to see what you're going to miss or what is added to your search strategy so if you're checking your results don't presume that it's done what you think it's done because in some databases I think you need to put quotes around your single terms as well otherwise it does truncation searching and it will also search for what they think are similar terms so just be careful it's not all journals are indexed in the databases and the papers might not be indexed accurately but also the papers might not be written in a way that you can pick them up so if you've got some sort of fun and interesting title but doesn't actually tell you what the paper's about it's going to be really hard to pick up it's not all journals are indexed in the databases but if they actually tell you what the paper's about it's going to be really hard to pick it up we did a systematic review a couple of years ago and it was called they walk the walk instead of a subtitle system review of it might have had more in the abstract it might have had indexed but you're reducing the chances picking it up. Your search strategy just might not find them, but also you might find some more recently published papers or older papers. So the database you're searching in only goes from 1990 say you might find some older stuff when you do supplementary searching. And there is some good evidence to support that it's good for finding the grey literature. So these are examples of supplementary searching. So citation chasing, hand searching, web searching, key author searching. And just to say I've done some of these as part of the big database search. So key author searching. So I was doing a project on antimicrobial resistance a couple of years ago. And in that there were three particular authors that were interested in. So we searched those on the Web of Science and Scopus for anything by those authors, as well as the keyword type searching as well. And I've also done one where there was a key paper on attention restoration theory. I think it was from 1997. The paper was published on it. So we did forward citation chasing on that one as part of the actual search strategy. So these are methods that aren't your standard database type searching. And you also have web searching, contacting organisations. I've done that in a project a few years ago on dementia gardens. We phoned them up, organisations and spoke to people and then they gave us other places to contact. So we kind of snowballed that list. But I think in the end, I'm not sure any additional studies were found from that. But it's really useful to do. Years ago at the E-Hill Conference, the European Association of Information Libraries, we did a workshop on supplementary searching and we wrote up a blog post about it. So it gives you all quite a few of the different supplementary search techniques and who does it and what stage in the review you do it. And then there are a few at the end which we didn't cover, like pearl growing and snowballing and stuff that we didn't have time to cover. So that's a bit on supplementary searching. And then I'm just going to mention great literature. So it's been mentioned already. So this is a literature which we think of as unpublished. And the main ones are conference proceedings, reports and theses. And I saw a tweet by Andrew Booth the other day that said we should be saying at the very start what type of great literature we're looking for. So if we say we're looking for theses, then you can say, OK, we're going to search pro-quest notifications and theses. So if you're if you're not interested in conference proceedings, then don't search web of science conference proceedings, you know, just for the sake of it, because if you're not interested in it. So, you know, be choosy, I think, is what you were saying and be explicit from the outset, you know, which is something I haven't done in the past. So that's useful to know. And where do we find it? I think people forget we can search for them on databases. So there are databases that find a specific. They do include. OK, and I'll show you a search summary table later, which shows that we included two theses in a project and they were both picked up, I think, from like info. So yeah, it's I think people forget and supplementary searching also picks up journal literature as well. So it's not one or the other. Great literature is a type of publication. Supplement, you know, citation, chasing and web searching is a type of searching. So try not to get the two confused. But there is evidence to show that supplementary searching does pick up more greatly. OK. Some catalogs, though, these are very UK specific from the British Library, electronic repositories. And I haven't used those, but most institutions, particularly universities, all the employees publications will be on an electronic repository, so they could be a valuable source. And there are networks and organizations with websites. They can be really valuable. So going on searching either in the publications catalogue or browsing their website, they can be that could be really useful. And obviously, the other supplementary search techniques that I mentioned before. I'm not going to pause because I think we've got about 10 minutes. And I just want to cover some updates there is why we do update searches, because you might find more recent publications. And often they say that if your review goes on for 12 months, then you need to do an update search. So there are different ways of doing this. I used to sometimes some people might put in a date restriction, you know, so just for the last year. So that's a publication date restriction. I'd be extremely wary of doing that because the new stuff that goes on to databases isn't always new stuff. They might be indexing new journals, which means they'll have a kind of back catalogue of new journals. Or yeah, they may have old information, they may have updated some some information. So I certainly wouldn't recommend using a publication date. But databases will often have a date of entry. And you can use that. But I have been awake on using that myself. I don't think that's that reliable, because sometimes it doesn't seem to work. I don't trust it put it that way. So what I now do is I rerun the searches and de-diplom against the original searches that we did. But you might want to think about where you search and how you search, because you've already done the searches, you've built your included studies. And you might have done like a 50 line search. But actually, from your included studies, there's a lot of stuff that you don't need. So you could decide to edit your search, I've done that in the past. Or I've decided just to search in certain databases, because I've evaluated the results to search and search strategy. And I've said, no, I'm only going to run it in one or two databases. So there is a potential for doing that. You do potentially miss stuff. But if you can evidence why you're doing it, it could be worthwhile if you're short of time. Okay. And this is just an example of in a protocol, what you might put in. So it says which databases you're going to search. And you can see from the question earlier, so it says Ahmed and Sinal via EBSCO host and based midline site info via orbit. And you'll see web of science only search in science citation index. So not social sciences or humanities and pro quest dissertations and pieces. And the other two fire pro quest. And it says relevant credit will be searched for and where we're going to search for that. And where we're going to do the forward and back with citation searching, and the organization's website, including but then when you come to that you would put down all of the websites that you've searched. I mean, this is quite old. You can tell because it says web of knowledge and that has it's no way of sign. So that that's an old name for us. This is an old one. And then these are search methods for writing up the systematic review. And again, things have changed now. So you'll see that it does say the platform that we use but what it doesn't say is the range of years for each database. So you need to include that now. That's, yeah, something that is really, really helpful, which is great. So it says that we exist. And that we run it. And we've got a documentary file of customer line search strategy. The guidance is now published all of the search strategies that you use. And we said how we did the backs and forward citation chasing using the included papers. Because often you don't see that you presume that's what they've done. Just because I presume that's what they've done. Because that's what we do. But they may have used other other ones. And it's just be clear. I mean, there's some search metrics that I'll show you. And then we said we searched 12 websites and Nexus news. And for this one, all of that information is in a supplementary file. And you can see we did another search later on, because we found out there were particular programs from from the screening of all the literature. There were specific programs. This was to do with parent to parent support in the neonatal unit. So then we then went back in and searched for those as well. So it's don't think you have to have been database search, that's going to get everything, you know, you may need depending on the question, you may need to be a little bit flexible about how you're doing it. But record what you're doing and be explicit and open about what you're doing. And this is this is a very old Prisma film diagram for another review. So again, Prisma has changed, I think it's last year. So have a look. I think I've got a link to the Prisma website. So you don't need to put a Prisma food diagram. So you need to know after you've searched, I think it now asks for all the databases and the number of hits you got from each database. So you need the total number from your searching, and then how many were excluded for duplication and then a title and abstract. Yeah, so that's just an example of a search strategy. But I've just realized at the bottom, that arrow, I think is going in the wrong way. We're going the other way. Every time you look at stuff you've done, you get Oh, you might have done that differently. And this is what this is what we now do at the end of every systematic review. We do a search summary table. It might just be a step too far for everyone. It might just be for the information specialist among us. But what we do at the end is we look at the included papers, the databases we search and then where those papers came from. So we use the unbiased files for this so we can tell whether the search picked up. And then you can see with this, these methods of supplementing searching, these three find nothing in these two find stuff. And what's interesting is that two of these that we found from backwards and forward citations were in the database, Medline and NBase, but the search didn't pick it up. So that's our learning point for me because I then go back and get a little wider and pick it up. And then you're learning from what you're doing, rather than just moving on to the next topic. And I've been doing these for quite a long time. So there are some databases I don't bother searching anymore. And it gives you evidence to see why you're not going to search anymore. You know, as an IS, I might think I'm not going to get anything from there, but the project team are very keen. So I can now have perhaps evidence, the reasons I don't want to search and something. And also what's interesting is, is some evidence to show that supplementing searching can take quite a long time, as does database searching, but for little reward. So that's something you need to bear in mind as well. So I was going to there's a certain strategy there thought for you to critique, but I think perhaps we've run out of time. This was published in the BMJ about two years ago. So I'm sure all of you can do something. Can I can I say better? More transparent, reproducible and unbiased, shall we say? Yeah. So I've just put some links to some guidance and help. Cochran, Campbell, CE, JBI, Library Guides. There's a link to Press, which is a peer review of electronic search strategies. It's a really useful checklist. And there's Prisma and Roses and their reporting standards. They're not standards on how to conduct systematic review, but they can be helpful. Prisma S, that's quite new. And that's for reporting of searches. So if you're doing a systematic review, do have a look at that to see what is now expected of you to publish. And it is much easier now because everything's electronic to be able to publish supplementary appendices and things. Because there can be a lot of information, a lot of words in it, that I wouldn't be able to go into the body of the paper. And as I mentioned earlier, platforms and hosts have instructional videos, which are brilliant. If you don't know how to do it, go and have a look. And there's some further reading. This one, if you're interested in searching, there's sure it's mainly around health technology appraisals, but they do have a lot of, they just list a kind of overview of stuff, but also lots of references. So if you're interested, that's a good place to start. And these two publications, one by Chris Cooper and ex-colleague and Ruth Garcite, who I think is presenting later on. The wonderful Ruth Garcite, I've done a few projects with her. That's a really good, these are both really good papers for guidance on search methods. And finally, these are a few of my favorite things. The SR Toolbox is fantastic. It's got, you know, everything is on there. I've tried lots of things on there. Some of it sticks with me. Some of it doesn't. There's a citation chaser, which Neil developed or was involved in. I'm not sure. Ryan, I use that now quite a lot for screening. If you're a reviewer, you do have to pay for that, but you can get a free trial of the Atlas, another brilliant resource for creating maps. And the SR Accelerator. They have a deduplication tool on there. And they have polyplot for transcribing your search from one platform to another. That's variable, but I have used it. It's pretty good. And have a go. But these will all be on the SR Toolbox, but these are just ones that I've used and I've found useful. Other ones I've used and they haven't done exactly what I wanted to do, but find your own and do use them. And I think that's it. I think I've done it. Oh, yay. Really, really grateful, Alison. I was just really impressed. You mentioned there's lots of things out there to go and learn, but I'm going to be bookmarking this video because I thought it was fantastic. I'm just going to give another copy of the link. So after this recording, the live stream will just close up and it will be immediately available to folks to refer to. Really grateful for a fantastic presentation. We've set up just a simple tweet. And if anybody has sort of additional questions, your tag there, folks can just pop future questions if they have a go with the search exercise that we had earlier and they might have questions. They can pin you over there. So just very, very grateful. Thank you very much, Alison. And thanks to everyone for their fantastic participation and interaction in the chat. Lots of great comments as well. So thank you very much.