 Wel yma, y cyfweld final yn y cesenni yw Tracy Weysgerba yw'r hipfer mewn maen nhw'r gwyllwyfydd yma yn y Bwrlen Institwyr Fyglwyr, ac Tracy yw'r cyflwyng yw'r cyflwyng gwyllwyr rydym ymlaen yw'r cyflwyng mewn cyflwyng maen nhw'r cyflwyng mewn cyflwyng maen nhw'r cyflwyng. Tracy. Rhaid i'n gweithio. Ddweud i'r gweithio. Rwy'n gweithio'n gweithio'n gweithio'r cyflwyng without shortcuts and not to press the self-destruct button on this little remote thingy in here that they gave me. Okay, so taking shortcuts, how can we use methodological shortcut citations responsibly? So before I begin, I think it's really important to point out that there was a lot of hard work done for this project and that work was not done by me. So I teach a participant guided meta research course every year where students from all different fields across the four Berlin universities learn about meta research by working together to design, conduct and publish a meta research study. This was the first year we did it in Berlin and these students all worked very hard on this project and so this is who you should credit for this work. I'll only go through a bit of our data today but we do have a preprint available on bioarchive if you would like more details. So methods are essential to consider when we're talking about reproducibility because reproducibility really starts with our methods. No one can reproduce your work if they don't understand what you did. No one can build on your work if they don't understand how you did it and if you're someone who cares about open data then you can't reuse data that are shared responsibly unless you know how that data were generated, which is your methods. Yet we also know from both our own experiences as researchers as well as from documented evidence that method sections of scientific papers and research articles are often lacking essential details. So one set of evidence for this comes from the reproducibility project in cancer biology where in their efforts to duplicate well-known cancer biology findings they found that no paper contained sufficient details to replicate experiments and they had to go back to authors and not all of them responded. So how do we document our methods in research papers? There are two ways. The first we might describe the method in the method section and the second we might use a methodological shortcut citation. So we're going to focus on the second today. So what is the methodological shortcut citation? Well this is when instead of describing your methods in detail you replace part of your description with a citation to a previous paper that use similar methods and hopefully but maybe not describes those methods in detail. So some context about this use of shortcut citations, some journals actually require this and some scientists consider it to be best practice and they would argue that we don't need to repeat methods that have been described in detail elsewhere. However others have had problems finding detailed methods when shortcut citations are used and I see people nodding in the audience so that includes some of you and have raised concerns that this practice is not great for reproducibility and these problems often look something like this. So here we see a couple of tweets and in both cases we have authors who are looking for a method and they go back and they go back and they go back and maybe there's a paywall or maybe it just says devices were fabricated with conventional methods or a variety of problems in suit and again I see people nodding so I assume some of you have been there and done that. So we decided to look at this across three different fields, neuroscience and yellow biology in blue and psychiatry in red and this is not the number of shortcut citations just the number of all citations in the method section. We looked at papers published in the top 15 journals that published original research for each field, 20 for neuroscience and psychiatry, 15 journals for biology and the first thing we did is just look at how many papers were cited in the methods and you can see that there are some papers that only cite a few but a lot of papers are actually citing quite a few different resources in their methods. So then we had to think about why authors were using citations in the methods because not all citations in the methods are shortcuts and so we developed this way of thinking about why authors are citing things. The first reason is a how citation that's used to explain how a method was performed and this is almost always a shortcut citation so how citations are shortcuts. The second set is a who or what citation and these citations are used to give credit to someone or to specify what materials or resources were used. The third category is from whom so where did I get these data or these material from? The fourth was a why citation and so this is usually used to provide context or justification so we use this method because others have shown that it works best in our patient population or because others have shown that it gives very reliable data under our experimental conditions and then two other categories were to specify a formula or value used in the experiment or other which simply meant it couldn't be categorized in any of the categories above. Okay so why did authors use citations in the method section? Well we went through all of the citations shown in the previous graph and we did our best to define why the authors might have cited and this is what we came up with. In neuroscience and psychiatry about half the citations were how citations or shortcut citations versus a third in biology. The other most common reasons for citing a paper in the method section were who or what citations to give credit or specify what was used and why citations to provide context or a justification for a procedure that was used and then the remaining three categories we didn't see very much of these so what we learned from this is that shortcut citations are very very common in papers and in fact in all three fields we looked at more than 90% of papers had a shortcut. We also looked at the age of the oldest and the youngest papers cited as a shortcut and here we can see that there are a lot of papers that are you know less than five or ten years old but there's also a very long tail and so there are papers that are being cited that are certainly much older than that. Methods age at different rates in different fields but this certainly suggests that some shortcut citations may no longer reflect methods that are in use in today's research. The next thing we did was a case series where we looked at a small number of studies five per field and we went through and we followed backwards to see if we could find a detailed description of the methods and that meant if there was a chain of citations we followed that chain until we thought we found a description of the methods and so there were many cases where shortcut citations were used appropriately and it was fine however we also encountered some problems. The first problem we encountered was that we couldn't identify the citation it didn't seem to exist it didn't map to any known resource or paper that we could find. The second was that we couldn't access the citation so maybe it was behind a pay wall it was an older article where there was no PDF or other version available or it was a book or resource that simply wasn't available to us. We may have had difficulty finding the cited method within that paper so perhaps the method wasn't mentioned or it was a book chapter with many, many pages and we could not find the method in those pages and lastly we may have had difficulties finding the details of the cited method so there was a description of the method there but it was not sufficient to allow someone to reproduce the experiment and then the final problem was that that paper also used a shortcut citation instead of fully describing the method so then we had to go around the entire cycle again which takes time and makes people impatient. So some limitations here. We did a very small sample size so I can tell you that these things can occur but our sample size is too small to tell you how often these problems occur. So what do we do about this? As scientists, as researchers, as scholars how can we use shortcut citations responsibly? We would like to propose three criteria that resources that are being cited specifically as a shortcut to explain a method should meet. The first criteria is that they need to provide a detailed description so they should describe the method in enough detail to allow others to implement that method. The second criteria is fairly obvious. It needs to describe the method that you actually used because if it doesn't describe the method you used you're saying you could do it this way. We didn't but you could if you wanted to. And the third thing is that the citation needs to be open access because if someone can't access that resource then a section of your methods is effectively missing from the paper. So effectively what we are arguing here is that if you're citing a resource as a shortcut it needs to meet a higher standard than resources that you would cite for any other reason and that's because methods are essential to reproducibility and when we cite a shortcut we're replacing a part of our methods so that information needs to be available and it needs to meet these criteria. So you might be thinking what if I have a resource that I really like to cite as a shortcut but it doesn't meet these criteria. I'm so glad that you asked. We would like to propose that in that case you can still cite the resource but cite it in order to give credit to the authors and not as a shortcut. Now that still needs you need to explain your method. So there are a couple of options here. If your method is very simple and can be described in detail in your paper you could simply do this. However it's important to remember that sharing something on a protocol repository can be more effective even for short protocols, a step by step set of instructions is more useful to someone who's implementing and that the repositories also make it easier to find protocols and to version them and fork them. And if you can't describe it in your methods you always have the option of depositing your protocol on a protocol repository and then citing your protocol as a shortcut. Protocols on open access protocol repositories that can be versioned and forked make great shortcut citations and that's exactly what we would like to see. Okay what else do we need to do when we're citing something as a shortcut? There are two things that are really helpful. One, if you modified the methods in the cited resource you want to describe how you modified them. What did you do differently? And if you happen to be citing a protocol then you could also simply version or fork that protocol to show your exact methods. And then the second thing that's really important is to make sure that people can locate that method within the cited resource. So you might need to in your citation provide the method name or the location within the cited resource. If you're citing a book or a manual or a large resource you might want to provide specific page numbers where people can find the information so they don't lose the will to live as they are paging through every page of the book that you have so kindly cited for them. Okay and the final question because I know this will come up during the question period. Some scientists believe that you should cite the first paper to use a method whereas others believe that it's better to cite a recent paper that used methods more similar to your own who's right. Good news everybody's right and you can do both. So these two beliefs simply reflect different reasons for citing a paper. When we're citing the first paper we're doing it to give credit. We want to give credit to the person who developed the methods. And we're citing the more recent paper that matches our methods or making a shortcut. So you just write your sentence so that readers know what's the shortcut and what's the credit citation. So experiments were performed using an updated version shortcut citation of the protocol originally developed by credit citation. Very simple. Okay so that is all I am happy to talk to anyone about this afterwards or later over drinks coffee etc and thank you all very much. Thank you Tracy very much great very clear and interesting talk.