 Hello, my name is Nathaniel Porter. I am a data consultant and data educator in the University Libraries at Virginia Tech in the US. And I'm going to talk about replicability and reproducibility along with one more thing will come upon. So why would we care about replicability and reproducibility? Well, it turns out science and especially squishier sciences like social sciences and education are is easy to mess up even with the best intentions and training hard to evaluate and even harder when you have to reconcile conflicting findings. Not everything fits in a paper available on request really if ever works and that's well documented and sharing not only our results but our processes avoids creating extra work or locking people who are outside of traditional systems out. Ultimately replicability and reproducibility can help avoid mistakes. You as a scholar avoid mistake to assess the credibility of your work and increase its impact. So what's the baseline the baseline is the traditional data and methods section it describes where the data came from what it looks like some basic characteristics, what methods you used, how you handle things like missing data. The ostensible goal is to access the internal consistency and the validity that is does the process test the hypothesis and do the findings support the claims about the hypothesis, but it's hard. So how do we build a better science then. Well, within this space of trying to promote cumulative reliable science and knowledge there are a few different things we can do here. Three of them I'll talk about our reproduction replication and robustness reproduction or reproducibility uses the same data, the same process to find out if you get the same results that is it verifies the integrity of the original process and the findings and that there weren't errors along the way, or things that can't be followed through new data replication, a bigger subset of replication just as reproduction is uses the same processes again, but a different set of data might be a different subset of the original it might be a different time it might be a different context anything like that to see, do the findings generalize beyond the specific sample we originally used, and then robustness checking and studies. We use the same or as close to the original data so different than new data replication, but switch up the methodologies the process is the way missing data is handled the way variables are coded that way indicators are created and things like that to validate that the findings aren't just dependent upon the quirks of one particular set of methods and methodological decisions. All of these are typically applied to quantitative studies, and most often done with those but the logic can generalize the qualitative and mixed method studies as well. So, to give an example. A colleague and I at Virginia Tech attempted to replicate and reproduce a study that found that racial minority group members among library staff report more ethnic harassment than whites among the same group. This doesn't seem like a super surprising finding but we wanted to see does it hold up. So, first, the internal consistency is it consistent no there was a mismatch that the method that was used was meant for count data and the data was not count data and interaction also was claimed to apply to men and women but actually statistically only applied to women. Was it computational reproducible no because the data wasn't shared originally but it was replicable and robust when we use different methods when we collected new data both from libraries and the general population. In this process of reproduction replication and checking robustness we learned how valid those claims were, how far they apply, even though there were problems in the initial study. So, how do you make research replicable and reproducible. These are not all the ways by any means but start by archiving and sharing data, documenting your processes, your analysis completely annotating and sharing code and original output. How do you do that, often the best way to do it is with the open science tools like the open science framework version control computational notebooks things that make it easier to document that to ensure you're sharing it and all of that. And then, as possible share everything during the review process, or by the time findings are published at least with a permalink or a do I in the paper something that can always be gotten to. Even if you haven't if you have 10 year old data, still better late than never to publish. So, hopefully this gives you a broad scale overview to get started there's additional information linked in the slides in multiple places and you can contact me at the listed email address for anything else. Thank you.