 All right. Welcome everyone. We will, I think we can go ahead and get started. There will be some folks that are still coming in, but we want to maximize the amount of time that we have available to talk about for today. I'm very excited. Hi, everyone. My name is Crystal Sultanpole and I have the pleasure of introducing the framework for open and reproducible research training. This is a project that is working to integrate open and reproducible science into higher education. So this, you're going to hear about a lot of really awesome projects today that work to advance research transparency, reproducibility, rigor and ethics through pedagogical reform and metascientific research. This team provides some infrastructure and didactic resources that are designed to recognize and support the teaching and the mentoring of open and reproducible science. I'm going to hand things over to Flavio to get us started. If you have any questions during the course of this webinar, I'll be helping to facilitate that during the question and answer portion afterwards. So free free to either use the Q&A option that's at the bottom of the screen or to use the chat and I'll be happy to help facilitate that after the presentations, but you're going to hear about a number of really awesome initiatives that are that are being done by Ford. So I'm going to go ahead and pass it along. All right, everybody. So I hope everybody can hear me and see me. Yes. So on behalf of the Ford community, I'm delighted to be here today and to have the opportunity to introduce you to Ford and its many incredible projects and team leads that are present here. I would also like to thank the COS, the Center for Open Science and the organizers of this amazing series of workshops and webinars and also, in particular, the brilliant Kristen Stengelball and Katie Corker for having facilitated this event. We will be presenting a series of four projects which collectively aims to show how big team science can and should be applied to open educational resources and meta science and how we have built, we think, a sustainable, sustainable community that aims to integrate open science into higher education and into teaching and research pipelines through pedagogical communities. So let's start from the beginning. What is the problem that we, the collective of Ford is trying to solve? So we think that as the wave of scientific reforms changes, the way that we conceive of research practices and norms globally, higher education has been left behind. And while academics are adopting best practices and higher standards for research quality and accessibility of research outputs, we're still failing to address how we teach, how we mentor and supervise through open science. So it's very common that people go through there on their graduates and graduate studies without ever hearing about open science and new and better methodologies, research integrity, robustness and all of that. So this has the consequence that it undermines permanently to sustainably redress the perverse academic incentives. But it also, we think, detrimentally affects how research evaluations are conducted, which erode research quality. So to address all of these issues, we funded Ford. So what is it, what is Ford? It's an organization that started five years ago. We just had our fifth birthday. It was started at a hackathon at the Society for Improvement of Psychological Science by a psychologist and a political scientist. So social sciences in that way. But ever since we sort of expanded to several other fields of inquiry and disciplines. Now we count with almost 900 folks, mostly early career researchers, and about engaged 500 people in at any given year on our projects. Our website is visited quite often. We have some nice presence in social media and partnerships with major open science organizations. And thankfully, our impact has been showcased at COS here, but also at the UK are and UK real UNESCO, the UK Parliament and NASA. So what are the four main goals? So the first one is to build together with educators a path towards the incremental so step by step adoption of teaching and mentoring practices that that teaches and mentors to gather with open science. We want to educate our students in the respective topics, but also prepare them to be good consumers of science after they leave the education, the higher education, right? For so for it also aims to generate a conversation about the ethics and social impact of education that is focused on openness. On epistemic uncertainty, epistemic plurality, as well as research integrity, robustness and credibility. We also try to respond to four tries to respond to calls to consider open science as inclusive open science, right? So try to build ever larger bridges and an umbrella for folks to be welcomed in. And finally, four tries to promote discussions on the perceived importance of different academic activities. So, for example, how publishing a paper is considered much more between quotes important than providing educational resources or doing great teaching. So we advocate for greater institutional recognition of producing educational resources and quality teaching. Because we're presenting on COS, probably a lot of people is familiar with the Center for Open Science goes towards scientific utopia, which states that we need three main pillars to achieve our Shangri-La of open science. How can we best do our science, which is opening scientific communication, restructuring that those nefarious incentives to promote better practices and crowdsourced science to promote large-scale collaborations to accelerate scientific knowledge, but also accumulate science. And we suggest, essentially, there is a fourth principle to steer open scholarship towards that utopia that is just familiarizing students, the future consumers of science, but also the practitioners, the people who will do science with the intricacies of open scholarship. So, in that effort, we founded FORT and folks will now present the projects that they have been working towards to achieve these goals. So, I pass on to Elena Hartman, who will talk to you about reversals. Thank you, Flamio. Okay. Hey. Yes, I want to present the replications and reversals in the social cognitive and behavioral sciences project that we are doing at FORT. Just a little bit of a background about the problem we're trying to address in this project. So, we know that replications of previous scientific work are at the core of the open scholarship movement, and they should be systematically highlighted in daily research practice. But it can actually be very challenging for both researchers and educators who are teaching them maybe the next generation of researchers to stay up to date with certain effects in or outside their field, whether they replicate or whether they don't. For the more so-called reversal effects, where the effect in the replication is opposed, so reversed, compared to the original study, those are kind of scattered in the literature and they are not really incorporated into research training or student education yet. So, for example, if you think of famous psychological experiments like the Milgram experiment, but also maybe some effects you have never heard of, the Big Brother effect, the Playboy effect, the so-called power-posing or the Marshmallow experiment, many more of those, we can't actually quantify as of now how much replication evidence there is for each of these. At least it would take us quite an amount of time to search for all the replications that have been done. So, for this year to alleviate that problem, we are now building a living crowdsourced curated list of replicated, non-replicated or reversed effects in empirical research across all domains of social cognitive and behavioral sciences. We want to show in an exhaustive way what evidence for or against a certain effect exists so far. And we include metrics such as effect sizes, sample size, number of citations and so on. With this, we want to give educators and scholars a valuable tool so they can keep up to date with prevalent effects in and outside their fields and they can use those in their students' replication projects. For example, when deciding which effects to investigate in a project with students. Here you see an overview of the resource and we have a huge spreadsheet with all the effects, one row per effect. And we have a few things to remember what the effects are doing. We have subterms, we categorize them into disciplines and of course we also note down the contributors to each of these effects. This is what an effect looks like in our Google Doc and also on our website where we constantly update new incoming effects that are added. And you can see that we assign a status that is obviously changing depending on when new studies are added and the overall evidence changes. We cite the original paper with some metrics and then all the critiques or all the studies, meta-analysis reviews who have kind of tried to find the same effect again. And then we list effect sizes. We're currently in a phase where we're trying to expand the resource that is planned until the end of this year and we are already drafting our first output piece for this project. We're also working on developing two or shiny apps that will further help educators to use our resource. So one will be called Replicability Annotator and here you will be able to upload reference list and identify replicated studies and their status. This is in partnership with the Replication Database and Lukas Röseler and Lukas Weierich. The other app, the Replication Explorer, will allow you to explore and visualize replication effects, as you can see here in this mockup, and will allow for investigation of meta-scientific questions using the effect sizes that we note down. So to sum it up, we have currently already over 470 effects spanning 22 disciplines. We're constantly adding more and the Ford website is regularly updated to reflect this growing of the resource. The good news is, we also applied to the Dutch Research Foundation Open Science Fund and our fingers are crossed that we get further funding to continue with this project, especially the development of the apps. And the good news for you is the work is still very much ongoing and new people are invited to join the project and contribute with entries. Since we have a manuscript in progress, which will be about reconceptualizing the many reasons why replications can fail, we'll also need feedback at some point. So these are two ways that you could contribute to the project and this just leaves me with saying that of course it's not of one person doing this, but it's me coordinating everything and a lot of project managers and a lot of contributors helping out with this huge project. So yeah, that just leaves me to thank you for your attention and I will hand it over. Good morning. Good afternoon. Good evening, depending on where you are in the world. My name is Sam Parsons. I'm always excited to talk anything, Forts. And it's my pleasure to be representing the community behind the Forts glossary project today. And of course, thank you to Center for Open Science for hosting the webinar. Really excited to share the project and hopefully have some of you join us. So without further ado, the Forts glossary project is very much an effort to bring together a community approach to crowd source and come to a general consensus on a number of terms, terminology and definitions, descriptions of the kind of language that we use day to day in our conversations around open science and open scholarship. However, a lot of this language can be an interpretable, can be a barrier of itself to those that are newcomers. And that's one of the things that we wanted to support and address. I'll have the link at the bottom of each of these slides to the glossary page on the website. Please do follow it. This will become more important later when I try and get you involved for the project. It's important to note there are many, many people behind this project. We had over 100 contributors and co-authors on our paper to share the glossary project. And we're welcoming to anyone and everyone that wants to support this effort. Now, the goals behind phase one in particular were roughly threefold. So it wants to reduce the barriers, as I said, to enter these discussions. One of the other aims was to kind of broaden the terminology used. Often we talk about kind of quote-unquote open science and that can be somewhat limiting. It can be very STEM focused. And it can be very exclusive towards, for example, the social sciences and humanities. So really opening up that approach was part of the aim. Now, not only in terms of broadening the kind of field-specific focus, but also to fully reflect the open in open scholarship. So therefore, there are a good number of diversity, equity, inclusivity terms and a lot of definitions and descriptions around social justice concepts that incorporate it in this because they are really essential, I think, to the mission behind open scholarship and thought. And thirdly, to provide a resource for students, for instructors to, again, reduce these barriers, make things easier to enter into these discussions. Phase one has been a fair success to give you a quick overview of the website itself. The Glossary Project, you can find it on the Fort website. We have a bunch of terms down the side. Somewhere in the 300-ish, I think, is live on the website. If not more, we have some basic information about the terms, but to give you an example of one of my favorite ones, Open Peer Review. Now, this is one of those terms that can be used in several different ways and often is itself leading to confusion in the discussions. If one means sharing the review materials versus signing reviews, now, that will lead to some very different discussions and it's really easy to talk past each other. So having that kind of information is exceedingly useful, in my opinion. Now, in phase two, we're interested in expanding the glossary to include more terms, more descriptions that will be useful to expand the reach, as well as improving and correcting any terms that may be in there that could be better. Perhaps most excitingly, to expand the accessibility, kind of interoperability of the glossary, we have a number of teams that are working on translating the glossary, at least sections of the glossary, into other languages so that we are trying to be a bit less, we're trying to increase the potential for diversity and accessibility, essentially. Now, these teams are in progress. We have them on our Slack channel. You can find them by joining that, as I'm sure you'll get a lot of information saying join our Slack channels throughout today. Please go there, follow the glossary. If you're interested in seeing or using these translations, follow the documentation and it will be there. Now, this leads us to the plan of action for this kind of phase two. In my ideal world, let's say somewhere before the start of the next academic year, it'd be really nice to have a couple of these translations live on the website, and it would be amazing if by the end of the year, we can update the glossary so we have the 2.0 version with all of the additional terms and a bunch more translations. That's kind of my ideal world situation. And of course, if you join us, that can become much, much more of a reality and it would be amazing to have you on board. To give a quick overview on the webpage itself, on the glossary page, as you can see, all you need to do is scroll down, hit phase two of the glossary. The document itself has all of the information you could possibly need to get involved in the project. Follow that, join us, ask any questions on the Slack. It's a very welcoming group that are involved. We'd love to have you. And thank you. We'd love to have you involved. Thank you very much. Thanks again to Open Science and the rest of the Fort's community for this. Speak soon. I hope everybody can hear and see me. Thanks for that. That was amazing. So I'm going to continue the passion for all things Open Scholarship. My name is Tom. I'm going to talk to you about the Fort's Summary's project. So another very exciting moment. And I guess the issue that's promoted this project is the fact that we're all very busy people and it's increasingly hard to keep up with what on Earth Open Scholarship is doing. Twitter every day seems to have a different controversial take on all sorts of practices. But also we've revised so many of the things that we thought were well established and assumed to be the norm. We're changing these all the time. And so we have a job as academics, as educators, as researchers to try and keep up with this. And that's a pretty big challenge. So the point of this is to try and find a way to ensure people can keep up to date with the Open Scholarship literature. And subsequently be able to use those practices, those ideas to inform what they're doing. And this is where the summary project comes up. So the aim here is to digest the literature and hopefully just facilitate people's understanding and adoption of Open Scholarship practices. So it is, as you might have already suggestedly guessed, we like a good spreadsheet and we like a big document. And here we've got over 200 attempts to summarize various papers, blog posts, sources of information or evidence on Open Scholarship. So this includes everything from opinion articles, empirical articles, literature reviews. And this is an example. You can obviously see them whenever you wish. Fort.org slash summaries. And you can see in addition to providing some basic information, including things like the abstract, how you might cite the paper. It also includes a summary of all the core details of the paper. So you can see under main takeaways, there's a few bullet points trying to summarize the article. There's some interesting quotes. We also do lots of signposting. So in this case, the first thing you might see is it signposts you to some other papers on similar themes you might find interesting. And again, this is an attempt to try and encourage people's incremental growth and understanding in Open Scholarship. What we do beyond that is we also map these summaries onto what's known as the Fort clusters, which I'm sure Flavio can tell you a little bit more later about, but in essence, what we try to do is group things that make sense together and then put these summaries tagged with these clusters so we can easily negotiate the same or look for papers covering the same sorts of issues. So for example, if you're interested in replication research, you can see this one. You can see talks about things like registered replication reports, large-scale replication attempts, purposes of replication. And you can find these summaries categorized based upon these clusters in an attempt. Not perfectly, but nothing really is perfectly fair, findable, accessible, interoperable, reusable, but that is our attempt to help people navigate and negotiate. The one thing I really like about this project is that in addition to acknowledging that there is a lot of stuff on Open Scholarship that's hard to keep an eye on, there is also so much in what Open Scholarship does in relation to EDI, social justice, accessibility. And so what we have covered is not only papers solely about Open Scholarship, but also those that relate to diversity, equity, inclusion and accessibility. And so this is an opportunity to place these a little bit higher on people's awareness in the hope that we can promote some positive social movements. You might think, gosh, that's a big effort, and it indeed has been. It's always a team effort with thoughts. And for each summary, it's drafted by one person. It's independently reviewed by at least two other people who then tend to make amendments, provide feedback, and then it's categorized. I think I've categorized a few hundred. It feels like of these according to thought clusters, so you can track them. And what we really need is you. We would love for you to get involved with team summaries. So revising existing summaries is really important to ensure we're really capturing what the true essence of the papers are, also with categorizing summaries. And we are working on a few things in the distance that we'd like you to keep an eye on. So we're putting together a paper describing summaries, trying to encourage people to use and engage with them. We're creating a strategy for how we can more widely disseminate these exciting, exciting overviews. So that is it for thought summaries. A really, really interesting project, and I hope I'll see some of you on the call today joining us to chip in and join. So I'll hand over to the next amazing project, and that's led by Magda, who's going to be talking about neurodiversity. Over to you. Thank you, Tom. Okay, so my name is Magda and I grow. And I'm going to tell you a few words about the work we do within team neurodiversity. So just very briefly, neurodiversity refers to the naturally occurring so non pathological variation in the human brain regarding social, cognitive and emotional functions. And some common examples of neurodivergence include dyslexia. Autism. So in our group, we have both neurotypical and non neurotypical members. The group was founded in 2021 by Mahmoud Tamara and Amelie. At the moment we have around 17 members. Not all members are active at the same time. Most of us are early career academics or students. We have a rotating leadership structure. And some of the projects that I wanted to tell you about that we have been working on recently include neurodiversify your curriculum. This is the first one that I would like to talk about because I think it's the most typical here. So basically team neurodiversity has developed 10 lessons which include lesson plans and interactive materials. We have developed them thinking about psychosis students, but they can really be used by any students and lecturers in higher education because they focus on developing academic skills and research skills. So we have done this to promote open scholarship and neurodiversity in academia and include neurodiversity affirming pedagogies. We wanted to raise awareness of some of the most recent debates relating to science. Invite students to engage in epistemological discussions and to reflect on the core values of open science. So to the left you've got a screenshot of what the research looks like and you can see a number of lessons, the lesson itself as well as lesson plan that can be used by the lecturer. So this is an example, a lesson which is on implicit bias and normative science. Here students are invited to read a text and recognize implicit bias in the academic text, so to reflect on it a bit more critically. We then move on to talking about deficit approaches and how they may stem from such language. And invite students to think about the ablest language that is used in scientific publications, especially on neurodiversity. So the project aims at raising awareness and the pathologizing neurodiversity, addressing the double empathy problem, so the problems with communication between different neuro types. It focuses on open science and participatory research methods and promotes universal design for learning in higher education. I am just very quickly going to go through some other projects that we're working on at now. Some of the main ones are database of neurodivergent researchers. So we invite academics who represent non-typical neuro types to become part of our database in order to promote neurodivergent researchers in the academia and role models for students. We also have academic privilege which is going to be in the moment presented by Sarah. We also present a neurodiversity reading list which is a starting point for anyone who would like to learn more about neurodiversity. We have written a position statement where we explain how important embracing neurodiversity in open scholarship is and we look at experience of ADHD years and other neuro types in the academia. Finally we look at ableism and challenging it and we do that by promoting participatory approaches and look at open scholarship and autism research. The last part is thinking about education, thinking about how we assist students and how we could make our assessment more inclusive as well as look at open scholarship and autism education. So anyone can join us in response to the neuro type and we have quite a few projects that we are working on at the moment. So there is plenty of scope for plenty of help needed. So I do invite you to come and talk to us. And this is it from me. Sarah is going to tell you more about one of our team neurodiversity project which is Will of Privilege. Thank you. Thank you. Is that working? So I am one of the co-leaders for the academic Will of Privilege and I am going to be sharing the backgrounds, developments, how you can get involved and wait until the end because there is a little challenge for you to take away. I have color coded each section so it is clear which section I am speaking about. I am going to start with another question. So the academic Will of Privilege, the origins are from our position statement from team neurodiversity which was about neurodiversity and open scholarship. And we were having all the discussions throughout the project and then towards the end we were wrapping up things like how do we organize the project and how do we organize the project. With authorship it is a key determinant of academic career progression from the number of publications, citations, authorship positions and they all helpfully influence the career trajectory and it has influence on hiring, funding, networking opportunities and even who we celebrate so we have that in mind. And also like the wider context of academia we are looking at the barriers faced by scholars and academia. So with those two things in mind we co-created the academic Will of Privilege and this is the first version of it which you can find in our pre-print and we have seven broad categories and identities and experiences which are relevant to academia and they are based on published evidence, expertise, our personal experiences and public consultation. And we used this as a way to, so we decided that all the authors would be joint authors and then we used this Will of Privilege to determine the author list. So we used a scoring system where there is more privilege as you go towards the centre so there are three levels and then we summed our scores and we also weighted each of the identity categories equally and then we used those scores to then decide on the authorship order. And that was a really interesting deep reflexive exercise for us. And these are just some of the sort of things that we discussed as authors from the authorship team. And someone said, oh, this could be actually pretty big because it was actually a supplementary figure in this position statement but it went sort of semi-viral on Twitter and LinkedIn and it's been used as a resource at the UK Research Integrity Office, local councils and then we were talking about the authorship project. So that's where it sort of comes from. And we're also sort of looking at the time like what is out there, what current authorship frameworks are there and we're all very familiar with credit. There's other ones as a new one with like merit. And there's different sort of frameworks about authorship, order, and just generally authorship. But these frameworks, while they're good at sort of building transparency and approved for this, these authorship policies do not embed justice, equity, diversity, and inclusion. So this is where the academic world privilege comes in and I would summarise it as a holistic equity-based authorship framework to examine social position academia on this sort of three strands to it. It's intersectional in the sense that we look at many different types of social cultural identities. And we also have an important part which is about that it's reflexive, that you think about the process of doing, using this framework and thinking about your sort of social position and what that means within yourself and also with the team that you work with. So in terms of developing the framework, we took on board a lot of the feedback from the students, the staff, the staff, so in terms of developing the framework, we took on board a lot of the feedback from social media, emails and various things. And then we've developed the shiny app, which makes it easier than just sort of looking at the figure and deciding where the shiny app allows users to customise the different identities. We'll have weightings, you can remove categories, add them and find out more about the categories. We're also creating a user guide to help navigate this process. And we also have some educational materials through a syllabus to extend this learning on these topics. So the current stage, we are finalising the manuscripts, the syllabus and the shiny app and we hope to pre-print soon. I won't give a definite date when, but it will be soon so have your eyes peeled. And in terms of net steps and once the pre-print is out, like those of you who are watching, how you can get involved, we'd love for you to try it out, reflect on the process, what things you found challenging or those sort of things in the process. And then also explore the educational materials that we provide, which will be in the supplementary. Also, we love, love, love feedback. So sending us questions or things that you found easy, hard, challenging, inaccessible, accessible, etc. We'd love to hear feedback. We'd also love for you to share it within your networks, on social media, with your colleagues. And also to discuss it within your academic sphere of influence with your colleagues, if you're an editor, whatever sort of direct position that you have. And we're also keen to get support for the students who are keen to get support for the syllabus and to extend these educational materials and crowd source to further extend the learning. And also we'd love for you to join our Slack channel for this project and details will be in the next slide. And now the challenge is to find 20 minutes this week, reflect on your social location and academic, so using the privilege and you can access it by this QR code. And then to ask yourself the question, which areas do I hold the most privilege and how does that make me feel? And with that, I will stop there and I'll go on to, it'll be Julia Volska, who will talk about landscapes. Okay, thank you for listening to my presentation about mapping the landscape of open scholarship literature across disciplines. No, I just need to find out how to. So the contents of this presentation are to talk a bit about the project motivation first and the project aims and then talk about the project plan and the community efforts. So the motivation for this project was to be able to understand the motivation for this project was because there's a problem of a solitary and fragmented nature of research. And often we don't really know what for other research is published and our field or like open science, open scholarship generally doesn't necessarily have to be in our field. And then there are information barriers that hinder the progress of science and also how to develop and develop our own science better. So basically that leads to the problem of reinventing the wheel and to a waste of resources. So there might be also some biased or misleading knowledge and there might be some lack of empirical evidence on consensus but also on the disagreement what might be open scholarship and what might not be classified as open scholarship. And then we've probably all heard of the general jungle fallacy so that two things that are actually different are considered as being the same or otherwise the two things that are being the same are considered as being different. So the aims of this project a project where to create and curate a database of literature about open scholarship across different disciplines. So I think it's probably fair to say that most people from the thought community are psychologists but not all of them are. Some people are different disciplines and open science and open scholarship exists across different disciplines and science and also in fields that might not be traditionally viewed as science and by creating this database we then hope this will allow for an easy and customizable query. So in the project we review and synthesize the literature that exists to map the landscape of this literature. So we aim to reveal the current consensus and also the disagreement and then hope to inform future efforts and discussions and we hope this will facilitate interdisciplinary knowledge exchange and collaboration because we can learn about open scholarship when reading literature on our own discipline but also when reading about something in different disciplines. So this is basically the project plan. So we aim to build a database and then review the literature. So we are almost at the end of phase one now and in this phase we build the base literature using a web of science as a search tool and then perform some bibliometric analysis to synthesize this literature and this phase is then our initial step towards building an interdisciplinary database of open scholarship literature and to map the landscape of this literature and then depending on the scope of this analysis we may then follow up with an in-depth narrative review and other complementary analysis. So this is basically the plan for phase one and we are almost at the end of the screening and then we will move on to phase two afterwards where we will then look for some disagreement because basically there are two screeners for each article and maybe so if I move on to the slide so basically there are some screener names whoever screens the articles then they put in their email and their affiliation and they have some screener ideas which are odd or even in different colors and basically then they blind code blind screen the different articles and so we look whether two screeners agree on one article or not and we have identified more than 2,000 articles and people but it's not clear whether all of those articles will be included and the final output or not because people look at these articles they look at the keywords and the abstract and if necessary they can look at the full text and then they see whether the article deals with open science open scholarship enough to be included or excluded so basically people say uncertain and particularly when they say uncertain then they can make some notes and say why they are uncertain and if there's like some disagreement if one screener would include an article and the other one would exclude it then and the second phase where we will move on to soon now then some other independent screener would then make a final decision so basically we are almost at the end of the phase one but moving on to phase two soon and always look for people who are happy to be involved and yeah obviously I also didn't do the project on my own there are some other community project managers involved so I would like to thank them and yeah and also thank all the people who already screened the article and if you want to join us and stay updated then as been mentioned in the presentations before you can join Fort Slag and our channel is called Team Literature Landscape and you can screen as many as little articles as you want and yeah so there are some credits to be considered to be an author but yeah we can talk about this in more detail if you want to so yeah thank you for listening and I will hand over to Max now okay thanks right so I'm going to talk a little bit about paper we recently finished and got just accepted and this is about the positive changes which came with the replication crisis and just to come back a little bit to my experience as a student there was a bit like a rabbit hole which I dived down when learning about the replication crisis and I was quite interested in this so it was not a problem for me I really liked the topic and I was digging down reading and listening to podcasts and really engaging with the topic because I knew this was important but I guess this is not the same for all people and it might especially be a little bit discouraging for some people hearing about all the things that have gone wrong in the field and at the same time of course outlining errors is really important maybe one of the most important things but I think the next step we need to learn from our past mistakes so this is why I thought yes, those positive changes that have actually happened in the recent years within the context of the replication crisis need to be acknowledged and also encouraged and then we need to build upon those so and just to underline this one point a little bit most that the focus was a lot on those on outlining the problem so here are two of the updated papers within the context of the replication crisis and you see an incredible and numbers of citations and those yes they outline the problem and now it would be great to move towards solving the problem and Simeon Visier has already coined a great term for this instead of like looking at the replication crisis as this crisis she said that it would probably be better than we can see as an opportunity to change and those key points of the credibility revolution are that there's a greater emphasis now on transparency and openness pre-registrations and direct replication studies but also that they are generally higher standards for the quality and I guess also to some degree the quantity of evidence needed to make some claims so so in other words there is already change happening because that term Simeon mentioned there is just not just an invention but really describing a lot of the changes that have happened but what is still or was still missing was a resource which was actually structuring and presenting the developments which are already there because it can be sort of cumbersome to dig through a lot of literature which is also not necessarily right away accessible and to even just get an overview so this is why we came up with a way of just structuring the changes which have happened and there are some general trends we noticed so there is an increased focus on working in teams for example so this is what we categorized as a community level change just as thought and there are many many different procedural level changes which each one of us can implement mainly focusing on methods and statistical procedures but also some statistical tools and then on the structural level there are a lot of policies and incentives which are great actually so just to give a little overview of the paper now so again it's supposed to be a resource which makes it easier to access this topic of the replication crisis and what to do with it and it can hence for example be used by students or educators and we provide a short introduction to the replication crisis and then give an overview of all those changes I just mentioned that you see in the figure you will also give a little outlook of what can come next and so great we have a paper but much more important is that it's actually being used so this is why we are now starting to prepare some educational resources and so first we want to give or prepare a short presentation which can give an overview so this can for example be used to present at your initiative or your reproducibility journal club and then when we're done with that then we really want to move to to create syllabi or one syllabus at least to facilitate the teaching of a more positive or constructive view of the replication crisis which will consist of a series of lectures and come with slides and scripts and yeah there's I will share the slides and I'm sure I can share also the link to the paper and this is all really in the beginning so I would really appreciate anybody who could help us with this so please come and join us now we said a lot of times Slack is the place to be and otherwise you can email me or Flavio or I guess so yes this was everything from my side and I just want to thank everybody who worked on this project it was really a fantastic effort and experience to work in Ford and I can only recommend it and now I can hand it over to Leticia who is talking about pedagogies Thank you Max so I will briefly introduce you to another Ford project which is pedagogies and as you hear heard many times today already like one of the major goals of Ford is to help people or facilitate the process of scholars and integrating open scholarship into education and mentoring and in their own research and we know it's not an easy task to do so what we try to do with pedagogies is that it should be we want that facilitated process that we actually learn from other people who are already doing an excellent job at this so what we do is we have a lot of exemplary instances of principal education in teaching or mentoring of open and reproducible principles and principal education is basically education that combines the content of any course or of any discipline with open scholarship tenants or principles and this can include a variety of things right it can be experiences of adapting courses to incorporate open materials or even perspectives on the social network responsibilities that we have as educators of teaching open scholarship pedagogies have two very important elements to it the first one is that every we actually contact educators who are doing an excellent job into integrating open scholarship into their mentoring into their teaching so that we can incorporate open scholarship into their teaching into the course materials so basically this is you can see here on the left side of the screen a pedagogy that we did with Julie's trend on a course that she taught on credibility revolution and everything is open so like we can easily incorporate or adapt into our own teaching so the syllabus so we actually conduct an interview with this educators to learn their experiences how hard it was to do this what were the experiences what is the advice to people trying to incorporate open scholarship into their teaching and mentoring what was the perspective of students learning about open scholarship and et cetera the process of the pedagogies is all community driven and we ask people who do you think is doing an amazing job in incorporating open scholarship into teaching and mentoring we follow up on the suggestions contact this scholars and educators in varied fields and after we contact them we announce that the next pedagogy will be with this person and then again we go back to the community and we ask the community for an interview with them and then we conduct we connect with this educator again like to share their materials online on Ford's website but also to conduct an interview as I mentioned to learn from them all the behind the scenes process and as the last step we actually put everything in the form of a blog post on Ford's website so you can have like different sources the and check it out on Ford's web page the pedagogies page we have one pedagogy with Julie's trend on her course on the credibility revolution where she shared the insights what the students thought about this course how she set it up and et cetera our second pedagogies also extremely nice with Gillard Feldman on the variety resources open science resources that involving students of his courses in doing replications and coming soon we did a pedagogies with the crepe and we had Jordan and two other students actually talking about their experiences in doing replications and this will be coming soon so stay tuned now that you know about the project I will quickly tell you you might be thinking it matters for three important reasons first as I mentioned is the aim is to facilitate implementation of open scholarship into teaching mentoring so all resources are open and you can learn from people who already experienced in doing that and you can borrow their materials and you can use and adapt to their materials the second key component component to the pedagogies and the third component is to foster social justice and in the hope that in the long term we can encourage the democratization of access to open education now again as in any project we are always looking for collaborating for collaborators and you can help in a variety of ways so you can help by suggesting educators in your own field who we can invite to do a pedagogies with when we kind of share in our social media channels that we are doing the pedagogies with this next educator you can share with us the questions that you would like to ask this person and of course you can also join team pedagogies and you will be very much welcomed we are a small team and I would like to also thank these three people Julia Volska who is here Georgia and Jacob who have been key and I will hand back to you Flavio all right everybody I hope you will see my screen soon and I will sorry because we have like five minutes I will be very quick but essentially going through a few initiatives at Fort that we think you should know about so we have the Fort lesson plans and essentially to support educators aiming to bridge that you know research to teaching gap we thought that it would be a good idea to produce lesson plans that everybody can use and can adapt to their own context so we composed nine evidence based lesson plans we are now updating our website to 21 total and we also made a shorter lesson plan so things between five minutes you can apply at any moment in your class that is sort of the contextualized in a way and now we are also updating into 65 very soon and we wrote an article to talk to you to tell you about what are these lesson plans so on the topics thinking of interpreting effect sizes register reports how to be critical not cynical understanding what dodgy papers are but we also talked about open science so you can see those at our URL for.org slash lesson plans and we are soon going to update another paper that we think you should know is that we made a review that has been just published at the Royal Society for open science looking at the impact of teaching and mentoring open science on students right so we use a team science that is going to synthesize evidence that investigates what happens to students when you teach a very open science so think we divided into three clusters so scientific literacy so think of knowledge skills competencies all that beautiful thing. Scientific engagement which is enjoyment of learning motivation future behaviors with science and attitudes which is just perceptions of science trusting science but essentially is that we need more evidence but there's some cool works in progress that we are aware of that will be able to answer several questions we have this initiative called for curated resources in which we pulled together the knowledge of several members of our community everybody can contribute online as well and we gathered almost 900 resources so you go to .org slash resources and you can use this search bar here to search any full word so about pVed is reproducibility broken science what is replicability any sort of word as long as it's whole and we will search resources for you and what is really great is that we collaborate with the center for open science and we are now together with the open scholarship knowledge to increase our resources to about 30 to 80 30 to 20% but it only means that we are just trying to increase redundancies and work together for your benefit there's also a fourth glossary that Sam Parsons couldn't be here today but we have a video so for those of you at home you will see fourth glossary initiative be edited into this but essentially we devised a useful research term so anyone can learn about what it is these new words that we are developing this is how it looks on the website there's phases there's translations we're expanding it as well so please check it out that video and we also have the fourth clusters which Tom talked about which is essentially a summary of how you can conceive of clustering or understand open science and we are now partnering up with Koala Lab with the open qualitative research lab which will together with us include an additional cluster on open science and qualitative research which is really really cool and we're going to also add research integrity cluster and a few other things we're just reimagining open science with a lot of resources so you can use it to compose that's what the community really thinks is the way forward all right with that I'll let you be and this for today I would like to thank everybody for their amazing presentations at Ford I'd like to thank Kristen Stento Paul for giving this amazing opportunity we are also very happy and I hope I didn't cross the one hour limit I just want to say thank you I think we do have time for one question that hasn't been answered thanks for folks that have been popping into the Q&A just really quickly somebody asked about the possibility of integrations into Zotero or other open bibliotools I don't know if Fort folks have talked about using Zotero at all for some of this work we do have a Zotero Fort library where we put in the bibliography that we used in our papers but when it comes to integration it's not yet but this is something that could be super interesting and we are just lacking the expertise and people power to make this a reality awesome if there's somebody able to quickly grab that Zotero link possibly and pop it into the chat seems like there are a couple folks that are interested in it otherwise I think all of the questions have been answered either through the Q&A session or anything else I've posted a couple of times the links to join the community and also the website and Twitter and stuff so if folks have questions they can contact anyone at Fort thank you all for coming