 Welcome everybody and thanks for being here. And so we're going to do a deep dive into preprints and open access. I'm Brian Cook, I'm at the University of Virginia, and I am the co chair of the moderation committee for at archive and I've been involved in with at archive which is a preprint server for education so I'll talk a little bit about that and we'll talk about other issues related to preprints and and open access. And I'll just, before I forget, throw out a quick, even though we can't see your faces, I think you can make comments or raise hands and things that we'd encourage you to chime in. If you have questions, comments, you know, would like us to expand on on something that we want to make this as relevant as possible for you so so feel free at any time to bump in and give us some input on. I want to try the chat and sort of interject but also say hi I'm Stacy. I'm a assistant faculty member and learning sciences and psychology at Worcester Polytechnic Institute. I'm also part of the ed archive steering committee and think my role is like liaison at the center of open science so I'm relatively new to the land of preprints, but I am a huge fan of open access and so I'm really excited to be here today with Brian and we're going to go over a couple of different things and sort of trade off talking, but please let us know if you have questions want us to talk about something in more detail or more depth. And if I think if we don't have the answer to your question we can always try and follow up with you later. But let me just, I can go ahead and start the side deck so today's agenda. I'm going to start us off talking about open access, but that is the rationale, publishing open access and some of these open access FAQs. I'm going to talk about preprints talking about the rationale benefits limitations and then like Brian said we'll, we'll sort of walk you through at archive. So just to sort of jump us off open access open access the rationale for this is several lines of thinking really. The idea here is that you want to make your work open and available to the public. You work hardest researchers, or writers or whatever it is right you want to make your work known and disseminated as broadly as possible, open access is a way to help do that. So by adopting open access promote scientific literacy I was just talking to someone in another session who said that they were in Ethiopia and no one has access to scientific literature over there because there's no library access. The publishing model usually makes it really hard to buy articles or buy access and so by sort of adopting open access practices you can make your, your work more widely available promoting scientific literacy. So people think this is the ethical way to disseminate work. And what I really mean by that is that researchers don't get paid to publish and reviewers don't get paid to review. Also, taxpayers largely front the bill for the research we do from our governments, or other sort of organizations and so it's kind of unfair to the taxpayer that their tax dollars go to funding us our work. They sort of, you know, are in the process of creating not knowledge but then they are charged again to access that knowledge so a lot of times when we think about open access or really thinking it's really ethical dissemination right we don't want to make people pay twice. Brian, can you explain what these various levels me. I think this came out of PVIVAR and I'm guessing I'm saying that right but I may not be. Did a, and colleagues did a very large review of open access publishing across different disciplines in science in 2018. Sometimes there's an additional category but these are kind of the traditional categories of open access. So someone would posit a kind of top category that sometimes called diamond or platinum open access and that way abbreviation is open access. So I'll start with gold and then go back up. So, gold is when a an entire journal is open access. So within the field of education for example, there is open AER a, and everything on that journal is is freely accessible by everyone if you can get on the internet. So there's, there's really no option to it if you're going to publish there it's going to be open access which sounds wonderful. Great I'll publish open access then. Well, open access is still. And so instead of having people pay to access the, the publications because they're behind a pay wall and people need to have a subscription or pay individually for them. They charge the authors in a PC or an article processing charge. And those range very broadly. I did a study looking at in my research, I did a study looking at the authors in APC or an article processing charge. And those range very broadly. I did a study looking at in my field of special education. We looked at journal policies and the APCs varied. They tended to be around $3,000 per article. So they are not trivial in in the news recently. Nature, which is owned by Springer has now they're developing some, some gold option journal options, and they are charging 9500 euros or, I think a little over $11,000 per article, which is raised a whole other issue or demand. 100 euros or, I think a little over $11,000 per article, which is raised a whole other issue or dimension of open access publishing so that ethical issue that Stacy talked about that we want to make our, our scholarship and our research as accessible and open as possible in terms of dissemination. But does this then limit who can publish in certain outlets and these gold outlets by who can afford to pay APCs and we'll come back to that a little bit later. Above it isn't on here but I want to mention the diamond or platinum open access is when journals do not charge authors so things are the publications are open access, but there's no APC for them. There's no article processing charge. Those tend to be small scale journals that are run maybe by a professional organization, and they just have volunteers running a website and they're more just kind of posting PDFs and and probably have a just a volunteer set of reviewers and editorial board members. And so those do exist they tend not to be very large high profile journals, but they certainly do exist. We'll talk about green, which is essentially anything that self archived so pre prints are an example of this, and there's different options that as authors, we can post versions of our own work on different sites and different locations on the web. Bronze is is actually, according to Pivovar's work in 2018 was the most frequently used type of open access. And so bronze is the work isn't licensed as open access. It is something that the journal or the publisher decides to make open access on their own volition. And this could be for kind hearted reasons because they want to make this a good research and scholarship available to the world. But sometimes it is probably because there's a special issue that they want to drive traffic to their journal website, or for whatever reason, but they decide to make things certain things open access on their on their website, and that may stay open access for a long time. It could change tomorrow and be back behind a paywall. So bronze open access gives us open access, but it isn't licensed which isn't then it's not clear about how you can use that in in different ways, whether you can freely use that. And it may cease to be open access very soon. Hybrid is becoming more and more popular, whereas a gold open access journal is the entire journal is is open access hybrid is where authors have the option and I see this more and more frequently. When something gets accepted for publications, they ask would you like to make this open access, and I would but then it also says well then you can do that you can make your article open access. Other things in the journal probably won't be in that issue or the journal generally, but you have to pay that APC then that article processing charge and again that tends to be in our field about $3,000. But it varies by journal and by publisher, but this is becoming an option more and more and some of the higher impact high prestige journals and it can be more expensive, and then closed is the journal is probably predominant, roughly probably 50% in in pivovar survey across disciplines about 50% of published articles were just not available, unless you were affiliated with an institution that that's paid for a contract, or that you choose to pay access to an article, and they're getting quite clear the publishers are getting quite creative about how to do that I see more and more like for 24 hours of access only $25 or something. Which is, I think is as Stacy mentioned is just a real shame that we have large oftentimes publicly funded research that we can't access without paying for. Oh, and so here is I've referred to it a couple times. This is a figure from the pivovar review. And so this shows it is, it's getting better there's more and more of these different options being done and this is a few years old now I would suspect that these numbers are continuing in the same direction, but even in the more recent years we see the majority of research is still closed with at any given time, there's a fair amount of material that is bronze access, some hybrid more and more gold, and I would imagine green is increasing in the most recent years more and more to So a lot of people have questions about open access because we want to make our work available, but like Brian just sort of laid out like it's complicated right like a lot of times to get that you know your work out if you go through a specific journal, you are going to have to pay an article processing charge 3000 to $11,000. The lowest I've ever seen I think is 700. So a lot of people try and find workarounds or ways that they can sort of identify the journals that are the most encouraging of open access and so one of the most frequently asked questions is how do you find these open access journals and articles. We all want the diamond one right like we all want the one that's not going to charge us it's not going to charge the reader it's just going to make it available but sometimes that's not always an option so we want to see what our options are. So, one really helpful resource here is the directory of open access journals, many of you probably heard about this. But what this is is it's a directory of open access journals you go in you can search by subject, or title. There's a lot of different filters you can put on and off like, you know does the journal require an article processing charge does it not. And this is a really great way to sort of find where you can publish your work in a way that makes it the most open access as possible. Another question we usually get is. Oh wait, Brian did you add this one. I did I snuck in a slide on you. This is a kind of the counterpart to looking for open access journals. This is looking for specific articles that may be open access. And so this is something developed, and they, they just search a whole bunch of different repositories for you. And so if you have an article name or a DOI, you just put it in and they search it for you. If they can't find it anywhere they actually have something that they'll set up you can email or you can contact the author. I'm assuming through the authors contact information from the article to to request the article from them, which is always an option which is a very good option. If you have on an individual basis, you can send your article formatted your PDF of the formatted article to people individually. Oftentimes though, you've signed the copyright with a journal you can't do that and just posted anywhere on the internet, but you can send it individually to folks so that's always an option, but I wasn't aware of this open access button until recently. I believe it is not 100% exhaustive. So there are things that didn't come back on the open access button that I've done searches, just through Google, and if I if I look around enough I sometimes I can find a version of it around to. And then is Sherry points out there's also the unpaywall option that that's out there. Yeah, so just to clarify Brian is this open access button to plug in as well to believe it is yeah okay. I can't wait to play with it. Yeah, but like as Brian mentioned, one thing that's really helpful. And I think if you're an instructor or deal with any students to recommend is that they can't access an article, don't ever pay. Try and email the author or look at the author's website a lot of times I found that works really well. Oh, thanks for sharing. Okay. Okay, so other questions people asked is their funding for open access author fees right these APCs can be a lot of money what sort of funding is around for them. Usually, pretty limited. So a lot of times you can write them into grants if you're trying to get funding for your research you can say I need X amount of money for article processing charges to make this open and available. Sometimes universities have funds specifically for this, but not all universities. Some journals have reduced APCs for students and authors based in countries classified as low income economies. There's a couple other sort of nuances Brian I don't know if you want to add anything to this. I don't, I don't know off the top of my head. But with the, a lot of the APCs have kind of a graduated that they'll have adjusted rates, depending on kind of career status and country that that you're that the university is based in so sometimes the APCs aren't as as expensive. But for journals that that require APCs to to make them open that is generally an unfortunate requirement there. There he is I think a growing back to the the APCs and as they seem to be growing higher and higher. It is almost just a transferring of the cost and we don't have time to get into it and I'll get all upset about it too but when you look at the, you look at the profit margins of the publishers that it's just remarkable. So, the people to charge the people that are reviewing the articles that are writing the articles that are conducting the articles, the to then charge them to publish it. It's a much bigger picture that that we're not going to go into here, but there, there is, I think a movement to kind of just blow up the system, and, and think about doing publishing in a very different way that is separate from the kind of traditional journal structure that that's been around for for quite a while now but may not be necessary. I wish we had more time to talk about that because there have been some new interesting models like collaboration psychology charging at cost but leaving APCs for like graduate researchers or early career paying reviewers like there's a lot of new models sort of happening around but I think for by and large, most of the time when you're talking about open access someone's going to try and charge you an APC. Okay, let's just pretend. You can't afford the APC, but this is the best journal for your work. Go ahead and publish in them but how can you still, even if you can afford to make it open access for everyone how can you still sort of make your work more open access for people right so if you can't do it through the journal through the cost there are other work around so let's talk about some of those. I think it's really helpful is research gate or academia.edu, whatever you you fancy in terms of research sharing. A lot of times you can post articles under your profile as a researcher and store the text privately. So, like one journal I had said you can send 50 free copies or whatever to people or if it's for teaching or instruction purposes like it's unlimited, and I would just hold a sort of a private text and then every time someone would hit that request button boom I'd approve it and it would shoot them out a copy of the article so that's, that's one way to sort of help make your work more open. Another is you can check Sherpa Romeo, which is a wonderful website that will basically allow you to look up the journal you're publishing in and see what you can post. And we're going to talk about pre prints in just a second but this is a great website to see sort of what can I get away with legally in terms of posting the work that I've done on this article. So if you go to Sherpa Romeo. This is the front page you can type in the journal title, or the ISSN. And then it will show you what the policy of the journal is. So here you have the published version so for this, this particular journal, you can post to any website the journal website etc. So this is the publishers PDF they formatted it it's very beautiful that's that's the published version. You also have the accepted version. So this is probably still that Microsoft or latex document but it's gone through peer review, but it's the accepted right all the material all the words are the same. And this accepted version, according to this website could be posted on archive bio archive social archive and a couple others. And there's this one I can't remember what the difference is accepted version pathway be I think oh this one for institutional repository or specific ones. You need you have an embargo of like 18 months. So there's just a lot of information here on what you can post and sort of where, but the submitted version which is what we typically think of as the preprint before peer review. So in embargo, you can pretty much posted to any of the preprint server so this website is a really great way to say like hey this journals policies are you can post this version of your article and these different places. Stacy can just tag on a second there. I'll highlight that embargo and it's here with a little hourglass and talking about time that we'll talk a little bit more about this but as you start to wade into some of the nitty gritty kind of legalistic elements of what can I do and what can I not do. It isn't. It is it is oftentimes a chore to figure that out for any particular journal. I wish there was greater clarity you could just go on the journal website and oh here are policies about open access but usually, you have to really search through their website and then oftentimes that takes you to. There's a link that goes to the publisher website and then you have to go through three more links there to go. I encourage people that they're not sure they can't figure it out. Email the journal editor or the the the the assistant for the journal. Oftentimes they're not entirely sure of all of these policies to but anyway one of the issues is that for certain things, you can do them certain things you can't and certain things you can do after embargo period. So, for example, you could post the accepted version. And so that's not the the fancy journal formatted piece just like Stacy said, this is probably a PDF of your word document, but it has elements of peer review that have been incorporated into it. The journal does have some say over what you can do with that because it's gone through their peer review, you've incorporated elements of that into the manuscript, but very oftentimes you are allowed to post that. And so here you can post that on a preprint archive at any point, and you can post it on different institutional repositories for example, but only after 18 months after publication, which in this is, I'm not sure if there is a rhyme or reason to it and the policies really vary across different publishers and journals so it's hard to just say, and this is the generally what happens. The submitted version where this is what you submitted it was it's a probably a PDF of your word document. It hasn't been submitted yet that you can pretty much do whatever you want with. And if it wasn't you, you haven't like dependent on peer review or anything like that. Right. Awesome. Oh, good. Oh, I was going to say, but there are there is a rare exception where some journals will not accept submissions that have been preprinted. It is fairly rare. I think it is very seldom policed, but there are some journals. I'm not sure of the rationale, but there are some journals that is at least their policy, not to accept submissions for preprinted articles so in that case if you posted yours, the submitted version. It's your right to do that. Legally, it may have you want to make sure that the journal or journals that you may be targeting don't have a policy that then prevents you from submitting there. Right. Yeah, that's a good point and we'll talk more about this. One thing I wanted to add though, talk to your librarian if you're not sure I learned way too late as a graduate student at UCLA that the entire UC system. Because I was like affiliated with them had all of these rights that I had no idea so like I could literally sign my first born away by giving my publisher or my PDF to a journal, and they're like we own this we own you whatever, because I was affiliated with the UC system, I actually had rights to upload that into their own repository so there's like a lot of layers here that are difficult to navigate. This is definitely one resource I think that that can be helpful but also your local librarians they might know things that are specific to you and your institution that are really helpful. It's a good point you see it the UCs University of California has have a very interesting history and recent history with negotiating with publishers and do have some unique arrangements with publishers. So let's talk about preprints. So this is sort of like where we're going to go from here so preprints we're talking about what they are the different terms that get thrown around why we need them the benefits, the limitations sort of how they're growing. If they're reliable or not, sort of how they relate to repositories. And then Brian is going to take us to the ad archive posting them consuming and then we'll do like questions and answers. Okay, so some terminology so we talked about this little bit of preprint is the most common term I think, but it's using a different and potentially confusing way so sometimes people call these white paper sometimes they're referred to as preprints. I think the open science community is pretty much adopted the term as preprints but there's a couple different ways of certain terms that get thrown around. So the preprint is the print that's posted to a repository before edits are made reflecting peer review from a journal. And to refer generally to both pre and post so if you have a preprint it's like what you wrote with your team or yourself or whatever. And if it gets posted it's it's considered like a preprint, it hasn't been printed by a publisher it hasn't even gone through peer review. The post print is the print that's posted to a repository after edits are made reflecting peer review from a journal. So basically, what this is is like, it's gone through peer review, but it's not the publisher's PDF version right. And then lastly this print is sometimes used to refer to generally just both pre prints and post prints. I think it's a little bit confusing but the idea here is that the preprint is before peer review and the post print is usually after peer review. Brian did I get that right. I think that's right the only thing to try to make it even more confusing is that I think oftentimes this is the kind of the sub bullet there mentioned preprint is often used, kind of generically and probably technically incorrectly people will refer to this kind of universe of print postings as pre prints and technically the pre print probably should mean just if it hasn't been submitted yet but oftentimes I think people just think about all pre print so when you hear pre print. It's just it's probably not clear what what exactly is being referred to unless you ask. Want to walk us through this. Yeah, sure. So lots of rationales for doing this that overlap significantly with the rationales for making work open access that Stacy talked about earlier and so a lot of it is just, we want to make our stuff freely accessible for everybody to democratize access to research and the knowledge base. In terms of for the researcher, there is research to show that making your work open access and in particular green access increases impact and I think that only stands to reason. So we're making it available to a whole new and broader range of researcher or other researchers and research consumers. So it can really extend the impact, especially I think the non researchers were probably less likely to have those institution institutional affiliations. That that's reflected that increased impact is is reflected in documented higher citation rates for for pre prints. Certainly combating publication bias where I mean I guess it isn't the file drawer analogy doesn't work quite as well in our digital age, but the idea of that that we just keep a lot of these studies with no effects don't get published they're just kind of on hard drives are in their file drawers. This allows us and there's actually been initiatives and other fields to try to empty the file drawer and get all of these unpublished all that unpublished great literature out there and available for people to search. So when you post to to a print archive, for example, it gets a DOI and then it turns up in Google scholar searches and things like that. So we can really make a big fruitful push for trying to flesh out the research base for things that that might not be published because the findings were insignificant or small or not interesting enough to make it into a journal. And so that's the next bullet point there. Making research available more quickly this is really made headlines and has become a bigger and bigger issue with the pandemic that in the hard sciences, getting information about COVID and the COVID studies out tomorrow is saves lives as opposed to waiting for months of peer review. And in the worst case. Well, I don't know if I should say worst case but in the extreme case, you know, peer review articles sometimes go through multiple rounds of peer review at multiple journals. So within a single journal I've had things that are in peer review for over a year. And so there are stories certainly that I've heard of things being in going through peer review for multiple years and so if this is a timely getting that out peer review obviously has a lot of strengths, but it takes a lot of time, and we'll talk a little bit more about that that obviously brings up huge issues that these prints have not necessarily been peer reviewed, but it gets it out there much more quickly. It can be a mechanism for feedback that people can comment on work before you submit it for peer review for publication. And so you can put it out there, invite people to look at it, or just have it's out there for people to comment on and get feedback for people to look at it. It's also a nice thing if you have say you're on the job market, or something like that and instead of saying oh yeah look at my CV I list a whole bunch of in in papers or yeah. And so here you can actually have here's a here's a DOI where you can look at some of my work that that is currently under review and have that available for both. Yeah. And just to jump in I remember like one tweet Brian knows like had was like this article just got published and it already has 17 citations because we published this as a preprint and so people were already like using the knowledge in their work and referencing that in their own work so by the time it was published it was already being used, which I thought was really cool. Right, which in kind of my view of how the publishing can be a nice kind of megaphone for work, but it. I like the idea of having a separate system for disseminating it making available making it available and people can use it, regardless of whether a publisher picks it up or not. And also, there aren't page limits, which most journals have associated with preprint archives so you want to add extra material, expand on your method. You can put it out there, have that extended on a bridged version on a on an archive or in a preprint, which I think is just fantastic. And I don't know how there's a bullet around the file drawer I put at the end because I already talked about that related to publication. Yeah, and I think that it ties into the slide someone wrote, when you post an empirical report on a preprint journal with no intention of submitting it to a journal for peer review. What's that called not really a preprint has determined developed yet to signal that a paper is a terminal preprint. I love this because, again, that idea of the file drawer like we spend time energy. If you're human subjects other people's time working and trying to collect data and for it to just sit there and never see the light of day is sort of like a huge disservice to all that time energy and resources and so I've had people who are like, this study like fell apart it doesn't have the findings to get published what I do and I'd be like, write it up as a preprint, write it up as a preprint post it there, you'll have a do I people can find it. Right, it's allowing people to find the work who are interested in the topic. And you know, it doesn't have the same impact on the CV as a published article but at least you're disseminating the work in a way that is open access. Yeah. Yeah, oh yeah great point share you can put those terminal preprints in your local institutions repository. Yeah, exactly. Another thing too is, we're seeing some shifts in the field so the journal elife recently changed their model to a preprint first model so basically this journal found after looking at who was applying to the journal and what was getting published, like 90 something percent of the people had been posting preprints ahead of time and so they thought well this is where the fields going we're just going to require this so with this. This journal, they actually want you to and require you to post your work ahead of time perhaps get feedback, have it open and then you could take that preprint and apply to the journals and I've seen a lot of energy behind and I'm just wishing that all journals would just accept preprints as submission so you don't have to like format it and do all these other things for you know specific to the journal but you could really just like post your preprint and then use that as submission. So I think there's a little bit of momentum here, which I really encourage, but it has yet to be widely adopted to my knowledge. Education I don't, I don't think it's been adopted in particular but I do that that idea of kind of blowing up the process a little bit this is maybe one way to do that and so publishing really then becomes just making it public and we make all of our work public and we have these print repositories where everybody's work is out there for everybody to comment on. But that then journals, then kind of can select and curate different works from that list that are available and in some ways maybe give them an additional megaphone but it obviates the journal as the the nexus of the dissemination of scholarship which I think is a real positive development potential development. Yeah, I think a lot of researchers would love the ability to just make their work with the click of a button available to everyone right instead of waiting years for peer review, and to get solicited by journals. That would be amazing. So I hope that is a direction we're moving towards. This is from people of ours review and I referred to it a couple times and this is so this is citation. There are other works that show other a an open access advantage in terms of like the, what's what's the social media impact on blanking in the name of. How often old metrics Thank you very much Wendy. Hi Wendy. And so, but this is just for citations and so, you know, on average citations get cited a little less than the bronze is actually a really interest there's a piece. I was that in a era open, but a year or two ago a era just had a mishap with things, and they made a couple entire journals just open for a short period of time bronze open, and they showed a real bump in terms of their downloads, which kind of makes their downloads for free and so it just kind of shows you make these open people are going to come get them and use them more hybrid is really up there. Green is really up there in terms of making these things. I would suspect some of this is selection bias people probably are making a choosing to make green versions of their work that is probably going to have some greater access maybe or greater impact, and probably be cited more they may be more excited to do that with that work. So that might be behind some of why people are choosing to, yeah I'm going to pay for that piece to be open access. It's got it might be cited more anyway, but it was, I always found this curious that there's a lower rate for the gold. And I think some of this goes back to the point I'd mentioned earlier about a lot of the gold journals are actually kind of small journals that are associated with small organizations that are just run by the volunteers and they're just going to kind of be low, lower sided, because of the nature of the journal. I guess I should pay attention. We have a great question but I was hoping to hold it off till the end because it's a spark a larger debate which is an important one. Perfect. I wanted to talk. So, some of the disadvantages some of the challenges. It's not peer reviewed. And so I'm involved with at archive, and we just make sure there aren't the, that the content is moderately appropriate, you know, we've run across a couple where that's not the case. But we're not going in and was that adequately powered and they didn't list that as a limitation and I think they really should have. There's none of that. That's not the job. On one hand, I think there's a caveat emptor and that's maybe that's okay as long as you realize that, but I'll talk about some, maybe some other approaches to dealing with it. But that's a huge issue and limitation to this that that the prints that pre prints aren't peer reviewed yet. Some as I mentioned as we mentioned some journals policies don't allow for submission of pre printed manuscripts. I think that is not too many, but it does happen and it's it's it's important to be aware. Make sure when you're thinking about posting a pre print that you're not going to then target a journal that won't accept your submission. There's no journal policies there. And even though the journal policies on the websites themselves aren't always clear. One other thing that that I personally, I'm doing a lot of work actually right now in the in the area of open science and in my little world of special education. There is a limited pool of people that are going to review works on open science and at least some of those people are connected with on social media. So if if I post something on on at archive and say hey I just we just finished this go read it I put it on Twitter and Facebook and great people go read that. Then I'm worried though that when they get invited to review the piece they they have to say no I've already read that I know who the author is. So I actually oftentimes wait till the first round or after peer review is done and wait till something's accepted, which is a shame I think for people in broader who are who are doing broader work and there's going to be no potential peer reviewers, then then maybe that's less of a worry. I'd like to see us maybe. Well I don't know. There are different thoughts on whether peer review absolutely needs to be blinded in terms of knowing the author, but that's another issue to think about. Yeah, Matthews I tend to be the same I'm I think I'm okay with with open peer review but I know there are pros and cons and different thoughts on that. And so part of this this is a the I think I'm being aware of the limitations are is an increasingly important issue as as we start to look at just the. Just the number of green open access and pre prints is starting to really explode in different fields. Biology, for example, physics has been that way for a long time. And I think we're growing in education. And so hope I don't think our, we did one specific to education it wouldn't quite look like this but I think we're moving in the right direction. All right, so this gets to be kind of the reliability of the pre prints then and this starts to speak to that issue whether not peer reviewed. Actually, a lot of the stuff that's out there as a pre print is peer reviewed. And so it is something that was a pre print. Either the journal allows you to update those those prints, maybe after an embargo period, maybe right away. You can also update pre prints as I'll talk about in just a minute. And so, when it is accepted when it is published, you can put the updated do I in in the, the, the title page of the print. You can put a link to the published article. And so, and you can actually note areas of potential change, but you can certainly clearly indicate whether this has been published or not so people can compare the published and unpublished versions. And so, this actually they were looking specifically at co vid in the Gahanau at all study, and they found that 68% of the pre prints that have been cited around co vid were subsequently published so they're actually out there and and published, and have gone through peer review. This is just an early version of that piece so most of the pieces actually do get published and stand up to peer review, but some some don't and that's important to realize about 4043% of the pre prints didn't didn't mention the publication status on the pre print. And that you go ahead and put this is this is under peer review. It just takes a few seconds but it does take some time then to update it when it has been accepted when there is an elect a published version of it out there, you can put that out there. There's more and more reviews and studies starting to be starting to be done, where they compare the published version to the pre print. And so, in this study 56% had the exact same data in the exact same conclusions, but 15% of the pre prints that had been published when they looked at the published version, it had different conclusions. And that's all just just this year already in 2022. Again, looking and co the co vid related pre prints and publications are kind of the hot area. They did find that that between 717% of the pieces had some changes between the pre print and the published article, but the majority of those articles that did have changes. They didn't really affect the main conclusions, they were, they were smaller changes. So in some ways, and different people have interpreted this as being very supportive of the reliability the pre print look most of the stuff ends up getting published. Most of the published stuff isn't meaningfully different, but it also highlights that there is some stuff that doesn't get some stuff I shouldn't say stuff, some pre prints that don't get published and even among those that have some of the pre prints, or some of the publications, there's a real change from the pre print. It's a it's a relatively small proportion, but something certainly to bear in mind, and, and recommendations to watermark pre prints to put something in the margins that that clearly identify the pre print and whether it's undergone peer review or not. Great advice. I just want to be mindful of time in about 12 minutes left. So maybe Brian you and I can like speed get to the rest of the slides. So fast from here on out. Now the other stuff actually is the I think, okay, yeah, quick. So there's lots of archives out there some of them, a general or multidisciplinary that education folks can post to. Then there are some that are domain specific sci archive used to be a big one for educational researchers to post to. But we've been going for two years now we have at archive which is an archive specific to educational research. And a lot of them are hosted on OSF so a little bit about at archive up to 931 pre prints posted probably more because that was a day or two ago. So to post on it, it is actually I'm not going to go through these for the interest of time and to because you don't need to. I mean, you go to the website, you click on submit a pre print you. I can do this without a tutorial, and I'm not good at this type of thing so it is, it is pretty darn self explanatory from a consumer perspective you can also search the archives that are on there. They have OSF has built in hypothesis which allows you to comment on the on the on the print so there is an element of ongoing peer review, and you can download any of the posted manuscripts as well. Most of this, there is. Yeah, so this is just one that I was going to demonstrate this little I thing is the hypothesis if you wanted to comment on it, you can also endorse the work and so this was one of my advisees had made this kind of cool flow chart about how to post a pre print and was a supportive I applauded the work there. It also comes with some metadata what's interesting this is just the bottom half I couldn't fit it all in one screen, but you can download it. It also shows that there were two versions and the you can download the previous version, it shows you when it was submitted and last edited, there are tags that you can provide as an author and then it actually gives you citations in the different writing styles. We talked about the flow chart. And that's already. Yeah, check it out if you're interested. So basically this is what we'll wrap it up this is what we talked about today I think I have one more thing for you which is pretty pre prints which I really appreciate. These are made by Brenton Wernick. They are on the open science framework you could literally Google pretty pre prints, and their templates that you can download to make your Microsoft Word document a little bit more pretty, a little bit more fancy if you wanted to up your pre print game. And actually I fooled myself into thinking that the pre prints I have on my website where actually the publishers PDF and accidentally sent them before. So it's just, yeah it's a really fun, great way to kind of make your pre prints a little bit more polished a little bit more fun to distribute to others, and I highly highly recommend it. And we, I think that's it so thank you Brian for helping us walk through this we had some really good questions in the chat so I just wanted to sort of start with a question posed by a Vaughn, which was, what are your thoughts about the need to educate the general audience about what pre prints are, for instance it's not always easy for media to understand the differences between pre prints and reviewed articles. I might start by saying, I don't know if Brian's frozen I might start by saying, there's a huge problem when it comes to dissemination of scientific research by media, just in general, even for things that are peer reviewed. But when it comes to pre prints. Yeah, it's a little bit more tricky because people might treat them with the same like level of rigor as peer reviewed whether that's true or not I'm not sure. But it is a little bit challenging, just in general to work with media who really just wants the bottom line the height the headlines, and not really caring about the new ones. So Brian I don't know what your thoughts are. Yeah, I don't know if I know that there is some work being done and I'm not going to do a good job. I'm describing it because I'll forget but how the media is handling this, and it is something that especially has come up with coven and the media oftentimes kind of trips over itself to either not mention anything about peer review, or to mention it as kind of just counting it, and, and, you know, something isn't of high quality, because it is or is not peer reviewed it is just a step to try to promote and serve as a gatekeeper for quality but there's plenty of pieces that go through peer review that are then retracted and are of low quality there's plenty of stuff that isn't published that that is of high quality so it's certainly no guarantee one way or the other I think just being as transparent as possible and about the publication status of it the peer review status of it, and being caveat emptor about all of that is is probably about as as good as we can do. And it's a trade off between making things, making our making work and research, publicly available quickly available, but then not having peer review, you know in the, in the big, perfect world, I think we would build peer review into a print server, a preprint archive, where we could have ongoing review and commentary there, but but we're not there at archive yet and most print servers aren't that that'd be quite an undertaking. There's also a point in the chat from Charlotte who posted a lot of great resources, which is have people heard of PCI registered reports, a community driven initiative that facilitates the peer reviews of registered reports via preprints. You're nodding like you I have not heard of this. It's absolutely wonderful and I think Chris chambers is spearheading it, and the idea here is is registered reports which is a whole other topic, but doing registered reports works largely like dissertations and you submit just a perspective and methods and then that gets accepted in principle, but doing that and so they're taking the peer review out of the journals hands and reviewing these kind of study proposals independently that then can be picked up by other journals, and it is I think really a kind of first move towards taking the review the peer review process and owning it amongst a community of scholars rather than having it being run through and for the kind of to some degree, supporting the profit of publishers, even though the academics are still doing all the work. That's great I just want to let everyone know these resources you're adding to the chat. I'm pulling into our slide deck that will be on the open science framework so the very last slide will include everything that people are sharing. So we won't have time to go over all of it, but I do want to make sure that we all can access it later. Question about at archive. And have we been in discussions about accepting pre prints in other languages besides English, and I think the answer is no because only Brian and Jesse are reading through all of the pre prints on a daily basis, and you're probably at capacity. I, I would have to check and sorry this isn't going to be a satisfactory answer. I don't know what our policies are about that. I know I've interpreted pieces using Google translator, and I kind of think we have accepted pieces that are not in English, but I could be wrong on that. I think it's important to do so. It's just we do have kind of a mediator role and we want to make sure content is appropriate even though we're not peer reviewing per se. So we have to be have some mechanism to figure out what it says. But I anyway I can't yeah I can't speak to it without having our policies in front of it because in front of me because I don't want to misspeak but I kind of think we do accept things other than English but again I could be very wrong and if so I apologize. It's a good idea. I think we should within our ability to do so. Alright, and Matthew is saying OSF pre prints accepts paper in Portuguese so maybe if you can't find the subject specific pre print, if your field that accepts a different language maybe there's a more general one like the open science framework or archive that you could still post. So I know when I went on the job market, I added all the pre prints I could to my website because I knew people were going to be looking at them and my old work was in transfer students and that wasn't what I was doing but that was predominantly what I had published. So I created my pretty pre prints, bark them clearly as pre prints and upload them to my website, which I think was really really helpful when people were trying to Google me. But also I sort of like had control of the pre print, even though it didn't come with the DIY man website. There's just another way to sort of use them to my advantage. The institutional repositories to or something that I haven't sufficiently explored but I know a lot of people use and feel feel very positively about that more and more. Almost all I don't know of all but most universities are going to have an institutional repository where I think you're encouraged to post the green versions essentially pre prints. And yeah, I don't, but I should and it's something I need to probably look into and make myself do. Yeah. Well, it's just turn three o'clock I want to thank you all for coming if you have any questions feel free to email Brian or I, or find us I guess in one of the, the hang out meeting rooms. But I'm sure all y'all want to go see other sessions so thank you so much for coming. And thanks for the resources you provided. Yes, that was fun. Yeah, thanks everyone. Thanks Stacy. Thank you, Brian. I think I've already told you for this.