 All right, so I'm here to talk about RIDs. I love talking about RIDs. It's basically all I ever talk about. Every once in a while I might stretch into a slightly different standard, but mostly it's RIDs. Okay, this is an emerging standard in journal publishing, and I wanna talk about some of the tools that make it happen. And by the way, yes, I have a conflict of interest. We just founded a company. Dr. Martone, Dr. Greta, and I are co-founders in a company called Cycrunch, which is a relatively new startup. So, there we go. Why are RIDs? Why did this happen? Well, now about 10 years ago, the Neuroscience Information Framework, while it was funded, got a question, and the question was, hey, antibodies are really hard to find for our scientists. Can you guys do some text mining magic to please make this easier? And we said, great, we'll try to do this. And then we ran into things like this. This is actually a paper from 2019, from last week. So we can, and I didn't have to look very hard for this paper to get this example. Now, what you find there is a lovely paper, good journal, and then there's this mouse anti-flag antibody here from Sigma Aldridge. So I said, hey, what if I were trying to reproduce this paper and I tried to find this antibody? So what I did here is I went to the Sigma website, which is Millipore Sigma, and I put in this mouse anti-flag, and then I came up with exactly 17 antibodies. Now, question. If it doesn't have an identifier, is it reproducible? Answer? No. All right. So how did RID start? We actually had a lot of involvement, you know these little buttons are really hard. So we had a lot of help from this organization called INCF, anybody heard of that? So there were also members of the NIH involved in about 25 of the major journal editors and chief, all got together in a room, some of them are still in this room today, and we got together at the Commander's Palace thanks to some wonderful funding for a very, very nice lunch by INCF. We found out at that meeting that journal editors, much like graduate students, will show up if you say free lunch. In 2013, NIDA sponsored a two-day workshop to get these people back together again, and again INCF provided a lot of the meeting support for that, in fact a lot of the mediation between these journal editors as they were fighting it out in terms of what can be done about this antibody problem. And the pilot projects actually started in 2014 with 25 journals. Now, we were supposed to get these 25 journals in any way they wanted to ask authors to provide RIDs for three months, for three classes of research resources. Now, what we also found out is that only two journals started on time, and one of the journals that didn't start on time, also in this room. We're currently in our fifth year of this three month pilot project, and we've grown a little bit. Okay, so, yeah, I think I'm just gonna use the little things. All right, what does it look like? What does it actually look like when you have an RID? So, here's a paper that was just recently published in a very nice journal. I think you might recognize this journal, Neuron. You might recognize this lovely paper, and you may or may not have seen what this particular publisher has actually done to all of their method sections. The method sections have starred out all of them now with the star methods key resources table. And the key resources table has a list of the various types of things that were used to produce that study, right? And in the case of this particular paper, there's a whole bunch of tools that should be relatively recognizable to all of you, pine, neuron, et cetera, nest, brain, and others. There are antibodies in this list, and there are others. So, what is one of the tools that can be actually really useful here? And the tool I'm gonna talk about, you may have heard of, it's called Google Scholar. It's relatively useful. You can ask it about any RID, and so here I've asked about some of the RIDs that you might be familiar with, or the tools that you might be familiar with, that do have RIDs, and especially I'd like to point out this one, because we're here. So, why not? So, PyMice was just discussed in the last session, and it's got some very nice references, again, showing that people are actually using the RID. Now, PyMice actually does something really interesting. It says, hey, if you're going to cite us, here's a format that we'd like to use. And this is actually a really nice thing to do for your tools, be able to just add this format. Hey, I want to be cited this way, and perhaps other people will actually cite you this way. Add Jean just joined the RID initiative this year, and they did exactly the same thing. So, every one of their plasmids, and they have about 20,000 plasmids, they have this little how to cite this plasmid section, and this particular citation format actually went live the day that we had added them to the RID project, so everything was actually added at exactly the same time. So, why bother? Why would we ever want to do this? Well, we ran a study at the end of what we considered the pilot project, which was at the end of about 2014, 2015, where we had gathered the first 100 papers, and we started to really analyze them. So, if you look at this pre-pilot, these are the same exact journals, but just the issue before the RID initiative started. And so, if you can see that, the percent that are identifiable is like, it means can you find this thing? Can you go into the catalog, and can you find the thing? And you can see that 50% of the time before the RID pilot, you could not find antibodies. But in the issue after, when we started asking, magically, you could find 90% of the antibody. So, this is a pretty big result, and we love to talk about this. It's true with organisms and it's true with tools. So, authors are quite good at being able to identify what they used as long as they're asked. So, some new developments have actually occurred. RIDs are now part of this NISO standard. This is the journal article tagging suite. This is the XML standard for journal articles. And version 1.2, which was just released in May, now includes this little bit of code. This is an XML wrapper. I promise there's no other code in my talk. But this is just the bit about, this is actually a mouse. This is an RID, IMSR. This is a Harlan mouse, 5669. This has now been immortalized by the National Institute, by the National Standards Organization into this particular standard. And by the way, for those of you interested in preprints, I am told that it is imminent that the bio archive, that all of new bio archive submissions should actually go into this XML standard theoretically within days to weeks, but I'm not gonna hold my breath. But it should happen relatively soon. So, all of you have tools, or a good chunk of you. How many people have tools that you've created? Yeah, that's what I thought. Okay, so if you create a new tool, remarkably enough, you can actually register for an RID. And all you have to do is either go to RID.site, or you just type RID into your favorite Google browser. If you are actually trying to publish with a few journals, there are a few journal editors that might make you do this. Most won't make you do this, but I would highly recommend it. This is what a page looks like once it's been created. So, this is an open neuro page. There's the RID. You can actually claim ownership of this page so you can control what's on here. This can be under your control. The curators will still get to it, but you can control what's here. There are some fun things like you can add your ORCID ID, and we're working with ORCID to push that information. We can have some relationships, so we know that open neuro is related, for example, to nitric. It's also related to data lab. And then there's a bunch of text mining pipelines, and also a curation pipeline that gets the RIDs and the mentions of this tool, so open neuro. We know that that used to be called open fMRI, and that has this particular RID, so that is actually being mined out of the literature, and it is being presented here. Of course, you can download this, and you can upload new mentions if we miss them, which we very well make. So how many journals are actually using our RIDs? So this is a slide that will soon be obsolete because we've moved to a brand new wonderful system of keeping track of them, but at the end of that first year in 2014, we did have actually 100 papers, 25 journals. We've got a little bit more now, so we've just passed the 900 journal mark, and we've got about 15,000 papers, and that's counting using the PubMed ID. If you count by DOI, there's about 16,000. In 2016, the project really started to go up fast, and that was because Cell Press had joined us, E-Life had joined us, and Dechronology had joined us, and then closer to 2017, AACR had actually joined our latest publishing group to join is Nature, so we are very excited about that. Okay, this is the elephant in the room. Okay, you can find antibodies. Does it mean anything? Does it actually help reproducibility? And we also had a cell line repository joined the RID project in 2016, and one of the reasons why we wanted to do this is because there was this great group of scientists called ICLAC, which is the International Cell Line Authentication Committee. This was a group that put together a list of naughty cell lines. These are naughty, naughty cell lines. They've been overgrown by HeLa cells. They have been misidentified, so somebody called a particular cell line, a hepatocarsinoma, and it was in fact a hepatocytoma, and if you don't care about what that difference is, then you might care if you get the wrong treatment for a drug. So for us, there was a very lovely list, and that list was of naughty cell lines, and there was another list of regular cell lines. We ingested both, we stuck them together, and we asked the question, if you put this list in front of the author, and you have him just look at this list with regard to before they publish the paper, will it make any difference whatsoever in terms of their use of those cell lines that are misidentified or contaminated? So could we put this in front of them before they publish, remarkably enough, the RID mechanism, without asking anything about contaminated cell lines, without asking anything else. We just said, hey, please go get your RIDs for these cell lines, and then we looked to see in these 634 papers that had been published by the time we did this analysis last year, what percentage of them were actually being used that were part of this naughty list? And we found out that out of these 634 papers, 5.36% of those actually used one of these naughty cells. So then we asked the question, hey, what about the general literature? And how would we assess whether or not authors are using these naughty cell lines in greater or lesser frequency in the general literature? So what we did here is we actually text mined using a tool called SciScore for about two million papers, and out of those two million papers, we found 150,000 that used a cell line. We identified the names of those cell lines, and those cell lines did actually match at a fairly high frequency one of these naughty cells. So again, looking at the difference between 16% use of naughty cell lines versus 5% use of naughty cell lines, we have now declared victory, because 5% versus 16, that's much better, right? So RIDs without additional help, just the fact that the authors looking at this list and seeing whether or not this impacts their paper, even at a stage, I mean, we don't control when they do this. So this could have been very easily after peer review already. It might have been before, we don't know, but it is associated, that act of looking something up is associated with a very significant reduction. This is a 67% reduction in the use of these bad reagents. All right, so looking at some future directions, so I did mention SciScore. SciScore has now been released to the public. This is a commercial tool that's funded through the SBIR mechanism in the US out of the office of the director and NIMH. SciScore is a tool that will find your cell lines. It will also currently find a lot of other information about the places where RIDs ought to be. It finds a lot of other NIH and a journal rigor criteria. So if you've missed, for example, the fact that you've done blinding or if you haven't done blinding, it will detract a point from your score. The beautiful thing is it actually does also have a score which we may or may not use relatively soon to try and create a reproducibility index. That is something that we're thinking about going into next, but that is the future of what we think we're going to be doing. So in conclusion, I wanna thank you. Of course, I wanna thank all the people who contributed to this work, but instead of that, I'm going to take my last minute to say, please use RIDs in your next paper. And I wanna see a show of hands really quick. How many people are working on a paper? Okay, that's the number of papers that I expect to have some RIDs. Thank you. I will find them, trust me. Please register your tools. It's not hard. It takes all of one to two minutes. And use the RID tags for how do I cite my tool? You can add it to your Zenodo or whatever else. You might be asked for it in your next journal publication. Thank you so much. Hi, my name is Maya. Thank you for a very nice talk. I myself liked this idea and started also being frustrated about antibodies, but to my surprise, I thought, namely that antibodies only had one RID, but I guess what, when I went to one of the vendors, I found out that one clone could have several RID because they sell them in different quantities. And then I was really a little bit frustrated again because what is the point to have an unique identifier if you are allowed to have three identifiers for the same product just because you sell them in different quantities? So I hope someone will enforce to them to go back to just one identifier per clone. Thank you. Was that from Cell Signaling Technology by any chance? Was that from Cell Signaling Technology by any chance? Could be. Yeah, it sounds exactly like it. I know exactly what era you're talking about. Yes, that's the data they initially gave us and it took us about six months to clean it up, but it's now clean. We're very excited. Thank you.