 All right, let's get started. Hi, everyone. Welcome to today's roundtable discussion, the importance of updating registrations. I'm Mark, and I'm going to be the moderator for today's session. Open Science aims to make scientific processes more transparent and results more accessible. Pre-registration is one of the open science methods where researchers report a study design and or their analysis plan before results are known with the goal of increasing the transparency of the research and reducing unethical research behaviors. However, as we all know, the future is not predictable and study design changes to reflect, they need to change to reflect these unexpected anomalies. A great example is with COVID, we've all experienced unexpected changes with that and how it's affected some of our study designs. Updating a pre-registration is a way to transparently report these changes. However, the community shares several concerns and has questions about updating best practices. So during this webinar, we're going to discuss the aim behind updating pre-registrations, what funders and other researchers are looking for when reviewing registration updates and how it's impacting their processes. So with that, let's go ahead and chat about our agenda. First, I'm going to touch really lightly on some housekeeping rules and they're going to dive directly into our round table. We're going to start with introducing our panelists and then go directly to our questions. Following that, we are going to have a session on Q&A and then we're going to wrap up with a video that demonstrates the updating process in OSF and share a community feedback survey so that way we can continue to improve these different types of workflows. So to go ahead and begin with housekeeping, if you have a question, you have a choice to either raise your hand or you can enter it in the chat. Hand raises will be answered during the session while those in chat will be addressed periodically or at the end of the session. So just kind of put those rules on out there. I'm going to go ahead and stop sharing and I like to go around and start introducing the panel. So I'm going to ask everyone to introduce themselves, state your role, your organization and if you fit more, funder, researcher or something completely different. So who would like to go first? Sure, I'll go first. I'm Katie Corker from Grand Valley State University in Michigan in the U.S. I guess I did a researcher bill and maybe a couple of other roles because I'm also involved with SIPPS, the Society for the Improvement of Psychological Science and a few other open science related roles. Thank you. I can go next. I'm Crystal Steltenpole. I am a research and evaluation associate at Dartmouth Center for Program Design and Evaluation, quite the mouthful. I fit more of the researcher role although like Katie, I'm also involved with various open science initiatives like the Open Scholarship Knowledge Base, Society for Improvement of Psychological Science and some other organizations. I'll go next. I'm Dr. Pam Deoskeen. I'm from the University of Michigan, the other side of Michigan from where Katie is. A researcher, also an editor for the Method Journal as is Katie mentioned. And I've big proponent of preregistration and I'll specifically talk about issues of using secondary and existing data. Morning. I am, my name is Kristen Elden Wiley. I'm a program officer and project manager at Templeton World Charity Foundation. I lead the foundation's effort to promote best practices in open science, both in the organization and with our grantees and our potential applicants. I'm the funder perspective. Yep. All right, fantastic. So depending on our different roles, we have different goals when it comes to preregistrations. So that's gonna be my first question. What is your goal or your goals for preregistration? Who would like to go first? I guess I can start on that. So when I talk to folks about preregistration and I think we can have conversations around rigor and transparency and all of that, but I also think of it from a really practical standpoint. When I taught, I would always tell my students the best person to answer questions about how you were thinking at the beginning of your study is you like six months ago, but unfortunately you six months ago that not available for emails, not available to talk to. So having a record of that can be really helpful in going back and thinking about how you were thinking about the study whenever it started. And kind of getting some plans in place. I also like to pitch it to people as a planning document. I think we've all had that experience or most of us have had the experience of being in a lab where a bunch of data gets collected and then you're like, okay, now what do we do? And that can be a really inefficient process. It can be a really frustrating process. And so like kind of having an idea of how you want to analyze the data beforehand can really speed up that process later once you actually have the data, which I think can lead to publication faster. And it can also just lastly serve as a way to get everybody on the research team on the same page. So when you design the study, you might be thinking about everything in a certain way, but then once you get the data, there can be a lot of ideas on how you should be analyzing the data and which direction you should go and what variables you should use. And so having that document kind of at the beginning serves as a nice reference point to be able to say, okay, this is how we were talking about it earlier. And maybe we should try this first. And if there are other ideas for how to analyze, maybe that can be a future paper or something like that. That's how I tend to think about preregistration. Yeah, that really fits with my practices as well. I also like to think about different parts of a preregistration when thinking about setting one up. So there can be an analysis plan that specifies exactly what steps you're going to take in the analysis, how you're going to sort of clean and prepare the data. There can also be specific research questions and hypotheses, but I think it's important to separate these two components. And sometimes people get into debates about the value of preregistration and really the debates boil down to whether they're talking about the value of specifying theoretical predictions ahead of time versus design or analysis choices. So thinking separately about each of these different components can be helpful in terms of thinking about the purpose of the exercise. Specifying the analysis plan ahead of time helps you to clamp down on researcher degrees of freedom. It helps you give more meaning to p-values if that's part of your study design. Specifying the predictions ahead of time gives you a chance to falsify those predictions. Again, if you're working from a framework that the goal is to advance theoretical understanding and different projects have different goals with respect to these endpoints. Yeah, I think I would only add to that. So we, I like to think of them as lab notebooks like you would do in any lab that you would be in whether it be physical or social science. And it's our way of making sure we've mapped out everything we've done so that somebody can replicate what we've done in the future. The other real advantage having done a few of these that I've seen, which I really to me makes the research much more fun, is really all of us justifying why we think we're gonna find what we find and really having the deep discussion about that. So not even anything that we actually write in the preregistration, but really engaging the research in a deep way because we're writing the preregistration. So we can't just say, well, I think this that has to be backed up by some type of research claim. And that actually makes for really interesting lab discussions when we do that. And I think it makes us all wiser about the science we're doing. And I really appreciate that in ways that I haven't seen when you just sit down, okay, we have to get this paper out the door. You're not really engaging kind of what the science is sitting on top of and what the prediction should be, what's exploratory, what's confirmatory. Is anything really exploratory because we're sitting on top of lots of existing findings from the research. So I think that brings kind of the fun into it. And then we can slap this lab notebook on top and say, here's the paper and here's everything we did to get to that paper. And I think it makes for, you know, just a really nice research package that we have to be able to give to the community. Yeah, something Pam said reminded me of another really valuable part of this practice such as narrowing down your focus. Because I think at the outset of a project it's really easy to get excited about all of the sort of different directions you could take. And one of the things that pre-registration really does is it really forces you to hone in on what is the primary focus of this project. Like what are the three things that we are trying to do here and really staking your ground on, these are the things that we're trying to understand can be really, really helpful in designing a solid study. It's really fantastic to hear all of this. It's from us that was probably a little more high level and probably something everyone's already heard before. But we, I mean, TWCF funds projects with the intention to push the boundaries of scientific knowledge. And for us, open science and pre-registration as a tool is important because it means it leaves some work or scientific research. And if we're gonna fund projects that are really make an impact in our contrarian discoveries, we are really gonna wanna make sure that it can earn the trust of the broader scientific community. And we see pre-registration as a way to achieve this. And then just so we've also had other, we have an initiative that has a very specific reason for requiring registration or accelerating research and consciousness which uses adversarial collaboration the idea is to kill off incompatible theories. So for us, pre-registration is crucial to this because we need the three leaders to sign off on what the different results of the research you can say. And pre-registration has really given us that tool for us to get three leaders to sign and commit to the results. Awesome, thank you. So it was interesting that you guys have already touched upon some of our follow up questions but I'm hoping to kind of dig into those a little bit more. So from what it sounds like, it sounds like updating has already been a natural part of the research process or some of you guys. So my first question would be is, how does updating tie into your research process when you're conducting a study? What points do you update and what do you actually want to write down and say this was an update to my study design? I'll start with secondary and existing data which I'm most familiar with. We just did a preregistration a few months ago and even though, I mean, this is kind of an interesting thing about preregistration and registered reports. So in the case of existing or secondary data, many of us already have some experience with the data set, sometimes not, but these are massive, I'm using population data sets or massive data sets. And so even when I think I know what I'm doing once you actually get downloaded the data after you've preregistered and you think you understand then you run into all kinds of issues related to how that data was managed or done differently than what might actually have been set in the documentation. At that point, we have to do an addendum because what we preregistered, we now cannot do because the data is not as we expected it to be. So we have a few of those addendums. The other addendum is updating and that can be related to something that we thought was continuous and it's being categorical. So we had to change our analyses or we've learned about a new analysis. With the analyses we thought we were gonna do, we've now talked about and it was actually incorrect what we preregistered or is not as good as another one would be. And so all of those, I think we ended up having four to five addendums were related to unexpected data issues that we had to change based on our preregistration and changes in the approach to the data because of those. Our role was we could make no changes until we justified why we were making that addendum so that every no data could be touched before we preregistered what we were doing and then we would do it. So again, it goes back to having those conversations beforehand, which I think are so vital. We could have just run the data and said, oh, this is what we should do. But instead we stop and actually think about what we're gonna do, put that as the preregistered addendum and then go and do the data. But it's really common in existing data that what you thought you were getting is not exactly what you've got. And so you have to make adjustments. Go ahead, Crystal. I was gonna add on, Pam had a good point that sometimes like you preregister one thing and then you think that that's the best way to do it and then you realize you don't or a lot of fields are updating a lot around methods and around analyses. You might have preregistered under kind of one paradigm and then you read a paper that maybe even came out but before or after the preregisteration but you find a paper that says, we've been analyzing data this way but actually it's probably less biased to do it this other way. And then you go, well, that actually convinced me but now you're stuck with this preregisteration that said that you were going to do it this one way. So having the ability to update your preregisteration to say, okay, we read this paper, we're convinced by it, we're going to actually be analyzing it this way. Also, sometimes errors creep into the research process. Maybe you have a pre-post design and you forgot an item on either the pre or the post on one of the scales that you're using for like a survey study. And maybe you didn't preregister how you were going to handle that because I think most of us don't think about like, oh, what if I forget to put something in there? But that happens and it probably happens with a little bit more regularity than we think that it does. And so being able to update that and say, oh, we realized like we're in the middle of data collection we haven't analyzed the data yet but like we realized we've left off this item, here's how we're going to handle that during our analysis portion. So like having like just those missteps or like you realized you didn't make a rule for how what you're treating as an outlier, you know, those kinds of things like being able to go in and say, realize that this wasn't a thing. I'm just adding that in there so that we have a record of the conversations that we have. And I really like Pam's talk about, you know, not putting it in there until you've actually justified, you know, the reasoning for you making those changes. Yeah, I think this experience of plans changing in the middle of a study is really common regardless of the study design, it happens in experimental work as well, whether, yeah, for the reasons that you all were discussing. So it's important to be able to update to address those changes. But this is actually one of the most persistent sort of myths about pre-registration that you're locked into the plan that you originally pre-registered and are not able to make changes. The most important thing is that whatever changes that you introduce after a pre-registered plan are made transparently. And so transparently means that in the study, in the report after you've made the changes, you disclose what changes have been made in a clear way so that the reader can assess how the final product is reflected. So doing the updating process in the registry is one way to add this transparency, but actually including it in the paper is important as well. I'll just add, I can think of a situation where, you know, the original pre-registration doesn't even need to be updated, but the document does, the final paper does. So in a situation where during peer review, a peer reviewer says, actually, I think you should do the analysis this way, not the way that you pre-registered. What I typically do in those situations, or I typically recommend in those situations, is that the authors report the analysis both ways. So they say, this is how we pre-registered it, this is how it comes out, this is why we think this is suboptimal or there's debate about this topic. And, you know, some people would prefer to see it this way, others that way. So you provide the results both ways and transparently disclose that it was suggested after the fact that the new analysis be introduced. That doesn't necessarily have to involve updating the original registration, it just needs to be clear in the document what's been done so that the reader can decide for themselves whether it matters. Awesome. Actually, I really love how you said that it should be a plan, not a prison. I think it's actually a article from Alex DeHaven, which we'll put a link here in the chat, but that's exactly what pre-registration is. It is a plan, not a prison where it's human, we're constantly learning, we're constantly getting new information. And that pre-registration is like this, what six years, six months me decided to do, but I am now wiser, I want to adapt, I want to change it, should have the flexibility to do that. So this is a great way to do that. So one of the questions that we have received from the communities is, what should you submit an update on versus what should you not? There's a little bit of distinction of, they're concerned that updating too much might say pre-registration is not as rigorous as it's supposed to be, but if you do too little, it's not as transparent. So that's kind of my question. How often should people update the pre-registrations and what exactly should they update on? Okay, I guess in general, I prefer there not to be too many updates to the pre-registration. For, I think, there's a question in the Q and A that says, when is it just a great, researcher's degrees of freedom? And I think that's absolutely the case. Like you should have a pretty solid pre-registration and then the updates should be about the things we've talked about. I mean, one thing that Katie brought up that I've been thinking about, which is have to update the pre-registration after you've had reviewer comments. And I actually, in my mind, we've been talking about this in the lab, that if my pre-registration is also kind of this lab notebook, then I actually need to have an update that's after review, what was I asked to do? Because I talk about this as reviewer or harking that we get a lot of reviewers saying, well, you need to do this. And in review, I think it is problematic how much you go off the pre-registration because the reviewers have asked you, you wanna get that paper published so you wanna do what the reviewers are asking you to do, but you also wanna be kind of careful about how much you're doing. And I think having kind of this post-review pre-registration update about what ended up being exploratory analysis or what you did, even if you compared it to these two analyses, what did you have to do to the data in order to prepare for the second analysis, which wasn't the thing that you had done before because that still means we're getting stuff in the data that has a lot of, not only is it research practices, and we now have reviewer practices coming in. And I actually think that's a pretty important part that we don't deal with in Pre-Reg is what happens when we have all this kind of reviewer harvesting coming in and I've like, do this, drop this group, change this. We can have that in the paper to say this is exploratory, but where is it in the documentation if someone needed to replicate that paper that we had to jump all of those hoops? So I actually think there needs to be a wraparound that after review, you should be going back to that pre-registration and saying, here's what we had to do, or even doing it through the review process. This is exploratory, reviewer is asked for the following. Here's how we addressed it. This is now fully in the document that it goes with this paper. So I really see us needing to have a package of information so that it's not, oh, and then the paper was able to go do whatever because the reviewer said to. And so therefore it's off to Pre-Reg that there should be, you should come back and say we were asked to do this and here's how we approached it, just like you would have said to the, maybe even taking it right from the reviewer response letter. Something to think about. Yeah, I do think have the registry and the paper synced up as much as possible. I also think it can be useful to look to areas of research that have a little bit more of a history with these kinds of practices. So in biomedicine because pre-registration has been required for clinical trials for many years, there is a set of guidelines, at least how widely followed they are is a matter of debate, but there's a set of guidelines called the consort guidelines and the consort guidelines provide pretty explicit recommendations about what to do about things like new outcomes being introduced. So if a new outcome is introduced, that needs to be disclosed in the manuscript, not just as exploratory, but as not preregistered. And I think even for people who are very experienced with these kinds of practices, the line gets very blurry between what was in the original plan, what was preregistered and then what ultimately shows up in the final paper. So it's very easy for a reviewer to say, oh, you should do the analysis this way or oh, have you thought about this outcome? And those analyses just sort of silently creep into the paper. The authors may think that they're being clear about which ones were preregistered versus not, but it's always good to go back to the registration, make sure that everything is synced up and to try to follow those guidelines as closely as you can. And I really do think that a lot of it is around justifiable changes. So being able to explain your reasoning for making changes, being able to, if you need to cite the other literature that's, you know, bolsters your reason for making those changes. And I think that can be a protection against you know, the concerns around researcher degrees of freedom as well, being able to say, like, here's, here are best practices or we've learned new information or we've realized that there is an error and like clearly outlining why those changes are made. And yeah, whether it's documented in the paper itself or in the preregistration or in like a lab notebook, which is what I kind of hope that we all eventually, you know, really move toward is like having these open lab notebooks that really kind of outline what that process is. Because scientific process is a little bit more iterative than I think we give it credit for. And so I think that that can really be important. I totally understand concerns around introducing and reintroducing those researcher degrees of freedom. But the purpose for me of a preregistration and especially updating a preregistration is I'm outlining my reasoning. I'm updating my reasoning and I'm explaining to you why those changes have been made. And you as the reader get to decide whether you agree with those changes or not, whether you think that there were good reasons for changing the analyses or whatever our plans are. Yeah. All right. So we're running out of time. So I wanna see if I can hit on one or two more questions. One of my questions is, let's say that you are reviewing research and you have noticed that there was a preregistration that was updated. What information would you expect to be in there if you wanted to be able to include that research within your study or if you wanted to replicate it? I think everything Crystal just said. Like, I mean, this is why I think we have to have like complete, that's why, like I said, like these lab notebooks versus, oh, it's a prereg for this one study and it never changed from the prereg, that they have a life, they're living documents. And even if someone were to come back and replicate and have an issue, just think about the fact that you could even put in this lab notebook your response to that response. Like, and so that when we go back and look over the kind of history of this study and the kind of responses, we have it all in one place. Now we have to like, check the universe for updates and look for even if something's been retracted, right? I mean, it's really hard to like, keep up with how these kinds of things have changed. So we end up having people citing like work that has been retracted, for example. So I think everything kind of Crystal said is absolutely true is what you wanna see. And you wanna know that this is a dynamic the papers are not done necessarily and that it's open for changes when those changes are needed or when there's a request for a replication or if someone finds something that they can't replicate because you forgot to tell some important part of what was going on. Now you've added that back in after a request so that other people can do it. So having living documents, I think is really the way to kind of think about our research in the future so we can crack it in the future. I think from a funder perspective, this all sounds like very much in line of what we would also would like to see. I mean, we just want to be able to track what has been changed and why it's been changed. But as a funder, we also work really hard to find the best experts in the areas that we wanna fund. And we don't want to necessarily impose on these experts as they're completing their research. As we said earlier, or you guys have noted earlier, don't want to stifle innovation and sometimes updating the preregistration and your research and your plan is necessary to meet that innovation. So I don't have anything else to add but that. Yeah, it's very much a new concept, something that is starting to gain awareness. And so it's one of those things that kind of have to fill out whether or not we're specific institutions, disciplines, et cetera. So we are just about out of time and I wanted to open up to the audience if they had A, Q and A's. So let's see if I can go and put our Q and A up real quick. It looks like we already have one that was already answered, which is fantastic. Thank you for that. Any of our other attendees have any other questions? So let's go with Charlotte. Does this change set preregistrations apart from register reports more? So for example, the intro method and analysis plan can't change with register reports after IPA through peer review, but this change for preregistrations allows for such changes. So there are a little worried about QRPs seeping through with those type of updates. So what do you guys think? So I know the three of us researchers are fans of registered reports and not speaking for myself. At least I think they are a much more rigorous process than I call them author managed preregistrations. So there's a lot of benefits of a registered report process over author managed preregistrations. The fact that they get peer reviewed before the study is done is a major plus. The fact that the details are pretty well locked down is a major benefit. However, having been part of several registered reports processes, I can say changes still need to get introduced. One of my own registered reports, we made a mistake with the stimuli that if we had been able to update the registration, that would have been disclosed in that way. Instead, it's in the paper. But errors, changes in analysis plans, these kinds of things, they still come up. One of the ways that things will come up is overlooked details. So although we try to catch a lot of this in peer review, we don't always catch every detail. So things like what you're gonna do about participant exclusion is a really common spot where we see people making changes after the fact because the plan that they registered and was approved in the in-principle acceptance was not detailed enough that when they actually, when it actually came time to analyze the data, they did enough deep plans. Thank you. All right, do we have any other questions? Any of our speakers, if they have anything else they'd like to wrap up with? I have a question for the panel, which is that we're getting a lot of these. Well, when does it become a QRP? And when is it an update? And I don't know. I don't know. I have this sense that if I saw 10 updates of which some of them should have been done prior, I would be concerned that those are QRPs. So I look for less and highly justified addendums, not ones that seem to be constantly changing. Oh, we changed this. We changed this. It should have been in the pre-reg. But I think that it's complicated. It's complicated as to when is it a questionable research practice after the fact? And then when is it something that should be pre-reg? I don't know if the rest of you have any kind of thoughts on that? I don't think it is solvable or knowable. I think there's always going to be debates about what is the best design choice, what is the best analysis choice, and so on. And just because something is declared, ahead of time doesn't make it the best choice. So the process of pre-registration adds transparency to the research process, but it doesn't add quality necessarily, right? It may add quality if sufficient thoughtfulness goes into the process and problems are anticipated ahead of time. But just because something is pre-registered doesn't necessarily mean it's better, just because something is pre-registered doesn't mean that there isn't still an opportunity for capitalizing on chance. The researcher judgment is still required to decide whether or not this particular study has value is valid, et cetera. And I would say too, with, I mean, pre-registration has been a thing in other fields for much longer within psychology, at least it's kind of a newer thing. And so I would expect more updates in this era. I would expect pre-registrations to be a little bit messier in this era as it becomes a more normalized way of a more normalized part of the training process, some more normalized part of the research process. I would maybe say then, you know, if I see somebody updating something 10 times, you know, I'd be like, well, what's going on here, especially if there's not a lot of justification. But I would say like, as people are getting used to this process, I think we have to allow a little bit of leeway for people to, I mean, this is a learning process. This is completely changing the way that some people think about research and it may be completely contrary to the way that they were trained. And so I think forgetting stuff and not quite, maybe you misinterpreted a term and you answered the question, you know, that you answered the pre-registration item a different way. You know, I think having those kinds of missteps or those kinds of half steps or whatever you would wanna call them, I think it's gonna be a little bit more common early on as people are learning like, how do you update a pre-registration? When do you update a pre-registration? What justification needs to go in? I would say for me, when looking at one, I would just wanna see the justification. I would wanna see like, does this logically make sense? Both in terms of their original thought process but also in terms of those updates, if they have to update more often, I mean, maybe it's a really complicated study, especially when you're working with, you know, for instance, I'm a community psychologist, we do a lot of work with communities. If you're doing like community-based participatory action research, a lot of stuff is gonna change throughout that process, right? And, you know, I don't even know if pre-registration really works in that sense, but, you know, more of a lab manual might. But so, yeah, I think it's really gonna be context dependent and I think we do have to take into account that this is something that's still really new for people and so they're still learning. And so for me, it's just, is that justification there? Is the logic expanded on? And if it's there, then I'm willing to give folks a little bit more credit than if they're just like, yeah, we changed our outlier from two standard deviations to three standard deviations or something like that, then I feel like, well, why? Right. Awesome. Thank you all so much. I really appreciate these conversations. Thank you for your time. That's gonna be our time-up for all of our questions and I'm thankfully actually out of them. So thank you all, I really appreciate it. Last thing I'm gonna do is we ended up preparing a video to kind of help demonstrate how you can update your registration on OSF. So this is a brief five-minute video. I'm gonna go ahead and share that real quick. Share my screen. We've logged in and are at the OSF dashboard page. To begin your update, you first need to get to the registration that needs to be updated and there are several ways to do that. One is going to your project, clicking your registration tab and clicking your submitted registrations. I prefer going to the My Registrations page because I have a lot of projects that I would need to search through. And My Registrations page aggregates all my registrations across all my projects into a simple list. To get to your My Registrations page, click the down arrow next to OSF Home, click Registries, and then you click My Registrations at the top nav bar. And there you are, it's that easy. Now, I'm submitting a preregistration for a study that is looking at sugar concentration on taste preference, but after looking at more literature, I've decided that I need to refine my sugar measures. To reflect that change, I click Update and I'm not changing any metadata, so I click Next. And I'm welcome to buy a new justification page. This page is important because it tells your readers whether there are funders, publishers, or other researchers why I needed to make a change. In this case, I needed to make a change because my measures were not refined to reflect previous literature. So let's enter that in, and there you go. Now I can change my preregistration. For this template, what I want to change is in Variables section, so let's go there. And more specifically, I want to change the manipulated variables question. So here I have 15, 20, 25, and 40% cane sugar by mass. And I'm going to add a 0% and 75%. So let's do that, there's 0% and 75%. And these changes are automatically saved. Now that my change is made and my justification is entered, I can now go to Review and click Submit Changes. And from here, it follows the same process as the original submission, where the admin contributors must review and improve the update. I am an admin on this registration. And as you can see, I can either accept the changes that are made or I can continue editing. If you click Continue Editing, it will take this back to a draft state where you can make more changes. If I accept changes, then it means that I as an admin have accepted the changes and ready for it to be archived. So I'm going to click that and now my decision is recorded. Now, let me show you why entering the justification is important. I'm back in my registrations page and right here is the registration that we just updated. So I'm going to click and open that. And as you can see, the justification that we entered is presented right here at the very top. This is presented to all your readers, whether or not they are researchers, funders, reviewers, et cetera. It includes the date the update was made and that justification that you had entered. You will also see that any of the responses that were updated will get an updated label next to that question. So previously, I adjusted my study design and as we scroll down, here is the question that I just changed where it includes the 0% and 75%. Readers, funders, reviewers, will know that this has changed because of the updated label that is next to that question. While you're exploring the different types of registrations and how to update them, we're actually going to see if we can get feedback as well. I want to put in a quick link. This is going to be towards a Google form. We will also include that within our social media postings as well. So as you start walking through the workflows, just let us know what you think. Our goal here at OSF and COS is to make sure that we make your lives easier. Research is already hard enough, technology shouldn't be, and we're always wanting to improve our workflows. But we can't do that in a black box. We need to hear from our users. So any feedback you can give is fantastic and it helps out a lot. This survey is three simple questions, but those three simple questions gives us a lot of information. So thank you to those who are able to do that. And then without further ado, thank you again for all the...