 All right. Good afternoon, everyone. And welcome to our April 7th, 2022 webinar, a guide to supporting early career researchers in open science. We really appreciate you all joining us and are excited for for this critical discussion today. A lot of this has been prompted from some of the input we are receiving from the open scholarship survey and from the community in general about the need to develop good solid open science open scholarship behaviors among early career researchers so that it will be able to last throughout throughout a career is as really valuable supports. So we are present of that and working to move forward in some of those efforts. So we're so glad you're able to join us this afternoon for that. Just a few quick tips and information we wanted to share on behalf of COS the Center for Open Science. We would be remiss without offering some of these resources to the community joining us today. The first is a STEM education hub, which as you see, you know is a place for education researchers to learn, discover, share and engage. In particular, in this hub, you will see the link that my my colleague is putting in the chat. Really as a go to place if you are starting your open science journey. There are a series of invaluable deep dive videos on preprints registered reports open access by a series of recognized experts that really is the place to start. And so we wanted to share this hub for you as a place to bookmark and return to the information is is is very valuable here. But definitely in particular wanted to point out the deep dive videos within this website. You certainly want to want to check that out. Next, we wanted to share the open scholarship knowledge base that is hosted by OER Commons on the hub and putting the link to that in the chat as well. If you were looking for resources on open science, you're looking for a tutorial guides information visit this website. It's another another website to bookmark visit again and again. That only allow you to search and to find resources on the hub on various open science topics that you're looking for, but it also allows you to add your own resources to support colleagues and peers. So we invite you to, you know, consider doing that and sharing some of your, your topics and journals and information that you're curating and make it available. You also can create discussion boards and guides reach out to authors. So it is a valuable valuable knowledge hub knowledge base to visit. So we wanted to signal that out for you at the beginning as additional valuable resource. And then finally, we wanted to take this opportunity to really stay connected to this audience and to the community of early career researchers and mentors. We are not looking to have a one stop discussion where, you know, the important topics that we're going to discuss today in today, and we realize that the questions will persevere long after this discussion and the need to connect people and to, you know, provide support for early career researchers. So in the link in the chat that's placed now, we invite you to share your information with us. If you are so inclined to want to engage and be connected in a more lasting way as far as potentially setting up discussion boards of early career researchers, adding to a newsletter list or, you know, just being engaged in a more holistic way beyond this webinar, please do take a moment to share your information. And then we hope to be able to engage with you outside of this. So we thank you for that. And now with those resource sharing and information about staying in touch. And to start moving forward with the stars of the show today. I'm going to go in alphabetical order, but wanted to thank and invite Jesse Fleming PhD student and special education at the University of Virginia, who will be speaking today Jesse is a, you know, great proponent of open science speaker at our open scholarship practice on open scholarship practices and education research in 2021 and was just extremely, extremely impressive. So we're really glad to have him today. Additionally, I want to welcome and thank umo papawala. She is a clinical psychology resident at the VA Bedford, and really did a lot of the really interesting work in the easing into open science guide guide for doctoral students and advisors that was really inspiring for some of the questions that we're exploring today, and it's just doing, you know, really, really tremendous work so Dr. Papawala, thank you, and we look forward to hearing from you today. And then next, you know, want to invite and thank Sarah and Lou Wilson. Um, and slide, I understood, was I outdated already so congratulations, Dr. Wilson on read your recent defense of your PhD, we all, you know, really applaud that going on doing doing amazing things and being a champion of open science and special education and we really appreciate that. You know, Dr. Wilson go ahead and use that that term was also a speaker at our 2021 on conference on open scholarship practices and education research, and we really appreciate you joining us today so thanks so much to all of you. And with that, I will one, as we move forward into the discussion and presentations, really, really encourage the audience to submit questions submit comments we will be monitoring that consistently throughout the webinar. You know, we don't want this to just be a, you know, one way train of communication so if you have experiences to share questions to ask it will also be beneficial to your colleagues so we really encourage you to do that. So with that, I'm going to stop my screen sharing and invite Jesse to please share any thoughts or comments you might have. Sorry about that. My, my zoom controls flip to another screen. Okay, should be good to go. Can everyone see that okay. Well, thank you so much for having us Marcy we're really excited to be here and have this discussion on supporting early career researchers and open science and really thrilled to be presenting with two really great early career researchers who have done a lot of really great work in this topic. So, I figured it would, I like to start out my discussions with the metaphor, usually they're not very good. But if you stick with it, you might get something out of it but um, when I think about easing into something I think about easing into a cold pool on a warm day and being from Arizona in the that's the only thing you could do you have to sit next to a fan or be swimming in the pool. But, but when it comes to getting in the pool there's actually a few different camps on this you know a few schools of thought. One group definitely is just jump in and, and just, you know, without thinking go in as fast as possible, cannonball, get trying to get the lifeguard wet, you know, and then the other camp which I was in growing up which I, you know, I'm proud of is I got in slowly, I'd like to let that that cold water reach regions of my body slowly and just slowly ease in to to the water and I think that in some situations it is appropriate, you know, to to jump right in, you know, head first and get right involved, get involved right away and without doing much planning but we really think that in terms of open science that it makes sense to kind of, especially for early career researchers to kind of ease their way into this and, and we hope that some of these first steps will be helpful to you as you kind of ease your way into the practices. So my, my partner, my, my wife she she's worked in the startup tech space for quite some time and it's a pretty interesting space to be in there's some, there's some interesting people that work there and they do lots of interesting things but they've done a few things right that I've observed kind of from the sidelines. First of all, the, you know, the fully stocked kitchens and catered lunches that they nailed that. But the second thing is, they all seem to have their why really like established they're all that they're really grounded in why they have their company and why they're doing what they're doing. And you know they it's plastered all throughout the walls of their, their company, their CEOs get up and talk about it constantly. And they really know why they're doing what they're doing and I think that's really relevant to open science. It seems like everyone has their almost their own like Marvel origin story. Moon Knight, if you haven't watched it, it's great. I really enjoyed the first two episodes, but it seems like everyone in open science has kind of come to find their why and I think that's the first step for early career researchers is to kind of learn why these practices are important. See what research is being done taken afternoon, read through a few articles and see if these are legitimate concerns you know look in your own field and if there's not research being done in your field look in adjacent fields or other fields where maybe open science research is a little bit more established but look and see it you know look at the reproducibility crisis. You know look at, you know, replication on p values and effects sizes look at pre registration and register reports and how they might impact findings and outcomes. And once you do that, then you can find your own why and also you know find out how open science is going to be relevant to you and to your own research questions. And so step number two is to start small and access resources. I think it's important for everyone to find small ways to get involved today. You know we really reject this idea this all or nothing approach that we sometimes see in science that if you're not doing all of it. Or if you're not doing it all that perfectly then you don't belong in this community, and we don't think that you have to be doing everything. And that perfect is the enemy of the good that you can find your niche, you can find one or two practices that work for you and start implementing those today. You know for me at first pre registration and registered reports and open data were very daunting to me I didn't want to touch them, you know with a 10 foot pole. But as I thought more about some of these open science practices, I thought to myself you know I can post a pre print of my work, you know I can I can share my open materials I can make my codebook and my procedures open, and I can I can take another study and replicate that and so start with what feels easiest to you and go from there. And the next it's it's really important to access resources there's so many great resources available out there. A little bit of shameless self promotion here but there there's a lot of resources for individual practices you know if you're, you don't know how to post a pre print you know there's there's a lot of resources on how to do that. If you're unsure of how to find a registry or a pre registration template that you where you want to pre register your study there there's you know supports for that and then as you get into, you know more and more practices as it becomes more a part of your, of your research, you know there's more general resources where, you know multiple practices are discussed and it's very helpful to think about how they, they come together how they interact with each other how they should be used across the lifetime of a paper and I'm excited for umo to go into depth a little bit more about her paper a little bit later, but definitely a great resource for you to check out. Number three is integrate open practices into your current workflow. So, umo is going to talk more about kind of how these practices relate to each other, but I think it's important to also think about integrating practices into how you currently conduct your research. So oftentimes when I talk to other researchers or other early career researchers, these practices are viewed almost as like an afterthought and an addition and add on it's like, oh we better quickly do this and throw it into our project or our study. But but really, we think it's important to begin to think how you can incorporate practices into your workflow so that they become, you know, just your common practice, your science and how it's being done. And really think about how that can be done across the lifetime of a project is really helpful. This, this flow chart here was created by Sarah Emily, and I think it's really helpful for kind of thinking through some of these things. But, you know, for a few examples of this. If you want to apply for a grant, submit a proposal, you might think about how you might incorporate one or more of these practices into that grant proposal. And given that many funding agencies now require you to do this, it's probably a good thing to do. If you want to be able to share your data at the end of your study, you're going to have to include that in your IRB include the relevant language. If you want to share your materials or your data at the end, it's important to, you know, use good sharing techniques, you know, fair principles when when thinking about developing those materials and data so that at the end of your study isn't such a heavy lift to share those. If you want to share your study as a as a preprint or postprint, after you finish your study, it might be helpful to, you know, target a journal and look at their policies on preprints and postprints before conducting your study or maybe selecting a different target journal. So to see how these practices, you know, we have to start thinking about them near the beginning of our study in the middle of the study and at the end, or as Sarah Emily says more eloquently planning, enacting and publishing and really kind of bringing those in integrating those into our current practice. And then our fourth first step is to turn challenges into advantages. Ryan Nosek, he uses this framework to discuss change in academia. And, you know, I think it's really helpful to think about is, is this change possible? You know, is it easy? Is it normative? Is it rewarding? Is it required? And especially for early career researchers, I think thinking about, you know, changes in culture and incentives, they're really important. But depending on your field, at least in my field, you know, it feels like a lot of these changes are still kind of far away. And so if these practices, you know, speak to you if they're important to you, if this is part of your why, then we have to find a way to kind of use these practices to our advantage and not just focus on the challenges that they present. So Maddie Pownall, she's an early career researcher and she wrote a paper where she used this metaphor and I really like it. You get two metaphors today. And she compared early career researchers to either Vanguard or Canon fodder. And I thought it was really powerful to think about, are we asking early career researchers to kind of lead, you know, lead the charge, lead this change for the next generation of researchers? Or, you know, are they going to be casualties in a failed attempt to reform academia? And so when, you know, these are challenges that we definitely need to consider and think about, but also consider maybe how we might turn some of these challenges into advantages. So some of the challenges are, you know, that they're very time consuming. Some of them can be very time consuming. Some of the open science practices and, you know, early career researchers, you know, given we're taking classes, conducting research, sometimes teaching, there's not a whole lot of time to engage in something new. Some of the practices can be challenging and difficult. Some required new skill sets. And for those of us who are new to research, that can be very daunting. As I mentioned before, the incentive structure really isn't there yet. I haven't seen many promotion and tenure committees talking about this, hiring committees. I've been following the listings in my field and I haven't seen a single listing, you know, touch on any of these things, you know. It is difficult to think, am I jumping into this without, you know, much incentives in place? And then lastly, a lot of our peers, they're either unfamiliar with the practices or, you know, they don't think it's important. And that is a challenge to, you know, not only do these, but also convince our peers and maybe even our mentors that this work is important. So what can you do to kind of flip the script and make these, turn these into advantages? So I listed four practices here. There are many more and many more advantages that I didn't highlight, but these are just a few. And so when thinking about preprints, you know, it's an opportunity to post your paper right away, post it on social media and ResearchGate. And it's open to anyone. Researchers don't need to send you a request, right? It's there and available for anyone to access. If you're on the job market and you have some papers that are under review, and if your CV is like mine and it's not very big, you know, these under review papers are really important to your CV, but the hiring committee, you know, they can't look at them, they can't access them. But if you're able to preprint those, they can look and see the great work that you're doing and better evaluate you as a candidate. There's a lot of research that points to, you know, additional citations and downloads and exposure for open access and preprinted articles. So that's a benefit. And then it's an opportunity just for more eyes to see your work, you know, opportunities for collaboration. In the most recent OpenScience Unconference that we had, I heard a colleague share a story about how they had preprinted their study, submitted it to a journal, and before the study was published, two independent research groups had already reached out to them to conduct a replication in their own field. And so it speeds up the scientific process in a good way by offering additional opportunities to collaborate and for others to see your work. Pre-registration, you know, I don't think we do this enough. I don't think we highlight it enough in the paper. I don't think we do enough of sending a blinded copy of the pre-registration for peer review, and we don't do a good enough job of discussing its importance in the method. But it's an opportunity for you to say, you know, tell reviewers and consumers of research that, hey, I'm trying to be more transparent here. I'm trying to increase the rigor of my work, and it really is an opportunity to distinguish your research in my field. I know pre-registration is extremely rare, and so it really is an opportunity for you to distinguish yourself as a researcher. A replication, I think replication studies are really great for early career researchers. It's an opportunity to reach out to maybe an author or a researcher who is seminal in your field and to replicate their work. Let them know that you're interested in, you know, replicating their studies, see if they want to join on and collaborate. You can use this as one of your dissertation papers. I'm using a replication study for one of my dissertation papers, and it's really nice to have, you know, the study protocol and method already laid out for me. And so, you know, a replication study I think is a really great opportunity for early career researchers to get involved in open science. And then registered reports. I'm not going to touch on this one too much because I know Umo and Sarah Emily are going to be talking more about the dissertation and how you might include some of these practices in your dissertation. So there are our first steps, our baby steps, if you will, to getting involved in open science. And hopefully this kind of laid the foundation for you to be able to implement some of the specific practices that Umo and Sarah Emily are going to share with you. Thank you so much, Jesse. If you don't mind, continue. Great. Thank you. So to get started here, I'm going to be building off what Jesse started to talk about today. And I'm going to start with talking about the paper that my collaborators and I wrote in 2021. And to start off with, why did we create this guide? So, after I went to the Society of the Improvement of Psychological Science Conference of SIDS, back in maybe it was the summer of 2019, I started to get a lot of requests from various grad students and other folks being like, Umo, how do I do things? And I started writing these really long emails to folks of the things that I've learned, the things that I've even learned from my own experiences. And it made me realize that I would maybe that other folks could find this sort of guide helpful or a way of kind of getting started or easing our way into open science. And I also found that there are so many resources out there. Jesse showed us a few and by no means is this the best version of a way to ease yourself into open science. But this is just one way that I, my collaborators and I wanted to try to gather all the resources out there and try to convince them into one place. And the QR code, let's hope it works, should take you to the OSF page that I am trying to keep live. So trying to add on resources, of course, the paper kind of stays in stagnant in time versus the OSF project page. I can continue to update and add on resources, kind of keep the links up to date as much as possible. So of course, if anyone has any resources, they would want me to be list there, I always welcome those via email as well. But that's a way to kind of get more in depth if you want to learn more about a certain area. I think I saw a question in the chat a little bit ago around how to get started and kind of what's easy or difficult. And that is actually something that's in this paper as well. We tried to talk about these different kind of open science behaviors or changes that you can make and gave it a rating of easy, medium and difficult. And our ratings are really based upon our experiences. So other folks could be something that I might say is easy. Someone else could be like, I think that's difficult or something that I think is difficult. Someone else could say is easy. So I'm not saying that this is, you know, scientific evidence of what's easy, medium or difficult, but just a way to kind of gauge maybe level of entry into a certain behavior or if we'll use Jesse's metaphor from earlier, the temperature of the water. You know, whether it's a little warmer or colder in some ways. So I really encourage you to take a look at those different things to see what, oh, this is what's easy, medium and difficult. If that would be helpful to kind of gauge places to get started. But things that I want to emphasize while talking about today is there's no right way to do open science. And you can really start with any of these behaviors and changes just as Jesse had mentioned as well. It's really about moving science into being more transparent and really thinking about things in a way that's thoughtful and really trying to move the field of science forward in different ways. And that can happen in all these different sorts of behaviors. So when I think about like some things to pull out from the article of like what could you maybe even start thinking about today at the end of this webinar. Something like project workflow, like how are my folders organized is the ways that my folders are organized in a way that's easy to collaborate and does it lead to less error. Is there a way that I could label my have a have a way of labeling even my files that would help prevent error or help to keep version control in a certain way. So these are things like, you know, we might think about organization that's open science. Yes, in any way that you can think about trying to move your research and practice into a way that that helps science be more reputable. I can't say the word now, but you know what word I'm saying what I want to say, or being more transparent would be behaviors in that direction. Other things that you could think about starting at the end of this webinar today would be like starting a journal club, creating a space that you can talk with others and read different articles together discuss them. That's a way to keep yourself accountable and learn with others. Something else could also be transparent writing. So when we think about maybe you're writing a manuscript right now maybe you're writing a dissertation you know thinking about okay how can I write this in a way that really shows my line of thinking and being transparent about my decisions I made the analytical choices you made. And there's a specific article that that I say about really that gets into the details about what does transparent manuscript writing look like. And so now I will, I think we can go to the next slide. Yeah, so this is the other piece that I wanted to chat about was the ways that you can engage in these behaviors across the whole research cycle. So, when you think about it's kind of similar to the similar, the graphic that we saw earlier thinking about like all the different places in the project that you can think about open science behaviors. And this is just another way of thinking about it from conceptualization to design analysis reporting, and then to dissemination thinking about the different behavior changes that you can make or ways that you want to engage in different open science behaviors. So, another way of getting started that I think could be helpful we talked a bit about about this earlier but pre registration so I think pre registration there's a lot of different ways of doing pre registration that can be from really detailed to writing the research questions like maybe couple sentences and posting that on your OSF page or putting it into some sort of way that you can have a time stamp of what those research questions are. And that still counts as thinking about your research questions and hypotheses before you start the project. There are many templates online as we saw earlier and that can help move that along and you can even use OSF to be able to put your pre registration in your project, and so kind of makes it like easier to access as well. Things like we talked about pre prints and putting it on psych archive or even using reproducible code even if you don't write code, and you do mainly point and click there's ways that you can still save your output. So that you can see all the steps that you did. So these are just like ways across the different processes of a, you know, from the minute you think of an idea till you're disseminating some of the behaviors that you can think about. Next slide please. So I'm going to talk about a few examples from my experience with open science. So I'm going to start with pre registration and then I'll get into registered reports so my first experience with pre registration I'll have to say went. I felt kind of sideways. And so I had started doing this my second year of grad school and I wrote this, the, my pre registration very, very detailed I was going to be doing analysis I'd never done before. And so I was writing everything out I'm going to, you know, this assumptions held, then I'll do this analysis this assumptions held and I'll do that. I did a very, very in depth type of pre registration. And so how did I know that when the time came for me to do the analysis that since I had never done those before, like, everything that I thought I understood from the articles and the textbooks that I was reading. I realized I didn't really understand. And so then I was like, Oh no, when I put did this pre registration I said I was going to do everything this way now what do I do. And then I realized that I really learned from this experience and my research questions still were there. And I was able to still do the analysis and make the edits that I wanted and I just was transparent about that when I wrote up the manuscript. I talked about which things were in the pre registration and which things I did made the decisions after and why I had to do that. And I was able to learn that even saying and making some mistakes in my pre registration. I couldn't make the project any less valid I could still do it, and just talked about things in a transparent way. The reason I bring this up as an example is I think that sometimes people, it might feel kind of like oh gosh, I can't do a pre registration because I get myself stuck in in saying that I was going to do something and feel that you can't maybe do other exploratory analyses and this is just a way to show that I you still can you just are transparent about which parts are exploratory and which parts were in your Then this example here that's on the slide is about registered reports so we're going to be talking more about dissertations next with their Emily. But with my dissertation I just wanted to share that I did do something called a registered report. Which means that after my dissertation proposal was accepted by my committee, I wrote a manuscript that included the intro method and planned analysis and submitted it to a journal to go through peer review. This is before I had started any of my data collection and the, the register report went through peer review and I was able to integrate feedback from the reviewers to better my project better my dissertation, added in some analysis some pieces. I did have to, you know, consult with my dissertation committee to make sure they were okay with those changes which they were. And after a revise and resubmit, my register report was accepted for that as long as I did what I said I said I was going to do, then it would be accepted for publication. And so I think that this is an example of a way of doing open science that is also beneficial for a graduate student and can be helpful so I, you know, defended my graduation in March 2021, and the paper was published available for folks to read by October 2021. And so within that same year, the, the dissertation my paper was out my dissertation was defended and it was available for folks to read, of course you can always submit your dissertation as a preprint as well which would get it out there and, and therefore people to read. But there is something about being able to, to publish it and have it go through peer review again within the year that feels really rewarding and made some of the work on the front end worth it. So, and I also think it was a better project thanks to those reviews. Let's do a little bit of an example of registered reports. Thanks. Alrighty, thanks so much you guys. And so I'm going to talk a little bit first about mentoring and kind of some of my experiences with mentoring, not vertically but more mentoring laterally, among other early career researchers, and then I'm going to talk a little bit about what I did for my dissertation. So my dissertation. So in the field of special education where I reside. There's only two journals that are currently doing register reports and I felt like from my timeline for graduation that wasn't possible. So I chose to do the pre registration route, rather than a register report and a pre registration so I'm going to talk through that in just a minute as well. So, mentoring for my advisor as well as mentoring among other early career researchers has been really informative of my adoption of open science practices, and has also been really informative and helping me gain some of my own expertise and where I feel like I feel comfortable in implementing these open science practices and mentoring each other within an early career researcher community and mentoring across career stages is essential and helping again build capacity across researchers and institutions. Some of what we talked about about institutions adopting these practices, or things more structurally being slow on the uptake and slow to kind of adopt them more broadly. I think mentoring is a great way of helping address that just because you bring in more people and you bring in a stronger community. And that means that the community as a whole can kind of press for these changes on a broader scale. So I think that mentoring is really important. And to not don't do it alone. And in some instances mentoring has happened with my institution, but like I said, other cases has involved researchers outside of my home institution, and has really involved us finding our people connecting with each other, and learning together and trying things on as a group. So I kind of have three recent examples of that idea of mentoring laterally so across early career researchers rather than mentoring from a later stage researcher. So for example, I'm currently involved in a registered report of a systematic review at a journal in special education and that was really cool because I was working with other career other early career researchers who had some expertise and other aspects of open science they had done open materials, they had done open data, but they hadn't done a registered report. So it was kind of cool to be able to compile like all of our expertise and kind of figure out the process altogether and kind of tweak it as we went. And so that's kind of one example. And then another example is I have a piece coming out soon about open science and qualitative special education research and qualitative research has definitely received less consideration and attention than quantitative research. So really what we're trying to do with this piece was, it was myself and another early career researcher kind of just brainstorming questions that we had about how could open science apply qualitative research. And we were very clear like this is the beginning of a conversation. But here are these questions that we have here are the things that we're not sure we have answers for yet. But here are the things that we do think we have answers for and here's how we might go about working as a community as a broader field to address the potential benefits but also the potential unique challenges that might come along with open science and qualitative research. And that really came about by just again, laterally collaborating with another early career researcher at another institution. And then finally, Jesse and I are currently writing up a manuscript of a survey of open science practices among special education researchers. And we've partnered with two other early career researchers who were interested in open science to develop and conduct the study. And that's really helped again build our expertise we partner with people who are more had more expertise and other open science practices than we did. And that allowed us to really adopt and implement open science practices across the lifespan of the study because we each got to bring our own expertise to the table. And that, for at least for me I'll speak for me has been really cool and really helpful in and kind of thinking about that workflow across the lifetime of the project. For the next slide. So what is your role in the movement and Jesse and I talked about this a little bit last year and just felt like it was worth kind of revisiting. And so as early career researchers you have to be okay with feeling vulnerable and trying something new and that and that is Jesse mentioned with the cannon fodder, you know the vanguard like that can feel feel like a lot and it can feel a little scary to be like I'm going to put myself out there I'm going to try something like my advisor doesn't know how to do let alone like I know how to do, and I'm going to just try it. And so you definitely have to kind of go out on the limit there to a certain extent. And you also have to recognize that these open science practices can take a lot of time of extra time and a lot of extra resources. I have found personally that the more I've done them the faster gets the more comfortable I am with the process, and I feel, I feel like I, I can reuse things that I've used in the past, for example, in my dissertation, I am doing open data with qualitative interviews. And so now I have a really nice kind of boilerplate IRB language like a couple paragraphs on how I'm going to protect the data I'm going to go about achieving consent. And so now I can use that in future products moving forward. And many practices may seem straightforward at first such as registration and open data, but these practices are complex and multiple steps and may require additional training. So part of that mentoring is also going out and getting that additional training and hopefully things like this webinar and other webinars in the future can help provide some of that additional training as well. And so, again, the easiest way to start is just to try to try something new. Preprints to me are a really, really great toe in the water. They're pretty straightforward. Open materials are also relatively straightforward. There's some involvement of copyright, but if you produce the materials and you can add a creative promise copyright license to it, then like open materials can be a really, really straightforward process. And the other thing that I think is more straightforward to is pre registration and I'll talk about why I'm for early career research that might be especially true in just a minute. All right, Jesse, can we go to the next one. And so I wanted to briefly just mention this for thinking about early career researchers who are soon to be transitioning or might thinking about transitioning to mid career researchers. And then the next little bit of time the next year or two to continue mid career researchers can continue the mentoring of early career researchers, they can model practices they can collaborate. Again, it's not just collaborating vertically but also laterally with each other encouraging early career researchers and each other and also integrating these practices into courses and assignments so doctoral seminars and doctoral milestones. You can also do this as an early career researcher. I'm currently helping lead a project for a doctoral seminar that we will be using and adopting open science practices across the lifetime of the project. And so in that way, even myself as a fourth year and finishing up I am mentoring first years and second years into how to adopt these practices within their own study. And then as we mentioned earlier starting a journal club and kind of getting involved more casually almost about open science and so less of a formal of oh I'm applying this directly within a project but thinking more conceptually and theoretically about the benefits of open science within research. Jesse can you go to the next one. Thank you. And so as I mentioned briefly, I did not. I felt like, in part because of COVID and in part because the timeline that I was trying to capture for my dissertation. The previous or the register report process wasn't the right fit for me, but I did want to pre register my dissertation. What was happening is that I went through my committee and I went through my committee defense the proposal defense. And once my committee had approved my proposal I transfer the proposal into a pre registration and I found this process to actually be really straightforward and really helpful place to start this was my first pre registration. And so I had already had to think through all of these questions for the proposal. And so it felt like a very very natural transition to get this into a pre registration format. And I did qualitative research again. So mine was a lot more open and less about like this is the analysis that I'm planning on conducting. These are the themes that I'm trying to capture. This is who I am as a researcher. This is my reflexivity and positionality statement. This is the kind of the overarching themes within the interviews I want to get at. And then once I had started conducting the study and my interview protocols became more firm and set. I updated my pre registration with to include the interview protocols and then as the study continued I updated my audit trail within my pre registration as well. And so in that way my pre registration really captures from the very first like yes sample of approval on my proposal to all the way up through data analysis and data triangulation and working to write up findings from the dissertation itself. Now that I've completed my dissertation. My plan is to again make my data open. I'm also making my materials open. I was really lucky and that another researcher in my field had actually shared a similar interview protocol or similar format of an interview protocol that I wanted to use and she had made her materials open. So I actually used open materials to then create my interview protocols and those will be made into open materials as well. And so when you think about the question earlier about what's most impactful. I think open materials really has a lot of impact on the field because it allows researchers to build upon each other's expertise. Rather than saying this is my interview protocol. I developed this and it's going to stay on my laptop and my laptop alone and instead it allows other people to access it build upon it expand it figure out where it can go where it shouldn't go you know the bounds of efficacy and bounds of usage. But I think open materials is a really really great place to start and something that has quite a bit of impact on the field. And so now that I am in the process of writing it up I will include a discussion of my pre registration and the ways that I will specifically discuss my audit trail and how the pre registration and study plan changed over time. And then after journal acceptance I plan on publishing a preprint of my article and these feel like a very small but concrete ways to systematically get certain open science within the context of my own work. And check the questions to see if I missed any questions. I just see we can go to the next slide. Thank you so much. Yeah, this is so interesting. I think there is quite a bit of interest on the timeline for fitting the for fitting registered reports into this process from committee approval to the data collection and thank you for for kind of your timeline but just wanted to throw that out about thoughts on how to fit that within the process. Yeah, I can. I mean I definitely. So I defended in the mess I think October. And so when I started writing my proposal in the summer I didn't feel like that was enough time because they'll go through at least one round of reviews sometimes more. So that's definitely something that if you're if you're starting to think about what you want your dissertation topic to be in your like second third year like that might be something that you can talk to your advisor about like I already know what I want this to be and what I want this to look like But mine was originally going to be a registered report and then COVID head and then you know everything was like oh we're no longer doing research in school so I definitely intended to have this as a registered report at the onset of kind of when I started thinking about my dissertation study itself. But it is important to build in several months of review or time. To give you back on that I think one thing that allowed me to be able to do my dissertation as a registered report was also that I was doing a meta analysis. So I wasn't collecting data and having to run a study per se. So that gives you like a little more flexibility when you're doing a meta analysis then the meeting to launch a study. So that also allowed for for a little bit of extra time that I had to wait from May when I defended my dissertation to actually starting data collection and the process in mid November once it was accepted. And so I also will say that like to be thoughtful about if you do are interested and are able to allocate a little extra time and maybe start a little bit early is also to think about the journal that you are submitting to and like what is their average turnaround time. That was another thought that I had and one of the reasons that I submitted to Calabra is that they had kind of they pushed to try to have quicker turnaround times and that's something that the journal values so that was something that also been aligned with my values and what I was needing. So something to think about. Awesome. Thank you. I also had an interesting question. Broad big picture question. But I, you know, do think it's interesting. It seems like it's kind of values based, you know, but kind of going back to, you know, the point of starting where you can, you know, and taking steps that, you know, might be feasible for you and start where it's easy and balancing that with the larger impact of the world and the urgency of the replication crisis and what, you know, the larger, you know, demands might need. And I think people may come down on different sides of that. But if there any like just big picture reactions on how you balance those two needs would be interesting. I mean, I think that is an interesting thing to think about. For me, I definitely do fall on the side more of like. A survival almost, you know, like these are things that I can do. And this is where I can get started. And I know it's not maybe changing the field and the way that I want it to I know that down the road that I hopefully can, you know, do a registered report, or, or, you know, conduct a study that might be more impactful to the field. But, but right now it as an early career researcher, I still think it is important to, to kind of provide input and where you can, you know, change where you can move it forward where you can. And then hopefully as you continue to gain experience to gain research experience, then you can, you know, hopefully, you know, do more of a registered report, or, you know, do more pre registrations, things of that nature. But it is a really good thought. And I definitely think that, like you said, Marcy, some people will fall on the other side of I'm going to do the most impactful thing that I can. And I think that's good too. Something that I find when I think about like, moving science forward or the field forward as I want to know more about all the studies that people do that don't have significant results. And when I do a lit search I want to learn about those so I can ask different questions, or I can try a different approach. And so when I think about like, by doing pre registration, and just like writing my research questions before I actually do the analyses or write out what I'm thinking and what I'm hypothesizing and then showing actually this was not significant. For me that's like one way of wanting people to see what I really thought before I saw the data. And so if that's one way I guess like I think about moving the field forward and helping to address this but in a way that feels like bite sized for me and where I'm at right now. I think the last thing I would kind of add. I talked about open materials is being helpful for like other researchers doing studies. It also helps like, you need open materials in order to replicate things like if you don't have the intervention materials, it's, you can't replicate it there's only so many so much room within a journal article to kind of talk about procedures and talk about what you actually use so instead of just like describing it if you can actually say like this is the intervention that I use or this is the form that I use, like that others can replicate I also think particularly within the field of education I know it's not just educational researchers here but particularly within that field of education. We have a giant research to practice gap there is a huge gap between research validated materials and what teachers and practitioners are using in the field. And materials can really help address that by attaching the research validated materials to an article and saying like this is something that has worked in this setting this is who it worked for. And this is something that we are giving you the materials so that you can use this in your own classroom in the field so I think that's another kind of impact consideration to make. I think one of the sayings is, if you can't run walk, if you can't walk crawl. So everyone if contributing as best they can wherever their certain statuses and position is of what they can do for the advancement of the field. So, any last minute remaining questions we are at the top of the hour. We do as I emphasized, not with this to be the end of the conversation. So hopefully we can follow up with additional questions and and discussion forums at a later date. So be looking out for communication from us and any last remaining thoughts from the panelists. Great. Well, thank you so much. We really appreciate it. Thank you, Sarah Emily. Thank you, umal. Thank you, Jesse. We appreciate it and thank you to all the early career researchers that are really keeping up the good fight right now. We, we salute you. Thank you. Hi, go science.