 And the moderator for our next session is Brianne. And so I'll hand it over to you, Brianne. Thank you. I'll just share my screen. We're just waiting for one more panelist. But I did hear from Kleber that there's a rainstorm. And his power was going out flickering lights. So hopefully. Oh, there they are. So it looks like everybody's here. Let's give another minute. Thank you to our European colleagues who are joining in the middle of the night. Great. I'll get started. So thank you for joining our session on empowering early career researchers to improve science. Our panelists today are Humberto Dabat. He'll be talking about his role in panlingua. We have Dr. Nafisa Jadavi, who will be talking about her role in reproducibility for everyone. We have Dr. Gary McDowell, who's working with light toller as a consultant. And we'll talk about his work with future of research. We also have Kleber knees, who will be talking about his role with the Brazilian reproducibility initiative. I'm the moderator today, Brianne Kent. I was an organizer of the event that we'll be talking about today that really inspired this panel. And I'm an assistant professor in neuroscience at Simon Fraser University in Canada. I'll welcome our co-moderator Tracy Weiss-Gerber, who is a member of the E-Life Early Career Advisory Group and works at Quest in Berlin. And she was the lead organizer of the event that has inspired this panel. It is very late in Germany, so she's not going to have a formal presentation today, but she's joining us just the same, which is wonderful. So this panel was inspired by ideas generated during a global virtual un-conference. And an un-conference is an unconventional conference, which really tries to take the strengths and benefits of the coffee chats that happen at a conference and turn it into the main highlight. So we're really trying to promote discussion, debate, participation in an un-conference instead of just having somebody give a talk and people listen like what's happening right now. So our un-conference had brought 54 invited participants. They were mostly early career researchers who had extensive experience in improving science culture and practice. And the details of the event have been published in an article. I'll give you that link later. But the results of the discussion and the outcome of the discussion of the two days are all posted in a pre-print on osf.io right here. And the pre-print is called Empowering Early Career Researchers to Improve Science. So I'll just start by saying thank you to all the participants who attended the event, a welcome trust who provided some funding, and a special thanks to my co-organizers of the event, Tracy Weisgeber and Constance, who hopefully is asleep in bed in Germany. So the un-conference covered four main topics. The first was why do we need early career researchers to improve science? The second was what obstacles do early career researchers face when working to improve science? The third was how can other support early career researchers working to improve science? And we left concluded with tips and strategies for early career researchers working on science reform, drawing from the experiences of the 54 participants, to say what worked, what didn't work, what do you wish you knew at the beginning? And so when we say trying to promote science reform or trying to improve science, we really mean a broad range of different topics. So some groups are working on trying to improve and modernize publishing with open access journal articles. Other groups are working on reproducibility of science. Others are really focused on changing the rewards and the incentives. Other initiatives are focused on public involvement and promoting science communication. There's also those that are trying to increase diversity in science and make sure that there are more perspectives from around the globe. And as well, early career research, their training and working conditions. So there's just a wide range of topics that we're referring to when we're talking about science improvement and science reform. So why do we need early career researchers to be part of the reform efforts? Well, early career researchers are the future leaders. They're the most diverse cohort of scientists, so much more diverse than their mid-career and senior scientist colleagues. Early career researchers, because they're early in their career, may be more open to new solutions than more senior scientists who have had their careers, built their careers in the system that is now. Early career researchers are also more often on the forefront of technical innovations because they're actively doing the science. They're still at the bench. They're still seeing the innovation that's happening and being a part of it at the bench, in the lab, in the field. And so they're really aware of where changes can be made and how improvements can actually benefit the science and how it's done. Some early career researchers may also have the time and energy to put into research improvement activities in a way that sometimes more senior researchers, more senior academics, who have a lot more responsibilities and commitments may not have. And importantly, early career researchers are the largest cohort of scientists. So if we are going to see improvements in science, if we are going to see reform efforts actually come to reality, we need early career researchers to be a part of it. So to learn more about the outcome of our result of the event, please see the preprint. We also have another document on OSF.io with the specific tips and tricks for early career researchers working to improve science. And we have an article which explains how we brought together scientists and researchers from around the world in an asynchronous virtual unconference. So I encourage you to please check out these resources to see more details. But today, we have a wonderful panelist who will each speak for about five minutes about their initiatives and their experience with early career researchers improving science. So I'd like, oh, and just to note, please put your questions in the Q&A and not the check. We'll have Q&A at the end. So first up is Dr. Humberto Dabat, who is a research scientist with a permanent position at the Institute of Plant Pathology in the Center of Agronomic Research of the National Institute of Agriculture Technology in Argentina. Humberto studies the interface of viruses and crops from a systems biology perspective. And for the past year has worked in the Argentine project on SARS-CoV-2 genomics. Humberto is a member of the advisory committee in open science and citizen science of the Ministry of Science of Argentina and has been an ASAP bio ambassador and E-Life community ambassador and affiliate at the bio archive preprint server and co-developer of Penlingua, a multilingual discovery and reading tool for preprints in the life sciences. So Humberto, did you want to show slides to share your screen? Well, I don't have slides to share. Okay, perfect. Thank you, Dr. Kent. It's a pleasure to be here. So the majority of scholarly work in biology is published in English. The language most of the world does not speak. To help remedy this key issue, hindering inclusive scientific dialogue, we built Penlingua, a multilingual preprint search tool intended to enable search and global access to machine translations of all preprints hosted at bio archives. At Penlingua, users can enter search terms in their native language and view search results linking to the full text of all available articles translated into more than 100 languages. But language is just one of the barriers affecting global scientific communication, especially among our communities. Latin America represents 8% of the world population, 4% of researcher, and 5% of global academic publications. 30% of our people lack access to the Internet, 30% is poor, 62 million live in extreme poverty. We are a region of asymmetries and contradictions with tremendous disparities, culturally diverse with one of the lowest global R&D spending. We are thrilling societies, we produce awesome publicly funded science, our salaries are ridiculously low, we are resilient, we are working, created minds, we are so poor. Doing science in Latin America is about passion, empathy, solidarity, community, and responsibility. As we wave our manuscript around with our humble results, balancing visibility, affordability, and institutional requirements, we fight to disseminate our findings, for months, sometimes for years, in this advantage, against all odds, despite setbacks. In addition, we are seeing a transition in the publishing ecosystem, the ground is moving, the advancements of the open access movement is a flag towards the democratization of knowledge. Nevertheless, we perceive that this flag has been co-opted by some players in the industry which have accommodated their business model in a way that could perpetuate the asymmetries of the color publishing and exclude even more researchers than the color communications. We are seeing a shift from paywalls to publish walls. We are observing the preposterous inflation and expansion of the so-called article processing charters which are not only unaffordable for our region, they are unethical. The discussions about APCs transcend open science, it's a discussion about constructive views of academic communication. It's about privilege and social justice. It's about inclusion. To encourage APCs to view scientific knowledge as a commodity rather than a human right, rather than a public good. Brazil spent $36 million between 2012-2016 on APCs equivalent to the cost of providing sanitized water for a year to more than eight of the 77 million Latin Americans who do not have access to drinking water. I live in a country where 80% of scientific activity is financed with public funds and where 40% of the population is poor. Our relevant students have incomes below the poverty line. This scenario implies that this is candle to pay exorbitant figures so that five publishing companies that exercise an illegal policy in an important market continue to accumulate wealth. We are expecting where with our 5% of articles we'll end up. How this figure may diverge as CCRs. We need to be a part on the sheets of publishing practice, encouraging the non-commercial roots of academic communications and supporting the development and maintenance of communication infrastructure led by and for the academy. It is becoming evident that many journals reflect an chronic 20th century predigital platforms elitist reserved for certain affiliations battling mostly mainstream science event for the few, a chat among privileged and highly funded research. Many white, many rich mostly men. I am failing to perceive how the academic publishing ecosystem values diversity in the world. In the last few weeks, they plan to take to modernize their journals. How are they working to make their venues more inclusive with more gender balance to be platforms that embrace more voices from the south where science is a global conversation. Beyond publishing our funders should be redirected in the resource we spend on subscriptions. 11 Latin American countries spend $100 million mostly public funds when access to academic journals last year we are giving millions to reach scientific knowledge that should be free. We should immediately cancel the waste of research in leonine contracts with the commercial publishing industry who has sequestered our scientific legacy. This issue transcends academic publications and involved research assessment practice. We are affected by monopolies of the indexing system and bibliometric indicators that unfailingly accentuated the dichotomy of mainstream and peripheral science resulting in, as Camero Nilo says, excellent in research as a neocolonial agenda. I think we have an opportunity in this context to break the vicious circle that commercializes evaluative cultures to better circulation indicators over journal metrics and redirect our indicators towards the public impact of our work beyond impact factors. We must remember that our research agendas must be aligned with the prosperity of our region and not with the creation of a defined for success in an order market. We are not in academia to accumulate publications in journals to advance our careers as individuals. Science is a collective enterprise that has to look towards society and understand its demand for knowledge. The way to strengthen our communication system is to align it to our society, to our needs, to our history. The real impact of our research has nothing to do with rankings. Our communication system is a trend if it is faced in society to the extent that it generates inclusion and well-being of our people and that, I think, is where ECR should lead a wave of change. Science is a shared enterprise, a global endeavour enriched by the multiplicity of visions, realities and languages. Everyone benefits from the development of a more inclusive ecosystem and seamless international scholarly discourse is a real possibility. Many barriers are stopping this utopia as ECRs we have the opportunity to transform research culture. Let's embrace this responsibility. Thank you. Thank you, Humberto. Our next panelist is Dr. Nafisa Jadavi, who is a neuroscientist and assistant professor at Midwestern University and research professor at Carleton University. Her laboratory investigates how the brain responds to different biological processes throughout the lifespan and specifically how maternal nutrition contributes to offspring neuro development, neurological diseases and aging. She is the chair of the advisory board for reproducibility for everyone. Thank you for joining Nafisa. Thanks, Dr. Kent for that introduction and thanks for including me in this panel. Are you able to see my slides? Yeah, looks good. Perfect. Thank you. So I'm going to speak a little bit about the reproducibility for everyone initiative that I've been a part of for a few years. Dr. Tracy has also been part of this initiative as well. So the reproducibility for everyone initiative is a community-led education program where we try to increase the discovery and adaptation of reproducibility tools at scale and so what we do is we run workshops at different scientific meetings at different institutions to educate individuals about reproducibility tools. And since the initiative has started in 2018 we've had about 100 plus volunteers that have run over 50 workshops across the world. These are international workshops that have included over 3,000 participants . So we've been really active in getting this information out in terms of different tools that can be used by researchers, early career researchers specifically as well as mid-career or late-career researchers in terms of implementing tools in their research laboratories. And why we started this initiative was that there was a lot of things that were missing in terms of reproducibility in that discussion in the biomedical sciences and in other science fields. The majority of researchers being left behind and in terms of scaling these initiatives and different tools that can be used to be reproducible there was that missing link and in terms of focusing on how an individual researcher's work can benefit was also missing and what we wanted to do was to include innovative ideas that could easily be implemented by researchers that attended our educational workshops. And so one thing that we end off our workshops with is we present a lot of information and we also ask researchers I know you're overwhelmed, we shared a lot of ideas but take one thing and try and implement it into your daily research program something like an electronic lab notebook or writing up protocols or things like that to help move your research forward in terms of it being more reproducible. And so these workshops do which I've hinted at a little bit as I've been talking is that they provide this overview of different open projects that researchers can get involved in. And what we try and do is we keep our workshops to about 30 to 90 minutes and we try and target a really large variety of audiences so in a number of fields plant sciences to the biomedical sciences and we try to hit all the different career stages because anyone can really implement the tools that we discuss in our workshops if that's something that they want to do into their research program. We recently published our work in eLife outlining our reproducibility for everyone initiative and what we do and how we do it all of our workshop material is freely available on our website. You can also, individuals can also watch workshops that have been recorded and we often get instructors or facilitators for our workshops that have been prior attendees. And so if you're interested in getting involved or learning more, please visit our website don't be shy and we're always looking for people to get involved and volunteer in different aspects of our initiative. We're looking to get a lot of people involved. We have some funding from the CDI initiative as well as other sponsors that I'd like to thank for supporting us throughout the years and currently to let us do our great work and to have a permanent staff member who does a lot of the infrastructural work and initiatives so thank you very much and I look forward to your questions. Oh I was muted just a reminder that if anyone has any questions please put them in the Q&A box and we will get to them at the end. So next up is Dr. Gary McDowell who has a background in biomedical research and co-founded the nonprofit Future of Research which seeks to advocate for and with early career researchers to achieve systemic change in the academy. He ran Future of Research for three years full time and has now continued working to help future generations of researchers to reach their potential in his new role as a consultant. Providing expertise on the early career researcher population to organization and providing early career researchers with strategies to effect change. Welcome Gary. Thanks Dr. Kent and thanks to everyone here so great to see you. So I have been involved with advocating for and with early career researchers for getting on for the last eight years now. First as Dr. Kent mentioned in the nonprofit Future of Research and that organization sought to communicate the issues faced by early career researchers in their academic environments and proposed solutions to overcome those problems. And we did this primarily by hosting conferences or workshops. We would gather lots of people in the room and we would talk through the problems and try to come up with solutions strategic ways of overcoming those problems. And then we as an organization would take those things forward and communicate those to stakeholders such as funding organizations and universities. I continued to work in the space as a consultant and as Dr. Kent said more specifically helping organizations think about how to better serve this population and generally just sort of working as a freelance academic in a space thinking about grad students and postdocs. And the issue of including early career researchers in science improvement, for me this is really an issue of representation. Most of the organizations and institutions that hold power in science and in the research enterprise their most powerful committees and structures are dominated by faculty dominated by senior faculty and also dominated by faculty from a select group of institutions and so are not representative globally even within countries even with all faculty. And so in order to have a realistic sense of what research looks like in order to make decisions in those structures it's really important to have representation and that includes across career stage and particularly thinking about the people who are certainly in biomedicine which is my background people are at the bench doing the research what the day-to-day looks like for them and what their environment is like in trying to succeed as scientists, succeed as researchers and to take the science they're working on forward it's really important that they be in the room. And so that was a lot of the work of future research included trying to get more representation of people into those powerful places into those rooms and as someone who sat in some of those rooms myself as the first young person in some committees it's really kind of scary some of the misconceptions that there are when those voices are not there so representation is very, very important. I think one of the key lessons that came up for us was the need to have a broad ecosystem of people affecting change and I think one of the things that was really useful in having someone in a full-time role like myself who was outside the academy who had left the traditional structure and was no longer subject to concerns about existing in that structure. I'm able to speak much more freely about what people are experiencing and I often try to speak from a place of gathering data and communicating that data very data focused and so that has been really helpful in trying to communicate things and I think it's important to try and consider having people outside the academy and within as one vector of thinking of who in the science ecosystem is involved in these conversations and that has certainly been really helpful for a lot of the things we were working on and I think it's important too to think about a broader connection of all kinds of people in this system trying to affect change including early career researchers because we find that not only is there the problem of general turnover of grad students and postdocs in temporary positions sometimes even with faculty but there's an issue that within advocacy there can be quite a high rate of burnout and so trying to spread work across numerous people and across numerous organizations really collaborate and work together and share ideas and share knowledge is really important and that's actually why the unconference that we all participated in was such a great event I was able to learn so much and hear about what was going on elsewhere and really I think this is sort of the next phase in the landscape of early career researchers affecting change of groups now who have gotten to groups with the issues of establishing themselves, of setting up and of starting to get their foot in the door and now I think it's important for us to try to think about all of us connecting together better and certainly thinking about being in the US researchers are told to be very independent and there's a big drive to be working independently and so I think it's hard for a lot of academics to try and think about changing the system with other people and together so I think we have to think about that a lot more thanks so much for having me here Great, thank you Cary and our last panelist is Dr. Cleber Nieves I'm so glad that your internet hasn't been affected by the storm Dr. Nieves is part of the coordinating team of the Brazilian reproducibility initiative PhD in Neurosciences and a Bachelor of Science in Biomedical Science from the Federal University of Rio de Janeiro his neuroscience research focuses on brain evolution and complex networks and since 2018 he has worked on the Brazilian reproducibility initiative and on the no-budget science hack week as well as on many meta-science research projects on issues relating to reproducibility preprints and translational research Welcome Thank you, thank you for the invitation I was lucky so the lights just went out like five minutes before this started but it seems to be fine now can you all see the screen okay so well thank you for the invitation Brienne mentioned that I'm from Rio de Janeiro and so I've worked in meta-science into main initiatives which one is the Brazilian reproducibility initiative I'm part of the coordinating team and well this is a replication effort much in the model of the ones that came before like replication project psychology or cancer biology where we're gathering multiple labs to reproduce experiments that were published pretty much in line with the previous panel about big team science it's one of those and this is ongoing and the other initiative which has a meta-scientific band is no budget science which started before used meta-science as a term as a name for this but we were discussing what would be meta-scientific issues in the university back in 2016-15 and this eventually evolved to become a more training focused initiative when it became a hack week which is now and we just had the third edition last month and well this is people gathering for two weeks or one week and trying to develop projects on meta-science or tools to improve science somehow and this has been going on both have generous funding from the CERPLERA Institute which is a private founder of science in Brazil so we're doing early career research so no budget science is more focused for training and it's more directly related to the issues we discuss in the virtual brainstorm paper the unconference but the insight I want to bring from the lesson learned from these two initiatives is that one thing that we mentioned in the introduction that ECRs are postdocs, red-edged students or even in our case even the PIs part of our collaborating teams actually are very young and these are the ones doing the experiments and discussing with us the coordinating team the nirigri of the protocols so these are the people who are actually caring about and implementing all the recommendations we do in terms of reproducibility and no budget science is very focused on training so and usually again as mentioned in the introduction the people who have time to engage in training in new fields usually are ECRs and on the one hand having ECRs being the ones who are engaged in improving science in meta-science is great because they are the future they hopefully will be here for a long time and this early exposition to the issues in meta-science will have a long lasting impact in their careers and they will hopefully spread that to other people and that's a great strong suit for ECRs but on the other hand these are the people who are collecting the data and the people who are caring about improving data collection and how we do science in general these are the people who are lowest on the academic hierarchy and they have unstable jobs and they don't have much certainty about the future they don't even know if they're going to be able to remain in academia and that is this reality varies from place to place but I think there's some universal truth here of course this is all based on my impression I'd love to hear about data on on this and one thing we do see in particular and it's made me think a lot is that from nobody's science we get to experience that people come and from those two weeks where they're doing the events the hack week and it's very collaborative and everybody's on Zoom all the time because we're doing virtual events now and during those two weeks it's very great and the projects move very fast but after those two weeks it's often that people just drop out of the projects because they have other priorities so this has made me think the survival of meta science as a future or the movement for improving science if you don't want to talk about the discipline itself but this whole motivation to improve science if it's really dependent on the motivation and we win free time off ECRs it's not really sustainable because as long as meta science activities are not rewarded people when push comes to shove they will prioritize the thing that gets them doing jobs and papers and publications and the things that actually make you advance your career in academia or even the things that give you opportunities outside of academia and I think this ties into one of the first panels in the morning today or my morning that was talking about how we should go about institutionalizing meta science and I think the institutionalization of meta science maybe is a good idea to create meta science departments but how meta science will become a more mainstream part of the academic structure will in large part be determined by or will determine how ECRs will come in to more permanent positions and how they will become a part of academia in the long term so I'll end up on that note and I ask you that you confirm or disconfirm my impressions that this is the case I don't know if there's much data on that so thank you. Thank you so much so now we'll turn well actually first I do want to give Tracy an opportunity to say anything or have any comments before we turn to the Q&A Yeah I think thank you to all our panelists for joining I think our goal was to give you a sense of some of the various different things that ECRs are working on and the power that ECR initiatives can have in changing so many different aspects of science but I would also ask you to remember that this is a really small portion of the participants are in our original event where we had a lot of dynamic discussions about not just what people are doing but how they are doing it and what we can learn from other ECRs and how we can be successful in founding initiatives and leading initiatives in building communities around their initiatives often in very difficult circumstances and often these are very widespread or global communities like for example Nafisa's example of reproducibility for everyone and so I think I would really encourage you to use this symposium to ask questions about how to do those things and what it is that people are doing. Great thank you Tracy so we have one question that comes around in the chat but I do encourage attendees to put their questions in the Q&A box so that everyone can read it so this question is for Dr. McDowell Dr. McDowell you had made a point that we need to get more organized in the way that we advocate for change across the various ECR or open science initiatives and organizations. In other industries unions are a powerful mechanism for organizing individuals to take action collectively yet we don't have a union for the global research community is it time we create an open science union to advocate for progress and researchers interest? Great question I'm very pro-union generally unions have great power in effecting this kind of change I think this gets to the point of we don't need to necessarily reinvent the wheel there are people in this world who are very effective at pushing for change and advocating for change and it would be great to use those structures and lessons in our own context I mean one thing that this may be think of is I'm really disappointed with a lot of scientific societies right now because I'm a member of some societies that have been advocating to block my access to papers because I don't work at university I don't have institutional access to publications and so you know I have to get them through SciHub essentially and yeah I find myself in the weird position in the states in the last couple of years advocating to the Trump administration which was one of the greatest champions for open publications in the states because of the pushback against it from scientific societies and publishing organisations and so I think there is more it would be interesting to think about is there a need to think about a society that is focused on science rather than academics and the way academia and academic interests I was really struck by something Humberto said that the point of science is to solve problems and to do this research it's not to publish papers and it's not careerism and that frankly is a major reason that I left because it's that is the direction that I sadly see things are in and so I think that this is a super important point of like the what organisations should there be broadly it could be a union but is it that there's a problem also with the structures that we have Gary so another question now is for Humberto sorry which obstacles do you think affect ECRs specifically in Latin America like what are the obstacles that are more prevalent for ECRs in Latin America well during my short talk I mentioned ECRs which is a very important issue in the region which affects broadly all our lives activities and puts in a different perspective our role as scientists what Gary mentioned just before is that we have a very important responsibility because we are planets but taxpayer money in countries where there is a lot of lack of funding and budgets for social and health etc so every time that there is some funding to do the research you really have to think what you should be doing with that money what is that research going to help your society the people where you live and of course during the pandemic that has raised it a lot it's like we have joined forces with many ECRs in our countries in different regions try to prevent the loss of life so that it's like a different issue before the pandemic we were thinking in very separated without the real community without integrating our capacities and I think that the pandemic has helped us try to understand that we are towards the same path that we all together want the same thing which is to try to generate knowledge that is useful for our societies so in that sense I think that one of the main issues that we have now is try to convince our leaders that investing in science is the best way to try to solve the problems of this society what we have to do is try to push into the authorities to try to provide more funding for our activities and being ECRs the most prevalent researchers all over the world as you haven't mentioned before we are like most of the people doing science so we should join forces to try to push to an agenda where we're funded is a reality so problems in our societies with a specific budget in the long term is more specifically thank you next question I don't want to just say I think this is something that I think about a lot that metascience is very focused at least so far on how we do science right that how we get the answers and how we get answers that we can trust and I think the whole point about useful knowledge and knowledge that is relevant to society and to local concerns is really about the watch like what questions do we want to answer and of course we need good answers but there's a reflection that comes before it's about the contents not the method that's such a good point so our next question is for Tracy I was watching your presentation on metrics today and it seems that follow-up of students is good in your initiative and they finish the project I think that has a huge overlap with the questions raised by Kleber how you and Kleber think that ECRs could be more consistent with these initiatives no budget science and your initiative with Quest so how could ECRs be more consistent with these and follow through okay I'll briefly provide some context for those who were not in the in the metrics webinar earlier today essentially so I started an initiative through the E-Life Ambassadors program called the E-Life Ambassadors meta-research team where participants learn about meta-research by working together to design, conduct and publish a meta-research study and I've since translated that into a six month course that I run in Berlin with students from four different Berlin universities and I think one of the things that's really important in the success of our projects and the reason students are able to finish is that they're all working together on the same project and so if students are sort of individually working on an idea they get to a point where they realize they don't have the resources it's going to take a larger team than they have and at that point it becomes difficult to proceed with the project especially if they don't have outside support from their supervisor or others in their lab group or community and so part of how our initiative works is the fact that everyone agrees that we have this goal coming in to design, conduct and publish a meta-research study and then it's also that because participants are often working in small groups no one wants to be holding the group back and so everyone is motivated to keep working and keep moving their part of the project forward and they see other groups doing the same thing in our meeting every week and we also use other ways of motivating so we encourage like we've done blog posts or other things so that students have an early output to share the work that they're doing or the things that they're learning the students have in the past found conferences where they can share abstracts and start to talk to other scientists about their meta-research and so those are some of the strategies that we use to keep people motivated in moving forward in our initiative and I think Cleaver can perhaps address the question more related to the no-budget client's week yeah I actually heard about your talking matrix because other people from the group were there and sadly it was very relevant and I should have been there but I was actually trying to learn from you because we actually have very very bad follow-up and we are now experimenting this is the third edition we always change the format a bit and we're experimenting now with regular groups so one month after two months after we try to follow up closely to the project to see if something comes up if people have a longer term commitment but I'll know if this works in six months maybe yeah I think the group mentoring approach can be a very powerful one as well as emphasizing to people from the beginning that they will need a team and to think about who they might be able to get and encourage to join their team I'll share a link in a chat for a couple minutes for a paper that talks about our initiative for running a participant guided learn by doing meta-research course which has more details for those who are interested thank you so the next question I think is directed at Nathisa I am a young neuroscientist and I have the impression that the scientific endeavor has become about selecting convenient data to publish convenient stories I think companies like this conference are very useful but I have the impression that tomorrow I will go to work and life will continue to be about finding significant differences and putting aside what does not go in that direction how to involve academic directives in the direction of open and reproducible science that's a great question thank you Danielle so yes I mean the academic currency is publications right and there are journals that are very pro-significant differences in telling the story but I think that there is a movement about publishing negative data and data that is not significant unfortunately this movement is really slow and there's a lot of resistance to it because we want to cure X disease or X whatever inserts your field of study here so I think it's definitely hard a hard place to be as a trainee when you're in a lab that maybe is pushing for publications and positive data and significant differences but I think what I would recommend or trying to do is to build a network of people that might be open-minded towards not that viewpoint and to get them involved potentially with your work and to get feedback I always advocate to my trainees and other trainees and students to build a community or a network of people and to get feedback on how you can you know sort of start this change within your area or within your school or program or whatever, wherever you may be I mean it can seem very daunting because the scientific world is you know very old set in certain ways but I think that if you're able to make some change where you are then that's great that's some people that were affected by your way of thinking about data and that you know it's not always has there doesn't always have to be a significant difference you know this is something that I've tried to do when I started my own group is to foster this thinking and that you know I am yeah the PI and my name goes on the paper and the grants and stuff but we have a discussion about the data and what it means and you know people we all work together I'm at the bench, students are at the bench staff are at the bench and so and it's my little lab but I think that you know I'm training students that are going off and doing their you know their future training in other places so hopefully I'm trying to sort of plant these little seeds of change even you know in an area or in a field that is very dominated by finding significant differences because I too myself and also a neuroscientist too so hopefully that answers your question or gives you some feedback on this very touchy subject speaking of touchy subjects the next question is and it's not directed at anyone individual but I can I'm sure several of you may have thoughts do you think there is a selection effect where early career researchers who stay in academia and become senior researchers are the ones who are least bothered by the system as people like yourselves either leave or pivot into meta-research how can we strategically get reformers into positions of power what any of the panelists like to tackle that I will give it a go because I have given this a lot of thought I think there's a broad distribution effect where the simple answer could be yes but I think there's a lot of caveats here and the first is there's a phenomenon in business research actually that people who resign from organizations generally there can often be a common factor that they are the people who care the most about it and just cannot persist in that space any longer they care so much and I think that there's certainly that element that played in for me but I think there are people who try to find a way of what it is that their values align with and whether they can affect change based on what it is that they can see being able to do and if you just said change is very slow and the academy which is true and I think this is a big part of why I just didn't have the patience for that and so that I think all of these factors come in for what determines but I think it raises an important point that some of the things I have heard that are the most awful about how the academy shouldn't change have come from very junior people junior faculty and because there are a subset who are super bought into the system because they have succeeded in it and there's a lot of this it is important not to try and have a broad brush of oh all junior faculty do this and all postdocs do that and all these people do this because there is a great spectrum of people bought into it and people not bought into it and I think you will have some stay and go I think it just takes a lot of persistence to stay in if you also see the system is broken and so I have my favorite faculty to work with are the ones who are like really see the problems and then are also trying to figure out how to deal with that but there are some who it's I think it helps a lot of people not to be bothered by it or that they just are bought into the system and that's their way of dealing with that issue can certainly arise and I had when I was leaving I had people say to me oh it's such a shame that you're leaving and you know you should stay or whatever and I was like well I don't really want to but also I think there is again this speaks to the earlier point of I feel I'm more helpful outside than I would be inside I feel I would have been a terrible PI trying to have like mentor people and do work and also try I don't think I would have been very good at any of it and so I think better distribution of labor and supporting each other is key there and so yeah but again getting and then we need to help the people who are still in the academy who are good people to get into those positions of power I think this is all again the reinforcement and the network needing to be strong yeah and like Gary said there are people at every career stage who are fighting the good fight you know there are there it's not we're here talking about the role of early career researchers but so much of what early careers are just can do is really working with some people who are in these positions of power so certainly you have people bought into the system at every stage and you also have people actively using their positions of power to also promote change so I think that was like a point would anyone else like to speak to that or should we go to the next question I can briefly add a point Humberto a very cute dog that was not my point my point was that the key thing that we kept hearing over and over in the young conference and you'll see it prominently placed in the preprint was people were saying over and over this work that I do to improve science is not rewarded it's not incentivized and it can actually it's actually in some cases perceived as a distraction that takes away from my ability to get grants and published papers and until that changes it's going to be very hard to keep people in the system outside of psychology it can be very difficult to find journals that publish meta research it's hard to find funders that will fund it there are lots of people who don't think it's research and there are very few places where meta researchers can get jobs within the scientific system and so I think solving that for not just meta researchers but people doing all aspects of science improvement is really critical to change I also think that just finding if you can a space in the system where you can make change is really important for me I started doing meta science because I was frustrated by people using bar graphs of continuous data just constantly and so we published a meta research paper on it in 2015 which went viral and contributed to policy changes in a lot of journals and for me that really just shifted my horizon of what was possible and over time I became less and less interested in the more physiological research that I was doing and more and more interested in you know not just how do I publish this paper but how do I do more meta science in a way that I can see it having an impact that I can see it changing journal policies changing laboratory behavior changing the way that authors think and engage with their data and so I think if you can find a place within the system you can that will help some of these feelings of discord and if you don't find that place within the system or it's just not something you want to look for and you would rather be Gary and help to improve things from outside I think that is also amazing Gary needs friends and we need more people like Gary and I don't think there's any shame in saying you know I need to leave the system in order to be able to do what I want to do I think if you need to leave the system do what you want to do just do that well said we need more Gary so I think this will probably be our last question and I think it's a good one to end on is can participants talk about how getting involved in reform and or meta research has benefited their careers we talk a lot about how advocacy work approaching for reform is seen as a distraction it's not incentivized but does anybody have an example of how actually participating in this work has helped their career I can say I want to hear from the panelists I think my work with reproducibility for everyone has really when I started my independent research group has really helped me set up an infrastructure where we can do reproducible science I hope that's my hope but it's been only a few years and so I think that's been really great the friendships that I have made being involved in these different initiatives have been amazing and the support and the different opportunities that have presented themselves have I think been really have enriched my scientific career and I'm grateful for what I've you know what I've contributed and what I've also received I guess as being a part of these initiatives so yeah thanks thank you absolutely I think for me it's growing the network which has led to new collaboration scientifically new grant applications invitations to things that even though often by some people it is seen as a distraction from science I certainly have had the same experience and there are new opportunities that actually do help my science by just building my network any other examples I mean I was just the most obvious tangible example was the work I did as a postdoc ended up in me getting a three-year grant to set up and run the future of research and I've continued to work in this space and it's only all the things I enjoyed about being an academic publishing papers, applying for grants getting rejected from grants which happened recently you can still do that outside of a university you can still get rejected from a grant but you can still do I think this is part of the interesting thing is that people think the academy is the only place to do things I've worked in the academy in a non-profit in a for-profit but essentially an LLC a business setup and all of these are just ways of having a structure in which to do this kind of work and you can still participate in doing all of these kinds of things and I collaborate and I'm still part of all kinds of networks because of this kind of work so my career I used to work on frogs nobody knows what my frog work now and I have this completely different academic career that's based in all of this great stuff so yeah it can take you in a more interesting direction great well thank you this was a nice note to end on I think something positive so thank you to our panelists thank you Tracy for joining this was really a wonderful discussion and it was great to see everybody so thank you and thank you to all the attendees who asked really great questions and took the time to join us today thank you