 Welcome everyone to April's ECR Wednesday webinar hosted by ELI, the series that aims to give Ulika researchers a platform to discuss issues important to you and your research career. We hope we find you well listening at home and today our speakers will discuss how Ulika researchers can improve training and research quality within your institutions. This webinar will begin with a panelist sharing their stories and in the second half of the webinar we'll be putting your questions to them. To ask a question you can type in the question box on GoToWebinar or you can tweet us at Eli community using the hashtag ECR Wednesday using that hashtag. Finally I'd just like to let you know that we are recording this webinar and we'll make it available on YouTube in the near future. Your chair today is Vinod Elangovan and I'm going to hand over to you right now over to Vinod. Thank you Naomi. Hello everyone thanks for joining us for our early career researchers Wednesday webinar discussing how ECRs can improve training and research quality at the institutions. My name is Vinod Elangovan and I'm a member of the early career advisory group ELIFE and I'll be moderating today's webinar. So just a quick word about a host of today ELIFE. ELIFE is a non-profit organization that is operating a platform to improve all aspects of research communication by encouraging and recognizing the most responsible behaviors in research. The role of early career advisory group or the picture that is sought refer is to influence and support ELIFE's work to catalyze a broader form in the evaluation and communication of science and in particular to represent the needs and aspirations of researchers at early stages in their careers for a research culture that is healthy both for science and for scientists. This webinar series Early Career Researcher Wednesday is one of such initiative that ELIFE has launched to help support the ECR community. I'd like to welcome our three speakers we have with us today. Verena Heiser from University of Oxford, Sophia Crovell from Charteret Medical University Berlin and Peter Graberts from Berlin Institute of Health Center. Thank you all for joining us today and I'd like to start or give a background to this webinar in particular or the topic that we'll be discussing for the next hour. So imagine if you have the immense power to improve the research culture in your institutions as an early career researcher not as a leader of the institute that will actually empower you. So for the next hour or so the speakers will inspire and empower you by discussing how early career researchers can influence and improve the research quality and research culture both at institutional level locally and also globally. So I'd like to before we open up the floor for speakers I'd like to let you know you can follow us on Twitter at ELIFE community with the hashtag easy events and you can also ask a question using the go to webinar question box or at ELIFE or at ELIFE community Twitter using E7 state hashtag. I'd also like to remind you that we have some expectations or community participation guidelines that we request everyone to be respectful, honest and inclusive and accommodating, appreciative and open to learn from everyone and in general do not attack, demean, disrupt, harass or threaten others or engrave such behavior. If you feel uncomfortable or un-belcome in any of these webinars so please contact ELIFE by email to events at elifesciences.org. This inbox is being watched by Anya Stas at ELIFE who is also working or supporting us from behind the scenes along with Naomi Penfold and we also reserve the right to ask anyone to leave or deny subsequent access to the future webinars. So please contact us if you need any help. Now I'd like to open the floor for speakers. First up is Verena Heizer. She's an intermediate fellow at Newfield Department of Population Health at the University of Oxford. She does a lot of interesting things including co-organizing Berlin Oxford Summer School and she used Open Science Framework as a platform to share her presentations about open and reusable research. So now I'd like to invite Verena to talk about her initiatives and enlighten us about how we can organize ECR activities that are not only limited to publishing. Verena, please. Great. Thank you, Vinod, for the introduction. I'll try to inspire and enlighten everyone so let's see how that goes. I'll basically talk about my experience over the last three years in getting early career researchers ready to change their environment, change their own research and a lot of this is from my own background but also a couple of things come from things that others have done in Oxford mostly and I hope it's going to be interesting to others. You can download my slides if you're interested from the OSF and you're also very welcome to email me, you know, either ask questions later or email me directly if you're interested in taking any of this further. So this is just my sort of disclosure form because I think it's a good practice but also to say that I'm actually leaving Oxford relatively soon. I'll be based at the Hansa Wissenschafts Clinic relatively soon in Germany to work on a meta-research project which I think is very exciting because at the moment my own research doesn't have anything to do with the sort of science policy stuff that I'm involved in at the moment and so I'm very excited to take this next step and this is also to show you that I'm super biased. I think there's a lot of stuff that we need to do to do research improvement and I work with a lot of different organizations to get stuff done. So I can talk about, you know, for an hour about what the problem actually is that I think we're pacing but I guess I just wanted to show you one slide that I think illustrates the problem quite nicely. So I guess most of us would be aware that we have reproducibility or replication crisis and lots of different fields in psychology, cancer biology or social sciences. We have a number of different papers that actually sort of give us some numbers to show that this is the case and I think we all can, we can probably all agree on the fact that we seem to have problems with the quality of the research that we produce. It doesn't seem to be as reproducible or replicable as we would like. So the question is what kind of, what can we actually do to improve the quality of our own research but also of the community in general and so I've sort of for this talk I decided to come up with a wish list for research improvement in a number of different areas and I'll just go through them in turn and I will show lots of stuff on the slides but I won't go into much detail. I think we can discuss the more interesting things later on in the Q&A. So first of all is training. I think training is such an important play such an important part here. I think we are not necessarily taught as much as we should be about research methods and this is across the whole cycle of a research project from how you develop your hypotheses to publishing sort of your results in the end and this is not just about good research practice but I think it's good research practice in combination with open research to use the slightly broader term than open science and this is about opening up your processes so that other people can see your data, see your materials and methods and I think this is just a way to be more reproducible as by really sharing the different parts of your research in addition to your finished manuscript. Then when it comes to infrastructure you know we can train people as much as we like if we don't have the infrastructure in place to be able to share for example our data then nothing is ever going to change so we need the digital infrastructure but we also need physical infrastructure and this is infrastructure for example to share reagents or other materials like cell lines but we also need ethical and legal guidelines and I think one of the things that would make a huge difference to people is actually having good help desk systems where you for example have a study help desk you can talk to someone while you're designing your study who actually knows how to run experiments properly and who knows a bit about stats and how you might want to analyze the data afterwards who you can go to to just get some useful ideas from but nothing's ever going to change if these incentives don't change so one of the things that really needs to change is how we hire and promote people this shouldn't just be based on the quantity of publications you have or the impact factor of the journal that you publish in and a lot of institutions at least in the UK have now signed up to Dora which basically states that impact factor of the journal that you published in will not be used in assessment procedures anymore but basically how this is going to be implemented is actually a big question mark and I think when it comes to incentives we also need to talk about working conditions and one of the things that I'm very keen on promoting is that we need to think about stuff like contract length for example I think it's ridiculous to expect from an early career researcher that they do a high quality research project on a one-year contract for example so I think we also need to talk about working conditions when it comes to research improvement the other question is about meta research I think we need a relatively good evidence base to understand the problems that we're facing and also whether the proposed solutions are actually working and I think this is a relatively sort of new field that's developing at the moment and for people like me who are now starting in this field actually it's relatively hard to find the right place to go because lots of institutions don't even have departments for this kind of work so I think that there's actually a huge space for institutions to get involved in developing meta research communities and then in the end it's about changing culture I'm a person who firmly believes that equality diversity and inclusivity need really to be at the heart of what we're trying to achieve here I think everyone needs to have a space at the table and this is not just about gender and ethnicity I think it's also about seniority and this is where I come from when I say early career researchers need to have a seat at the table where we make discussions and decisions about how we're changing culture nowadays and so I just wanted to go through a couple of examples mostly from Oxford also things that we've done basically so that people can get an idea of what you can do as an early career researcher even if you don't have any or don't have a lot of support be that from your PI or from your department and so these are just a couple of training examples so and this is not stuff that I haven't necessarily been involved in in all of these but these are examples from my work and from others in Oxford so people have set up seminar series or journal clubs and this is what Sophia is going to talk about in much more detail later we've run software data carpentry workshops one day reproducibility schools where we basically got lots of speakers together to talk about reproducibility and and how to improve reproducibility and then the sort of longest event that we've run in terms of training was really the Berlin Oxford summer school which is a five day event and sort of shameless advertising here we're going to run the Oxford Berlin summer school this year again it's probably not going to take place in Berlin because of the current situation it's probably going to be virtual but if anyone wants to apply you can do that until the end of May but there are also examples of where people have built their own infrastructure for their labs so this is a great example from a good colleague of mine who has spent a lot of time trying to develop infrastructure for her own lab where they now have for example standard operating procedures so that every person who comes into the into the lab actually knows how to do things and can read up on things in addition to obviously being able to ask the people around here to help you and others have gotten really interested in meta research and this is just another example from Oxford where people have gotten together to do a meta research project that they were interested in and it was a research that they didn't do as part of what their job actually is but it's something they did on the side simply because they were interested and I think this is a relatively common theme here that a lot of people just got really excited about different aspects of improving research culture and basically just went on and did it and I think we all realize that we can't do it on our own I think we can't change culture on our own so it's for us I think what we realized over the years is that we really need to get together and this is how reproducible research Oxford developed out of all these different initiatives when we sort of formed our sort of core group we are now expanding massively into lots of different departments into lots of different divisions and I think one of the most rewarding experiences has been that we actually get people from social sciences from humanities from across the board involved in this and we realize that a lot of different areas have problems with research quality but I think the solutions are actually relatively similar at least in some areas and the great thing is that we now got funding for a coordinator it's a full-time post and so we can now take this forward really well in Oxford and I think it's been amazing to see how this has developed over the last years and we have the local network of the UK reproducibility network which is really a group of amazing people who are trying to ensure coordination of the efforts across the sector because really what needs to do is needs to happen is really the whole system needs to change and you know yes while we can change our own practices as early career researchers I think what needs to change is how institutions view what we're doing how funders give us grants and all these different parts of the scientific ecosystem need to get together to really think about research improvement at the broad level and why do I firmly believe that early career researchers need to be at that table well I think what we have to offer is three things but I think there are more but maybe boils down to three things one is passion I think a lot of us are really really keen on changing things and are developing really amazing ideas for doing that but we also developed lots of expertise over the last couple of years and we're actually at the forefront of this because we have to implement the solutions so we have the expertise to make things happen and time and by that I don't mean that we have too much time on our hands I don't think any of us do I think we're all totally overwhelmed because we're tending to work 24 seven anyway but what I mean is that we don't have additional activities like administrating a lab group for example so in contrast to PIs if people actually give us a bit of time to develop solutions we can actually do this much more efficiently and then we can share expertise in our labs we can run training activities and I think we're all really keen on sharing ideas around infrastructure and other things that I've talked about because I think this is where we are at the forefront of making changes and the only thing that we need is really some job security and I'm not even talking about permanent posts here I think what I'm talking about is that these activities that I was talking about that you'd arrange training activities for example but this can be an official part of your job I've done that on the side and it's been sort of really eating up my life and so we really just need a bit of job security we just need to make this part of our jobs we need a tiny bit of money and this is just really to run training activities I'm not talking about salaries here and a bit of support be that from your PI or from you know department heads or even from institutional heads I think would be great and incredibly helpful and I think we're starting to get there at Oxford certainly and I think lots of other institutions are waking up to the fact that they need to do something about the quality of the research that's produced there so what can you do as an early career researcher I think the first thing to do is maybe try and find training opportunities and I've just put up some on on this on the slide here develop your own infrastructure and this could be just infrastructure for your own project it doesn't have to be for your whole lab but this just has to be for your project I think learning things like a good data management practice is just brilliant for yourself as a as a researcher and will be beneficial to your sort of future career definitely then the the other thing that I think really makes a difference is if you've managed to find like-minded people and manage to raise awareness around the issues of research improvement by running events and I think this is what we've seen in Oxford is that when you're working on your own and you think you're the only person in the world who cares about this it can get very lonely but as soon as you develop some activities and realize hey I'm not the only crazy person out there who's interested in this I think it can be very exciting and very empowering and then if you want to go crazy yes you can start to lobby your PI you can start to lobby a department and if these are things that you're interested in then by all means do email me to to ask more questions and I've put together sort of a short document with some tips on what you might want to achieve this is not published at all it's not published anywhere you can just go to the OSF and have a look I think it's better than nothing so have a look at this if you're interested so really just to say thank you to all the people that I work with I'm sure I've missed out some and so yeah thanks to to all the people who are supporting what we're doing and thanks for your attention if you want to get in touch here's my email address and you can find my slides on the OSF thank you thank you Verena that was more than inspiring you actually hit the nail or hit the hammer on the nail thanks for that um up next is um Sophia Krugel um so Sophia is a PhD candidate uh at the Metaresearch Innovation Center uh Charity Medical University Berlin and she's also a co-founder of the Retreducibility Journal Club um which is a movement um but she'll talk about which now has over 70 groups around the world if I'm not wrong um and she's going to actually tell us how we can transform your frustration into action um and you know change things using um you know initiatives like Journal Clubs uh Sophia please thanks um oh god why do I feel like an old person um all right um yeah thank you for that introduction um so Verena has already um sort of introduced some of this uh I guess at the side so I'll be talking about reproducibility um which is an international and ECR led Journal Club series um oh all right what is it right as I said it's a Journal Club initiative and a podcast um which is very much grassroots so when we started this um it was Amy, Sam and I um Oxford we were kind of a little bit frustrated maybe or we were um hoping to have um more space for discussion surrounding open and reproducible science um and more sort of power behind those those discussions as well and so we just put together a reading list for a first term so for eight weeks um and got started with this relatively um quickly um because it turns out if if you're just meeting and drinking tea and eating cookies no one can really stop you doing that um now the core team that we have is um still the three of us and then um Matt Jackery, Katie Jax and Jade Pickering just to make sure that I've mentioned these lovely people um who make up right the core of disability now and of course all the local Journal Club leads without which about who none of this would work because it's now actually quite a big community um the main idea behind this obviously it's a Journal Club right so technically it's it's about reading papers together and discussing them and learning things from them and that is the case right you are exchanging ideas um with other like-minded people um and you are having quite interesting discussions but at the core of it I think this is about building a community um which I'll talk about a little bit more later as I said it's turned into an international organization it's actually more than just 17 more than 17 institutions now it's 88 institutions in 20 different countries across the world which is a bit mad so this is an older map actually because the I haven't been able to make make the new one work in the way that I wanted to um but yeah so it's just it's all a bit mad um and we've got a lot we've got a huge community across the world which is really really nice because you get to hear so many interesting stories as well um we are funded currently by the UKRN um so Luna has also mentioned that the UKRN disability network um yeah so you can find those you can find them at UKRN.org um right so I've told you very vaguely what this is why should you why do I think that you should host uh or join a journal club um in any in any case or a reproducibility one in particular um so yeah as I said you can read interesting papers you can learn more about the interesting ideas and skills surrounding um open and reproducible science together with people um of sort of your career stage mostly um yeah so you do that with these interesting people key part and then thereby you kind of without really even really realizing you create a community um around these issues um a community that can help each other um so that's kind of basic idea um main aspect supposedly the papers um a lot of the reproducibility journal clubs start with a common reading list that we put together for 24 weeks in total um but they're kind of largely focused on psychology social sciencey um topics we're in the process of um creating uh different reading uh reading lists for um for different areas as well because it's uh while it started in a psychology context this has spread uh beyond that um and then after sort of a couple of common papers um journal clubs mostly move on to a mix of um again sort of these these papers mostly surround mostly focused on open and reproducible science and papers that are more relevant to specifically their area um either in either also focusing on open science um or even just um just any paper um and discussing them those papers um with that kind of lens of open science and reproducibility the goals of this are of course to learn more about open and reproducible science um kind of in a space where you don't have that pushback um or that potential for pushback from above or above you know above in the seniority kind of sense I guess um and ideally uh you can also use these journal clubs to um sort of through criticizing um the papers that you're reading be that the the open science papers which aren't sort of um completely uh like you can still criticize them just because they're focused on open science doesn't mean no perfect they're not because it's it's still uh research on the humans um and and sort of the the journal papers that you might be reading um and by criticizing that that research um or really engaging with it um you I find you also often learn a lot about how to do better research yourself and then of course you know outside of the papers that actually just basically teach you new skills um or new things to care about kind of thing and as always you're forming a community um of kind of visible proponents of these ideas um making like you and a whole group of people turning you and a whole group of people into these kind of experts potentially um in in your larger community which can be really powerful community um which I as I said I think is actually the main aspect of reproducibility it's a kind of trojan hole that's where like here's knowledge and actually end up with friends and and support and a platform for change as I said we first got started um out of a frustration a liquid visibility out of frustration with the state just growing with um the way uh that that's sort of yeah that these these issues might maybe were being discussed um with science um as I also said it was super easy to set up for us um and very effectively or efficiently transformed our frustrations into a community um into a platform for change um it's now even easier because we created this artifact I think that I think that is helpful and yeah what you end up with is a visible local community um of at least your local journal club but ideally also extensions of that including potentially even your lab um because you might by being more visible you might end up um even convincing people that you didn't think were convincing sort of invincible that's a word um and on top of that local community that you can see quite easily um this is because you're commuting it I guess with the people that join you you're also joining a global community um you're joining that um by a twitter which is big um and as we've got a reproducibility slack mostly for the organizers um but also more generally and um initially originally there was a there were plans for a big meeting um an in-person meeting later this year actually next month which is now online but yeah basically you you end up with a whole like a whole host of communities um to support you um and to make you feel like you're not the only one thinking about these issues caring about these issues which I personally have found so unbelievably helpful um but so that you don't have to just take my word for it here are some quotes um kindly collected by Jade um I'll just read some of this um so the the question was what does reproducibility mean to you um someone said it's been a brilliant chance for me to try building a community in a workplace with amazing support um someone else values it because again the social support the collaborative opportunities that came out of this and the knowledge exchange um I like that it allows people to designate some time in their schedule for thinking about the big uncomfortable questions that might be making most of the rest of their schedule obsolete but hopefully just means they can do better work sorry this sounds like an advertisement but I just I enjoy this um and it's showing departments that ECRs want to be a part of a big change even if there's no university leadership yet we are the change that is needed in research practice and culture now um yeah Wolf as I said you can start one you start to general help yourself we've got this artifact which I'm linking to later as well but currently because of the general global situation you can actually just join an existing reproducibility um journal club online because lots of journal clubs have moved their meetings online instead of just canceling them because you know we still need to clean up science and um learn more about these important skills surrounding open or robust science and so we put together a calendar first of all with the reproducibility um journal clubs that are opening their online meetings to wider participation not usually just the link you do have to email someone that's the threshold um and then you can sort of join these discussions um there's um not a ton on here right now but a couple are happening um and it's quite interesting um and sort of to go with that we also have some tips both for attending um and for hosting a reproducibility online journal club um you could even actually just start your journal club remotely which probably um minimizes that potential for pushback even more um than just starting a journal club at any other time yeah that's it um my that's my sort of broad introduction to reproducibility I've got some uh general links uh here and um uh yeah thank you oh thank you Sophia for for sharing your wonderful thoughts on this um so now uh we can move on to our final panelist um and uh you know Peter Gravitz he's a research fellow at the Berlin Institute of Health Quest Center which is an institution that runs projects on reproducibility and openness of biomedical research um Peter during his medical studies actually co-organized a community of students that campaigned for better conflict of interest policies on campus um and that kind of bottom up campaigning you know changed into top-down regulation for conflict of interest uh policies so he'll talk about that uh so I'll invite Peter now please awesome can you hear me here we go great thanks thanks you know for the kind introduction um I'll be talking about conflict of interest policies at German and medical schools and what we've done there um just one note on my own interests as you know said I'm a research fellow at um the BH Quest Center I'm also a member of the university's allied for essential medicines and meet these if I which is a no free lunch group and I occasionally consult for site ink how did this all start um effectively it was a couple of years back um when I was still at medical school and I started attending my first scientific conferences and I saw the various same professors giving talks at these conferences as they would at university however one thing is they were talking a lot shorter not longer than 10 minutes that was good um and also they they didn't one other thing in a different way they showed a second slide disclosing additional sources um of money for example and it made me think for a second hold on in the past five years they haven't done this why do they do it in their 10 minute talk right here um for for a while this this was just a uh uh weird moment um but when I then entered into the clinical phase of um of my studies and uh did clinical rotations as well I came across pharmaceutical representatives more and more often there were little presents handed to me and I started talking to some of my my colleagues um who had really similar stories to tell um of little gifts happening and context with industry so we did um I think what's what a lot of people would do um we looked at the literature and um did the journal club more or less and we were surprised in finding out in fact we're not the only ones and 88% of students at German medical schools have received a gift from a pharmaceutical company or attended a sponsored event so um this is something that almost every medical student is confronted with but we realized at the same time wait we never talked about different interests in in in medicine or in healthcare in general um and this is where after after having read the literature even more we we also realized there are solutions to this um there the the data's all there it has been shown that comprehensive gift policies uh at medical schools that reduce gifts and also contact times with pharmaceutical representatives improve the prescription or prescribing behavior of residents in later times so if this is clear why is nothing happening um and we decided to kick off a campaign um to go outside of the of the general club um focus and um start a campaign with three very student focused goals we can't tell i mean conflict of interest it's a it's a it's not such an easy topic people are usually they really quickly get um feel feel attacked but that's not what this was all about it was all about the students and us um and getting the best education possible so we asked for a second slide all the professors should show us a second slide we asked for a part of the curriculum um to to talk about and touch upon interests and healthcare and different interests and we asked for policies regulating conflicts of interest at medical schools and if german by itself didn't sound aggressive enough we also added a now in all caps underneath it we also ran a little study um we thought in the beginning oh this is just going to take four months it's nothing more than a little survey to all deans of german medical schools asking um do you have any curricular teaching activities on conflict of interest and do you have policies or parts of policies that regulate conflict of interest can you please send them to us we thought oh this will be fast in the end it took us a year and a half um um and what we did is we um also did a web search that um in case we didn't get um any responses from any of the deans and we rated all the policies that we received um the results were pretty as we kind of expected um they were pretty scarce there were only two schools having parts of policies or one policy that would regulate the criteria we set up beforehand um and they received only 12 out of 26 maximum points um my own university charity it's mentioned there um put four out of 26 points what we then did is we um we talked we started talking about this and um we hosted a conference we got people together in person in berlin this happened last october um and um talked about our results talked uh shared our own stories um how we were um dealing with with the lack of regulation or what our own experiences and um we were fortunate enough to also have um parts of these studies of our study covered in different press and that national press um so that created a bit of a buzz um but the question really is so what um did we change something the culture at our universities really change well there is a small sign of success um the institute for medizinische und pharmaceutische Prüfungskragen so the body that actually sets up um the questions for the final exam before becoming a medical doctor they introduced an agenda item on interests and potential conflicts in interest in healthcare so now it will be part of the major exams medical students need to take to become doctors and if it's part of the exams as all students know well um you probably need to learn about it beforehand so um that was part of the success other than that uh and this is what we're doing now as um i've learned of reproducibility does it as well we meet online and strategize further further because ultimately it's about um talking to deans um calling in meetings asking hey can we support somehow with setting up um more teaching activities um did you send a reminder emails that professors should disclose the conflicts of interest and also um this is what we want to do next we are not the only ones um doing studies like this in the past the US and France have have done rankings like this and belgium is at the moment doing it there's a group in Italy um in in australia and in in um in danmark we're running similar rankings and um yeah we can help you do this and we've done this before and we try to rank and continue talking in the language of institutions to put them under pressure to change the culture we have because that's what universities care about rankings and students um thanks to everyone we're on twitter as well um this is a map with all the different chapters of universities allied for essential medicines who supported the study um and i'm looking forward to further questions hi everyone well we wait for vinnode to um unmute i think we're having some audio issues over here um we can kick off the questioning so um amazing talks all three of you i mean that's just so inspiring to hear all those stories we have um a couple of questions that are already coming in from the audience so i'm going to start with those i know vinnode had his own question as well but the first question i can see here is from mirth manpay um do you think online platforms such as pub pier where anyone can post critiques on published and unpublished papers anonymously are beneficial for reproducibility now um sophia and verena you spoke specifically about reproducibility so i wonder if one of you two could start this off verena yeah do you need us to yeah no i'm happy to go for it um i mean that's a very very specific question not sure i'm i'm the expert on pub pier at all i think um what i'm not a big fan of is actually anonymous anything that's anonymous um if we're talking about anonymous criticism i think that's actually really hard i realize that it sometimes protects early career researchers because if you criticize people openly um that can be quite hard and difficult and sometimes detrimental to your career at least that's how people perceive it um but i think anonymous criticism in general is not necessarily helpful um if you don't know where it's coming from you know what kind of background people have it's really really hard to to go for that and on top of that i'm not sure if it was pub pier or one of the other platforms that kind of work like this but isn't it the case that you kind of you have to have a certain number of publications to even be able to join this and to use it no i have no idea i have no idea i just don't know well because i think it is and if it's not for this one i mean like it because it doesn't make sense right because you don't want just anyone to um to suddenly i don't know say something about like sort of like anti-vax things or something um but on the other like on the one hand you know that makes sense on the other hand it also means that it's actually not that helpful for ECRs i guess um and might might end up well it's not like but it might end up like a sort of um probability contest which you kind of have with these on the open science framework there's a there's a new feature where you can clap for people for preprints right so it's like i'd rather have i'd rather have uh discussions like that than than the than sort of clapping for each other but yeah no i agree it's kind of it can be dangerous if it's anonymous um and in this particular case it doesn't even have the added advantage of being open to us. Peter do you have anything to add? Not really at this but i just want to stress and emphasize what Sofia already said i think it's important that there shouldn't be a barrier for for criticizing or for critically appraising any research that's out there so if you if there are journals for example that only accept your your viewpoint if you're at least um having a PhD or a master's while the internship matter not the person where it comes from so that was the case that's um that's a big downside. Oh well i'm sorry do you want to maybe maybe it wasn't pub peer if it's not pub peer i'm so sorry. Um we might actually try to go to uh mirth to unmute in case there's anything else to add there um but mirth does say um that it this question was indeed about the anonymous nature of the commenting. Have we had any luck getting to mirth no i think maybe maybe we can't oh no here we go mirth can you hear us okay we'll move we'll move along um we have another comment from a listener who's not going to have to save the q and a but just wanted to raise that um uh of course within acrs we're including lots of junior faculty as well so you've got a lot of your postdocs taking action but also people who are new pis and and and actually it's quite a diverse mix of people in terms of career stage that we all lump together into ecrs and we've all got different ways to um change the system and every and there's a lot of new pis who are very keen to change the system as well so that's an excellent point um there's a question here from honor pollard uh what would you like to see from funders to help with reproducibility and research quality jump into fear what wouldn't we like to see um god i mean probably um an important part of this is is actually rewarding um open and robust science um or at the very least not disincentivising and activity right because if you if you're in a situation where you're sort of when you're trying to get grants and you and you know that you have to um you essentially have to have published in certain journals to have the best um chances chances at that lottery for example and then you um are doing a registered report and putting lots of lots of work into it but you actually only get one publication um out of it and you know that that sort of puts you in a bad position that's that's not a great state right like it's not great if the if someone who puts a lot of effort um into um creating a reproducible um a robust project um has worse chances than someone who is focused on um getting their their research into um as fancy a journal as possible um so i guess that that's that's one big thing also more money for meta research of course well i should have i should have had an uh conflict of interest side as well where marina have already has like all of the points of this yeah i'm i'm sure i have lots of ideas of what funders could do um i think the the most impactful thing that funders could do would actually be to have small grants for the kind of community building activities that we're doing um you know we've done most of this without funding we've been quite lucky in that we got a number of different ps who started to support us also with money um but i think having small grants and i'm talking about you know 10 000 20 000 pounds it doesn't have to be much that you can apply for and for which you don't have to be a pi i think that's another bugbear that i have with a lot of uh applications that you basically have to find a pi who runs the application and i think that's slightly ridiculous because the early career researchers are the ones who want to run this and yes you know i don't have a permanent contract but i tend to have a contract for another year or two so i can you know i'm i'm going to use that year that year or two um and i'm going to use up the funding during that year or two so i think having to find a pi who fronts the applications that's always driving me crazy we might be going to honor for some follow-up there honor you with us i think it works now um i'm sorry about that i think i'm unmuted thanks guys that was a great answer thanks hona binode back over to you thank you uh please go ahead with the questions that was there uh i was just able to join uh sorry about this uh bad uh connection oh yeah no worries um so there's no more questions in the question spots right now so if you'd like to send them in on the question spots please do um particularly if you're interested in how you can make change in institution and and some of the tips that our presenters have shared have inspired you um you can also put them on twitter in the meantime binode you had an excellent question for the speakers earlier that you shared i want to i wonder if you want to follow that up um sure i could um so i'm just trying to pull out that question again um let me see i have it here if you'd like oh oh yeah it's something you've talked to a lot of us about already a binode's question is um there's a there's a quote from akaya windwood you can't sustain a movement if you don't sustain yourself and what you've all talked about here today our efforts have have grown into much bigger efforts and i imagine are quite demanding so how do you sustain yourself and your energy to build a movement against a lot of resistance okay want to go first i'll go i'll go first um all right um i have used this word a lot already today but community as always right it's just like if you if you have if you have that community of people that you know is fighting for this as well um then you can yeah and it's quite easy to just um to reach out to them and um and get that support the support that you need in difficult situations um otherwise of course just like just logging the um the sort of smaller successes that you do have along the way so that you can sort of look back at those if if things are tough but mainly community it's just um that's that's that's why i think it's uh it's so important to create spaces um where you have that support to empower each other to create change thank you um are there any more thoughts on that i would like to directly add on onto this i think um in order to sustain yourself as a person but also the the movement you're trying to build one needs to think about this from the very beginning and it's not that easy um i mean as researchers or specifically as early career researchers we taught in python and r and some unlucky of us still started but um we're not taught about how to how to manage people or how to manage expectations or how to even build um a community and maybe this can be connected with the question before what what are what am i looking for um from funders it's not it's necessarily money of course no one i mean setting up online platforms or infrastructure i don't want to pay for this out of pocket if coming having a community coming together but it's it's more also about the skills that are needed to sustain a movement or any effort so i would i would love to get more um more input on maybe training on how to build communities how to moderate how to set expectations etc so this this could be something that that funders could step up and and help help these little groups flourish i'm also very happy to comment on that actually um first of all so first of all i think i personally haven't actually run into a lot of resistance um i think that's been really an interesting feature there and it's something that maybe we can talk about as well so i don't know how the others feel about this but i think in the beginning a lot of us were expecting resistance and i think it hasn't been the case actually for most of us i think um the the barriers that you the most the biggest barrier that you get people are just simply not interested and you know especially people high up the food chain who are just not interested in what you have to say but i think this is slowly changing as well and i think a lot of institutions a lot of p.i.s are waking up to the fact that they need to do something the funders have woken up to that fact and i think the funders are actually pushing for change a lot i think this is one of the things that in the uk is happening a lot that the funders are actually the ones that are driving change and and because they you know pour so much money into the system i think they have incredibly much power and they're using it in a good way at least some of them are and so i think i haven't really run into a lot of resistance and i think the most important point is that you don't get hung up on areas where you either find resistance or where people are simply not interested i think i've realized that you know if you can't um go one way then you will always find it some way around the obstacles so you know don't go you don't always have to go through the obstacle try and find another way and i think this has been i think this is one of the tips that i can sort of share with people here is that you know it doesn't really matter if you know the one person that you actually wanted to get to to be behind you isn't behind you or you can't convince them you know there will always be another person who who may be interested in this and i think in terms of sustaining myself i think this the journey has been really amazing over the last couple of years and to see to see that this community is building to see that that so many people are getting involved in this has been incredibly rewarding and i think it's sometimes even more rewarding than sort of publishing your paper because you're like yeah you know i did a bit of research and you know here's my paper in the end but actually to see to see that your your part of something bigger can be incredibly rewarding just by itself thank you for that nice thought so i mean this also kind of reminds me there are three solutions to a problem either accept it or change it or leave it i mean things that you cannot you cannot accept you need to change it and things that you cannot probably change alone you have to leave it to the community to solve it so maybe on that note i'd like to ask your final closing comments from all of the panelists and before that i since i didn't have the opportunity to thank Peter for his really visually engaging presentation thank you Peter for that and i also like to take this opportunity to thank all the panelists for today to for sharing their insights into building a community initiative so i'd now like to hear your final comments on our closing remarks on what we can take from your initiatives and how we can build upon each other's work would someone like to go first i'm happy to go first if you want yeah i think my my take home message is probably every little helps you know i think this is there's a big supermarket in the in the uk that has this as a tagline and i think it's actually true i think it it doesn't matter how small it is that you know make a change either in your own practice you know start with something small start with something you don't have to change the world you don't have to change every single practice for your research project start with something that actually works for you that that's actually you know good for your project maybe that something that you can convince your PI of whatever that might be and if this turns into something bigger that's great but actually it doesn't have to i think we don't have to aspire to build a community i i think i never did certainly i'm not sure how the others feel about this but sometimes you just you know if you find a small problem you want to change it and suddenly it snowballs into something much bigger and i think this is probably the situation that we've all found ourselves in so you know every little help starts small and if it turns to something bigger great and if it doesn't you know don't don't be disheartened at least to change something on a small scale for yourself do you want to add on that so yeah i know i can i can just i can just go on but i guess like in a in some in some way i then maybe disagree because i actually i think the main thing is the importance of community but just maybe not commute but maybe you probably don't we probably still agree with each other because it's i i guess i do see community as a much smaller unit as well it doesn't have to be a big thing just yeah look for the people around you that you can create change with and yeah i want to encourage people to sort of create spaces that work for them and that work for like their communities or the the issues that they're facing i personally found reproducibility and the reproducibility community incredibly helpful in navigating both sort of the challenges the changes that are currently happening in science and the challenges that are thrown at you by academia and beyond and so i think if you can claim that space at your institution and use it to help each other and to effect change so yeah community surprise thank you for that it's also good to have agreements on disagreements and that's fine so peter your final closing remarks it's hard to follow up on this and be the last one but my attempt would be my attempt would go along the lines of university specifically in a lot of institutions are about you in our case and medical school or is only a medical school because of the students otherwise it would be it would just be hospital and you your your opinion and your voice is super important if not crucial it's really easy to not listen to someone far away but if you're the main stakeholder at a place well people will listen to you if you say something and also especially early on in a career one has the advantage of not too many strings attached you can say things because they're right and you can help move institutions that are big ships that have been around for centuries to steer the way into into the reaches that is right and change the culture of being eventually it's it's your institution it's your home and they probably will listen to you yeah thanks for that but that we come to the end of this webinar so I'd like to thank all our speakers and Naomi and Anya for helping with the back end and to everyone who tuned in today and contributed to this discussion and also if you're asynchronously joining us on youtube please leave us your cool initiatives and comments and it was also a wonderful opportunity to learn more about your initiatives on how we can improve both training and research culture at institutional level and also the global level and thank you for that and our next ECF and state webinar will be held on May 27th and we'll be discussing improving conferences I hope to see you then and have a great day stay at home thank you