 Good afternoon, everyone. I'm Lisa Cuevas-Shaw. I'm with the Center for Open Science, and I'm delighted to be moderating today's panel session on reforming science institutions. The purpose of this session, as you can see from the program, we have a number of more progressive existing institutions that are reforming their cultures to change reward systems, open their practices, foster collaboration, and accelerate discovery. So this panel is intended to introduce and allow a few leaders of these more progressive existing institutions to just discuss the opportunities and challenges that they've experienced as a part of this reform movement and to highlight successes and failures for others to emulate or avoid. So allow me to introduce everyone. So we have Schell Gentemann from NASA. We have Ulrich Dörnagel from Charity and the Quest Center for Responsible Biomedical Research at the Berlin Institute of Health in the center. And Mira Schulten, the Chair of Utrick University's open science platform. So please join me in welcoming our panelists. So I'll share a little bit about the lineup. When we got together to plan for this session, we wanted to make sure that the presentations went at a pretty fast clip so we could really privilege the nature of a panel discussion. And so as you're hearing everyone speak to essentially three points. What is the scope of their reform? And when we're talking about reform, I should probably at least broadly define it as we are talking about open scholarship and open science. And so each of them will address how they approached or how they are approaching that aspect of reform within their institutions. They're going to be describing those opportunities and challenges of the work to date and then highlighting those successes and failures and potentially what's next on the roadmap to amplify successes and avoid failures or pitfalls. And so as you hear them speak, please feel free to start thinking of some questions. After they present for about five to seven minutes each, I will start to offer a few questions. But I welcome everyone to come up to the microphone and then I'll start to pick on folks to share their questions. So without further ado, let me welcome Shell. Thank you everyone and I am glad everyone is still awake. I am here to talk about NASA's year of open science. I'm Shell Geneman. I am the transform to open science lead at NASA headquarters, which is across all of the science mission directorate. Science is the key to unlocking the secrets of the universe. It unifies humanity in asking questions. Science is unifying us what we measure large and small. And NASA has declared 2023 as a year of open science joined by 13 other federal agencies representing over 90 billion in science funding. You can find out more about that at open.science.gov. We're really here to build with the open science community and that's one thing that I want to emphasize. As NASA comes into joining the open science movement, we do really see ourselves as joining an existing movement. There have been decades of work in open science developing the infrastructures, the protocols, and the practices that allow and enable a federal agency to come in with that visibility to an existing community that is developed and is powerful and we want to recognize and have respect for that. One thing that the federal agencies for a year of open science have come together is to agree on a definition of open science for the federal agencies. And this is growth out of the many different existing definitions of open science and we see it as sort of an evolution of process, right? As we start doing open science, we say, well, open science is open data or open science is public access. We really want to emphasize how open science as we see it has grown and transformed as a concept and how important it is for it to be equitable open science. So the definition is the principle and practice of making research products and processes available to all while respecting diverse cultures, maintaining security and privacy, and fostering collaborations, reproducibility, and equity. And so within this definition, what we have is a recognition that open science isn't just the final product. It's what goes into that final product. It's the principles, it's the practice, it's how you do your science, it's who you engage with and how you do inclusive equitable science. And it's respecting diverse cultures, it's maintaining privacy with this goal, with this goal of increasing collaborations, reproducibility, and equity. So within NASA, we're putting open science into practice. We have four areas that we're working in and this is NASA's open source science initiative. And we've called it open source to sort of build on that collaborative energy that open source software development has. This year, it's $30 million. It increases to $50 million for the next years out pending appropriations. So with those, we have policy. NASA has updated, they've coalesced all of the existing scientific information policy into a very exciting new document called SPD 41. I did not have any input on the naming of that. And that takes all the information policies that were scattered around NASA, brings them together, and then that was then iterated on. And SPD 41A, again, I wasn't consulted, was released, which actually starts to require, instead of shoulds, that policy is full of shals around open science. You shall share your data at the time of publication. You shall share your software. You should apply a permissive open license. You shall publish open access. And this is for forward looking policy. So that development, that publication, that work, that's part of this. We're also developing the infrastructure investing in data repositories in software development, investing in core services that help the computational infrastructure that we need to do science. We're also funding. We have funding opportunities for open source software tools, libraries, and the core infrastructure that has so far been limping along with funding here and there. As a federal agency, we want to recognize the importance of all of that primarily volunteer work within the open science community and start to fund that at a federal level. And then we have the community. So this is the transform to open science mission. And this is our outreach and working with the existing open science community and building new bridges to our new open science community. So community engagement, this is where we're trying to work with both our existing scientists and broadening participation in science. It's a $40 million five-year mission to accelerate adoption of open science. It has three goals to support 20,000 researchers to earn a NASA open science certification. This is a digital certification that goes with your resume. It goes on your LinkedIn. And we see this is really valuable, especially to early career researchers to motivate them to learn about open science, to double the participation of historically excluded groups across NASA science and to enable five major discoveries. And we're doing this by creating an open science curriculum, by doing outreach and efforts with historically excluded groups. And we have sessions at many of the national society meetings in 2023, which is why it's really hard to get hold of me right now. This is really exciting. We have the Open Science 101, which is a community-developed introduction to open science with inclusivity, accessibility and diversity at the forefront. If you get out your phone, you can scan the QR code. That'll actually prompt you if you want to sign up for the email list. It'll send you information about where our GitHub repository is. We have discussions enabled. And we really want to have as many of these discussions and these questions about open science in the public. And we want to hear from you, from the community, what are we doing right? What are we doing wrong? And how can we support the open science community? Thank you. Thank you. Ulrich from Quest. Thank you very much for the opportunity to talk a little bit about the Quest Center in Berlin. It's what we would like to call a complex and systemic behavior change intervention in a academic biomedical center. The Quest consists of roughly 50 very enthusiastic people from very different fields and at different career stages. We are very happy to have, from our institution, about 2.5 million per year in funds that does not include third-party funding. And to give you an idea about the ecosystem in which we want to afford this culture change, actually, just two numbers. These are about 5,000 researchers and 8,000 students. And it's probably the largest medical university center in Europe. Our work is based on a framework which, when we started, we gave us. We basically start from three principles. We posit that the advancement of knowledge, in our case of biomedical knowledge, of course, needs to be trustworthy, which we define as robust and rigorous, needs to be useful. That is, of course, useful for scientists, but also for society. And it needs to be ethical to humans, but in this realm, also, of course, for animals because there are a lot of animal experiments going on. These principles, we believe, are underpinned by processes and they are obviously context specific. In our particular case, for example, they include things like randomization blinding to make work trustworthy or pre-registration, timely reporting to make it useful for scientists or for patient-relevant outcomes and patient engagement for society. And ethical, I guess, it's quite clear. It's about risk-benefit analysis. It's about data protection for animals. It's about the three Rs. And so, if you have these processes, and that was just a list of those processes, we then have selected activities to underpin them. We group them or you could group them into competences that we are trying to bring in or motivation that we want to create and opportunities. And I'll have just a few examples in a second. Importantly, we are accompanying this with meta-research to find areas in which we should become active and maybe even more importantly, areas to investigate what the impact of our activities is to possibly adjust them. These are just a few examples of our approaches. So we are helping our researchers with quality assurance just to examples. We are providing electronic laboratory notebooks to all of them and help them use them. Error management in clinical practice, just another example. Education, I guess, it's quite obvious what this entails. In terms of open science, we are helping our researchers with fair open data, research data management plans, data management plans and so forth. Key to all this, I believe, is to change the reward and incentive structure. We are happy that our leadership is pretty much aligned with all those things and so we were, for example, able to adopt narrative CVs or narratives in the promotion and tenure committees with, for example, a specific narrative on open science practices. We have rewards and special, so if a researcher at the Charity, for example, has open data, we reward it with a little bit of money for extra money for research and so forth. In this realm, very important is we find is to engage our patients in the type of research we do, clinical trials, not just as subjects but also in the design of the clinical trials, defining the outcomes and so forth. And we are helping with our researchers and clinicians to do this. I have to say that Germany is actually quite a diaspora in this and we are way behind, for example, where Britain is in this topic. And I already mentioned it, we do meta research and you could also call it implementation research on those things that we do. And we act as a think tank because we have been, I think we have had quite some success so other institutions are now trying to find out what we are doing and obviously we are keen to help them. I was asked to not only talk and brag about what we have achieved but also mention barriers and challenges and as you can imagine there are many of them. I just chose the top three ones for me. A very important one I think is that the brains of our scientists and large part of the leadership and the administrators are what I believe and that's of course through socialization but they are almost hardwired to equate the impact and even purpose of their research with journal reputation and third-party funding. It's very hard to get this out of their brains. You can argue that responsible research is actually a state of mind but it needs additional resources and we are happy that we can provide those. This is a very special situation. No other institution in Germany has these funds available so this is nice but on the other hand so what we are doing basically is we are piloting a lot of activities and then to get them in a line function to make them sustainable is quite a challenge because then even the resources that we have are not sufficient and this would bind our resources to just maintaining where we are and not getting further and so I guess that's a problem that many of us are facing that get money from foundations for example to start a project but then when it comes to making it permanent everyone's looking away. So thank you very much. Thank you very much also for your invitation to speak here today. I'm Mirosch Holte I'm an associate professor at the Utrecht University and also a chair of the EU Open Science Platform. So today I'm actually talking on behalf of all of my colleagues we're approximately already 250 people at the whole university dealing with different topics of open science so I'm presenting on their behalf. So my short introduction is going to be about what open science is at the Utrecht University and the purposes or the aims that we try to achieve opportunities and challenges and the progress so far and the way forward. Now what is open science at the Utrecht University? Well very shortly it's a program which is there and I think listening today to different presentations I think we actually mean lots of different things when we talk about open science so I try to define it briefly. So open science started perhaps in the still oftentimes the word is oftentimes changed intelligently used with open access but it's much more to eat so we defined it broadly we defined open access sharing data and software public engagement open education and the system of rewards and recognition for that so this is how I mean it and why we do it that well at least and these are the key purposes. First of all promoting more but not quantity but quality in research and I don't know how it's you know about your expertise or discipline but what I do is law and economics political science and governance studies so when I search for some articles especially I remember when I started my PhD to do that I felt like you know in this all the way the picture to the I don't know left for you it's a lot of articles you're getting lost there and it's you know it's a lot to handle and what I nowadays value a lot and sometimes is a few review articles just literature review what has been there what has been said and how to move on and this is what I'm trying to also contribute with my research so not more necessarily articles and publications but different things databases blogs blogs or review articles and you know really trying to move forward promoting scientific innovation is our next aim in research and education and of course it's not possible just to do it on your own if you need 20 years to get a database to really get something moving towards it would be a pity if I would be also spending another 20 years to get to that data so we need each other we need to promote scientific innovation and that's only together together in terms of different scholars together but also disciplines and last but not least we believe that science is there also to make the society better well like with an invention of the light bulb it helps to educate the whole society right now is it all going smoothly no of course there are opportunities and challenges to each of these purposes so I tried to put it really in brief here so promoting quality yes it gives us an opportunity to rethink establish and establish what actually quality should be in the 21st century we're living in a digital world with lots of technological and other challenges it needs to be rethink rethought but as my challenge what I've come across so far nobody likes change everybody wants immediately the menu this is the new quality criteria no we're not there yet but that's the discussion that we are going to take and it takes a while promoting innovation I think it's a great idea to share publication data everything openly to you know to improve the progress but we are stuck well with the ideas if I'm going to share data with you and you're going to be promoted tomorrow that's not an incentive that I would like to have so I'm not going to share and that system needs to be changed but also unfair competition sometimes and knowledge security against the big tech geopolitical situation at this moment etc and last but not least it's very it's opportunity to have a stakeholders together a co-creation agenda setting but there is also debate somewhere from behind pure science and danger no in my opinion open science is for everybody but not only science for science now our progress so far we've been there for quite a while since 2017 we've done the first run of the program this year we are finishing up and actually from the next year we hope to embed it in our daily work life so you know it's like doing a reference list at the end of your publication yes you just go with the stakeholders engagement or something like this so it should be normal to your life it shouldn't be a bubble program or that community those guys in open science program we are not know it should be our daily work life but of course it's also something that you can you know cherry and peak in a sense that what suits you know not everybody has to go and do the stakeholder engagement also if it's just personally doesn't fit you you can block flock you can have a small meetings with those who you work for etc we have developed a number of principles in our promotion system so we we've established a new assessment and development cycle each year you get to your boss and you are developed and assessed and at the end getting promoted according to the triple model what we call so basically it's we promote team leadership and impact in the three primary tasks that we are doing research education professional performance and our approach is holistic so it's all those themes that I told talk to you about open access but also with good data reliable data engagement with the public or relevant stakeholders to make science for society doing open education and actually also educating the new professionals in that direction with the relevant recognition reward system in the Netherlands within the European Union well and hopefully with the world I have I'm running out of time so if you have any questions also on the pictures I will be happy to respond to those in the question and answer session thank you okay thank you can everyone hear me okay it's on okay good excellent okay so if you have any questions please queue up I'm going to kick us off actually I think one of the things that we talked about when we were preparing for this panel first of all it shouldn't go without saying just to point out we we tried to get some diversity in terms of institutional representation so we have US federal agency we have obviously Berlin Institute for Health and Quest Center very different unique kind of circumstance and then I'll say in air quotes more traditional university situation but but certainly there are certain dynamics to that but the other thing that we talked about was just the fact that all of these are are fairly well supported and funded so I just want to say that that's we make that acknowledgement as a group that that's not always the case and if there are questions about that we should raise those as well in the discussion so the first question I have and I'll maybe throw it lob it over to Shell is one that's related really to implementations implementation oriented I guess the question that you know you you have a fair amount of funding but you still had to make choices in terms of priorities and as you've kicked things off we've talked about the fact that there are resource and investment tradeoffs in any reform effort and so can you describe you know for the group here what are the resource and investment tradeoffs to consider when implementing this kind of reform what have they been within the NASA kind of paradigm yeah I think one of the first really important lessons that I had going into this project was from other people in leadership positions at federal agencies who had been responsible for programs and the first thing that they told me was they will love to give you a title but you have to fight for a budget so don't accept a position without a budget because then you won't be able to do anything meaningful so the first thing was just getting a budget luckily NASA has high level support for this initiative and this sort of came into that what had already been thought of as an existing budget and so we came into this with a budget and we had to make some decisions about what we would be investing in and what we did is we did a lot of community outreach and we were doing a lot of community forums talking to everyone who would talk to us and through that outreach process listening to the community about what the needs were is really how we came to developing this outreach this curriculum and the main areas that we're working in so we heard we want more visibility we want more support so we decided we're going to invest in communications so that we can provide high level cover high level support so people aren't afraid to take what is a risk and next we heard we don't know how to do this you can't ask us to do this and not support that journey so we're developing a curriculum for this we also heard there aren't any incentives in place how can you ask us to do this if we're going to not get tenure and we've heard earlier today about professors who struggled with tenure so we're working really hard to announce new incentives in open science this year and then finally they wanted to see coordination among the agencies so that if they had some rules that they were following at one agency they wanted it to be easy to to apply to multiple agencies and not have to reinvent or rewrite sections of their proposal to because NSF had something different than NASA which had something different than NOAA so trying to and that's where the year of a year of open science that coordination on the federal agencies came from so you really i mean in terms of those investment choices it sounds like you really relied on the community was there were there any hard choices at the outset in terms of those potential trade-offs or things that or a matter of timing at the beginning of a project you often have more ideas than you have but you also have funding because you haven't committed any of it so i'm right at the point now i've spent all my money and i feel like a hungry hippo just trying to get more so that i can fund because there's you can't put enough money into open science right now i think we're at a moment in history where if we really lean into this we can transform to a more equitable inclusive scientific future so right now i am having to make hard choices about what particular things within that portfolio i can support now and what i have to push for a year or two i don't know if you want to comment on because we've had a little bit of a chat on trade-offs what what has that been like or what have you observed at quest because you've ramped up this reform effort we were very lucky that the institution there was a moment when we could secure this this type of funding which was kind of a a gift from heaven away and this is now baked into the into the institution so we don't have to fight for it and the number was someone made it up in fact i was when when i was trying to convince them that we need an initiative like this i i was much more humble because i didn't think that there's so much money in the system but they they said well okay 2.5 million and i didn't complain but now we are learning and that i already indicated this that uh and that's the hungry hippo i guess if it needs a lot of it needs a lot of money and the but we are now i think well equipped to to pilot a few things but once they are working and we have some confidence in that we're going in the right direction take for example research data management this should be done by an institution and not by a special group that has these fancy ideas should be part of the what the institution provides to its researchers and then you have to scale it up 8 000 researchers we now have one person doing it but for 8 000 researchers you would need i don't know 10 and then of course they are not so happy with our suggestions so i and that's really uh a big big problem i think um because we get a lot of exposure they they the leadership is pointing at us they are proud that they are bragging uh when they are with colleagues from other universities um but this is just for these for these uh special activities and and once it is no longer that exciting but routine then uh so i guess that's that goes very deep the the only response to this in the end and that's why i think it's a very very big problem is um it it it touches at least in germany um with the financing of the universities and universities in germany are under financed this is a state financing system of universities and the the financing is extremely tight and if you come with something new they tell you well we can do it but then we have to get rid of something else and then you a lot of hands are going up and no no not mine not mine i think so um to to make to to really scale up what we are doing in germany at least we would need a reform of of the entire university system well that that's actually a great segue and this is is getting a little bit more to the meta level or a combination of meta and implementation but maybe to mira what have you learned thus far about scaling up and sustaining the efforts that that you have that you tracked uh yeah i would say three things um first of all uh is getting started i think any of us if you you know try to go sporting or give up smoking i think it's the first thing it's difficult is to start once you've started it's kind of already becomes the the the part of your daily job it becomes we have it in the strategic plan of the university for the next years as well we have it now in in our research evaluation we just had a number of programs the different faculties evaluated by external research group which has followed our approach so it becomes kind of a normal thing to do um the second one is communication and it's it's funny thing that you already talked about how important it is to invest in there but the funny thing is what i see the most effective is when you communicate really to people many times you meet someone at i don't know new year's drinks a colleague you haven't seen for in the corona times and then oh yeah open science and sometimes it's still vague and and oh is it just publishing openly and once you just said a few things like no it's about actually quality innovation and and and getting you know the why of of you're doing anything then becomes so clear and then oh you know and then they pass it through so it's very important also this personal contact from the people involved uh and here say and the last one at least i think support and also in terms of uh not putting it as a oblige you know top down okay tomorrow we are open science university these are the things you know the checklist to do no it's actually going to the relevant people especially the the the leaders of research group education institutions etc and that they actually talk with their people so what are you doing why doing it and what support do you need to do that because i've run my cell phone project for students i call it the not waste students research recycle so we have done so what can you do with your papers you know for search students it's pity that it goes all to the archive and basically to the bin you know garbage bin um so once we've done our idea i decided to check the whole university so you know what are people doing with their students and we found at least 30 initiatives just already people teachers doing that within different faculties disciplines etc so it's not like we are not doing it it's actually making it's more explicit and given the necessary support to the things that we already sometimes doing without even know that's great i want to dig in i want to make sure there aren't any questions okay you mentioned just the um research evaluation you know a promotion tenure you know that's been a topic that's come up a little bit but can you describe a little bit more how that change process was because that always seems like a tremendous obstacle with that can be politicized very entrenched what was that process like or has has that process been like well we're still in the process i must say i must say but it's it's moving and what we've done so far is first well define these elements that we want to use in our sort of annual talks with all the employees we've decided to skip the the division of academic and support staff because it was kind of you know and also there's a whole debate also within the Netherlands about everybody is a professor so it doesn't mean assistant associate full professor everybody could use the title as a professor in specific circumstances etc this is still a debating but what we are trying to do is that first of all to put this system as a sort of a development cycle it's not like tomorrow everybody is a professor and it's not realistic and you don't even want that you want to develop you want to find things in your daily work what you like and what you think hmm i could you know study a bit more about it etc so develop yourself so these are the things that we are inputting and now into place and the next step is to you know how it's going to be promoted into this different you know it's always called the new wine into the old bottles of these three divisions which we have in the Netherlands assistant associate full professor and at this moment we actually i mean it's it will be announced in july so i'm not allowed to say that yet but we are getting to to some reforms to get rid of some of the you know complexities how to get from one to another but it takes a while a lot of discussion and sometimes debates differences within the disciplines is it really true i'm not sure actually but yeah it's next time okay who was first okay i'm over here i've earned a camera with with plus thank you very much for these presentations and i guess my question is probably for the the europeans who are on the on the stage i mean you've both described very impressive programs in your in your universities and i gather that you both benefit from strong support from leadership of of the university and and i'm contrasting that in my head with what we heard this morning about the i don't know how to characterize it but the the state that we have in in in the u.s. and in particular with you know i mean we heard this this point that we have two u.s. universities that have signed dora i don't know how many there are in the netherlands and in germany but i'm sure it's way more than that and and so so my question to you is more about how do you how do you think about the efforts that you are you're leading at your university and how can that spread within your national or european context because i think what we hear a lot from leadership of american universities is that it's a collective action problem nobody wants to be the first to do it and then and then actually if we wanted to change culture it's very because of the movement of researchers and all that it's very difficult to change culture in only one place it's not going to be very efficient if it doesn't change elsewhere and so how are you thinking about the scaling not within your university but but across the higher education system in in in your country thank you question thank you very much and i can start by first of all congratulating the netherlands because i think you have a national plan sort of the universities as i understand universities academies the the ministry they they all came together so this is the perfect breeding ground for for scaling it up nationally so you're you're in a in an excellent situation there in germany i think we have big hopes to the quara process which is a european union led initiative in fact the european union has started it but now it's it's it's spreading into a participatory thing of of universities funders and so forth and i think this or i should say i hope that this will be very helpful in this regard it's it's a little bit like dora but it goes way beyond dora touches upon all the elements of assessing researchers and i think this can be or i hope again i have to say i hope that this will be very powerful if now more institutions are signing this so our institution has signed it others are signing it now too and that to me is the process i think where this collective movement actually can take place because now you can point to others they are doing we are doing it um and so we will see over the next few years i think that's that's again a process that will take time but over the next years i think we will see some progress because leadership looks at those things the funders are picking it up the european union has is already very strong on these things local funders at least in germany are starting to to take note at least um because it's coming from this high level of the european commission and of the funders of europe so that will be my my hope how we are going to scale this up now just to add i would congratulate also the us on this at least already starting with the open science here i think it's already getting here um and honestly from my own research on compliance with laws and policies on the one hand yes you need this top-down approach the policy the money the the law or whatever and it is helpful indeed in the dutch system we even have it nowadays in the cabinet of agreement that we invest in open science and we need to have a new recognition and reward system at the same time please don't forget there are different levels of the reform so that could take a while but it is still already happening if we are here today i guess but there is also a very important level of of individual researchers and then individual collaborations i mean nothing stops you um of thinking okay why am i doing something that what i doing i was also once you know when i was finished my phd i was checking 200 case notes of students all the same you know and some points they were so nicely written case notes and i really thought wow it's a pity that it goes away so i decided to start this projects and i collaborated with others who like the idea so i mean for me it's part of open science it's just making or adding purpose to my work to my daily work and what i've done is that i started to reward those students who write nice uh case notes and asking them to put it to a wikipedia page to build up the societal knowledge and they get an extra bonus point for their grade i mean and this makes also fun for yourself because students also interacting and they see differently etc so there are i mean different levels where you want to achieve it but also don't forget about yourselves again i just want to add that there is a coalition of over 90 universities in the us it's called helios and if your university isn't part of it you might consider advocating for them to join but this is to advocate in advanced open science at universities uh there's several open science project office and there's also there was just a call from a philanthropic organization to do more open science offices at universities so there is there's a lot going on in europe and the us i think and if your university isn't part of it advocate for them to do so i just to call out there unique the point of a lot of times in reform you don't there's the i don't want to be first right especially when there's a lot at stake at that institutional level so i think it's a very good call out so thank you brian brian no sex center for open science my questions about project management uh to what extent have you adopted traditional project management strategies of well specified goals and implementation plans in advance versus more agile strategies where the goals and implementation is responsive to what's happening on the ground and why do you regret not choosing the other one thank you um i i can just start i mean we have uh various various ways of dealing with these things that we're doing with these projects in some we have a clear um theory of action which i think then dictates a full program in a way and also an assessment and monitoring of of checkpoints within the program and it's an evaluation that is formative so we are kind of in a circle of of while we're evaluating we're changing our um what we are doing some of what we are doing i think is done more agile or i could say that that's a fancy word of saying it's done at hoc um because um there are opportunities that that are popping up in terms of funding there are opportunities popping up in terms of people who are suddenly i mean there are positions that change in the university so you and and your theory of action may not include this so um i think we are in a way i think we're pretty opportunistic uh in how we are trying to achieve what we are what we can do some mix i would say we've started off with one page then objectives then goals then key performance indicators we're doing risk analysis so we're doing a more traditional project management approach uh and that is in part because we're within nasa and there's very formal guidelines about what a project is how it is organized how it is run how it's implemented and while we were more agile at the beginning as we were developing it's becoming more formalized as it goes into it in part because of the way that government funding is organized and the way these projects are spun up and in some ways the speed that they're able to move with is far far less agile than i ever would have anticipated so there's some adjustments going on and if you ask me in a year i'll have a better answer as to how this is working out um it's it's a struggle to do something this unconventional within a conventional agency but i'm glad i'm doing it within nasa which has so many tools about risk assessment and being able to agile respond to new information especially when you're developing missions that that i think is helping our project i think to add for us i i guess it's also both and you just split the the terms your short term goal mid term long term and a short term goal also because of the funding you need to be a bit more specific but then it's open-ended because you still have to define what is now public engagement it took two years for our group of representatives of all faculties to really write an open access paper of what public engagement should be could be etc so it's it's open afterwards now we just reform into the next stage of the program so i guess it's it's both and the why you wouldn't regret that you asked well it's not regretting but i think it just fits what we are doing as transition so you cannot be you know very strict as any research grant application as well you you think you're moving to aid could be at the end as long as you can justify it's it's also scientific discovery so it's a transition process so i think it needs this kind of approach wonderful contributions in the talks that you've given i'm thinking about this a little more and like i'm i'm i'm sure what she'll said is essentially open sign seems to be all of these cool new things we have like technology we have collaboration we have diversity equity inclusion all of the things i think she'll like really summarize this nicely all of the things that we all want and that we all think is cool but while we can spend like eight billion dollars to shoot a james webspace telescope into space and operate it we don't seem to be able to find really the money to do the transformation that would help us all and so my question is is the problem maybe not our transformation to open science but that science or the way we practice is is fundamentally not collaborative and that we're not only trying to transform to open science but that we are trying to do a more foundational transformation of science itself i'm sorry if i stumped it no no no but for all of us or uh whatever i mean i think it would be an interesting discussion why is something that we all clearly want so difficult to fund this transition takes time and the mindset difference i think it's not just you know putting the law into practice and then thinking that tomorrow there is going to be 100 uh percent compliance is what i said you know we've we've issued beautiful working papers on the new recognition and reward system a few years ago and then two three years later i mean it was the corona years but okay still it was online you could read it i meet colleagues at the professor level who were you know still wondering so it's really important to communicate to discuss at a very individual level because each of us we are situated in a particular school within a particular discipline sometimes a particular method of working and not that many people ask themselves a question on a daily basis why is it so is that okay no we think oh we we just go with the flow and this takes time i must say so far um and i invite people to talk to me also against like hey i would like to see counter arguments um and i'd like to to get into those debates but um people just you know need time and when they hear more and more about it they start thinking trying to see to what extent it could apply to their situation to their group research group etc so yeah i think that that's my experience so far just time but can i challenge your assumption that we all want it i mean all in this room wanted but i guess a big problem is that not everyone wants it or so uh that's that that i think is is a big a big hurdle here because uh we we need to on the one hand convince others that they really want it we need to demonstrate actually that this paradigm these paradigms that we are preaching partially i mean i'm partially preaching is is really worth it and that that we are that we need to create evidence that that this paradigm changes uh all those things that we are actually uh we believe uh a lot of a lot of for a lot of the uh things that we are discussing here they are extremely plausible um and they're also very plausible ways how we get there but in the end uh we have to demonstrate that this is actually true so um uh it's it's not only about money i guess although i have brought up the money issue i think it's more about the mindset i i would also say that we have an opportunity i think that it hasn't there was a lot of activity around open science around a decade ago and it there was a lot of financial investment especially within europe and it didn't take off uh i think i hope that this is different now and i think it's different in part because of the advances in the cyber infrastructure and the ability the tools i just don't think it was quite ready i don't think it was mature enough you know i can publish an interactive notebook and share my entire workflow including the data in about 10 minutes now and that was not possible even five years ago uh so i think the infrastructure is there and if we're taking this opportunity i also think we're at a moment in time where if we're going to reimagine how we do science we need to reimagine who's doing science with us who's in the room and who's at the table and we have a chance to create a more equitable inclusive future and i think that's also part of why it's resonating more with people now is that addition of the equity component because so many of us for various reasons have been on the outside looking in and it doesn't feel great and we have an opportunity now to not just invite people into the room but actually just change our definition of the room to make it so that everyone has a sense of belonging and i think that's really important when we look at what we want out of the next decade as we move into open science i think you was up next you're kind okay hi mario malichki from stanford university if i can provide a little bit more data on that um 90 institutions as you said stine healers but there's more than 4 000 in united states and when it comes to quara only 500 signed up quara right now and there's more than 14 000 so we're not even close to 10 percent of institutions following this but my question here was for you for those who are actually implementing changes how did you manage to balance the diversity and inclusion opposed to open science we've seen reports that only 10 percent of faculty in united states are the ones that haven't completed their phd in united states and in europe reports seem to be less than that that the faculty who haven't completed their phd's within that country has gotten those positions so is it more important that we are advancing hiring and through in tenure promotions for the open science practices rather than diversity and inclusion practices i think that they're reinforcing i don't think you have to choose one or the other i think they're mutually reinforcing and if you do things in an open inclusive way you'll achieve your diversity and you do it thoughtfully with design using evidence-based approach we saw the picture with the spaghetti earlier uh if you were in that talk it was very engaging um but i think you can have both and i agree you know we're maybe we're only at 10 percent but this is actually a really good place i think to be at right now and again this is a year of open science this is the start of building out what will become a new era uh well so far what we have been doing is that um we have quite an umbrella of issue on the open science at the uteric university so um what we've done so far is trying to also you know find focus within you know specific areas open access we have actually achieved the goals that we've done i think quite uh already uh very well so we find it really like a successful project um a few publishers only especially for the native language which is still a sort of a negotiation stage but for the rest we are really happy so we are kind of doing the step-by-step approach and diversity inclusion it's not like it's not under the umbrella but it's one of the issues that we have discussed for instance in our promotion recognition and reward system and there is a huge debate about also actually the fact or the sort of question mark if it is necessary for a scholar to be successful to be moving around uh to you know experience different uh research institutions uh different cultures especially within the european union where we also want to have a free movement of of people etc so i mean i don't have an answer yet it's part of the debate but not necessarily at this moment it's not necessarily our main focus but it's certainly not included and it's it's certainly in a debate how to integrate especially international EU citizens who put us a lot uh into the the the the promotion system and at our university i think we i mean i work at the department of international european law the faculty of law economics and governance and we i think we are very international okay we'll take one last question we got started a little bit late so we'll take one more and then we'll do kind of a one minute wrap up uh hi i'm howie from university of florida i wanted on the point about the the telescope and um it reminded me of this article recently about how google constantly develops new products and then they discontinue them and it's because you can get promoted and recognition for building new and shiny things but they you don't get promoted and rewarded for maintaining them after they've been created so i'm kind of curious about kind of the thoughts of the panel on moving forward what your you know perceived to be the challenges and maybe opportunities in sustaining uh transformation and reform well if i may uh well in this whole debate of the last five years or something where i'm there the system of recognition and reward is not the same as the promotion and that we have had a lot of discussion about it so um uh it's not like i'm not i mean i'm associate professor would i like to be a full professor yes of course uh is it now uh recognition and reward not necessarily 100 for me because i rather be recognized for bringing a good idea uh for you know being cited being invited to talk about my topic etc then i feel really recognized rather than simply putting the the title before me although it would make me a bit perhaps happier i would say so it's not exactly the same and what we're looking for is how to make people incentivize to do good research because what you have if you don't have any recognition and reward is that you have a research group someone as a leader gets all the credit and the old system i would say or bad system and the rest is not even cited or even named and actually what you have at the end is counterproductive result nobody wants to go to work everybody takes a lot of coffee pauser you know and and etc you don't really get innovation and anything so you want to get rid of that now for many people it has something to do with the title that you want to have but then you have to ask what do you want to reach with that title do you want the title professor before the name or do you want specific functions that the professor is performing so now we have just moved and we have a very interesting block of a colleague of mine from the the dutch royal academy the young scientist there about simply it was really like five things which they distilled from the real job description of a professor what we would like to have with the title of a professor i can i can send it to you the link very like very simply explained so i think this is how a debate we are doing right now is that simply think in the title the function descriptions what we can do and when we people can be doing it to be satisfied with what they are doing on a daily basis uli or or shell in terms of sustaining very simple minded i think the the key to all this is is the the rewarding incentive system if we the moment we reward and incentivize what we want and we heard from iran mccurran today that there is this mismatch between the values that we want and what we actually reward if we if we bring this into into a match we will on the one hand all progress with these with these agendas and and then they will be sustainable they're scientists are smart i mean they are they're going for for what how they want to do their research but they also want to be successful with make a living personal living with it and that this is this is the mismatch we are having right now and if we if we are able to manage this i think we'll be we have reached our goal we have sustainment through policy so the new policies that start with forward looking grants in 2023 uh the those go into effect they start getting progressively more uh uh integrated into the proposals and actually start being evaluated in 2025 so there's this gradual approach and then those those changes will be sustained those requirements will be part of the reward system because if you want nasa funding if you want pay tax payer dollars from nasa you must be practicing the open science as we define it by our policy but then we're also going further so the open source science initiative is now part of the chief science data office for all of nasa and again it's 30 million this year and then it's 50 million it increases and that is to support open science infrastructure policy and development so this is providing funding for open source software libraries things like that to sustain the develop to sustain these projects that are a core part of how we do science you are touching also slightly uh upon another issue of you know having research projects especially grants for five year projects which do not really allow science so sometimes to to bring the results what we want and this is also part of the whole reform we cannot go with that because this is what i know from also myself colleagues you get the funding for five years and then stop and then all the open science i run a blog what should i do now i cannot fund it anymore so it doesn't make sense so this that's why we need a reform thank you so much okay we are at time i had a couple of other questions but we will have to do that over happy hour i guess please join me in thanking our wonderful panelists today