 Welcome everyone. My name's James Wilson, Director of the Research on Research Institute, and it's my pleasure to be moderating this session, which is focused on the issue of the moment, how we build ourselves back out of the pandemic which has dominated life in almost every sphere over the past almost two years now. We want to focus in this session on the ways in which the ripple effects of the pandemic will play out in research systems, cultures, the approach we take to the funding and evaluation of research, and the ways in which research priorities are set over the short, medium and longer term. As we said in the blurb that accompanied this session, we wanted particularly to try and raise our sights, if we could, as a group, to beyond the immediate effects, which of course we've seen already in terms of a lot of agile responses by funders and by researchers themselves of course to the urgent challenges of the pandemic. But to raise our sights to try and think, well, what kind of mark will this whole episode leave on research five, 10, 15 years from now. So that is the task we've set ourselves. We are joined by a fantastic panel. I'll introduce them all now at the start and then more briefly as they as they make their opening comments and interventions but first up we're going to hear from Matias Eger Matias is of course president of the Swiss National Science Foundation, and also played a very central role in the Swiss national science community as opposed to COVID as the chair of the key scientific advisory committee in the early months of the last year. Following Matias we're going to hear from Channeta Jones, now vice president for research at the Michael Smith Foundation for health research in British Columbia in Vancouver, Canada, and before that, here in the UK at the Welcome Trust. And also former co-chair of Rory, my own Institute so it's great to have Channeta here. And then thirdly we're going to hear from Karen Salt. Karen is deputy director for research culture and environment. So really crucial brief in these times at UK research innovation the National Research Funding Agency here in the UK. And also to be joined by a fourth panelist, Ganson Pillay from the National Research Foundation of South Africa. And as soon as he arrives we'll bump him on to the panel and hope that he does, he does join us. But we've got plenty to talk about among those of us that are here already. What I'm going to do is ask each of the panelists an open question we thought it would be better not to have big formal talks to kick off a topic like this. We've got some opening questions and then we'd of course greatly encourage panelists to pick up and respond to one another. And all of you as as participants and attendees to also make liberal use of both the chat box and the Q&A box at the bottom of your screen so that we can scoop up and hear from as many of you as well on these issues. I do, you know, pop questions in as we go along. And we will, we will turn to those once we've once we've gone through the initial round. So, that is the plan. Let me start them by turning to Matias. And as I say you've been centrally involved in the response to COVID-19 from a range of different angles. From those different vantage points and particularly your ongoing role as president of the SNSF. How do you see the longer term effects of COVID-19 on research funding systems and priorities, both in Switzerland and internationally. And in particular I wanted to pick up on something you wrote about last year you wrote a thoughtful piece on the potential pitfalls of what some have called COVID-19 of research funding systems a sort of lurch to, I guess in a sense fight the current or the last battle rather than some of the other battles and challenges that may lie ahead. Can you say a bit more about that particular risk and whether you see funders like SNSF and others avoiding it or falling into it. So Matias, over to you. Thanks James. And so perhaps what I would like to add to the introduction is that I am an epidemiologist and an infectious disease epidemiologist so the COVID pandemic really was something that concerned me, both as a chair of a funding organisation but also as a researcher, because I'm still research active at the university and I discovered that my field epidemiology, which up to then basically was, you know, a difficult thing to explain to people and many people it had something to do with skin diseases and all of a sudden everyone knew about epidemiology and actually many people were described as epidemiologists and during during that phase and talked as epidemiologists. As some people have said, you know, the population of Switzerland is 8.5 million. We now have 8.5 million epidemiologists in Switzerland. And as a funding agency, of course, we felt that we need to respond to this very quickly. And we launched a call in March already. And we took the money for this call out of our reserves. And that's important. And I'll come back to that. What we what we also observed is that everyone in the research community got interested in COVID. And the government told us that we should launch another call. And then the second call was sort of additional money from the government, which was aimed at biomedical questions. And then a third call which we actually will be launching in a few days, which will address social science humanities issues. And an important point to make is that we didn't want to eat into the budget, our normal budget. We didn't want COVID to take over. We wanted to keep the funding for everyone at the same level. And to avoid, you know, what you described as COVIDization that everyone jumps on this funding opportunity and the quality goes down. We wanted people to carry on to research what they what they want to want to do. Now, what this crisis in Switzerland really brought to the fore is that although Switzerland is quite, you know, a prominent has a prominent research and science ecosystem and does well in research, there is very little exchange between politics and science. And the same thing we have observed in universities of applied sciences and industry, that there isn't enough, you know, people are sitting in their silos. So in the context of applied sciences, we've launched a pilot program where we fund people from industry to go back into applied sciences universities. And we are thinking about starting schemes that do the same with the political sphere. And that there is more exchange between science and and and politics. And in that context we, and Johnetta actually alluded to this we had an exchange recently with the William T grant foundation, which is mainly funding research on inequalities in youth, and they have a program which is called the institutional challenge program, where we, where they found research institutes to build research practice partnerships with public agencies or non profit organizations. So I think the longer term impact and I'll close with that. The longer term impact I see coming is that we will invest more in building bridges. And because it was very clear that building bridges in a storm is difficult. And there is not enough exchange between science and the policymaking bodies in our country. So that's, so that's an encouraging sort of counter trend to COVIDization in that sense. Can I just for turning on to genetic. Can I just ask, I mean, one big question of course that hangs over research funding post COVID will be for public funding agencies in particular, because there's enough money to, to, you know, keep on funding research or to fund research at the necessary levels after all the other economic knock on effects of the pandemic have been addressed including of course the huge needed in health systems and I just want me from a Swiss perspective, how does that look in the sense how is the, the, the broader settlement with politicians and the public with respect to the funding of research because I mean obviously you as a nation fund research at a high level certainly high level than we do here in the UK. So I think that level will stay. It will not increase very much. But I think it is largely protected parliament will protect that level of funding. The situation chains is complicated by the fact that we've recently been excluded from the European programs which is not the topic of today, but which also offers, perhaps, of course it's negative, you know it isolates Switzerland, but in the short term, it may, it may also lead to increased funding to minimize these negative effects. And of course, the goal will be to be associated. Actually the UK is in the same situation aren't you because of political reasons we don't want to go into, but it essentially we want to be re associated with the European programs as soon as possible. In the meantime, there may be some opportunities for additional funding instruments in order to counteract the negative effect of being excluded, which we witnessed already in 2014. Great. Thanks. We may we may return to that question. Yes, we are. We'll try and avoid Brexit as a topic I think that we'll learn. Yeah, there are there are definitely parallels on that from with between Switzerland and the UK. Chinetta turning to Canada. Within the Michael Smith Foundation for Health Research you were very quick off the mark last year in going out proactively surveying and talking basically to your research community those that you fund and those also that you don't in the wider British Columbian health research community about the effects of the pandemic, and particularly some of the differential effects that the pandemic was having on on different groups very obviously those with, you know, family or childcare responsibilities those with health challenges of one type or another. And so I wonder if you could just tell us a bit more about that process and what you learned from it. And then also the extent to which any responses that you've made to the particular features of that situation have them been carried through into some of your longer term policies and strategies. The adjustments that you made during the sort of most urgent crisis phases the pandemic persist over a longer period in the way that you approach funding. No thanks James. So to your first question what did we learn when the pandemic was first declared a public health emergency we like many other funders and as Matias explained we paused non essential activities and immediately deployed funding to support COVID research that would inform an effective public health response. But while we were responding to urgent short term research needs we were acutely aware that there would be longer term impacts caused by the pandemic's disruption of research. We said we proactively consulted research leaders, the research community and our partners to understand the impact of COVID on research from different perspectives and much of what we heard and learned and mirrored what was reported around the world. The majority more than 80% of researchers reported that the pandemic had a negative impact on their ability to do research, and only a small proportion of researchers about 14% reported that they were able to take advantage of the new funding and shift their research to focus on COVID. But the pandemic as you said had a disproportionate impact on specific groups, for example women and caregivers were struggling to balance responsibilities at home with research activities and maintain research productivity. On behalf of students and postdocs reported the pandemic had greatly impacted their training and reduce their ability to network and collaborate with others. Early career researchers who were pre tenure or new faculty were affected by delays and grant conditions were struggling to build or maintain their teams and limited to no access to clinical populations or communities, marginalized groups that would have benefited from participating for example in clinical trials. So how did we respond again like many other funders we offered you know general support like cost of extensions, relax and grant conditions to give more flexibility extending eligibility windows. We also made a number of very specific changes to remove any unintended barriers to funding and to those who may have been disproportionately impacted by the pandemic. So we revised our application forms and invited applicants to share how had COVID impacted their research plans their productivity and any other challenges even personal that they were facing. We also made new guidelines and training for peer reviewers on how to take the applicants, COVID impact statements into account, provide more equitable and fair review of all applications. We diversified the membership of our peer review panels and strengthen peer reviewed guidance on the assessment criteria to specifically address any issues of inequity, and we pushed compliance with the principles of Dora. We're working with our panels to recognize the full range of research outputs and focus on the quality and the impact of published works, not just the number. So in terms of what longer term impact you know the pandemic believe in funding strategy and priorities and this is going to be true I'm sure for all funders across the world the research system is not the same. And that has changed the use of virtual technology shifts and research priorities, and as Matthias alluded to an expectation that research should be poised to dress the soil challenges like a pandemic, or I would say climate change, and now that we're in recovery this period represents a critical opportunity to reflect on lessons and insights that we should take into the future. There's a critical role for meta science to inform a post pandemic research system so just a couple of topics and I'll end with that we're kind of inferred by my comments the first is around research careers and career pathways. We know that postdocs and early career researchers are in critical stages of their careers, they were already sensitive to career transitions. The loss of research productivity will make it extremely challenging for researchers to succeed in a highly competitive research environment. Disruptions by the pandemic could cause significant setbacks effect productivity and their prospects for securing funding or jobs in the future. And so there's a risk that the research system will see a loss of critical research talent. The second topic is funding strategies and instruments and I've mentioned a few of the interventions that we made in our own funding policies programs, but even rapid response of funding the impact that mode of funding will have, and whether that'll be appropriate for for funding going forward, long investments and some areas research will be affected by the shifts and funding priorities we're already seeing that on national levels around the world. We've spread adoption of open access and data sharing for public health emergencies accelerated development of COVID vaccines and treatments, and it made the case even stronger for open research and will this persist post pandemic. And the last point topic I'll make is around systemic inequities and racial justice, longstanding inequities and research were present before the pandemic, the pandemic only exacerbated these inequities. Calls for action on systemic racism have activated funders higher education institutions publishers and others to take ED and I and racism more seriously to understand how inequities marginalized communities to make changes toward more inclusive research and more inclusive inclusive research systems. So this James is a significant window of opportunities to accelerate and make meaningful progress, and to evaluate and scale up initiatives that are working, as well as monitor challenges or unintended harms on under the agenda. Thanks James. Excellent thanks to that that's great to hear lots there we can come back to in discussion. I just wondered, you mentioned the role of meta science which is something we will I'm sure talk about as a panel shortly but from your perspective in terms of the changes that you introduced in the most urgent phases as it were. How easy was it to simultaneously build in the kind of testing evaluation that would be required to sort of draw robust conclusions from on the efficacy of those sorts of interventions in the heat at the moment I mean I know, talking to other funders that while many introduced measures of various kinds it often was very difficult to also put in place, you know those kind of evaluative infrastructures around those initiatives for completely understandable human reasons I just wondered what your experience was of that dilemma. Well, that's a great question I think to our benefit we were already thinking about strengthening our ability to evaluate to analyze to collect data on on new, not only what we were doing but any potential new interventions or new programs. We were doing that already for really prior to my joining the foundation, or my colleagues had done that prior to the pandemic so we were actually really in a good position to be able to collect data and to be able to analyze data now by no means as robust as if we had more time to prepare, but what we wanted to do is we were very conscious of the fact that, wow this is a natural experiment, we're doing some things differently, it can we can we evaluate to some degree or at least monitor and understand what we're doing a new intervention is working. And I think we were really pleased and what we were able to to to see, you know, kind of before and post or intervention. I think the other thing we didn't know, you know, research is not just, I mean, research system for search funding is also a social process, we're not doing things by ourselves. And we weren't really sure whether some of the changes the interventions which we would normally not have done in the way that we did, we would have probably done wider consultation explain communication of what we wanted to try, we just did it. And so we were pleasantly surprised at how receptive our communities are our partners or panels were to just get on, because, and that's why, you know, I don't remember the phrase Christ using crisis as an opportunity, despite all of the other, you know, the trauma and bad effects of that. I think we saw that and so we seize that opportunity so going forward I think we will have some information but I think if anything it's given us the license and probably honestly the courage to continue to innovate and experiment and try things and to understand what works. That's good to hear again I'm sure we'll come and revisit some of that with others shortly. Karen, turning to you, if I may. You know you've got this role now at UKRI which for those that don't know UKRI is the big unified National Funding Agency here in the UK with a budget of around 9 billion pounds a year. So you've got the role there of developing and embedding strategies for research culture integrity, EDI. Security. They're adding to your list as the weeks go on but I mean all of these issues were already rising very fast up the research policy agenda for very good reasons. Covid has then happened and in a sense has put even greater urgency behind several of those agendas as we've just heard from Chinetta. I just wondered again from your perspective, the extent to which you know if we're kind of if we had to roll forward in five years or 10 years as it were and look back at the Covid period, whether you think it will be more of an anomaly in the sense that there were lots of quick things that were done in response to it but perhaps the effects of those won't be as deep-rooted or whether it will be an accelerant whether it will lead as Chinetta was intimating in her remarks just now to some of the bigger longer term shifts that many including yourself have argued for in the way in which we organise and manage and govern and operate our research systems. So your thoughts on that. It's a small question. I should also know open data is an open research is also my brief and I'll sucker back back to that kind of in a moment. I mean, it's interesting that the you know thinking of it this way because I'm a long term person working on long term researcher interested in and you know systems and transformations and governance and kind of changes at various different scales and levels. You know, I'd be remiss if I didn't look at that question you pose and kind of go probably a little bit above. I think we've got a whole set of things because often it's not necessarily does something accelerate something or is it an anonymous kind of moment. It's what do we learn from it. And where does that learning go, you know, do we end up with a kind of institutional memory or a memory within government that kind of recognizes all of this and kind of stores that so that you might have a shift of the top of an organization or a change in government you haven't lost that set of memory and knowledge. And in fact you have that bank and you can you can turn to so you know, I think what just mentioned about that learning is sort of like you know that is the type of stuff we really need to be be harnessing I think across the sector in multiple ways that where does that kind of information sit, you know where can you go and kind of pick this up kind of later kind of over time. And I have some more specific kind of responses other than the quibble with with anomalous and and accelerates. And I think part of it is sort of start starting with the recognition that, you know, this is the same system. It's the same structures, it's the same funding, and it's the same people. Yeah, we're trying to move in maybe additional funds of money from different places we're trying to move in always and increase the inclusivity of the people who are in it. We are making up new structures in various ways. And we're definitely thinking about systems transformation across the piece, but it's not as if we've invented a new R&I system to handle COVID. We didn't we took our existing sets of structures, and our existing sets of people and our existing sets of systems, and then try to address and deal with that now we're all interested in transformation across the piece to make this a place where, you know, things flourish and and people can thrive and great ideas and discoveries are just part and parcel of the packages of what we do. But that is also meant that we've also had to recognize that the system just like everybody else has had to respond to pretty seismic shifts shifts in terms of I mean you guys are in my house. I don't really give talks globally for people aren't in my house, but that has become the norm of COVID. And, and that has offered up so many more opportunities for all of us to be where we live to reduce our carbon footprint to really be a much more accessible kind of community to really bring lots of different people together. Now, is that a COVID type of thing, because we've talked about making these become the kind of fundamental ways that we we exchange and do business. And will it last, will we revert back maybe next year or the year after depending upon we're just back to the normal hanging out go into the meetings having the dinner is doing the various things that are just the kind of normal what we understand as the professional world of doing research and innovation. And I, you know, that that's a question of whether that we will we will get that learning and how we will actually embed that within within what we do, because the shifts have been profound. While you come into my home to be able to have this there are others who have small children pets, aging relatives all sorts of other things all around their homes, while they're also trying to do their business that's transformed their pattern of how they work. And the capacity and the productivity level which which they've been able to engage, because they've had more demands on their lives. People who've had long coven folks who are not vaccinated in any capacity who would like to be, but because of various different issues around supplies where they live globally, they do not have a vaccine necessary readily available in front of them that everyone in the population can have all the way through to those who have but but but can't get back into their environments, because they have other sets of restrictions so we, we've got a lot of different seismic shifts much much less the economic ones, which are very very big, building upon past economic problematic that our system is ultimately responding to. So if we add all of the kind of tensions, the inequalities that we know that have already been within our just normal societies, the challenges around capability and capacity. We get a situation where it's not really just this question of where should we invest our money in terms of this co post coven moment, we've got people who are really deeply tired, they're exhausted. Some who are dealing with huge levels of anxiety, some who are trying to figure out what is the new normal for them look like in terms of trying to do their work and they're trying to do this in often unstable environments where we've seen really shifts and employment in various different areas. And we're dealing with places that are really going to have to grapple with financial sustainability in terms of moving forward. Now how do you measure change in that environment. You know, it's sort of like let's throw everything at it, and then go, Okay, well what has changed well, you know, everything for some people, their daily lives has changed. I went to dinner with somebody last night, where we were masking and we're rearing the gel and we're doing the things we're sitting the distance we're having. And we still worked, but it changed the dynamic of how we were going to do that I don't know if that will stay essentially moving forward. But I think that there's some, there's some interesting things that this moment has done in terms of the spotlight that it has shown. And I think this is the question I have as opposed to an accelerant and an anomaly is what do we do with these things where the spotlight has been shown in really important ways. And I think the first and most critical one is the one I had mentioned in the one I think Jeanette has mentioned as well as Matthews is people. You know, whether or not we're thinking of it from a research a career perspective in terms of what people have been referring to it as the loss of the COVID generation or the COVID decade, or the lost generation, all the way through to the individuals who are just overworked right now. Right. Whether or not they're part of our health services or other sets of key workers in food delivery or other sets of areas where they're just they're just what is going to happen for the future of our of our communities, and then those who are working with R&I. What does a career look like given all the demands and the shifts that people within the UK context, a number of people within R&I who were who haven't health background volunteered for the for our national health service at the start of the pandemic. That was not necessarily as a researcher, they volunteered to potentially do all their sets of services and other sets of roles. So, again, what do we, what do we understand now about our people that are in the system and moving forward. The pace is the other part. All of these seismic shifts and changes have really have already taken an accelerated sector into overdrive, often because now the level of complexity or the level of challenges whether or not we're now adding back in to climate change on top of trying to think about, you know, a global pandemic and thinking about the education of our youth and understanding large levels of mental health. We are adding these buckets of these sets of problems. So we almost as an R&I sector have to work faster and faster with the various forms of solutions or interventions and thinking through and that has a cost eventually to the well being of those those staff members and the well being of those communities. Much less do we have the structures. Matthew was talking a lot about structures. You know, do we have the right set of structures now that we're going to need to be able to work dynamically and agilely moving forward. We don't want a future pandemic, but we would want to get to a place where the types of seismic shifts that we're talking about that we have a system that ultimately can adapt and respond to those and do it in ways that are sustainable moving forward. So we have to then say we stop doing other sets of work to be able to accommodate it. And I think it brings it to really big light the final spotlight for me which is decision making. I think this is going to be crucial for us moving forward, and that'll be decisions for what we prioritize, how we fund who we find who we invite to the table, much less than thinking about the decisions about how we go about doing research. And what is what does research and innovation ultimately look like and we've been doing a lot of work I think across the globe to think about this from a from an assessment process, all the way through to thinking about the inclusion point that you know to your raise, but you can we can recognize now that this this is shining spotlight on every type of decision making that we do at the very simple level to in terms of who we partner with. We've got all the commissions we've got all the way to the much more complex ones, which is you know what we invest large amounts of money, and we probably got to get underneath that to think about how do we make decisions in times of crisis, and in times of crisis, and in complexity that does it in an equitable, inclusive and fair way consistently, and not necessarily be pushed into emergency response mode, as the only way that we engage in terms of moving forward. Anyway, those are my four that I that I find, if I were to look at an anomaly or an accelerant, I would say you would get a mixed bag across those four types of things. But the proof will be what we do next. And I do, I agree completely with you know that this is our moment to really get underneath those to really think how we will go about doing this, and then how we will share that learning but also put in place those sustainable structures moving forward. Aaron, thank you that's great very rich. Multi dimensional contribution there with lots lots to unpack and talk about. I mean just coming back on a couple of quick things before we pull to the panel and can I just again encourage attendees to use the Q&A box or use the chat box and we'll come to you in a moment. So bump people up to be able to ask questions in person if that's something you'd prefer so use the raised hand function if that would be your preferred mode of interacting with the panel. I think you captured very well that the sort of tension between the dynamism and innovation that's accompanied all of these quick changes and also just the fact that lots of people in parts of the system are at this point. Pretty burned out by it all. You know in some of the discussions around the changes to research culture the move to preprints all of the stuff that we've seen through the pandemic. There's perhaps, do you feel there's perhaps a slightly naive assumption that that can become the new normal without enough recognition of quite how difficult it's been for funders as well I mean all of the emergency response stuff you know to operate in that kind of crisis mode as as many organizations have had to do is and of course the NHS here in the UK and equivalent health systems around the world continue to do in large part is is is impossible on a long term basis. So I just wondered whether yeah you had thoughts I guess on how one tempers some of the more positive enthusiasm for for us doing everything differently now with the more prosaic reality of just, you know there are only so many hours of day and we've all got lots of other pressures and we're all pretty exhausted none of us have had a holiday. Definitely. I mean there are you must have you must have this challenge I mean as a as a funder you know with. Absolutely, and I think I think some of it is. Quite a lot of the opportunities and the transformations were building upon existing things. So, I was talking about open data well we know that the, you know, huge amounts of data was able to be shared very early and relationship to covert because of the decades of work that people have been doing to instill open data protocols and processes and to make them available and then others were able to build on the back of some of those moving forward so it wasn't like we invented that to to share that that that knowledge of those sets of information. I mean even going all the way to you know thinking about the vaccines that we have the proliferate that's also built upon a whole series of sets of work that people have done in terms of getting to a place where we have vaccines. So, I think part of it is a recognize that some of the places we could go whether or not it's pre print or various other sets of things, we do actually have a lot of material. We've got a lot of initiatives and thinking kind of underneath that and in some ways, the meta science concept is is this space where, well actually, why are we working in silo across all of these domains and thinking about them distinctively where where and how can we draw this knowledge together. So it's not, you know, this entity over here has to go and set up something completely new. They could borrow and piggyback off of something someone has started over here we could actually coalesce and be much more effective and efficient. And because we're actually bringing bringing and drawing things together and research culture is a great example, where there are lots of things that folks are imagining should be now newly created to deal with research culture when in fact there are some very simple things. And one, one, one that is often talked about is is workload planning, but very simple, you know, clear articulations of workload planning can go a long way for folks to really recognize their value and their contribution. And to have that as a substantive kind of conversation. It almost sounds kind of archaic to kind of go workload planning to sit down and put a model together and talk about what sort of fair workload that people might have across various scales and abilities. To distort that, if you start to think that you have to work certain sets of time period or you can only do certain sets of things that you have to do things. You can see the knock on effect of what that might have for the types of ways that you you balance out your own well being in your mental health, much less than the types of things that you think well I have to have this article in this journal in this with this ranking to be able to move forward I have to do these sets of things. And then we get the same set of distortion and then we see stuff like where our reviews that we've done just recently on integrity have often pointed towards workload issues. As one of those stressors that can drive people into a various different practices around research integrity now that's interesting right because it's not a we didn't start that by asking people about workload. Tell us what your workload look like but but by exposing asking the question about integrity, we could see the join up across this other space and it goes all right well maybe the intervention is not so complicated. And it's not this massive thing that we need to do maybe there's some base level stuff that we can put in place. I think moving forward so that's not to say the folks who've got the pie in the sky naive types of things thinking as you describe our wistfully thinking about a future that we can't ever deliver, but I do think we've got the components already for some of this transformation, and we can also do things at very simple basic level to start to put in some of the conditions that we need to transform culture, because that's the thing about culture change, or cultures is a lot of it is about practice, much more than it is about policy, and these two things kind of go hand in hand together. Thanks Karen. I'll put in the chat for those that haven't seen a link to a SNSF meeting coming up next week in fact on best practices and research funding so do take a look at that. Now I am not seeing as you get any hands nor am I seeing any Q&A unless I'm missing some vital bit of the tech infrastructure here. Who are you going to come in? I mean I can keep asking you all questions so there'll be plenty to talk about, but please do everyone else, you know, use you put your hand up or put a question in if you'd like to ask anything of the panel. Yeah, I know I just wanted to, to, I guess, reflect on some something that the Karen said which I think is really integral, not only about child research systems, you know we'll work going forward but also in terms of you know meta research and it's that this concept of preparedness and agility and adaptation which I think is so key, you know, and so in terms of preparedness, you know one of the things I think we've learned, you know to Karen's point is that, you know, there were things that were localized relatively quickly, you know reserve funds for example that Matias mentioned infrastructure, which is, you know, is also about people and some of the technology enabling platforms, but we also knew we fell short of that like things when we reflect, we would have like to respond and quicker or more effectively etc. So, you know, really, instead of being in this kind of emergency responsive mode I think it is a huge opportunity to think about preparedness is, as Karen said about building really the processes of the infrastructure, the structures in order to be to better respond to the kind of the next seismic shift that we see because it's going to happen. So I think there's something really important about that and the agility and adaptation I think is really interesting, because again, it's it showed us what was possible and then made us reflect on well why, why are things the way they are, do they need to change and and and there should give us the the kind of really the the opportunity to say you know what we can bring about change, the pace of change doesn't have to be as slow as it has been regardless of what topic you care about. So I think there's something really important about that but also from a meta research and evaluative learning perspective, I think that's really important understand is that I think that from at least for many funders we're realizing that this kind of static approach to the change that we do our funding deploy our instruments and so forth, put out money, turn away work on the next thing and then we'll check in five years to see how things work. It's just not adequate anymore. So really having to adapt approaches for how we evaluate what works and so forth I think will be really important but also understand again from from people out there who care about studying. And the research system is that we are constantly changing and unfortunately we're doing it without data or evidence. A lot of it is instinctive. A lot of it is reactionary. And what I think there's a huge opportunity for mental research is to get us to help us to understand perhaps what what happened, you know, descriptive I think work is important as has this role, what we could learn from that and importantly, how can we experiment and make insights into trying new things and doing things differently. And so that I really love that that concept because I think that that is the direction of travel. And I add just very briefly something to this I also felt that Karen's point was extremely well taken and what I would add is that perhaps research funders should get together and develop a research agenda around the impact that has had on, you know, research and the lives of researchers in their respective countries and to do that at an international level to, for example, address the incredibly important question about integrity how that has affected, you know, increased so you could hypothesize that, you know, people were cutting corners who knows, but also on research culture, to what extent has covered perhaps helped to move towards a more healthy Dora compatible research culture or has it actually, which I fear, will it actually throw us back because things get harsher and jobs get scarcer etc. And these ingrained, you know, criteria will sort of take over again. And that's my little input so I would like to collaborate with other funders on on these questions, perhaps within Rory, metal science as well I don't know I mean James is the leader there. And just to look at COVID impact on different, different developments and issues, the ones that Karen raised among others. And if I could just add one more thing, you know, I think that's great. I can't, I have no power to speak for my organization or every organization in the world to commit to do this but, but if I could be of help as we start to imagine something I think it would be fundamental and and in fact I would imagine, you know, being able to do something at various levels so so one conversation I started having quite quickly with with folks from a very but various publishers and also with James were changes I started to see it at the method of methodological level. Due to COVID and, and, and it's really interesting to start thinking about this this way. I mean, I'm not sure if Nessia has already mentioned about people unable to access or be participating trials, but then imagine people unable to go to sites or unable to go to certain sets of places or they can't, they can't access they can't go to digs. There's still a lot of travel that doesn't happen. And so for those who those who have sites that are that are quite international that that might involve a lot of community sets of interactions, or working in various kind of environments and spaces. And so it's really interesting to do that work or that work looks very different at the particular moment. So what's going to happen to the methodologies that we're going to see the types of projects the types of questions and queries people might actually put in applications for investment for, due to the nature of the actual physical conditions, and the types of real conditions that we're grappling with with restricted travel with tests and various sets of regime and likely a system that's going to continue to do that for a number of months, and kind of moving forward. We already started seeing people move to more data driven types of research around certain sets of things, especially bank data that they might have had, who couldn't necessarily start to do new experiments in to prep for the types of things because that might have been restricted getting into the labs, because the lab based work was restricted for a period of time. Last year in certain areas around the world. And these are the types of questions that just from a funding perspective it's like okay well we if we do if we come up with new challenges to create these really good grand challenges and sets of work. We're restricting the types of things that might come in, just because of the nature of where we are right now in terms of dealing with the pandemic. And that means we've got to start thinking about that from from on our side as we're starting to move forward. In terms of what are some of these long term impacts and effects. And I definitely, I definitely think there's something there because that you know we might get to a place where we look back at the papers, and the types of patterns and discoveries and we can see how we're moving forward. We can start to see it quite because specifically in the types of work people are doing, not just necessarily the areas with which they're they're researching, but the how. And I think the sooner we can get underneath that the sooner we can start to help an arm, especially postgraduate researchers who are starting to grapple with these methodological issues right now in the midst of their thesis projects. I'm moving for much less everyone else. I'm just in BTS in terms of your proposal I love the idea of a fund is coming together and collaborating. Excuse me. Excuse me and sharing information but I'm going to put a shameless plug for two things what is that in Canada we have started to do this already so there was in response to what we were seeing and learning and recognition that we really wanted to seize this opportunity to take these lessons forward, the National Alliance of Provincial Health Research Organizations, NAFRO, the Health Charities Coalition of Canada, and the Canadian Institutes of Health Research. We are members Michael Smith are members of NAFRO. We all came together to identify what was happening. And what was emerging emerging through COVID, both in terms of the community perspective but also the kind of the broader funding research ecosystem perspective. And part of the insights and the work led to some concrete recommendations about collective action to benefit the research ecosystem going forward. So there's a report it's embargoed right now I can't share that but this is my so my first shameless plug is when it comes out, read it. My second shameless plug is for the Canadian Science Policy Conference because I believe that's where the report the findings of the report will be I can't remember exactly when it is but I'll find the link and post it in the November isn't it I think November. Yeah, I think someone, so there will be a panel or discussion or presentation on the report finding so Mattias I think it's, it's, it's the start but I think there's, it'd be really really useful to see whether some of those themes or learnings are true in other national contexts and whether you know there's an opportunity to kind of share that learning more broadly. Yeah, thank you although I mean that's great obviously, but I mean it's music to the ears of someone who runs a research on research institute here you're calling for more work on these things but and when we are certainly through Rory, you know, as I think you will know doing doing work on aspects of this. We've got a very interesting study that my colleagues Ludo Waltman at CWTS at Leiden and Steven Pymfield have been doing looking at innovations in peer review systems, more on the publication side than the grant giving side but that will be coming out in November and has, we've done a lot of new data gathering from a whole range of different scholarly communication organizations so that's another thing to look out for along with the report from the Canadian system. We're getting a few questions coming in which is great do keep them coming I'll come to some of those in a minute but I just wanted to pick up on Karen's point about some of the challenges posed by the pandemic to disciplines and methodologies that rely on mobility. And just ask the rest of the panel about that but also I guess expanded a bit to reflecting more broadly on how we may see the broader landscape of international collaboration changing as we come out of the pandemic. I mean clearly as a sort of high level there's all sorts of calls coming out of this for more better coordination between research funders in relation to pandemic response and all sorts of other aspects of, you know, health and related research. We also of course in a world where travel is restricted are going to see, and we are already seeing in some of the data, a retrenchment in terms of the volume and at least the patterns of collaboration that exists in that people are of course able to continue collaborating with those people that you know internationally and you know long term collaborations will will persist for quite some period. It's much harder to set up new collaborations with people you don't, you've never had any face to face interaction with much hard also for earlier career researchers who perhaps haven't had the opportunity to establish themselves and for whom international meeting for the conferences or other things can often be quite critical points in the development of a career and the building of a network in a discipline or subdiscipline. So I just wondered how you know what your thoughts were. I mean clearly you know the picture there is changing movement is gradually coming back in but it's going to be quite slow. So just sort of any thoughts really on on the collaborative landscape. Channette or Matisse Karen either of you want to come in on that. I see Karen's microphones on so I'm assuming Karen you go and I'll follow. Okay, that's, that's great. That was mainly because I hadn't pressed mute. More than an urgency to speak. But, yeah, I think what you've outlined James is is is interesting I have a lot lots of different questions and I think responses, I'll try to focus on the kind of one. And I think the one I find really interesting around this is, especially for trying to talk about what will what will collaborate to practice look like for the future for example, especially in the dynamics that we're describing. But I think we've kind of baked quite a lot of stuff into our systems around collaboration. So, you know, there are certain sets of things that if you put an application into X you get to bring everybody to X, you can sit around and, you know, hang about a particular place, work on I mean Volkswagen Foundation has this and Hanover. And in Italy there's some in California, they are site specific types of places that allow people to co locate to essentially build upon their collaborative practice and their sets of exchanges, all the way through to people who have network grants right. And the full purpose is to actually bring multiple sets of people who might work in various domains and then they might all meet in a particular place because and not be. Well, all of those are kind of really interestingly now called into question aren't they in this moment in terms of much less than thinking about the finance model that might live underneath some of this. And what does that mean from a financial perspective, if everything turns to zoom, and you're in your no longer kind of co locating and I'm in the midst of trying to set up an international meeting where one of the calling cards of the of the meeting concept in prior days was to bring everybody to a secure location to be able to talk openly with each other. And then be able to, you know, revert now, obviously thinking about carbon offsetting is another sets of things and climate change would be part of the question, but now we're even moving to a place like is that a value add to be able to do that. I mean, that will then if it's not a value add and we could do zoom the cost should change in terms of what the collaboration might look like. And this is where I think we start to move into some thorny kind of spaces around what will collaboration look like in the future there are some things we can handle we can deal with like, bringing all of us together by a zoom and various things that just makes sense and I can do that. I mean, we can all agree to come in. But then there are other sets of things about financing behind all of this, the type of value in time that I'm not too sure we've worked out just yet. And in terms of the sustainability model about what that might mean. And, and I wouldn't want to rush to try to find a solution about accessibility and bring everybody together and how great all of this is that doesn't really think about that second part of the conversation with the extra costs that a lot of conferences and other areas have picked up to now bring specialists in to help them manage an international meeting, all via digital technology. Now, should we have already been at this place in the future and the past, probably, but this is an interesting set of dynamics to try to put in front of us, I think around collaboration, because it just just it just changes the nature of the and probably make means we can get to some of the juicy stuff. Kind of underneath that as opposed to just presuming we're all going to fly to some location or we're going to all collocate some and bring people together at the culmination of our project or whatever it might be. And just to add, I think, you know, the presumption that now that we have digital technology virtual solutions new ways of collaborating and to suddenly this has opened up, you know, research and ways that are hugely positive but I would say that they're essential but not sufficient. And I'm going to just mention three reasons why that's the case. One is around workforce impact. So what we're going to learn through our COVID study talking to researchers and so forth that the inability to hire recruit internationally was had a profound impact on many research labs and research teams. And so it's not just about, you know, mobility or collaborating internationally or virtually it really is about that the ability to be able to bring new talent from around the world to be able to foster that talent and new settings and ask questions in a way that you just can't do by only, you know, working with local talent. The second way is really on the impact on early career researchers which I mentioned briefly and their lack of being able to network or collaborate and and that again will have profound impact. So as we all know we've sat on, you know, various different, you know, panels, when researchers are assessed, one of the things that they're evaluated against is how big are their networks how how many collaborators what additional expertise there are they able to mobilize in order to support some of their research ideas. And the third area I would say is knowledge translation, the how important it is to have those face to face interactions interactions between, for example, researchers and policymakers or publics and patients or communities, because those relationship building that trust building is so essential to be able to ask some of the research questions that you need to ask. So I would say that, you know, the lack of mobility lack of in person meetings and interactions are really critically important and I do think that's going to change and have a huge impact going forward in terms of how research is done. Matisse, did you want to add anything on collaboration and how are you approaching it at SNSF in terms of meetings and funding for the meeting? Perhaps one thing to add is that at the SNSF we've been flying in people to sit on panels from the west coast of the US etc and from all over the world. And we want to stop that. And we really want to move for the panel meetings, the evaluation meetings, we've got about 80 or so different panels. We want to move towards a virtual panel meeting scheme. We also would like to learn how to best, you know, manage these virtual panel meetings. We do think that they have advantages that go beyond the carbon footprint in the sense that you really focus on these proposals. You don't have chats during coffee breaks that may not be appropriate about these proposals. You are perhaps, you know, less easily influenced by nonverbal communications etc. So there may be actual advantages to having these panel meetings virtually. But we don't really know. And that's another research topic. We're doing a literature review on this at the moment. On the other hand, we believe that the more strategic meetings, we would like to keep them going in person. And of course in Switzerland is a small country that the public transport is very well developed. And these are main, often exclusively Swiss, or only very few. And there we think face to face is important. We also think that if a panel is new, face to face, one or two face to face meetings may be required in order to create, you know, a base of trust calibration, perhaps to some extent that people understand what the criteria really mean etc. And then we would switch to virtual or online meetings. So my perspective is more from the funders, rather than from the researchers but I think it is an important topic and there's quite a bit of resistance against this. People actually like to travel to places and you know, but we'll see how it goes and we need more evidence on it. Yeah, it does strike as another good topic for more actual empirical work to see what what effects this all has. I mean, having chaired various panels myself during COVID I mean I think the classic evaluation panel can work very well online in some ways as you say it is it can actually help to even out some of the other factors that may that may be less desirable in an evaluation context but it's the more creative stuff that it strikes me it's very hard to recreate. And if one thinks I don't know for others on the panel if one thinks about the, the best conferences you've been to over, you know, the years as it were. Quite often, you know, they're the ones that have a lot of other elements that go beyond what you can recreate in zoom I'm thinking of me something like a classic US Gordon conference or something where you know you go off. You know, white water rafting in the afternoon with academic colleagues or whatever these are actually quite formative experiences in building. You know, both collaborations but also friendships that can sustain you then through the ups and downs of periods like the one we've just gone through. To add to that James I mean, you know, I've been at conferences where part of the purpose of actually having in a particular location is to bring in various archives or to talk to various of the community organizations or to introduce people to, you know, another or a system so you actually got to tour labs and facilities or understand things and it connected people who may be yes those who were regionally based in New each other, but it also brought in people from other areas who didn't really know the capacity and the capabilities around so yes the water white water rafting and the, you know, the drinks of the side various other things or coffees. But I think also that that ability to connect on a wider basis, and to, and to really link people into to other sets of things and I do think, you know, the challenge we're going to have as we move forward is, and I think, you know, we underestimate two things in this moment. I think the first one is restricted travel. So people who actually have a grant, who are actually approved to go someplace and they can't get into that country, or they can't get out of the country. So I have had colleagues that stuck in places, waiting to be able to have the ability to leave because COVID laws and restrictions have changed while they've been in country. But the second one, which is one we haven't mentioned just yet. And that and that is the fact that it is a miracle that my tech has stayed as stable as it has over the course of this call. And I live in a place that supposedly this is a, this is something that I should be able to trust is going to be consistent as opposed to living some other place where there are differences in my electricity usage over the day and we're trying to maximize these differences. So you end up in a situation where the digital divide, as we call it in the UK, or the massive swings and challenges for people around technology are acute. They're so acute that we just, we just know them as normal now, in terms of people trying to work on every device they possibly have if they have one to try to link into a meeting. So that can't be the normal way of going forward. And we're going to do something about that, because that's going to make collaborating kind of hard. If you don't know if you're going to be able to stay connected if you dropped various other things and we keep adding more platforms, thinking that's our solution a mirror board or another set of things, and we're still having the difficulty of people just being able to connect. There's no norm anyway globally, but it but this is my, my other point around about technology technological possibilities moving forward is we've got to really address that. And I think really understand that because it can't be the that we just rely upon it, because it's just not going to university operate consistently for people and then we'll end up with disadvantages and inequalities and I don't think we want that. I would agree with that but can I also offer at least one counter view, which I think has been a positive. And this is probably along the lines of, you know, really trying to diversify. You know, who's actually talking and representing and speaking to a lot of these key issues. I mean, frankly, I mean I'll speak to myself. I have lost track of how many different meetings I participated in the last 18 months, but, and it's exhausting to define the ability to be able to be in spaces that pre COVID I would have never been in. And as a person of color, you know, and being able to bring that representation that voice to the table is has afforded me an opportunity in ways that I couldn't. And so I would agree. And so I figured, but there's the dilemma right because not everyone, many voices who are who are not heard and are not represented, don't have the ability to be able to access technology in this way. So one of the positive things I would love to see going forward post pandemic. And I really, I'm really kind of really. Your suggestion about, you know, some kind of commitment to staying some meetings virtual really creates an opportunity to be able to do that maybe that's something we should all consider, because otherwise the very few of us that exist that are that are accessible that are willing. You know, we can't be in 100 different places at once and I would hate for us not to be able to continue to participate at the levels or more to invite others, just because going back to the old way of being able to communicate. No, absolutely. That's a very good point. Great. I want to move to to some of the questions that have come in from from the wider audience colleagues who are with us and again to encourage people to either raise their hand or to put more comments or questions in the Q&A box. I think McKinney has asked all of you what you consider to be the most important metrics for the health of our research systems and cultures, and for how well they're integrated with political industrial and cultural domain so another big question and I guess how those metrics or the choice of metrics has changed in the aftermath of the pandemic. So any thoughts on that I guess it does take us through to, you know, beyond some against some of these more immediate responses to some of these deeper systemic questions you know will we actually see the ways in which we measure performance output success or otherwise in the system, altering in a more permanent way as a result of all of this. Anyone like to take that. Yeah, I mean, we can have a whole another meeting on that topic, right. I mean, I think it's a brilliant question. It needs to change. Right. I mean, you know, I mentioned briefly Dora, you know some other and I think the tea is also mentioned it I mean the metrics that that are in play right now are so inadequate. They are so flawed and so forth and and you know you would you would have liked to envision that the, the kind of new technology platforms allows to do something more sophisticated which really has to be much more qualitative and mind you so the ability to be able to bring more insight deeper insight meaningful insight and what's happening how who's effective so forth, I think is really key and unfortunately busy heat there's a positively there's a huge need and there's a huge right opportunity for the development of news so called metrics. I'm not really a big fan of that word, but new measures new indicators to be able to tell us whether the research system is thriving. Now having said that, you know, we do have some things available to us. And there's ways that I think that could be helpful, but I think we all have to really appreciate that that metrics don't make decisions that metrics can further inequities etc etc I know I don't have to sing to the choir about this. So I think it's a great question but I think it's a difficult one to answer. You know Karen talked a lot about research culture and behaviors and so forth and that's, you know, that's really becoming more of an interest is not just in the what's in the how it's in the who. And I think that that's going to be really critical in terms of what we'll be able to access in terms of evaluation capabilities on those aspects of research. Thanks Janetta. Mattias, any comment on that. I mean, I think, Jeanette has dealt with the sort of first part of sacks question which I think is really crucial the second one is about getting out of the silos. And you know considering that we have decades left to deal with climate change. And that question is is incredibly, incredibly important. And it is complicated I think covered has perhaps taught us a few lessons in that respect that we can apply to the, the climate change situation. There is this tension between independence of science and having influence on political decision making and trust. So the more independent you are. The less influence you tend to have and the less you are trusted. And to overcome this situation where you have independence in really truly independent input into policymaking that is heard. And that is taken into account and trusted. And to tell you I don't know how to get there. What I what I see happening is that the people who have influence are not independent, or not as independent as they should be. And those who are independent are sometimes also quite diverse in their opinions are chaotic, don't communicate well, and are therefore, you know, have little influence and and are not trusted. And so I don't know how to get there what you what you talk about Zach. I think it's, it's really very important and and and also very difficult I would love to hear from you, how you think we could get there. Thank you. Just to mention, as if to bear out some of the points that the panel I think Karen in particular was making, and Channeta as well just now about the inequalities around access to technology and its effects I have just heard by email from from some of the people who just joined us from South Africa. And he's just now just got back online after a massive power surge in the Western Cape, where he's currently based so he sends his apologies and I guess that's an illustration of at least that broader point that we were discussing so I'm sorry we haven't been able to have a perspective from because I was very keen to, in a sense invite a perspective from outside Europe and North America on on some of these dynamics. Let me just go to another question that's here, conscious that time is running tight and again there is still time if anyone wants to raise any others. We've had an anonymous attendee asking questions about the tension between academics essentially remaining competitive in terms of competing for funding and all the costs associated with that. And some of these broader more positive dynamics we've been discussing in research culture and I guess this speaks a bit. Mattias to your point about whether, in a sense we are going to actually come out of this in a better state in terms of some of those tensions in research culture or whether the macroeconomic context for many countries and for many research systems is actually going to deteriorate. And this is of course an acute issue in all of the systems that you're you play key roles in in terms of how much as funders one can ameliorate so shave off the rough edges of some of these pressures. When, you know, the broader dynamics in terms of the funding of research the funding of higher education system are, of course, partly influenced by research funders but but subject also to much broader influences and forces. In Australia would be one good example of a system where we're seeing a lot of redundancies in the university system right now, as a sort of consequence of COVID or at least the reduction then I mean international students because of COVID. So just so any any thoughts I guess on on, I suppose the broader put it in grander terms the broader sort of political political economy of the research system and how that will all won't be affected by the pandemic. Karen I'm going to pick on you for that. This is big. Okay, I can do this. I'll actually circle back to the first question for for just a minute and, and, and then and then move to and kind of figure out a way to loop this one in because I complete agreement with with Mattias and and Kinesia but I think one of the things that I would probably add to that first conversation we were having is that, you know, I, we're in a really interesting place now as we start to think about research assessment and you know metrics around journals and various other aspects. And then UKRI is moving forward with utilizing the resume for researchers or a narrative CV, which others globally are also doing and really seeing these as ways of transforming the ways people package up and kind of talk about their work and their research in really, powerful ways. But I think there's a real big difference between all the things that we're thinking of doing at that kind of individual level. Promotion assessments these writing of things, and even the peer review of that and the kind of project type of work. And then the ways we're understanding the system, or the ways we're trying to understand institutions. And we're, we've got, we revert to type often when we're talking about measurement there, where we have KPIs and we have different kind of, you know, seven year plan we have lots of stuff that normally just we chuck at it so in some ways similarly to what we, you know, thinking about inclusionizing and rethinking the way we assess and understand and evaluate kind of work. I almost think we need to do that around institutions and that kind of systems knowledge. That doesn't mean that we don't want to just get, you know, we shouldn't be gathering the information, but we really do need to be critical about how we're doing it and what, and what we know about what we are what we're seeing. We don't make massive jumps in claims based on, because we don't normally have a lot of really good information as giving us understanding about organizational structural processes. We have outcomes, but we don't necessarily have a lot about those processes. And that's kind of bringing me back to thinking about the question that's just been posed. Because, you know, it, there's one thing about thinking about all of that political economy and various stuff and things within the system that you were outlying James. But one of the ways that I've been starting to talk about both collaborative collaborative practice, if you will, our collaboration is in the system in general is to think about it as a collective knowledge economy. What would change dramatically if we talked about research and innovation as a collective knowledge economy. What would change about that published in Paris about the, about the, the drivers and the sense of things because we're so dependent on peer review and assessment. We're so dependent on people giving and contributing their time and mentoring and engaging and interacting. We're dependent upon the impact of what that's going to have societally. And that is a collective, right, and that means I should be extremely proud. James, if you get a grant or if you get, you know, an opportunity, I should be proud of that for you, as opposed to thinking that I've lost something, or, you know, snippy because you beat me out to the punch of something your growth helps my growth. And we don't think about this out of departmental kind of institutional level we don't think about this instead we we're all chasing stuff, whether it's in an institution or it's at the funding level, and it just breeds this idea that everybody is in tension with each other. Even though we're highly dependent on each other in terms of what we're doing and I really do think the more we can think about the value in those contributions and really start to think about that collective knowledge economy. We can transform how we start to think about how the system operates and works and yes the assessment and the other sets of things underneath that, but this is where I think culture can go. Thinking about this from a culture perspective means we're not necessarily just thinking about norms and behaviors or different sets of things where we just try to punish this group, or restrict that group or resenction that other group because people aren't acting in a particular way, but it's transforming how we even talk about it, and transforming how we even understand how people engage with it and interact with it. And for me that's going to be the most powerful place to really stop the stop thinking about stuff so really in terms of how we think about our system because right now that really encodes a lot of the behaviors because they're so transactional almost in terms of how people think about almost everything that they do and that's unfortunate. Yeah, just a brief comment to follow and I love the way that Karen just kind of phrased that and it really is about approaching evaluating research from a systems perspective but I think you know linked to the first question of the previous discussion on research systems research culture I wonder if if there is something around being able to codify conditions or characteristics that you would want to see from a system, if all of the players in that system are behaving and acting in a certain way. I really love that this concept Karen of this kind of collective knowledge economy collective impact and so forth. And this is I think is, is, is probably bubbling up in these national research assessment conversations around rather than putting continued pressure on individuals building individuals responsible for achieving impact which we all know doesn't work in that way is to really to transfer the accountability to the institutions to the systems to as that includes culture for how do they promote enable create the conditions for that impact to occur. So I think there's something really rich in and what you just said and I think that is perhaps maybe where the focus should be as you said not at the transactional level, but really the conditions the broader conditions for what a system a healthy system or knowledge economy should operate. And we should perhaps mention that China is on the panel that is overseeing the the the current review of the ref the research excellence framework in the UK so it'll be very interesting to see if any of those kind of shifts manifest themselves in the coming months as the national funding exercise here gets its periodic. I can neither confirm nor deny. I'm sure I'm sure it's too early to say these are all I'm expressing personal views that we just record. But the minute I mean you were quite in a British funding context. Absolutely that is that that's it. If you want us to choose a single arena to sort of tweak the balance between individual and collective then that would would very obviously be be one. Mattias any quick final comments on this or I was going to come for a final quick question. Yeah, just very quickly I mean I mentioned the WT grant institutional challenge scheme which helps institutions to sort of break out of the silo and and create partnerships at the individual level, funding agencies can open doors to researchers to get out there out of their silo. They can, you know, fund placements pairing schemes. They, they can be open to co designed research where, you know, institutions like the one supported by the WT grant challenge are actually involved in designing research together with the academic researchers. And that, you know, I think might help to open doors to the academic research community out there, see things, see other opportunities and not be sort of completely dependent on the academic world and academic, you know, next academic career step. And it would also, you know, respond to to sacks integration point, which I really think is a is a very important one. We do need to get the academic system better integrated with other sectors in in society. Thanks Mattias. So we're drawing to a close let me just end with a final very quick question to each of the panelists sort of 20 seconds each. We threw or at least in Europe, North America with through what we hope were the very worst phases of the present pandemic, but we also are clearly confronted with lots of uncertainties many of which we've touched on in the last 90 minutes. As you think about these rippling effects through research systems and cultures over the the next three, five, 10 years, where you sort of locate yourself on a spectrum of optimism to pessimism in terms of what this will all mean for research so I just feel really for how glass half full or three quarters full or empty your feeling about post COVID research systems and cultures. Gosh, I was hoping Mattias would go first. I mean, I'm happy to take a glass half full perspective. Like I said, I think we know now it's possible. And if anything it's invigorate us broadly as a research community system. However you want to characterize us to do things differently. And I really hope that that much of what that was positive will persist. Great. Thank you, Karen. Well I'm a certified pessimistic optimist. So I strongly like the glass half half empty and half full of probably at the same time. I think I would, you know, completely echo the points raised I mean for me this is this is a really good powerful defining moment with a lot of transformation and a lot of really hard graph from people and some great creative thinking. And I, you know, I feel like my job is to figure out how to help that keep going and to flourish but also make sure that people, you know, are capable to stay whole as much as they can to be able to move forward and we just don't keep asking more of more of people. So that means I'm a bit pessimistic about trying to make the systems change. And I think probably more pragmatic about how we go about doing that. So it can be sustainable in the long haul and not just a big punch, kind of at the moment so a bit of a bit of non answer answer for you. That's great thanks Karen and finally Matias. I think, basically, the global, the challenges that we are facing and have made it very clear that what we've been doing in the past is unsustainable. And we have to move in that direction. If we don't do it in a, you know, in a calm or calm in the sense of we have decades left, we will be forced to do it. So in that sense, I'm an optimist. Great. Good. Well, that's a nice. Thank you. Great. Well, thanks to all of you who have contributed comments and questions. Huge thanks in particular to to our panelists, such as Jones Karen salt, Matias Eger. And apologies again that Ganson pillow was was unable to join due to technical difficulties in the Western Cape. I hope this is at least helped us to define some of the shapes of debates which you know that going to run and run for quite some time as we totter out of the the chaos of the past 20 months. Do stay in touch with the community around the meta science meeting, which I think is in all sorts of ways trying to contribute insights and analysis to these questions. There are a number of great sessions still planned through the rest of this week so do keep logging on and joining in we've had some great discussions here today and earlier on this afternoon. Thanks very much everyone. The video of this will also be up online in the next few days. And we look forward to seeing you at the next one. Thank you.