 So hello everybody and welcome to this session about excellence in the research system. It's great you've been able to join us. Thanks very much for coming along to this session and thanks to our panelists also for agreeing to take part. I'm Stephen Pinfield, I'm Professor of Information Services Management at the University of Sheffield in the UK and I'm an Associate Director of the Research on Research Institute which is one of the co-sponsors of the conference. And I'd like to give you a very warm welcome to this session. We do want this session to be genuinely discursive so although there will be some introductions to the topics from panel members giving us their perspective on the notion of excellence in the research system. We do also want to hear from you. Now you can do that in a number of ways. Please do contribute to the chat right from the get go if possible please. So we've got some things coming in that we can address and we'll be adding those into the discussion. Also please if you have a question that you'd like to be addressed by the panel, please put it into the Q&A and we'll keep an eye on that as well. During the second part of this session and we've got an hour and a half for this session, during the second part we are also hoping to invite members of the audience to switch on their cameras and mics if you're able to and to participate in that by asking questions or making comments. So very much encourage you to do that and we'll see how things go. So thank you very much for joining us we are talking about excellence in the research ecosystem and this is a topic which has become a one of increasing debate and discussion we know that excellence is becoming more and more embedded into the research ecosystem through research evaluation systems through grant application systems and various different other ways in which excellence is now really central to many decision making processes in a research system, not least in publication, for example. And yet at the same time we know that it's controversial, it's contested, a number of people have critiqued the excellence regime, as it were, for creating all sorts of negative consequences. And so what we'd like to do today is to sketch out some responses to this discussion on excellence, to think about some of the ways in which the excellence agenda impacts on the research system. And also to go on to discuss possible alternatives. Now this is a real challenge because although the excellence notion and the excellence regime have been critiqued. So someone has emerged in terms of definite proposals about how it can be replaced and so what we want to do is not just deconstruct but also think about constructing as well what might go in its place and so I'll be interested to hear from panel members about that. So let me welcome our panel. First of all, Lizette young who's a researcher at CWTS Leiden University and is currently on a project as part of the research research Institute. I'm Laura Revelli, who's coordinator of a Latin American forum for research assessment follec, the Latin American Council of Social Sciences. We're still waiting for Lara to join us. So I may have to tweak the running order of speakers but we'll see how we go. Welcome to Cameron Naylor, who's professor of research communications at Curtin University. And finally, Zena Sharman, who's director of strategy and the Michael Smith Foundation for health research. All of you are very welcome. Thank you very much for joining us. We're looking forward to the discussion. A note about how we expect things to go. So after this brief introduction, I'm going to hand over to each of the panel members to talk for about five to 10 minutes to introduce their idea, their ideas on on excellence. And after each panel members contribution there will be some time for Q&A on specific issues that they have raised. And after we've heard from all of them, I'm hoping to engage in a panel-wide discussion on the issues that have come up and that's where we're really hoping to engage with your questions. And also, as I say, there will be opportunity for you to contribute as well. So please do bear that in mind. Keep your comments coming in on the chat and do ask questions using the Q&A function as well. So that's the agenda for today. And I'm very much looking forward to hearing both from our panel members and from you, the audience as well. So with that as an introduction, let me hand straight over to Lisette, who's going to be our first speaker. I'll stop sharing. Lisette, you've got some slides you want to share, haven't you? I do, I will share my screen. Okay, brilliant. Over to you. Thanks very much. At the beginning, it was working just now. It seems to be a bit stuck. Well, this also works. Okay. Thanks, Steven. Thanks for the introduction already. Today I'll be talking to you about the Transforming Excellence Project, which is part of the Research and Research Institute. The other members of the Excellence Project team are Steven Spinfield, who just introduced and this little cat hair. Yeah, Steven moderating our current session and also Thomas Franser, who is not here, but whose input has also been very valuable to the project. So the background to the Excellence Project is the observation that the notion of excellence has become an increasingly important part of the research ecosystem over the last 20 years or so. And the term excellence has become widespread in science policy research funding and evaluation activities. Excellence initiatives are generally associated with competitive programs and concentration of funding on what are supposedly the best of the best. But what excellence means often, yeah, remains rather ambiguous. So to get a better grasp of what notions of excellence are doing in the research ecosystem, in the Excellence Project, we set out to explore different uses of notions of excellence as these appear in research funding organizations. And we specifically look into the concerns and tensions that rise around these different uses and the strategies to mitigate them. So as part of the study we conducted a literary review that is now available as a working paper through the Rory website and I'll post a link to that one in the chat later. But the core of the project is actually an empirical qualitative study in which eight of the Rory partner organizations kindly agreed to participate. So together with our key contacts at the participating organizations, we developed a case study protocol to guide the data collection process at each site. And we asked participants to submit what we call instances of excellence that they identified within their organizations, and this data collection exercise resulted in a wide range of resources. Among the materials are publicly available documents such as strategic plans, mission statements, program guidelines, but also we got some internal documentation that reflects work in progress, for example, on the development of new conceptual framework strategies and criteria. And in addition to this document analysis we also conducted two interviews for case study site and we did an online consultation with representatives from the partner organizations to collectively discuss or preliminary findings. Now, the next slide summarizes some of those preliminary findings. And one of the things I want to emphasize is that despite being happily critiqued notions of excellence also come with affordances. They make certain things possible. So notions of excellence give for example shape to a range of activities around funding decisions listed in the blue square, like ranking, comparing, guiding, framing, valuing and selecting. And in the submitted materials we also identified different ways in which the term excellence is actually used. And we roughly categorize these different uses that we encountered as being rather descriptive, aspirational or objectivist. But we focus our analysis mostly around problems or matters of concern associated with excellence. And this list that you see in the slide is definitely not exhaustive and I don't have time to elaborate on all the points mentioned on this slide. But in a bit I'm going to zoom in on the matter of equity, diversity and inclusion, also to highlight one of our observations, namely that the notion of excellence can be considered a particularly sticky concept. So to speak. And this idea of sticky technologies has kind of been conceptualized in the field of science and technology studies as sitting between what is called fluid technologies and immutable profiles, and where fluid technologies are open and flexible, they're made to adapt to changing conditions and able to change shape and maintaining their function. And the famous example of that was the water pump described by the lab and rule, of which the parks could be easily replaced. And the pump could still work and maintain its function immutable mobiles as discussed by Latour on the other hand, are made to hold their shape while traveling think for example about the list with test results going through the hands of different doctors in a hospital. And then sticky technologies sits on what in between these two. So thinking through stickiness kind of helps as the researchers to focus attention on what makes the technology of their comments or concept flexible, but persist, and also helps to address the frictions generated by the limited adaptability of sticky things. So in the context of research project thinking through stickiness helps us to shed light on the difficulties of transforming the concept of excellence. We suggest that the baggage that sticks to excellence are in particular ideals around competition and meritocracy. So sticking is of excellence not only means that there are things that stick to it. Excellence is also made to stick to other things as well. And in that sense, I was kind of picturing excellence a bit like the ball that plays a role in the catamari as a ball to which things stick and as it becomes bigger, it comes to absorb more and larger things. So in that sense, excellence sticks for example to the mission statement of an organization as a guiding principle for all aspects of the work and excellence sticks like a self almost like a self adhesive label to arrange of initiatives in the context of research funding. So if you have a program to assure standards of excellence in very review or national research evaluation exercises like excellence in research for Australia or the UK research excellence framework and stickiness also kind of hints at the attractiveness of excellence, like who doesn't want excellence. The attractiveness is also sometimes strategically used by funding organizations, for example, when someone was explaining to me selling a new funding program to the government. But importantly, stickiness also comes with friction, and it has been argued that the concept of excellence has been at odds with academic values of creativity, openness and integrity. And today the focus on excellence has been become associated with the range of problems in the research ecosystem, such as homogeneity, Matthew effects and hyper competition, as Stephen also already mentioned. So research and funding organizations are very much aware of these issues and have been trying to adapt their understanding of excellence in attempts to mitigate such problems. The things that is currently high on the agenda of many funding organizations are matters of equity, diversity and inclusion. Yeah, so excellence has actually rather been associated with exclusionary practices that favor those who fit a very specific image of the ideal scientist and work on forming dominant epistemic practices. The above mentioned issues of homogeneity, concentration of resources and hyper competition are also increasingly addressed in terms of an EDI discourse, emphasizing the lack of opportunities for those researchers and most of working that fall outside of the norm. And as one interview we responded to a question about limitations to the notion of excellence was like it's not exactly Matthew affecting, although that's the outcome of it, but it's you know the people who look like me issue. And, and more recently attempts at the need to bring excellence and diversity closer together. So the notion of inclusive excellence has gained track in North American research and higher education institutions. And from the university sector, it also made its way to research funding organizations, for example, in the new strategic plan of the Michael Smith Foundation for health research. It reads diversity in research is important to cultivate talent and promoting inclusive excellence which in turn drives discovery. So the notion of inclusive excellence to summarize can aims to counter the idea that excellence and diversity do not go well together. But interestingly, I find that by doing so, diversity seems to become rather subsumed by excellence as diversity here is made into a pre conditioned for performances for performance. So, promoting EDI is another goal in itself, like, for example, just to improve the well being of researchers and create more political research cultures. Instead, matters of EDI become articulated in a performance discourse where diversity comes with the promise of increased productivity and scoring higher in the rankings. So in the words of Sarah Ahmed, diversity kind of becomes a technology of excellence. So the concept of inclusive excellence does strategic work to open up the discussion around matters of EDI and research cultures. It does not so much seem to transform excellence. And excellence somehow manages to hold its shape and rather provides the conditions of possibility for diversity as the language of diversity is exercised as the late language of merits to quote Ahmed again. So the politics of competition and meritocracy that sticks to that stick to excellence are carried over and give shape to how matters of EDI are getting replaced in strategies of research funding organizations. And again, this is not to say that EDI initiatives are not important for that they do not need to make a difference because I think they do. But it is to say that if we intend to transform the research ecosystem, we need to start more thoroughly questioning those sticky things that hold excellence together. So, thank you. That was my contribution. So far I'll post a link to the working paper in the chat in a bit and I will stop sharing my screen and then over to our next presenter. Thank you very much, Lizette, and always good to have feline participants as well. So, yeah, that's, that's, that's great. Thank you. We have got before I hand over to our next speaker and I think that's going to be you Cameron if that's okay. There is a question in the, in the Q&A about how measures can be taken not to tokenize EDI as a technology of excellence. What's your initial response to that I'm sure we'll come back to issues of this sort but it'll be interesting to hear from you, your initial thoughts on that. Yeah, thanks. That's a, that is a great point. Yeah, do I have a direct answer about that? I don't know. It's, I think as I tried to end my presentation with this it's really, I think about questioning those things that lie behind it right so I think it's starting to start with the things that hold excellence together such as competition and bureaucracy. And I think that is kind of key to, yeah, actually open up the research ecosystem and make it a bit of a more welcoming place for everyone. So I would, I would say, let's start there. Instead of holding, holding on too much to do excellence itself. But I'm sure we can come back to this in. Yes, I'm sure. I assume that our other presenters also have a lot to say about about this. Yeah, sure. And I'm sure we'll come back to it as an issue. So let's carry on with the presentation shall we so every one of the panel members can make a contribution and Cameron, I'd like to invite you to to go next. Thanks very much. Thanks. Thanks for having me. I thought it would be interesting to pick up on two aspects of the report but I'm going to note something that's sort of frame frame what I'm going to say first isn't a criticism of the organizers. I've got to observe it's five o'clock in the morning here. I'm in Western Australia, a couple of hours south of Perth. And one of the things I've noticed since I've come back to Perth is that despite the fact that pandemic has led all of us to be more sensitive to the fact that people are in different time zones. There's still the people particularly in this time zone, who seem to whether they're getting up at five in the morning, or being on meetings 10 o'clock at night and the interesting thing about that is not that I object violently to talking to people at odd times of the day that gets retiring at times. But I'm actually sitting in the world's most populous time zone. The quarter of the world's population live in GMT plus eight. And yet, our meetings, our thoughts, the narrative we have around where people who do research and work in our community are is still built around the presumption that the center of gravity is somewhere in the mid Atlantic. So build on that idea of narratives and stories that we tell ourselves. Another thing that I want to do is acknowledge some speaking to you from Nungar lands in Western Australia unceded traditional lands of the Nungar people and recognize their elders past, present and future. And they're continuing connection with the land again. It's a story in Australia and some other places now we tell ourselves about how we're recognizing those contributions and expropriation of resources my university and the wealth that it relies on is largely built on the theft in many ways of resources that we dig up out of the ground and sell overseas. And at the same time that's a narrative in a story we tell ourselves about equity diversity inclusion that we're improving our history in this space our spaces in space. Though it's a very rhetorical storytelling kind of thing. I'm going to focus a little on this idea of two things one I'm particularly interested in the institutional ideas of excellence. The stories we tell about excellence and other things in the academy and the work we're doing at the curtain open knowledge initiative in fact if I shift over to my right you can possibly see our tagline, and if I haven't reversed it. But we're really focused on change through change in narratives. So this idea of changing universities through changing the stories they tell about themselves. And we think is really powerful because what we've found in our work on change is that the people in power at the top of organizations are not antagonistic to many of these ideas which I think a lot of us advocates for change have assumed often. But they simply don't receive information in the form they need to be able to make different kinds of decisions because that information is simply not available. Again, for instance in my organization and any number of other organizations we've looked at information about equity diversity inclusion simply isn't there. People can't tell about whether we're retaining people from disadvantaged backgrounds because we don't record a lot of that information and certainly it's not recorded by by the institutions at judges and institutions at funders. The thing that is recorded. The thing that does seem to matter to those institutions that judges and funders. These conceptions of excellence. I love the idea of the sticky concept that wraps things up and gathers things to it along its way. And to use that as a way of asking the question well how might we change that how might we either make it less sticky. Or replace it and the technological conceptions an interesting one right so as soon as you start thinking in terms of fluid technologies or modular technologies where bits of it can be swapped out and put in place. We have to address the question of whether we actually have to do wholesale reform and rip the whole system down and start again. I mean I think about excellence and I guess I'm on the record has been pretty critical overall of it as a concept over the years. It seems to me that in many ways the idea of needing to replace it is a little bit is feels like a bit of a misdirection or a bit of a the wrong kind of approach. I think there are two ends to this one is that it's a conception and a set of stories a set of sticky concepts. That is so corrupt and corrupting that if we do not get rid of it. So by going to actually destroy the Academy and the things that it values we can talk more about that but but as an example I had in in this piece. Oops, which I just sent to just a panelist sorry excuse me and send that to everyone. This book where I really made the argument that conceptions of research excellence particularly in low and middle income countries was really neocolonial that we were actually recolonizing the capacities interests and directions of the research efforts in these countries and that's you know that was damaging four to five years ago when I wrote this piece. The context of the issues around vaccine availability patents issues sitting around that has become a real issue. The piece that I mentioned in that in that in that chapter is looking at the relationship between the amount of research being done. This was just a simple topic search in Web of Science at the time. And then I looked at the number of articles being produced out of South Africa compared to the mortality rates for particular health and other issues in South Africa. And what I observed was that there was a good correlation between the cause of death and the amount of research being done on as counted by numbers of articles, but only for things that also concerned Western audiences and northern audiences. So for things like HIV or things around blood pressure and heart disease, there was a good correlation. But for things around respiratory tract infections and in particular car accidents, the amount of research didn't follow the kind of issue with the burden that was being faced. So I think there's an argument that you're replacing it. No, we need to completely destroy this start again, build from the ground up. And then at the other end, and this notion of excellence as an aspirational thing, I think is an interesting one. It's almost seems kind of obvious, right? It's just something we need to grab a hold of ourselves. We just need to use the tools we have at our disposal to refine what it is we're looking for at an institutional level. And so I'm thinking here of things like particularly the scope framework for the iron on research evaluation working group, but also the strategic research assessment process has been developed in the Netherlands where the first step is always defining what you care about. And if we're going to take the sort of pragmatic end of the approach and try to reform what we're doing and grabbing control of that process and actually really actively defining what it is we care about upfront and why and what makes for excellent research in our views seems like to me the way forward. And so I think that's a point I'd like to discuss both the balance between the individual, as we often talk about excellence and its impacts on issues like EDI and other things as a concern of the individual or for the individual. So I would like to think about the distinction between that and the institutional and cultural aspect. And we talk a bunch about that in this book, which of course is open access available from MIT Press, where we talk about transforming universities. And then think about the pragmatics of this. How do we embed more of the choices around what really matters and have conversations about really matters because I still go back to the thing I think I've been saying for about a decade now. Every time I see the word excellence, I see a person or an organization that's avoiding having a conversation about values. And we deploy these words, words like excellence, like peer review, the very things that we study as metascientists, I guess, across the academy, often in ways to avoid conversations about the detail of what we're trying to do. So in a sense, it's really just a back to basics. What do we care about and why and how do we get those things embedded more in the culture and narratives of what we're doing. And I'd love to talk more about that in the conversation. Thank you very much indeed for that very thoughtful contribution, much appreciated. We do have a question that's been put and I wonder if you could give us an initial view about that but once again this is an issue I suspect we will come back to. So, the question is, whose values should lead the discussion of a change governments, the research ecosystem, civil society. What's your comment on that. Yeah, so, so this is, this is the core challenge in most of these discussions, right, you've got more people who want to be at the table than are at the table in these conversations. And there are more people who should be at the table who don't even know the table exists. And so, one of the things we say in the book actually, it's probably the hardest bit for people to write their heads around is. So we take the view that knowledge making is about bringing communities together. And core to that process is coordinating the interactions between communities and this coordination piece of bringing communities together and actively going out and seeking the communities that ought to be engaged. Is a core function of a knowledge institution in the 21st century. That includes having the capacity and the expertise, not just to recognize that a particular group may be impacted or involved or should be consulted or in fact should be directly driving a particular piece of research. But it also involves the thing which we're still very bad at, which is genuinely building trust with those communities so they feel like their engagement is a value to them. So the answer for me is everybody needs to be involved, but we still don't have the mechanisms and systems in place except in some very narrow slices of the knowledge making ecosystem where we're good at engaging genuinely engaging not just consulting and then ignoring but genuinely engaging with the interested parties. Lots of great work being done over the last 20, 30 years with patient groups, increasing efforts being made in particular sort of disadvantaged groups, including in Australia, engagement with Indigenous groups, but a huge amount of trust that we need to justify and over time earn. And that's a that's a decades long project to build the systems that's capable of engaging all of the communities that should be involved in these conversations. Thank you very much for that. Let me now turn to you now, Zeena, if that's okay and invite you to make your contribution and just to remind it to everybody in the audience. Thanks very much for your questions coming in. After Zeena's spoken it doesn't look like our fourth panel member is going to arrive, although I'm still hopeful, but I do hope she's okay. But nevertheless, we've got quite a few people in the room, and I recognize quite a few names and quite a lot of expertise associated with them on this topic so it'd be really great to hear from you as well. So, but before I try and solicit any further questions and control you into contributing. Let's now turn to Zeena Zeena over to you. Thanks very much. Thank you so much, Stephen. And I'm really glad to be here, and I'm coming to you from unseated ancestral pouch and tribes territories, which is important to acknowledge the land we're on as Cameron did as well, especially when we are connecting virtually, and I'm thinking about this in particular, because my family and I recently moved to these lands so we have shown up as white settlers on an as uninvited visitors uninvited guests on these particular territories. So moving into that knowing and the accountabilities that come with it. And when I think about the theme of this panel, and this notion of after excellence, a few questions came to mind for me. So my first question is, Well, what is it we're after exactly. The second question is, Why are we chasing after it. And the final question for me is, Well, what comes after excellence. It's important to reference the fact that I've worked in the health research funding sector at national and provincial levels in Canada. Since 2008, and I've been an overachieving nerd since well before then, and I'm trained as a health researcher. I was someone who came up through academia so spent many years thinking I was going to become a professor before my career pivoted into funding. I'm conscious of how excellence has gotten under my skin by virtue of being trained, working in and being socialized into professional identities in context deeply invested in an imagined and fundamentally inequitable meritocracy. Excellence as a quality and a goal is pervasive and research funding and academia, as we've heard about from the other speakers and as you can see in the really excellent literature review that's been cited here. Excellence feels like it's everywhere in these contexts, though it often goes and defined or is operationalized in such a way that perpetuates systemic inequities and entrenched research and academic cultures that hinder the flourishing of people and ideas. As a person living on stolen indigenous land, I am acutely aware of the ways in which my own perceived and socially constructed academic and professional excellence in quotation marks is entangled with systemic inequities, white supremacy and colonization. Excellence as a perceived quality is not equity equitably or fairly distributed. And I can certainly reflect on the many ways just at a personal level, I've profited from this. I understand from experience and from witnessing the inner workings of academia and research funding agencies, how being deemed excellent often leads to an accumulation of privilege on top of privilege. And I see part of my work and my own accountabilities and all of this as challenging and uprooting the ways in which excellence attempts to work on me, as well as challenging and uprooting the harmful ways excellence works in the systems I'm part of shaping as someone who works in the research funding sector. And I know we've talked about excellence as sticky. I know it's also been described as a boundary object. I sometimes find it helpful also to think of excellence as a container when considering this question of what we're after when it comes to excellence. A container can hold a lot of things. Someone made it out of something. So I feel curious and often skeptical about what's inside this container, we call excellence, who made this container, where did it come from, and what work is this container intended to do for us consciously or otherwise. What are the unintended consequences of using this. And that brings me to my next question. Why were chasing after excellence. Just, why are we after it anyway. Who does excellence benefit. What does this notion of excellence enable or constrain. This is another place where I found the research teams literature review helpful in articulating a set of historical and contextual conditions that shaped the evolution of what has become what I often perceive to be taken for granted notion of excellence. You might shift according to context, but I think it often plays out in similar ways. Here I'm thinking about things like the rise of neoliberalism related attempts and certainly successes in the corporatization of academia and universities. This notion of running a more efficient research enterprise and attendant tendencies towards quantification, as well as again something that came up in the literature review that was both surprising and ultimately extremely obvious to me, which was the clear connections to eugenics in some of these early notions of excellence so we can, we really I think need to understand that the tangled and toxic roots of these concepts. So with this, I finally come to this question of what comes after excellence, you know, thinking about this context thinking about this history out of curiosity, I recently looked up the etymology of the word after. I learned the word after wit, which is a word from the 1500s that has fallen out of use which is why I had never heard of it before, but it means wisdom that comes too late. So in the spirit of after excellence and after wit, I am cautious of attempts to replace excellence with a concept reported to be more equitable or inclusive. As they say in the healthcare quality field, the system isn't broken. It's working as designed. And although I certainly have strategically marshaled concepts of inclusive excellence. I think it's also really imperative that we not only look at this container we call excellence. We also need to interrogate the conditions under which it was created and is recreated. And what this might mean for the success of efforts to simply create a replacement container without fundamentally altering the means production. That's my metaphor is excellence isn't the disease. It's a symptom of larger problems. So when contemplating what might come after excellence, we must consider the preconditions that led to its creation, and what we see existing in the systems that exist today. I really appreciate what you spoke to around the need to think about creating different stories. And then I think this question of destroying excellence and building from the ground up is also an important site of inquiry. When we think about again, the institutions in which we are existing and of which we are apart. So, these are some of my questions, and I'm really interested to hear your questions and reflections. Thank you. Thank you so much for your great for that contribution as well. So we've heard from all of our panelists and thank you very much for your contributions. And there are a number of issues that I'd like us to pick up. And also, I'd very much welcome contributions from our audience as well. You are able if you put your hand up to contribute and we can switch your mic and camera on or you can once we once we allow that to happen. So we're very much welcome. Well, be great to hear from you. While people are doing that please add stuff to the chat as well and also to the Q&A and we'll keep an eye on one of those. While people are thinking about that. I'd like to pick up on this idea of universality or situatedness of excellence. And the idea of excellence is often an idea of its universality. The idea of world class which has been mentioned in the in the chat, for example, but even notions like rigor and so on often assume a sort of normative understanding of what constitutes rigor. I'm just interested in whether any of you panel members first of all have ideas about the extent to which excellence can be considered a universal thing, even if we have to redesign it so Cameron you made some comments for example about the subject matter of scientific education is if you like is that the nature of its situatedness or is there something more about the very kind of knowledge types that are being created, which makes it more situated so what's what are your perspectives on that camera can I ask you to address that first as you mentioned that but very interesting from other people as well. I'll take a pop at it at two levels. One is, and again there's more on this in the book so I'm not going to regurgitate the book particularly five o'clock in the morning social theories of knowledge five o'clock in the morning and not my strong point. One of the things the argument we make is built on this idea that knowledge happens when communities groups come together. So the situatedness is everything. It's the experience of the group and the people and the processes and the institutional systems including this concept of, for instance, the scientific method again scare quotes required. That we work within and that our institutional forms that have served us well, to some extent in the past and caused other problems as well. But I'm certainly will sit on the side of saying that the Western knowledge production system has some things that are good about it and has done done a good job of creating certain kinds of knowledge and that we can enhance some of the good aspects of it. So bringing in more consensus more situatedness more contextualization and more of the broadening of experiences. So that's, so I think that that's one of those unhelpful answers where you say well the context of context is everything. But it's also not a very helpful thing to say in the abstract. Because then what do you do with it. And so I have a very concrete example of how I've dealt with this right at the moment I have been writing a document that is intended to become a set of principles around research evaluation that will be used for research evaluation. And in that document, I started with two principles, which adopted from other work on these things that many of the people in the room have done. And those were suitability. So anything used to do evaluation needs to be a suitable approach for the thing you're evaluating. But in the context we're working, we need comparability. We need to be able to use this information to make choices about resource allocation where, you know, there are the dollars are as good as some game. Or where is the attention going who gets the support. And I think what was what I thought was useful in setting up this document was I set those things up deliberately as attention. I'm not saying you should have this and that, but you need both and they are intention. So your choice to use citations as a measure of something excludes is useful because you get numbers and numbers can be compared across different places if you're careful about how you do it. You're also excluding whole classes of research and knowledge making activity that generate things that do not get cited and do not. So, yeah, the thinking that as a tension as a tension that we're working with, and as something that could be productive. This is the point by negotiating how we might choose to measure ourselves. And, and again, to see this point who are we doing this for and how are their interests served that we start to address those questions. So I guess that's that's one version of sort of very abstract and theoretical and notionally inclusive but things that are really inclusive tend to be exclusionary in practice because no one can actually grapple with them down to the concrete where we're trying to figure out how to actually make this work on the ground in practice. And we'll come to that because we've got an interesting question on if you like moving to real world applications which I'd like us to address in just a moment. We were hoping to hear from Laura Revelli from the Latin American Council of Social Sciences who would. I've no doubt given us a another perspective on this whole question because a great deal of work has been done I know in the number of Latin American context in this whole in this whole field. So, if there are any people in the audience are able to give different perspectives from different sort of global contexts, I'd be really interested to hear from you please do raise your hand if you're able to make a contribution and and take part in the discussion. So you know, Lizette either of you want to address this question of the extent to which excellence can be a universal. I'm suddenly having flashbacks to first year philosophy class which was in 1997 so that's not particularly helpful to me in this moment. I'm not going to tell you about my essay on Plato's cave, but that's not not at all relevant here I know enough to do about 1997 to tell you that. I mean, you know the thing I can think about in terms of the universality versus situated in this piece is, I mean, certainly I think that that I don't think there's any universal notion of excellence I mean that that's just not a real thing right excellence is made up it's a thing we make up and perpetuate in different kinds of contexts. And I mean I think that the place where the university out universality versus situated kind of gets interesting and challenging in terms of what does it mean in practice you know and Cameron I think you gesture to this and what you were just talking about in terms of the work you're doing is thinking about the work that we do in a research funding agency context so I've worked in the health research funding sector specifically currently at the Michael Smith Foundation which is a provincial health research funding agency and British Columbia Canada. You know we are actively thinking about what are the ways in which we might disrupt some of our own practices and assumptions in service of a more equitable but I would say also impactful if we want to think about the kinds of impacts we're actually seeking to have forms of health research and and doing things like shifting for example the way in which we design assessment criteria for applications shifting the design of some of our funding programs in terms of who actually has the capacity to be able to apply for them. Looking at how we actually interact with our chairs and scientific officers and our peer review panels more broadly to enable them to hopefully enact this notion of of excellence or whatever proxy we're using in ways that may be more equitable more aligned with specific goals we have for example around funding people from smaller universities universities and more rural and northern areas you know folks who are black indigenous people of color or other groups that are systemically under represented by virtue of the oppressive context in which they're And the thing I would say there is, we are engaging in these kind of ongoing practical experiments to varying degrees of success, something that I think is a constraint that I am often challenged by is that we are up against the vastness of academic and research culture, and that that operates in incredibly potent and invisible ways, within the context of the peer review system, so I think that that that is an interesting again another tension or challenge that we need to grapple with here is that, you know, as a funder it's not just us in the room we rely on the peer review system but that in and of itself is so entangled with research culture, which of course is also so entangled with with excellence and all the stuff attached to that so I wanted to offer those reflections in part because I did see a question in the Q&A as well around kind of how do we get academics who applied this to themselves in a more kind of practical way and, and just to say that I am I am inside that struggle, unfortunately with excellent colleagues at the Michael Smith Foundation so we don't have the answers but we have experiments, and we're always keen to learn. Thank you. Is that have you got any comments on that? Well, I think to the initial question I think Sina made a very important point that indeed like excellence cannot be universal it's made up somewhere and I think precisely all these uses of excellence where it's presented as a universal concept actually precisely obscures that you know that excellence has a history that it comes from somewhere that is from a very specific place and that it carries some very specific ideas about what research should look like and what researchers would like. So that's one thing and another thing I can add to this is that in the research project that we're doing, looking at all these different ways in which those vaccines are used in the context of research funding organizations that there's already such a wide range of uses Yeah, like looking very closely at that it seems to not make much sense to think of excellence as something universal because even within one organization, it can do so many different things and it comes to mean so many different things. So, yeah, that's what I wanted to add. Yeah, I mean that that that's really important, I think and, and thank you very much for the questions that are coming in please do keep them coming in and on the chat as well. Gabrielle has made a really interesting comment in the chat for example about more and more countries joining the excellence regime as already excellence agenda. At the same time as it being questioned in other countries, which I think is an irony anybody who can talk to that we're really interested to hear hear from you to get your impressions on on that. Well, while waiting, I'd be I'd be very interested to hear more about how, if you like the the abstract or apparently abstract criticisms of excellence can be turned into on the ground action. And I'm particularly in relation to the question that's come up a couple of times. Equity diversity and inclusion. I mean, you commented on very briefly on the idea of inclusive excellence, which has been around for a little while as an attempt to moderate some of the negative consequences of as they're seeing of the excellence agenda. I don't think that has any future as an agenda or do you think it's trying to simply only moderate and and that's not enough. What do you feel about the, the issue of EDI and excellence. I think that's I guess I would give it a yes and and I think in saying that which I will expand on. I know that in the literature review. And again I really am I'm hyping the literature view because I think it is actually a really fascinating and useful document for folks who haven't had a chance to read it, but it does give I think a bit more of a history of where that concept originated. I would say I'm seeing it and also my three year old is thumping around outside the door so if you can hear that. Yeah, but I have a small joyful child who is being attended to by other adults right now. Just to give you the live zoom reality of my excellent life. And so just to say I mean I would say the inclusive excellence concept I think has a longer history that I am less well versed in. As I understand comes out of I think the US. I've seen it used more widely in Canada in the last several years, during a time when we have seen, I think for a number of reasons including human rights challenges by academics who were systemically underrepresented in major government funding schemes. We have seen more attention to equity diversity and inclusion by federal funding agencies and by universities. Certainly that has been a big shift in the many years I've been working in this sector in Canada. And I think it's been a success right in the sense that I think it is leading to some material changes in the allocation of resources, for example, broader discourses of equity diversity and inclusion, but as I've seen people way smarter and more skilled than me, you know who generally again are black indigenous and folks of color, who have been deeply embedded in the work around equity diversity and inclusion and critiquing equity diversity and inclusion for generations. And that's really about, you know, how do we even question the notion of where equity diversity and inclusion came from, versus for example a more liberation oriented approach you know approaches that are more transformative and looking at actually questioning and shifting the institutions were inside. So, I mean I try to bear those complexities in mind when also thinking about what are the ways in which we can move strategically through the work of actually affecting systemic change. I don't have easy answers to in terms of doing my best to work in a way that feels aligned with my values and personal politics, trying to work strategically in large and complex bureaucracies that often don't want to move. And so when I find those windows of opportunity or those leverage points in the system, I am interested in utilizing those, like for example notions of inclusive excellence but I try to do it with a lot of questions around it and never assuming my own rightness and all of this because I think again I also try to be attuned to the specific violence and harm I can do as a white cisgender woman. In this work of equity diversity and inclusion or trying to bring forth a more inclusive notion of excellence I mean and I think it's important to situate ourselves in this work as well as being institutionally and conceptually and theoretically situated in all of it. So, I mean again I'm interested in these these strategic questions right how do we do the work of change in ways that are actually sustainable and ultimately connected to to what I see as a larger political as well as intellectual and academic project. Lisette I want to come to you in just a moment to ask you about how your research has given insight into how different funding organizations are looking at this question of EDI. But just to give you notice Cameron, I want to move on to you after that, maybe to address this question but also to address the question that's been put in the Q&A about the relationship between, or that gives rise to the issue of the relationship between universities as institutions and their other communities, and the extent to which universities have tried before, historically that is, to if you like retain autonomy as a as seen as a strength, whereas you seem to be seeing it as a weakness and I'd be really interested to get your view on that in relation to excellence but let's go. That's just to give you notice I'm going to ask you about that but Lisette, how was your research, given us insight into this question of EDI how are different funding organizations in the case as you've been looking at dealing with it. Well, yeah, one note I want to make about this and that is that, of course, and it was already slightly mentioned that this idea of EDI and term equity diversity and inclusion or diversity, equity inclusion that's different different acronyms, it's also not used everywhere. So again, it's also this specific way use very much and we need like a North American context, you can find it in the UK as well but if you look at already other countries in Europe that are where organizations are, it doesn't appear everywhere. But overall, I think many funders are currently concerned with being more inclusive in their own practices and are quite reflexive about this, starting conversations. But I think it's what I find quite interesting is that there's quite some differences in the way in which funders position themselves in the research landscape regarding excellence, transforming excellence, transforming the research ecosystem and also around the matters of EDI, like some funders position themselves more like, well, we facilitate a process of decision making, we facilitate a process of peer review and how decisions are made. It's very much left to like the scientists or the peer reviewers and we're not going to intervene so much and on the front is taken more firm position and are actually trying to actively intervene by for example training reviewers provide bias training or by working all sorts of other ways. So I think that that's quite interesting to see so sort of like how different funders sort of position themselves in the landscape and think differently about what is the responsibility of a funder in this context. So, yeah, that's that's that's what I'm observing right now. And yeah, it's, it's, it's, yeah. So there's a range of positions that are being taken and that's not necessarily that it's only the really independent funders who are being more interventionist. Sometimes also government funders, so it doesn't seem to have a very strong pattern what I do seem to see is that the funders that have a very wide portfolio so that discover many different disciplines. Yeah, it seems there to be a little bit more difficult to make big interventions, kind of like the more specific the work and the focus of a funder is and the more sort of the smaller kind of the field that it seems to more concrete that interventions are possible, which kind of makes sense because of course if you need as was mentioned already before a funder doesn't operate in isolation right so it's part of this larger system and also scientific disciplines also working different ways. So I think there's a there's interesting complexity going on there. And somebody's commented in the Q&A about the possibility of linking funding specifically to EDI as well which I think is happening to a certain degree but not, it's not prevalent amongst many funders that I'm aware of. Cameron, over to you. First of all, do you have any comments about the EDI issue but also I'm very interested in this issue of the extent to which institutions should regard autonomy of decision making within the academy as something to be protected or whether it's something actually that is a weakness. Can't hear you. Let me unmute myself first. Let me start with the abstraction and I'll tell you a story. And picking up particularly on what Zeta was saying I guess my work has always been the way I have always worked has been through systems and institutions that exist and through trying to make change because I've always believed that if you want to make sustained change, ripping everything down tends to just recreate the same problems. When I was in high school we had a had a poem from a poet who probably no one outside of Australia has ever heard of called Bruce Doar, which was actually about military coups in South America but the poem the poem was called only the Beards are And that stuck with me ever since the revolutions tend to lead to the same power structures the same people ending up in power. And at the same time I think to again to Zeta's point, the institutions I'm talking about they are explicitly patriarchal colonial institutions. That was their purpose. That was what they were designed to do. This is the system working as designed. Again, from where I'm sitting, the idea that our universities in Australia had graduate postgraduate teaching is a relatively late thing. And we have still the Rhodes Scholarship still exists as a thing where the best and brightest go back to Oxford because that's where you get proper education and training. So changes is dangerous if you're trying to do it within those institutions. And so actually in the chapter in the African minds book, I did explore this issue of autonomy, autonomy versus responsibility versus engagement with society and government. And I made the argument that this was a real tension that needed to be the autonomy autonomy and academic freedom that's not a word I would normally choose to use. I think are an important part of knowledge making in a small liberal society. The freedom to explore the freedom to engage the freedom to criticise and think deep without things as important. And the autonomy of knowledge making institutions from government interference is an important part of that and something that is worth protecting and guarding where we have it. At the same time, gaining that autonomy is a question of building stakeholder trust and having governments engaged in believing that that autonomy is something that's a value and something they want to protect. And again, I well, I could rant for a long time about the government in this country but in the UK and the US certainly government interference in universities is looking in many ways to be a similar level to places we would normally regard as not liberal democracies. So it's complicated. But let me let me give you the story. So a couple of years ago, we ran a workshop in Mauritius we brought a bunch of people we had that workshop in Mauritius because it would be easy for people to come easy earth for people to come from Africa. For these are reasons, you know again thinking about the ways in which people do get excluded from things. And we had someone come and the both myself and Stephen Curry were in this meeting, and we were talking about research evaluation so you can imagine Stephen Curry and I went in sort of rankings are bad university rankings are terrible. Obviously, these people will agree with us that rankings are terrible and horrible things that universities shouldn't engage with. And we had a very eloquent black senior academic from Ghana saying, actually, no, rankings are really important because they provide an external point of leverage for us to tackle issues of corruption and nepotism in the system. They give us a means of pointing to an outside space outside the power structure as we operate in. And I'm put slightly putting words in their mouth now but essentially those rankings as external and universal conceptions of excellence were actually giving them autonomy, which is a kind of was certainly a cup call from my perspective. So I think the problem is we have to build institutions, and we have to build new institutions and better institutions that manage the tension between autonomy. Trust, but also responsibility, I think autonomy comes with responsibility. You don't lose your autonomy if you don't take responsibility for those engagement and needs to serve society. Again, I appreciate that's pretty waffly in terms of what that means in terms of direct action. But it's, it's where I've landed it's just hard work and we've got to get on and do it. Thank you for that. Now, moving on from that, Sophia in the Q&A asks an excellent question that I'm not sure I can do justice to. So Sophia, if you were able to switch your mic on or do raise your hand and I can activate your mic would be really interested to hear from you. So let me try to, well, let me draw your attention to it and try to gloss it and maybe invite you to comment, those of you that would like to do so, because it does both pick up on the EDI issue that we've mentioned in very direct ways, but, but also talks about the possibility of coordinated action in this area as a means of addressing those kinds of problems. And I think that that's quite an interesting thing to ask you about the extent to which collaboration across reasons and across funding agencies is a strategy that's realistic here in order to address some of the problems we've been talking about. You'd like to address that. So you know, would you have any comments on that around the possibility of coordinated action I was actually just going back and rereading the comment because it is so rich. I mean, I don't know. I mean, and the reason I say that is I think I want to name the fact that I have been trained in and worked in and lived in Canada, my entire life right and so so again Cameron even thinking about some of the comments you offer just just as introducing your piece or your remarks around kind of where you are globally situated and I think these very lopsided and colonial and white supremacist kind of dynamics of power in terms of again where where it is believed that excellence originates and certainly I mean if you look at the history of where these notions of excellence originate you mean there are these kinds of interconnections so. So yes, do I believe in the power of working collaboratively to change and disrupt systems. Yes, do I think that's necessary. Absolutely. Do I think it's possible and feasible sometimes. And so yeah that that is something I do I do feel curious about because I would say, you know even just through the work I've done professionally. We do a little bit of collaborating with Europe, you know it often tends to be in EU countries and with the UK, sometimes with Australia with the US, but that's about where it ends, and it's very white, right and and I just I'm trying to be very explicit about I think it's important to name these kinds of dynamics and so, to me that's insufficient collaboration and and even thinking about in the Canadian context there's been enormous and really important work that has been done for for decades and generations around indigenous health research in Canada and really ensuring that our health research funding enterprises is actually equipped to assess indigenous health research in ways that are culturally safe that really respect the methodologies and priorities and knowledge of indigenous people and communities in Canada, while the lands clonially known as Canada. Because again the funding, the funding enterprises is is ill equipped to do that and is having to do a lot of retrofitting again because of sustained activism and leadership by indigenous folks working in those contexts right this is not out of the good will of institutions so I'm thinking of that example I think specifically even about sort of what are those intranational kind of opportunities and challenges and conditions, and then what does that kind of raise in terms of the possibilities around collaboration across indigenous cultures, and I mean of course the larger question of why do we have borders in the first place right and thinking about that as a racialized capital capitalist and kind of colonial, you know, violent entity in and of itself thinking about the work of people like Harsha Wally and others so you know there's layers upon layers of this and, and I'm conscious I think of the people in the zoom room that are saying okay but what does this mean for me in my everyday life and and I think these are ongoing challenges for all of us, touching into this kind of ways and definitely curious to hear what others think and what others are doing and experimenting with in their own context, I mean I certainly do not have the answers. Thank you very much for that now Sophia you very kindly raise your hand so let me switch the button allows you to talk and you should be able to join us so just introduce yourself please and make some comments about the that arise from your question. Yeah, thank you so much hi, I am a researcher a postgrad at the University of Sao Paulo. And I'm really just starting to work in in questions of meta science and, and like scientific research. So you guys actually know a lot more than me but something that I've really thought about a lot of working in Latin America and, and with us being with an arm and see a little read a leak Latin dex that there are a lot of efforts in Latin America to define excellence, through through open access and through very transparent like standards for research that sometimes to proceed the ones that exist globally. Like I said in my comment and oftentimes will go ignored I mean even in the histories of open access you might see a mention of of the CLO library but not much more than that even though, like 70% of research in Latin America is open access. In Brazil is one of the top producers of medical research and research in general in the world. So I just wonder, you know, if funders or organizations that you guys work with if there are more sort of direct kind of interventions that have to be made for for institutions in the global north to actually recognize the quote unquote research excellence in the global and especially to the panelists who work like in South Africa and stuff we're in, in Africa where I know this is the same issue there. It's just something I think about when we talk about EDI, you know it doesn't always translate to to the reality and to the kind of ugliness that exists, sometimes the prejudice that the liberal is regard for for academics in the global And of course I, you know, I would say there's there's so much interest in the global in Latin America for example to collaborate with researchers in the US or in Europe and other places but there is always seems to be very little on their part to come down to Brazil or to Mexico to to work to to collaborate in the same way. And I wonder why and I wonder if there are, you know, like I said sort of direct interventions through through funding or other for through universities that could really promote those collaborations and hopefully kind of show people that maybe they're what they thought was Latin America was Africa is not actually the case. So that's my comment and and thank you guys so much for your for your discussions and and for your critics criticisms. They're really, they're great. Thank you for your contribution as well. So, leading on from that we've got a little over 10 minutes and I'd like to invite all of our panelists now to continue in the spirit of direct interventions right and and and to say where we go from here. We have one or two bullets that we'd like to finish on that are action focused would be really good. And I'm padding here so you can all think hard before I pick a one of you. Lizette, why don't you go first. I have a little bit more time. But I think I, well one of my bullets could be relatively specific and it was actually a policy on a comment in the in the Q&A that wasn't addressed yet. And it was this one. Somebody asked about whether the race equality charter is connected funny, but with the key questions so our funders giving preference to some protective characteristics of EDI rather than others. And I think this is an interesting one. So this is very specific about the EDI theme again. Generally, there seems to be a lot of interest in gender gender seems to be considered quite the easy one right so we make a policy specific for female scientists and we're good. Generally, that tends to be a little bit too simplistic. Gender is also not that that's that simple, because it's also in transit also in a very binary idea of gender. Yeah, and, and, and also that and that again is also a question about data gathering actually. So we come back to this point that Cameron also made like often we don't know right so what they so I think that's a very key point related to the EDI to very critically about what kind of data is being gathered and because sometimes data is gathered without really thinking so what are what questions are we actually asking when we are gathering this specific data sometimes the questions only come after so then the data that has been gathered already shapes the questions that then can be asked so I think we should sort of switch that around really things of what are the critical questions to ask here and then what kind of information that we need and maybe the methods that we have for that right now cannot answer the questions that should be dressed. So I think that's one, one key point I want to want to make. And it's really not that practical but I hope a little bit more practical. A comment about data and availability of data is very practical even if it's difficult one to solve isn't it. Thank you for that. Cameron over to you next. Let me let me start with the one that won't sound as practical but it's actually I think the thing that has, again, I know you're coming back to this but tell the story about how what you're doing matters to you or matters to other people and do that in every place you can. So that can take the point from writing a paragraph at the top of your CV or thinking about how you're writing an application letter or an abstract through to, you know, thinking creatively about the way you present the data in your CV if you have those options if you don't have those options maybe complain about it. But telling stories it's stories in the end not data that drive the change and capture the imagination of people with the power to make a change. And of course then to turn that on his head of course most of us or many of us particularly those of us with jobs in universities or in some of these institutions are the people in power. We recognize the power we have. And so we should listen to other people's stories and see where we can learn from. So to pick up from Sophia's point, you know, if you want to learn about how to do research with impact, go and talk to an African researcher, because they're much better at it than anyone in the Anglophone, Northern Atlantic world. They're really good and it's really embedded and really just part of the culture in a way that you don't see in Northern Europe and North America. If you want to learn about how to make open access work at a large scale, go to Latin America. Talk to the folks at Cielo and Redlich. And, and learn, you know, actually learn from those successes and they're huge massive successes. And so then amplify those stories. And, and then think about if you're in that position of power, what data is available to you, how can you bring the evidence to bear about how things are not working or how things might change. So for instance, an interesting thing in terms of in the UK, racial versus gender diversity in UK institutions, if you look at the correlation between the number of black people as recorded in the HESA data in UK universities, there's actually a positive correlation with the size of the institution, which correlates reasonably with the prestige institution. There is a negative correlation with the proportion of women. And so there's something, we talk about gender as being something that's been tackled effectively in the UK. I'm not sure it is. Also, look at the data, look critically at the information that's there and ask questions about about these narratives and whether they hold up when you start to test them. And yet, and I would say, maybe betraying my my origin in the sciences, you advocate for more transparency, and more of appropriate transparency to make some of these things available and self reporting on how well we're doing and making progress on those things. So stories, my bullet point would be tell stories, listen to stories, and then look for the evidence and test those stories against the evidence you can find or not find. Okay, thank you very much indeed for that. Zina, finally over to you. I mean, I would say the first piece of advice I have comes to comes from a therapist I had many years ago, which was specifically around I remember her talking to me about kind of a behavior I was hoping to change in my life. And she said, Well, you know, sometimes, it's good to start by just noticing, you're noticing that this is happening maybe you're not ready to shift it. So, with credit to Lorraine my beloved therapist of many years ago, I will say, I think one of the things we can do and this is something we can each do in the context that we're in and that some of us may already be attuned to but I think probably many of us are not actually foster this practice is practice noticing how and where excellence in quotation marks shows up in the context you work in. And I'm curious about this, because again I think part of the challenge of this is that excellence does have that tendency to fade into the background it becomes part of the air we're breathing the water we're swimming in. And to me that that opportunity to notice right to actually start to discern a pattern to unpack this taken for granted construct, you know can again be a place of actually beginning to then understand where can one productively take action to begin shifting those dynamics so that that is the first one practice noticing excellence get curious about it. And then the other piece I would say is, think about how and where you might have the opportunities to productively disrupt these notions, you know, especially when they are showing up in harmful ways in the context in which you have influence and be willing to experiment. We all have the capacity to do this in different contexts and at different scales. One example I can think of is the people I know, you know, basically everyone I'm thinking of is a queer person a disabled person you know who is an academic working in a context of universities during the pandemic, who have chosen to engage in queer abolitionist feminist pedagogies that are actually creating far greater humanity, I would say in the in the institutions of the university. So this notion of actually again rather than perpetuating what can be an incredibly harmful notion of excellence and how you might expect a student to show up around meeting deadlines or performing a certain kind of performance in a context of a global pandemic. They are taking different kinds of approaches in terms of engaging with different forms of grading or no grading at all. No attendance policies this incredible flexible very thoughtful pedagogy. That to me is one example of what that can look like at the level of an individual instructor, which I know is work which I know can be disciplined by the university but I'm seeing people in my life experiment with this and incredibly generative and very interesting ways. And again, I think that there are similar call to actions obviously for those of us in a funding context and, and to not get stuck in fear of getting it wrong. And that doesn't mean we have unfettered permission to do whatever we want and cause further harm. But I do think we need to get out of that that that stuck place of perfectionism and having to get everything right and be willing to try. And that I think this is another another place that that I think we can all do something and that that is how we shift system. Thank you very much indeed for that and I, I really like that note of productively disrupt. That's a great note to finish on so thank you very much for that. Thank you to all of our panel members it's been a real pleasure talking to you thank you for your questions. For those of you who contributed and I hope it's been of interest to you we need to carry on this conversation. I think I think you can. At least I hope you can see how the discussion has been shaped by your comments. So thank you very much for those. And I hope you've enjoyed this session and indeed the rest of the conference. And just to finish off by saying, Cameron you win the prize for the only member of the only member of this zoom call who hasn't had a household member trying to disrupt this zoom call. So thank you very much for that. Thank you and yeah, have a good rest of the conference and thank you very much to everybody who's joined us. Thanks. Hi, this is Leslie from the Center for open science thank you so much to all of our panelists for what has been a really thought provoking session so so thank you very much. Thank you for your time on attending. I've just popped in the chat a reminder of our slack and remote spaces so please do feel free to hop over to those spaces and continue the conversation. And the next session will begin in half an hour. So thank you very much to everybody. Goodbye. Goodbye. Thanks so much. Thanks everyone. Thank you. Thank you.