 We're going to go live just now over to you Matthew. Yeah. Welcome everyone to this conference 2023. Our panel discussion on building community practice. My dog is just about to sit with me. So we have a dog in the picture, sorry. This session is being live streamed to YouTube and there should be automatic subtitles available at some point in the future. If you have any questions for our presenters and our panel, you can ask them on the ESAGathon Twitter account about this session. We'd like to draw your attention to the code of conduct, which is available on the Esmeralda website at www.esmeralda.org. So we've got a panel here. Excuse me, of experts are talking about the communities of practice. So first of all, I think I'll ask a question. So I'll ask you each to just introduce yourselves very briefly and then tell me what is how do you define community practice in evidence instances? So let's start with you, Vivian. Sure. Hi, Matthew and hi, everybody out there on YouTube and in the Esmer Conference Black community. I'm editor in chief and interim CEO of the Campbell collaboration. I'm based just north of Ottawa in Canada, still have a lot of snow and I've been working in the field of evidence synthesis for going on 20 years now. I think if you said define a community of practice, I feel like it's a community of people with a common interest. And that's kind of the the the gel that brings communities together that be it they're interested in methods or they're interested in editorial practices, which is a lot of what I deal with as editor-in-chief. So how to improve the quality of the editorial processes. But I think the community is people who share a common interest methods or otherwise and and have some way of connecting on to talk about that common interest. I think that's how I would go with it. Let's stop there. Thank you, Andy, do you want to go next? Hi, thanks, Matthew. Yes, and hi, everybody. Yeah, my name is Andrew Paul and I'm a CEO of the Collaboration for Environmental Evidence and also editor-in-chief of the Journal Environmental Evidence, and I'm based in North Wales in the UK. So I've got very similar sort of very similar sort of roles to Vivienne, I guess. And yeah, community practice, I agree with everything that the Vivienne has said. I think often the community of practice is formed because there are particular challenges, particular. Work to do, I suppose, and that community is is, you know, interested as Vivienne said, but also wants to improve things in some way. So or make things more available or make things more understandable for people, want to, in many cases, train and enlarge in the community, build capacity to do things which they think are worthwhile things to do. And but they do have that, you know, everybody has that connection. There's some some common ground there for everybody in a role, maybe different roles to play for different people as well. But it still remains a community because everybody recognises this common goal as Vivienne was mentioning, you know, this common task that we all we all have. I suppose. So I'll stop there. And then James. Hello, Herman. Good morning. Good evening. Good afternoon, depending where you are. And yes, thanks for the invitation to join you today. I'm James Thomas and I'm based at the epicenter, which is part of UCL in London, where I do lots of systematic reviews over the years to inform policy and sort of a second area of interest is the use of computer science technologies to make the process more efficient. And, you know, we've all seen many communities and practice in this area over the years. So I think, yeah, a shared sense of goals is obviously key, but also sometimes about values that the community shares both the the understanding of a particular issue being a problem, but also some shared sense of how that problem might be to be addressed or how the community itself is going to address it. So and sometimes communities of practice are relatively loose coalitions of people who either either virtually or physically group together to do a little bit of work together of a fairly sort of amorphous in some respects and others really work quite tightly together, sometimes very, very intensively for a short period of time and then sort of things things lie down a bit. Or, you know, some communities of practice, I suppose that's one of the things that has been involved in and obviously Cochrane is a large community of practice which has been going on for a long time. So, yeah, they take many shapes and forms, don't they? So is it a question to all of you, really, is do you think that it's something that sort of become comes out organically that these things develop? Or do you have to be actively steering the community in a certain direction? And there has to be some sort of top down control or pushing it together in the right way. Yeah, anyone cast that? Well, I can have a go, I suppose. I think the most durable, I suppose, sustainable communities of practice are largely bottom up. They often have to start from a very few people, of course, as the Cochrane collaboration did and as all our collaborations did. And, you know, CE started from a very few people, but then the word spreads and people get interested. But at some point it switches from being, you know, one or two leaders to a community that is basically operating bottom up if you want to sustain it over a long period of time. I don't think a top down approach is terribly easy to maintain. But obviously, you have to have leaders. But like all organisations, there's a delicate balance, I think, between, you know, the sort of leadership roles and the innovators and those who can come through and do or make really radical changes to the community for the better. And so you have to, I think, try and strike a certain balance in that. A lot of it depends, of course, on how big that community of practice is. As James was saying, I think sometimes it's sort of got an intensive work amongst a relatively small group of people. And that may work completely differently from a very large global organisation, I guess. Yeah, I think some communities of practice are people who are interested in the same kind of thing and just like to come together to talk about it together and advance their thinking, but not necessarily achieve a goal beyond meeting and talking and advancing knowledge in that way. Whereas others have to have the objective of actually achieving a particular goal or set of goals. And I think what you see, the two are very different in terms of their objectives. And so what you see are situations where there's a challenge in sort of shifting from a group of like-minded people who want to do something then to something which actually becomes slightly more formalised because they're this I'm thinking here as communities where there's a specific goal that's being aimed for. And then often what you see is some of the fun leaving the community of practice in some respects. You see some communities to sort of go through various growing pains where moving from a set of like-minded people trying to achieve a goal within their own sort of ability to achieve it to something which scales up, which then needs tiers of governance and funding and all that kind of thing. And then at that point, yeah, there are there are challenges to be met, but also then the community itself has to undergo sort of shifts in the perspective in terms of what it feels like to be a member of that community. Yeah, I kind of I agree with both of you. I think one hundred percent, Andrew, you need leaders that are going to move things forward. And I think that's quite tricky to do from a top down way. So I had an experience many years ago. We we tried to start communities of practice on community engagement and research. And you you can't really you have people have to come together because they have like-minded. They're like-minded and they want to talk to each other. So we we tried to start these discussion groups and it just failed. There wasn't someone in there that wanted to lead it. But then in another part of that network, there was a leader who started a group, kept it going, you know, because they got the right people. So I think there's that you have to have that bottom up that that brings people together. I think a little top down to encourage, promote, facilitate the discussions could be helpful. So just give an example that in Campbell, we I really wanted to see our information retrieval group reactivated. So I I asked several people until I found two people who who said they would co-chair the information retrieval group. And once they got it going, it's an amazing group. People come together just for a chat and then they identify goals they want to achieve, which was a little bit of both the things that James was talking about. So I think a little top down to find the people who want to make things happen. And then then that sort of community identifies its own goals and things that kind of keep it going, which I think is what Andrew, you're talking about sustainability in the long term, you really need that community to have the interest to keep it going. Is there something about the the nature of the subject or the people there that makes it possible to sort of form a community of practice? Because, you know, information specialists. I don't know, as a sort of like a discipline, it's about valuing the knowledge and the contribution that other people have already generated. A lot of it is around that. And information specialists seem to be very good at forming communities to practice together in a way that other academic areas are slightly less good, possibly, let's say, where, you know, there's a there's a sort of almost a point of principle of just basically arguing with one another. And, you know, you can you can you can come together to have a nice argument, I guess. But you know, in some areas, you don't necessarily see that same buy-in to actually let's let's let's talk together about achieving this common goal. Well, I don't know, I can't speak to all academic. To hear what Matthew and Andrew think. But we had a similar experience with our methods editors, who I guess are also really, you know, detail oriented methodologists. So they're the statisticians within Campbell. And similarly, I asked one person to, you know, really lead. Took a while, but now when they meet, they just can't stop talking. And they come up with ideas for things that they need to do and then go off and form a new sort of sub sub community that gets those things done. So. But you're right, maybe content academics are a bit different. Haven't had to do that in Campbell yet. There's interest in that you found, you know, that you've been able to sort of top down seed the community of practice. Because when we were first start talking about this, I was thinking that I was I was struggling to think of, you know, someone who's come along and said, you know, you know, you should form a community of practice about dot, dot, dot. Yeah, a couple of doesn't always work. Yeah, a couple of observations on that conversation. I mean, what one is that, you know, with leadership comes responsibility. And I guess, you know, they when a community practice gets to a certain size, as you I think said, James, you know, you need you need things like budgets and you need administration and you need, you know. And so that you need to bring in people with different skills to be able to scale up a community. And that means that some people are either in burden with things that they don't necessarily want to do, like look after the finances or something like that, they want to do the exciting things. You know, that they originally that the whole thing was originally about the whole community was originally about. And yet it become inevitably it becomes fragmented into the number of different roles and so you need different people. The other thing is that sometimes, you know, communities get so large that they they fragment because the the overall aim is is so big. People get interested because they people get into the community because they're interested in particular, particular subsets. And those subsets then are more exciting than the overall thing. You know, people get get more excited by going into certain aspects of evidence synthesis or or certain subject areas. And so the community fragments and do you keep it together or is it good that it fragments? I mean, that's that's another question which I don't really know the answer to. You seem that you know, you want you want the community as a whole to stay together to some extent, but you don't want people to have to, you know, span the whole the whole community, you know, the whole subject of the community necessarily as a place for people in the in the subsets. So yeah, it's it's quite complex, really, when you think about it. I think that's something that we experienced a lot in the evidence since this hackathon and as my conference is you tend to have like a satellite, a core and a satellite people who some people are always in the core. Some people move between the core and the satellite of driving things forward. Obviously, Neil is the main core of the conference, but there's people who drive the whole community forward. So you end up with communities of communities of practices, if that makes sense. I think that that's maybe something that, yeah, I think maybe on the larger scale that all you guys work at is probably something that you have is these communities of communities. And maybe we can talk a bit about how to how to engage those together, maybe communicate together to communicate within those communities of communities across that, if that makes sense. Yeah, it certainly makes sense to me. Communication is, you know, once a community gets to a certain size, then how you the one part communicates with the with another part and keeps the context, you know, and the reason that the overall goal in context, if you like, is is a real becomes a real challenge. I mean, even with even with social media, we have it at disposal these days. It's it can be really, really difficult to keep the family together if you like to keep those connections, those personal connections that mean that people can engage with each other to feel part of that community. And you can easily start to lose people if you're not careful to keep that communication level up and make everybody feel valued as we talked about values. You know, everybody's got their role in the community and just constantly remind people that that that that role is is really important and everybody needs everybody else to make it work. Yeah, I agree. And I think, you know, on the flip side of that is sometimes the boundaries around the community are sometimes either sort of barely barely sort of vaguely drawn or really quite sort of drawn with quite a hard line. And you think about the society perception methods, for example, which until relatively recently was a closed community of practice in that it had a fixed number of members who and people were invited to join it. And the the the objective there was to keep discussions focused and to, you know, to really get the sort of intellectual focus of the community, you know, to maintain that well. But what it all about that also then meant was that people outside that community were just like, well, you know, I want to join the community of that community of practice, but I can't. And so, you know, but it's, you know, I can I can also see that I meant, you know, the objectives that the society originally had around that was also partly to keep conversations but also organizationally keep it manageable. And so if you have everything, everything wide open, then you can quickly sort of overwhelm whatever organizational structure might be sort of might might might work for a group from sort of 30 or 40 people. Suddenly, if you've got hundreds or thousands of people wanting to get involved organizationally, then it becomes a real challenge. Yeah, I mean, it's interesting about the size. I was thinking about a different idea, Andrew, about the idea of personal connections, because I think that's a really important part of communities of practice that people make, you know, individual connections that bring them back to that community practice. So there's a common interest, but I think the personal always makes a huge difference. So, you know, we're all connected to Neil and we come along to the SMART conference because, you know, we know Neil. So I think that personal relationship makes a big impact on communities of practice. But I don't exactly know how we how you articulate that and how you form and maintain communities of practice. But I think it makes a big difference. It's part of maybe feeling valued, Andrew, is that you get these personal connections which have value to you and make you feel valued. The other thing that Matthew, your question about. Starting to think about, you know, communities of communities of communities is I was thinking of how, you know, we bring together communities of different topic areas like environmental evidence and Matthew, you're from ecological background. I think ecology background, Campbell is social sectors. We also work in partnership with Cochran and JBI on things like evidence based health care day and next year, the Global Evidence Summit. And I think there's a real strength in bringing communities of practice to together at that higher level that are rolling it up. Yeah, I think. Sorry, James, go ahead. Well, I was just following up on Vivian. I think that is really interesting example of a community of practice and and how they evolve because, yeah, obviously, Neil is very central. But then also, you know, as they are, obviously, is our as in the language. And, you know, in the first to begin with, I didn't really feel that I had much to say on for Esmar, really, because I mean, I can code in a number of languages, but I are is truly appalling. You know, I'm basically a Stack Overflow or now, you know, the chat on being just, you know, you put in what you'll give me the code, but my I'm really, really bad at our, you know, and so. That boundary for me said to me, well, this this isn't really a community that is, you know, is for me. Whereas, you know, you look at all the abstracts, you listen to the the presentations and everything, of course, it's it's it's all the stuff which I'm working on every day as well. So and, you know, we're here talking about communities of practice, we're not talking about our code yet. Anyway, I might go quiet at that moment. So, you know, it's interesting of evolution of interests, even in this little community of practice here. And I think it's it's quite important. Possibly, though, you know, this is an interesting question, isn't it? That that's nobody sort of essentially is saying, you know, if you if you're not talking about, you know, this particular package or this library or, you know, I'm doing this in our, then it's not it's not eligible for discussion at this this meeting. And so it's enabled this to be a really sort of wide ranging and, you know, really sort of vibrant community, you know, that gets together or something like this. But, you know, it might well be that there are some our coders here who are sort of thinking, oh, you know, I just just give me the codes. What I'm interested in is that part of it. So there's that tension, isn't there? There's the there's the sort of not losing the sort of like the focus of what makes it different and special from some other meetings, but then also all of the other interesting discussion that goes on around it. And I'm happy to talk about our code if you like. Please do. I think that's what I really like about Esmeralda is that it is open and the R is often whispered. So we don't make a big point of it. And it's a community of different disciplines all trying to make useful things for people to use. But we also need the people who are using them to be a part of our community. Otherwise we just make things that are useful to us or they look cool and the cool the codes nice, but they're useless. So that's why I think it's really a good community to be a part of because we're trying our best to involve the end users as well as the people who like to play around with code like me. Yeah, I think that's an important point about the communities that we're talking about. Because I think all of us have a strong service side to what we do. The usefulness of what we produce is one of the main goals of what we do. We wouldn't be bothered with it otherwise. So I think that's the idea that the community serves a separate community is something we think about. But we're always trying to bring those two communities together. Those that produce the evidence synthesis and those that use it use the synthesis. We've certainly tried to bring the users into the community as much as possible so that we've got that direct feedback about what we're doing. And it's not always easy because they belong to, in some senses, they belong to a different world. They don't do evidence synthesis. They don't get excited by doing evidence synthesis. They might get excited by evidence, but not the way it's produced particularly. And I guess that might be the same with Esmark that some people aren't that interested in writing the R code, but they'd love to be able to use it to do whatever they want to do. So that's a common theme running through these types of communities we've got experience of, I think. I completely agree. I think that's absolutely key, isn't it? Because in this field, everybody's got a constituency that they hope they're going to use what they're doing. So there's obviously tool development which is pointless without people actually then using tools. But the people who are using the tools are doing evidence synthesis to be used to inform decisions, which again, arguably are relatively useless unless you've got people who are actually at least meeting the things at the end of it. So, yeah, and it's interesting. I don't know how far on how broad a community of practice around the use of tools in evidence synthesis needs to go to inform that tool development. Because presumably there are tools obviously for doing the stuff about the integrity of evidence synthesis. But then there's tools around presenting the findings and enabling people who are users of evidence to then interact directly with some of the evidence. And so, you know, there's really quite a broad constituency that this, you know, to do it ideally needs to be also to be connected up in order to inform actually how the tool development should proceed. Yeah, I was just thinking that in Campbell's, we have a number of people that commission Campbell reviews and across totally different topic areas, you know, so crime and justice, education, international development. And with our annual conference, we really try to bring those different commissioners and users of evidence together. And they really feel like they don't have, you know, a community of practice to talk about their challenges in, you know, timeliness, rigor, working with evidence synthesis. So it's something we're thinking about activating at this year's Campbell meeting is how do you sort of create a forum for that kind of person to get together with other people who have challenges in using evidence. So, and somehow connect them, I think James with the people producing the evidence, right, so that we can be producing useful evidence, which is what we all want to do. I think they're, they're a community that hasn't been really super well engaged in, in Campbell and in evidence in this communities in Cochrane, you know, there's been a, I think pretty good success getting practitioners involved because they actually many practitioners actually do reviews and practice, whereas you don't see that so much with decision makers in policy, you know, the Ministry of Education or the, yeah. So I think it's a, it's a audience that somehow could create a community practice, but might need a little, we might need a meal had a way to do that. Yeah, that comes back to the sort of like the leadership and the sort of like the visionary leadership that some, you know, you know, that a lot of communities of practice can't really do without. Yeah, I agree. Yeah. And I think that goes back to sort of top down bottom up like there's a bit of both. I think these communities, they need that stimulus that could be from the bottom or from the up or could be from the bottom or the top, but is that stimulus that leader that kind of makes it happen? Yeah, and I think sometimes it's sometimes it's making that transition from the sort of like dependency on, you know, it's like an individual person almost where something is sort of coalesced around moving from that to something which has got a more formal governance structure, you know, like a group of people and then, you know, people then want, well, how do the people end up on the group who gets to make the decisions and then, you know, that's it's making that transition that you see, you know, communities of practice sometimes fall to bits because people can't agree on how that should that transition should occur. I don't know if there's there must be research on communities of practice making that kind of transition. I mean, I remember Cochran making it way back in the day, my first ever colloquium is quite a controversial one with everyone arguing with each other, but it was making that kind of transition from a relatively small close-knit group of, you know, everybody knew each other and everything was relatively informal to the time when you're getting lots of people, it might be hundreds of people then involved and suddenly those existing sort of informal ways of making decisions about things then just don't cut it anymore because you know, you can't just depend on everybody knowing everybody else and saying well, then that because, you know, then you don't have sort of equity and you don't have accountability in terms of how the decisions are being made. Then, then, you know, then it gets, you know, like I say, it gets a little bit less possibly a little less fun, a little less exciting because then you're looking at governance and, you know, steering boards and that kind of thing which some people find very exciting, but often not the people who actually just want to get the job done. Yeah, I think that, you know, some of some communities, of course, have grown in such a way that they kind of democratised, I suppose, is the word. I mean, you have a membership, you have more formal membership of the community, you're either in or you're out and the leadership is kind of a result of voting for who's on the board or who's going to be running the show and it's interesting to think about when communities really need to go in that direction so that it's obviously more a choice of the membership of who's leading and who's in particular positions and, you know, and the organisation of the community will then, you know, it is more of an organisation then by definition, I guess, will go in directions which are dictated by who's, you know, gets to the most popular, who gets the power, I suppose, to change the direction and I guess, you know, to an extent Cochrane's gone in that, he's gone through that process, hasn't it, to a certain extent where you do have, you know, changes of leadership that have meant changes in policy to a certain extent, whereas I don't think any of ours maybe got quite gone through that same thing, I don't know whether that's the case in Campbell, maybe, you know what I'm sure of. But you know what I mean, you get to that certain stage where there are different views as to where the community should go, what are the priorities of the community and how should it function. That's almost inevitable if you're successful in building a community, it's almost inevitable that point will come, I think. Yeah, I think exactly that. If the community is successful and if it grows, then getting to that point is kind of inevitable and from that perspective, it's kind of surprising that it's such a surprise that the challenges that might do seem to be a surprise every time, but from what I've seen they do seem to be. We've had a couple of questions come in from the audience. Trevor Riley has asked us on YouTube how, you know, what the panel think about a smaller group of people who want to become a community practice. What do they think about them trying to sort of to become a subset of a larger community? Are the pros and cons of the approach of either going alone or going in with a larger group? I think Vivian is keeping quiet, but I think there were discussions in the early days of Campbell as to in terms of how close it should be and whether it should be part of Cochrane and sort of Cochrane. I can't remember what those discussions looked like, but I think in general, you can kind of predict the pros and cons. You get essentially, you can piggyback off the organizational structure of another organization and that can be really, really helpful. So there was, we've had various Cochrane tech meetings of piggybacking on the back of the fact that Cochrane was organizing a meeting of the Cochrane and Cochrane and Cochrane, and then we'd have sort of parallel tech discussions which worked very well, but I think when you're doing that, sometimes there's a danger that your community of practice becomes a sort of like a subset of the interests of the other community. And so, you know, the text is a nice example there in that, you know, it was fine if you were interested in evidence synthesis tech in health and then, you know, areas around health but then, you know, piggybacking like that didn't draw in all of the people that, you know, potentially part of Esmar, who are interested in other areas, not health, but still interested in the tech. So, you know, it was easy to organize that because we had, you know, existing structures and meetings and everybody, people were likely going there in some respects anyway, but yeah, it didn't work if what you're wanting to do is bring in a much, much broader cross-section of people interested in that specific subject. Yeah, I think the Campbell Cochran discussions are at least two that were quite serious, one at the very beginning and one in around 2012 or so. And Campbell both times felt that their sort of community of social science practitioners could be lost by joining forces with Cochran. So, I think it's James, I totally agree the idea that joining forces has to bring this sort of brand identity but you could get sort of subsumed into a larger community practice could be the risk and yeah. I think that there's so many new organizations coming up in evidence synthesis that there's probably a good rationale for seeing if some can work together rather than creating new communities that generate new requirements for meeting. So, I love the idea of the Global Evidence Summit where Cochran, Campbell, JBI, Guidelines International Network get together for an annual meeting that just seems that you could join up those interests but I totally agree with the downsides. Yeah, so from a perspective of being a small group I think there are definite pros and cons. I guess it all comes down to knowing what you want as a small group and how it compares with the larger organization that you might join. James said by joining a larger organization you get various benefits potentially. The larger organization might have some funding that will help you grow your particular area of interest for example but the cons are that you're going to be constrained by what the larger organization will tolerate in terms of your interests as you develop. I agree with what Vivien has just said that in many ways it's good that there are a lot of smaller organizations growing up in the evidence synthesis area but on the other hand there's a point where if you get too fragmented and then there's duplication of work etc etc and you want these relationships or communities of communities that you were talking about Matthew earlier on there's a mechanism for those communities to come together maybe it's an annual meeting where you can maintain those links and understand what each is trying to do and making sure that you're not replicating effort because for sure there's so much work to do in evidence synthesis we don't want replication of effort that's one thing we pretty much all agree on. You talked there about maybe having an annual meeting of all these communities other ways we can sort of promote communication between disciplines and communities of practice I know I don't think we're unique at realizing there's other things out there in the world and we try and reinvent the wheel every week so I think other strategies we can think about that would help us communicate better between the different groups and different focal points of the communities. Thanks to Neil Andrew and I ran a joint series between the collaboration of environmental evidence and Campbell which I think is one way to work together I mean I think there are things you can do together that could promote those communication links I guess the ones I mentioned were conferences but one that Campbell hasn't engaged in very much and would like to do more in the future is how to advocate for evidence use so how to advocate for rigorous evidence for decision making it seems like it's something we could all do better together than alone so I think there are ways that we could work together I think it's a little bit going back right to the start of the conversation where James you said one of the things about communities of practice is having common values and goals and I think kind of identifying those values and goals those are where we might be able to work together on things and I think that's a great point there are times when there's real strength in communities coming together and showing a united front that evidence synthesis is important and it's being neglected in many ways and it's difficult for people to develop careers in evidence synthesis and get funding evidence synthesis is difficult to use evidence synthesis in many ways because we've never really spent any money on communication of these kinds of outputs and so the fact that we are all connected through various mechanisms including the new global evidence commission and through evidence synthesis international gives us an opportunity to have more impact at the political level I guess to argue for more funding and really for just more attention I suppose to be paid to evidence synthesis in terms of decision making than has been done before obviously for us in CE it's about raising the flag for evidence synthesis and important global issues like climate change and biodiversity loss where I think it's quite alarming to look back at the last 20 years to see how little evidence synthesis has been taken into account as it is being at all important in some of these areas so if we partner with Campbell and Cochran and other evidence synthesis organizations we can get more attention for evidence synthesis in those kinds of areas because they cut across all of the disciplines really because there are so many health and social issues involved in climate change that really it's just as big as the environmental issues so that is really where I think the larger community beyond our individual communities the big community really becomes a big resource if we can all stay together and show that united front which I think largely we do I don't think there are any big differences that I've detected in the approaches of the collaborations across the sectors we agree on 95 and 99% of what we do I think the sort of criticisms of the sort of community practice is often that they get to a certain stage and I haven't told Andrew this yet but a collaborator of mine said that CE were policing the systematic review in environmental evidence synthesis he didn't know that I was associated with CE in any way at the time and I found that quite shocking that someone would think that I don't know how that message got to him and how it got across but I think do you think there's a danger that when you get to a certain size then people might see you as something negative I guess I better answer that one so when you're trying to develop a community of practice that basically upholds certain standards like we all do you know we're trying to demonstrate why those standards are important otherwise why would we be trying to uphold them obviously that's something that is part of our community of practice that rigorous evidence synthesis is important and so there are inevitably certain things that we have to try and communicate to the very large body of scientists that are potentially conducting some kind of evidence synthesis and I guess what they're referring to by policing is the fact that we basically produced a number of publications demonstrating the variation in standards that exists even in reviews that are called systematic reviews and we're not the only ones who've done this it's been published in the health sector and in other sectors as well that there are lots of things out there that are called systematic reviews by the authors that we wouldn't recognize as systematic reviews because they don't conform to our standards by a long way and unless you actually point that out you're going to try and gain the attention of a very large community of people who you'd like to see doing systematic reviews how are you going to get that point across in any other way but it's not policing in a sense we don't say that these things that are other types of reviews have no value we just point out that there are certain weaknesses that people ought to be aware of when they're going to use them I think you're highlighting a really critical issue around the external persona and the external profile and the way that a community of practice interacts with people in related areas and it's up to a point that we're actually in what we've been doing in evidence synthesis if what you're doing is you're saying this is what reliable knowledge for informing decisions looks like you're almost by default saying that if it doesn't look like this either we've missed it in some way or actually that's not such reliable evidence for informing decisions communities of practice in this area are actually setting themselves up as targets for people to shoot down and so I think that the interactions between individuals and the community of practice corporately and people wanted to throw rocks in either direction is a fascinating part for historian history of science in this area to look at because you know it's obviously not been straightforward we can all point to various situations which have been uncomfortable within the communities of practice where either the communities have come under sustained attack from outside or where the communities themselves have not been agreed in terms of how to address a particular issue which may be within and or without the community and I do think that that's there's probably lots of other examples outside our fields of where this happens but I do think there is something around if you're setting yourself up as being specifically around the advocating for the use of reliable evidence that you've got standards there both to maintain but also standards there which are up there for criticism yeah 100% agree and I think James that last part for criticism is really important because that's I think something that you know allows responsiveness right to that criticism so while still keeping the standards maybe there's room for innovation and methods but and totally agree with you Andrew that we set ourselves up as advocating for rigorous evidence then we need to maintain those standards so the transparency about what those standards are and the openness to constructive criticism I think is really important for all of us and it probably drives the communities that there is that debate like that that makes it interesting and exciting going back to some of our earlier discussion about what brings people to communities of practice I think there is you know if it was always the same the same people might become less interested we've had another few questions come in this is from Valentina Costa she's asking about a role for everyone each community practice we mentioned earlier having a role for everyone but is that something that people themselves choose their role or something that the community chooses roles that need to be filled well I think I think largely we would like to believe that people join the community because they have a particular interest they have roles in mind that they want to fulfill I mean we have lots of different roles within CE and we constantly need people for all of those roles I think there's plenty of room for people and it wouldn't be usual for us to make the decision it would be the person who was coming in saying I'd really like to help you with this I'd really like to get involved in this aspect and our normal reaction is yes great brilliant this is who you need to talk to so I think but of course there are always specialist roles within any community that you may want to have to try and persuade people to take on those roles because there aren't many people who have the skills so there's always that element of persuasion to take on key roles that exist within a community I think I agree but I also think it's around the stage of evolution of a community that you're at you're describing a community there that sounds relatively mature in that there are some defined roles that you would like people to come and fill and in order to part of it is around the organisation and making the community sustainable and just keeping the show on the road as it were and if you step back to sort of early sort of a community practice earlier and it's an evolution I guess it's sort of theories around the formation of teams suggest that things are really quite fluid and that a good team player will sort of look to see what's needed and we'll decide okay so I can fulfil that role so I'll go and do that in a much less structured or organised way and that just takes us back to that moment where the team starts to grow and actually then team thinking of things as being a team works less well with those defined roles and then translating into a wider team and into something with more definition is tricky I totally agree with both of you I'm going to say from thinking about Campbell as a community practice or Cochran I'm not sure, I don't know CE as well but I know that people look at it from outside and think how do I get involved it's not totally clear how you do it I think that community practice to be sustained have to renew membership and I think one of the at least for Campbell one of our challenges is how to bring packed people when they get involved with Campbell and keep them involved in Campbell and there is a role for the organisations to do that I think I love when people ask me for a role Andrew just like you yes come this is for you I think though we have to make sometimes the opportunities for the people to see though that there is room for them to get involved and it's something we're planning to work on in Campbell for the next couple of years to make that easier to get into the organisation and to have a role that makes people feel valued right because I think what keeps you involved is that there's something you're doing that excites you interests you makes you feel valued I think you've put it's on a key strategic issue of importance for community practice around how you how you get people involved and how you rather than thinking okay so you figure out what you can do here but actually how you define the roles how you sort of think about both in terms of people's skills and how they can contribute when they first join but then also how the community can support skill development and sort of evolution in terms of what they're doing and what they're contributing within the community you know there's been various sort of strategies within Cochrane on that over the years from sort of okay so you know I'm not ready to be a lead author on a review at the moment but I want to get involved how do I move from here to there and sort of thinking about how you can organise contributions within the community in order both of you to contribute to the community but also for you then to sort of progress and evolve in your own right in your own skills Sorry to say our time is up it looks like we had a really good discussion thank you all for joining in and