 Llywodraeth gael y byddwch yn ysgrifwyddon yw'n gweithio i'r wneud yw'r mhwng o'r ffordd. Mae'r mewn amser yw Ruth Galside. Mae'r mewn ysgrifwyddon yn cyllid ddod yn cynllun ysgrifwyddon yma yn y ysgrifwyddon yn ysgrifwyddon a'r cymdeithas gyda'r gyfnod i'r mewn gymhwyllus ym Ysgrifwyddon. Mae'r rôl efo'r cyllid ysgrifwyddon yn cynnig ei ddod o'u cymdeithio ar y rôl o'r rhan o'r tyfu. I'm particularly interested in complex questions that require the use of a range of different types of evidence, and particularly I'm interested in qualitative evidence synthesis. I'm a co-convener for the Cochran Qualitative and Implementation Methods Group and have been working on different methods of qualitative synthesis for about 10 years now. So, I think my presentations are going to be perhaps slightly different to the ones you've had this morning, so I'm going to give a kind of overview of the methods first and some of the assumptions behind it, the sorts of questions that it answers, and then give some examples. In this case I'm giving one example, but we can talk about other possibilities at the end if that's helpful. So, if I can make the thing move on, there we go. So, what is a realist review, the basic questions first? So, this is a type of systematic review which is often referred to as theory-driven, and its goal is more about explaining and understanding what's going on, rather than judging and summarising the evidence which is out there. So, it takes a slightly different perspective to either reviews of quantitative or qualitative purely evidence. It's sometimes referred to as a logic of inquiry, a mechanism for looking at things rather than a methodology, and it can incorporate both qualitative and quantitative evidence into it. Some of you may have already seen this helpful diagram from the epicenter from James Tominson colleagues which tries to explain different types of synthesis approaches from the more aggregative over here on the right to the more configuring or interpretative, and they think of these, if you look at the row along the bottom, in relation to theory. So, while a meta-analysis and content analysis is looking for the sum of what's going on and looks to test theories, so you're looking to test hypotheses and see whether or not they're true. And although I think there's a lot of words and conversation about reducing bias and how objective these kinds of reviews are, anyone who's done them will tell you that there is interpretation about what goes into the review, how you define the question and so on, that these tend to happen before and after the synthesis, not in it, so they're ways of framing the question or using the results from the synthesis. And then right over on the left-hand side here are the more configuring or interpretative reviews, and they've got metre-ethnography here as an example which I'm going to speak about yet next. And in this case the interpretation happens during the synthesis to build meaning. And then you're generating theories, so you're using empirical data to propose theory. And then realist, which is where I'm speaking today and I think you've already had a presentation about framework synthesis, you're exploring theories. So there's just one way of thinking about the different approaches, their relationship to interpretation and their relationship to theory. So some people have said, well, you know, there's so many methods out there, there's loads of methods within quantitative systematic review, there's lots of methods in qualitative evidence synthesis, why do we need another method? And it's been proposed as a way of thinking about complexity in questions which are subject to systematic review, you can also have realist evaluations, so for complex interventions in primary research as well. And it takes a different kind of perspective to traditional positivist research, and particularly in the recognition that interactions between mechanisms and context are really important. So it's not just does something work, it's how does it work in this context with this mechanism. And it wants to explore, create and sometimes test mid-range theories of impact. And a mid-range theory is somewhere between the nuts and bolts daily stuff and the grand theories of everything. It has to be something which is explanatory and slightly abstracted, but also needs to be quite close to the data. And according to Mark Pearson and colleagues, realist approaches are grounded in the realist philosophy of science which holds that it is possible to discern generative mechanisms but they are thought about within the social systems in which they operate. So we're not looking at these very sort of pure positivist ideals about what works and thinking about social systems often as things which bias the findings rather than thinking about them as potentially very important aspects which may be the reason or the way in which something becomes successful or not successful. So it does have a different sort of epistemological and ontological background. So having said that realist evaluation and realist review is thinking about complex interventions, just some definitions around what complex means and these are taken from the MRC report about what complex interventions are in the context of health. So these are mostly sort of public health type interventions that have been considered in this way for the health background. So this is thinking about the fact that there may be interacting components both in the intervention and control groups there might be a number of different behaviours that need to be changed or impacted on by an intervention and those behaviours might be both those who are receiving an intervention but also those who are delivering an intervention. It may also be that there's lots of different groups who are targeted. So for example in a big obesity programme there might be lots of different types of activities that go on under that activity which might include trying to influence schools behaviour, the behaviour of councils or local planning people, it might be targeting children and adults, people who are obese and people who are not and those types of complexities in an intervention soon add up to very different things. It may also be that there's a lot of different outcomes which you're hoping to influence and that they may be very variable as well and also that there may need to be a degree of flexibility or tailoring in the interventions to ensure that they work. So I'm sure all these aspects are very familiar for people who are thinking about conservation and environmental management where the interventions that you're looking at are almost invariably complex and so realist approaches are potentially very useful. So they all really came about trying to what we call unpack the black box where there may be very long and complex chains of influence so where A doing an intervention leads to not just C, D and E but maybe X, Y and Z further down the line that these are very complicated and that in the past we've been quite bad at articulating exactly what is going on in the black box where the miracle occurs. So we're hoping to unpack the black box, we're thinking about chains of causation which may not be linear, may be very long, may have feedback loops within them and importantly we're thinking about interventions where people are what makes something successful or not successful and those inevitably become complex interventions and also that these may be very highly context dependent. So one way of expressing this is to say that the causal relationship between two events can only be inferred where underlying mechanisms of action and the context in which that mechanism occurs is fully understood. And in realist terms this is the way that that's often thought about that it's the mechanism of action within a specific context that leads to a particular outcome. So the implications of that are that the same outcome might be achieved through a different mechanism of action in a different context that they may not be the same in all circumstances or that the mechanism which is used may have a different expression in different contexts to make it happen. And you sometimes hear these referred to as CMO configurations. The language in realist evaluation review is quite specialist and I think people find it quite off-putting. So the context mechanism outcome configurations are one way that people talk about realist reviews and realist evaluations. So the types of questions that realist review aims to understand and again this is a very common expression or set of words that you hear in relation to realist reviews. It's not what works like it is in an effective review. It's explicitly what is it about this program that works for whom and in what circumstances. So it's focusing more on how and why things work or fail and trying to understand how those things come about. So this is a diagram which tries to capture that. So a program intervention may change the context or the mechanisms which changes the interaction between mechanisms and outcomes and produces a set of outcomes. And the other thing that realist reviews try to do is to think about both intended and unintended consequences. So one of the problems with complex interventions is that they sometimes do things that you were hoping they wouldn't or hadn't even thought they might do. And thanks to Andrew Booth for this slide. In terms of what the processes are, this slide shows it's called a traditional Cochran review but any traditional quantitative systematic review has these steps. And realist reviews also have a fairly structured set of activities that go along with them, but they may be different to what you're used to in a standard systematic review and they tend to be a lot more iterative. That original slide that I showed you with the configuring versus aggregating systematic reviews, it said for a traditional review the interpretation is done before the protocol is written and may be done after the analysis has been done but it doesn't happen through the synthesis whereas for a realist review these are iterative processes so that the interpretation may be happening as you go along. So that means that the early stages are often about trying to define your terms, trying to understand what the correct question is for a particular topic and to articulate candidate theories and theories are just explanations here. Again the searching may be much more iterative and inclusion criteria might change through the process as you realise that things that you didn't know were going to matter do matter and things you thought might be important maybe don't have the kind of information that you want. And you don't necessarily appraise the quality of studies in traditional ways either because it's a very pragmatic way of thinking about quality so studies might be included in the review because they are providing a particularly relevant contribution to the theory development and so it may not be the traditional ways of quality assessing. Similarly extracted data may not be the same from all papers which again would not be normal in a traditional systematic review. So the data is synthesised to achieve a refinement of programme theory and the programme theory is to determine what works for whom and what circumstances. So again it's a very precise definition about what you do and what your outcomes are but may be different to other ways of doing it. The other things I think are probably more similar dissemination and recommendations in the same throughout but it does have a sort of different focus. So these are the key steps terrifying the scope searching for evidence appraising and extracting data synthesising evidence and disseminating but again they may go back and forth you may do some stages several times and refine them through the process. I won't go through this in detail but I can share these slides. Again this is just another way of thinking about what questions are being asked at different stages of the review and you'll see that it has a focus on defining what things are what's the nature and content of the intervention what are policies trying to do for example and the searching is all about trying to develop programme theory so it's all about trying to think about explanations for how things work in particular situations. And I've taken this from a paper by Rycroff Malone and all. So one of the other ways of thinking about what's going on is to realise that interventions and policies are theories and that's a core informing belief if you like of realist review so that if we propose doing something even if it's not very well articulated and there isn't a programme theory or there isn't a logic model or there isn't even maybe a textual explanation about how we think something's going to happen then are always implicit if not explicit theories behind interventions we're assuming that if we do this then something else will happen as a result of it. So I'm just going to very briefly present for the last few minutes on a realist review that I've been working on here with colleagues in the UK which is looking at social prescribing and social prescribing is becoming increasingly popular in the UK, it doesn't have a particularly well-developed evidence base and the idea is that within primary care for some patients the appropriate prescription is not necessarily a drug or health technology that it may actually be something social so for people who have for example ongoing mental health problems or have stress that what they might need is encouragement to join social groups who are doing artistic or physical activity activities or involved in charity work or whatever and this has been taken up with great enthusiasm because the NHS is running out of money and without very many ideas about how these might work and one of the things we said was that actually the first bit of that process is completely not understood nobody said what the service referral needs to look like so it doesn't matter how effective your knitting group or your physical activity group is if you don't actually get transferred from primary care into the thing so we wanted to use realist approaches to try and explore and explain different methods of referral into social prescribing and say why they may or may not work so we were primarily trying to develop theories about how we might best support people into social prescribing so we did a lot of searching but in two main phases so the first one was to identify different processes for social prescribing and to use these to develop program theories and by program theories we developed a series of if then statements as a way of expressing those and then once we developed the if then statements we asked our advisory group to prioritise the list of if then statements and we then did targeted searches to try and find evidence which would support or not support the if then statements that we were developing so we had a broad set and then a very targeted set of searches overall we extracted information from 109 papers of which quite a small proportion were kept conceptually rich and developed a lot of our theory so the others perhaps just offered a little bit of supporting information we then used these to develop program theory which were expressed through these if then statements and we identified that there were three key stages of referral how you get how you enrol people, how you engage somebody in an activity and how you keep them going so we used our advisory group to prioritise which statements we should try and develop further and as an example these are the sorts of if then statements which we've come up with which propose program theory for how these activities are most likely to work so for example the first one if the patient believes the social prescription will do them good then they will be receptive to a referral and we further developed that with a text in green to show what influences whether or not the patient does believe these things some of which may be modifiable within the consultation or beyond so these are our kind of program theories and we use that to develop a kind of overarching diagram which showed at what stage different aspects need to be in place in order to try and ensure that people were transferred from primary care into the community and this is our main output from the realist review so I'll wrap up but just to say that Ramesses is a project which is trying to develop and support both realist and metanarrative reviews there's a project page there's a really active gysg mail discussion list which the address is there called Ramesses and they're incredibly supportive of you if you are doing a realist review and you're struggling and you want to know what should I read about this, what should I do about the other it's a very responsive discussion list they've also published the Ramesses groups and reporting guidelines for realist reviews which again are very helpful for conducting them as well as for reporting them so I really recommend if you're doing a realist review or just like to know more to get yourself on the gysg mail discussion list and that's all for me