 Thank you It's a it's a great honor for me to be here and addressing you and Malcolm's left me a little nervous about proceeding But let's get on with it. Um, so, um, the Principle question I'm interested in is can we have evidence good evidence for causation in the single case the answer I want to defend is yes, and then a second. Do we need to establish a counterfactual to do so? No Now I realize that this is a rather ecumenical group and it may be that I'm going to be Preaching to the converted, but I'm still going to proceed to defend these two two claims So I'm going to talk about I've constructed this Chart of categories of evidence for the single case. I'm going to talk about it We don't need to understand it yet. Just that's what the focus is going to be the point about the Categories which are categories of evidence types for the single case is that they're all things you would take for granted as good evidence for single case causality They're familiar they're tried and true methods of evidence in singular causation We see them in legal proceedings all the time Which is the primary case where I think we see a lot of causal attribution are in a single case We make we use these in daily life all the time when I try to figure out who left the dirty dishes in the sink It's I use these kind this kind of evidence when I accuse my granddaughter Lucy of having done so I know she says why might what why do you think that's right? Well? I have categories of evidence that I can produce We do this and you do it in case studies and causal process tracing and lots of we Lots of other places Nevertheless, we were told that these categories these can't this kind of evidence can't do the job It can't warrant single case causality claims Why? well Skeptics and especially the people I've been engaged with recently Randomized controlled trial advocates the skeptics say you must establish a counterfactual to establish a singular causal claim So here's Ian Chalmers one of the big gurus of evidence-based medicine the need to compare like with like and fair test It's been recognized by some people for a long time Ian Chalmers doesn't like my Worries about RCTs because he really thinks that you do have to establish a counterfactual or here closer to home This is King Kohane and Verba Nothing can be learned about the causes of the dependent variable without taking into account other instances when the independent variable takes on other values And that lots of I think you're familiar with this view, okay? so Skeptics say you must establish a counterfactual to establish a singular causal claim Then they say and you can't do that Okay, so I aim to the contrary To show that these non-contrafactual evidence types are evidence for singular causal claim I think it's obvious that they are but then I've got engaged with these people who claim you know what they're doing is rigorous and That there's nothing rigorous about this other stuff. Okay, so So I'm going to use their own trick I'm going to use the same device that shows that randomized controlled trials can make causal claims Which is a potential outcomes equation or what I've generalized this to to something I call a Situation specific causal equation model, okay, so I'm going to talk about that But it's really the potential outcome equation that justifies That RCTs make causal claims and I'm going to use potential outcomes to show that these evidence types are good evidence for causation in the single case, okay So here's the the outline what I'm going to do evidence types first. I'm going to explain what a potential outcome equation is I'm going to then generalize these to what this scam the Situation specific causal equation model I'm going to show how these evidence types fill in the scam from which the claim derives So that's going to show why that they are really evidence for the single case I mean something we take for granted but that seems to need showing and but before that Why bother? Well, one thing is for post-hoc evaluation attributions of blame and reward we Want to be able to establish singular causal claims? Moreover, I think more importantly or I'm sorry equally importantly many of these evidence types double for X anti-prediction when there are reasons to think ahead of time that your cause will produce the targeted effect in this very situation Thirdly singular causal claims can corroborate or probe general causal claims Oops, okay That I think people Many people will begin to bulk out. How can a single case corroborate a general causal claim? We know Putatively that we can't generalize from a single case Well, yes, that's true, but You can't generalize from any cases You can't generalize from 10 or a thousand or even 40,000 we all know this right there are more than 2,000 swans in the Thames They're all white In the right season there are more than 40,000 swans in the UK. They're all white Still in Sydney Harbor as we know Just not all swans are white, okay, so I just want to make a remark about generalizability I think it's a really bad idea that we're going to generalize by just you know induction which is what the generalization generally means Generalizability depends on projectability it depends on having got the right concept that will hold across the class of objects you're interested in Does not depend on sample size So here's an experiment that I was a participant observer on at Stanford when I was there 20 years they designed this experiment to put some gyroscopes in the space They would preset if they cut if there was space time coverage or the gyroscopes should couple with them and process very very infinitesimal procession that had the measure and They so they were testing the general theory of relativity using these gyroscopes And they drew conclusions from three gyroscopes That's because they knew that the kind of thing they were studying was projectable I mean if it worked if these three gyroscopes possessed and coupled with space-time curvature, you know, they all would Moreover, okay, well over the much-vaunted RCTs if we want to look at it the other way around An RCT can only provide evidence that the treatment caused the effect for some individuals So what the RCT is actually doing if it's establishing something causal It's not establishing a causal generality. It's a call. It's establishing that to the extent that you could trust the results that The treatment caused the effect in some individuals in the somewhere in the study population So that's really is an indirect way of establishing some some Singular causal claims So they establish singular causal claims in an RCT, but it's kind of odd because We know some people now some individuals in the study population The causal claim is true for although we just don't know for whom so it seems to me they're worse than You know just dealing directly with it. So I call randomized controlled trials when I'm not being very nice to them Where's walley studies? Okay, so now here are my categories of evidence. I think there are more that to fill in But I'm going to talk about them very very quickly because there will be familiar so evidence This is part one now on the categories evidence that C caused the in an individual eye There's direct evidence that looks at aspects of the putative causal relationship itself to see if it holds an Indirect that looks at features outside that that bear on the existence of this causal relationship These aren't distinctions written in stone right these are way of organizing the material I've isolated five kinds of direct evidence Many of these are familiar from Bradford Hill, you know who did this for medicine One of the things I'm interested in is I think that this skim business Shows that some of those really are symptoms of causality and some of the things he suggests I don't think our symptoms of causality. There are symptoms that you didn't make a measurement error But so it is nothing useful. So the character of the cause did see occur The cause occur at the time in the matter of the saws to be expected had it caused the Character the effect does it occur at the time in the manner of the saws to be expected had C caused it I think you sometimes call these two together congruence. I've seen that in a lot of the social science literature Presence of support factors Moderator variables was everything in place needed that was needed for C to cause E Presence of intermediate steps Mediator variables were the right kinds of intermediate stages present and the operation not just the presence of the intermediaries But you know that could accidentally have been present that they actually operate where they really caused by C and did they really cause E indirect evidence causal potency is see the kind of thing that could cause E Elimination of alternatives, that's a very famous one What else could have produced E in this individual specific situation and what evidence is there against them? Symptoms that the cause operated not just that it was present, but it really operated what further features should hold if the cause had Acted as needed and absence of derailers were there features that could stop C from resulting in E and are they absent? So that's a kind of that's what's in my catalogue and As I said, I think these are kind of familiar categories. I hope that by talking to you You'll be able to add some more and we can figure out whether they fit the schemes seem or not And if not, does it need revision or? Etc. Okay, so what these are I think tried and true categories of Evidencing for evidencing singular causation the singular case. Let's go on towards potential outcome equations And I start with something I think you'd be familiar with is epidemiologists pies that were introduced by Rotman Where he said and we I saw this in the session this morning that QCA session The idea is there's more than one way to skin a cat So there are very different pies that each one of which is sufficient to contribute to the effect but then Each pies itself made up of a number of factors that have to be there Present together in order to bring about or make a contribution to the effect So here's an example from one of my colleagues in North Carolina on homework Where homework can improve certain kinds of outcome learning outcomes for students, but It needs a lot of other stuff to be present to for it to work now The epidemiologists are the only people who were thinking about Causal pies at the time a couple years before that J. The philosopher J. L. Mackey wrote the cement of the universe and He said exactly the same thing in philosophers language Which is that? Causes singular causes are insufficient but necessary parts of unnecessary but sufficient conditions are sufficient condition is pie right a pie is sufficient for an effect but not necessary it's unnecessary for it and Then the parts of the pie are necessary in that pie But they're insufficient by themselves to bring about the effect. So exactly the same thing as the causal pies And Mackey wrote it using Boolean logic and this looks just to me like QCA that the effect here X on the left and X happens if the first conglomerate Which is a 1 and x 1 or the last one a n minus 1 and x n minus 1 I've divided them in those two categories because he was interested in that we tended to focus on one feature like the homework and So that gets labeled the X but there's no difference between the homework and the other factors that help the homework work You could think the homework is helping the other factors work at least, you know so far as the What's been done so far? Okay, so that's That was what Mackey did. He's doing yes. No variables So really talks about it being sufficient of causal pie is sufficient for the effect to occur We tend to think in terms of more continuous variables so The idea here is that the causes different different Pies can contribute and they might all add up to the final effect, so this is just generalizing from Mackey and Rotman to This equation where the alpha is a constant and the others are variables so before that sorry and The when I talked about moderator variables earlier That's what what's represented by the a's I call those support variables They I mean if you've got a salient cause you're interested in then Mackey called them auxiliaries and I'm calling them support factors. So that's what I was referring to earlier, okay? Okay, so I Want to put a little red C in front of that? All right because and in all these cases What's supposed to be happening is that we've got an effect on the left and These pies on the right that are sufficient for the effect But it's not just that they're sufficient to guarantee that the effect occurs which could be spurious correlation They just co-occur together. It's supposed to be that the stuff on the right is the cause of the effect on the left so And the idea is That we're dealing with cases where in the real world right there this situation there are a set of causal possibilities of And what the formula represents is it represents all the things that are present that? you know could contribute to the effect in this very situation and Sometimes some of them will be actualized in this situation and those will be the real causes So now potential outcomes equations potential outcomes equations are just those Mackie formulae and We use potential outcomes equations to justify what's going on in a randomized control trial and all that's happened is we've taken our our Mackie formula and we've taken our effect that we were interested in and called it why and then we've been only interested in one particular term here one particular cause say the treatment and Then we've lumped all the other stuff out in the w so x is the focal cause W is the contribution to y of all the other causal pies and beta is the net effect of the support factors So that's what you a potential outcome equation looks like and I see it all the time in Discussions of randomized control trials when people are actually talking about doing the statistics on them Okay So what justifies that randomized control trials estimate treatment effects? Well, I'm not going to bother to give you the little argument here, but you start out To do this you start out with a potential outcomes equation and notice it's supposed to be causal I'm we're supposed to have the causes on the right Okay, and then um you argue if certain conditions are satisfied which the design of the experiment is supposed to help You were sure you are satisfied, but if these conditions are satisfied then All right, you can start with the potential outcome equation And you can show that the difference in the observed mean outcomes is an unbiased estimate of the study population average treatment effect and So that derivation my only point here is that derivation starts with a potential outcome equation and it ends with a Justification that if it were an ideal RCT where the assumptions were really satisfied, which they never are You would have an unbiased estimate of the study population eight Okay So now I come to scams which is by part three um I Called them here structural with a really situation specific causal equation models And I'm not quite sure how I managed to slip in the word structural I think I was talking to some economist And I just I don't know type what was in my mind rather than what I ought to have been saying So situation specific causal equation models um what they are they look like this and you can see that What what they really are is a Generalization of the potential outcome equation But for just different outcomes that might occur between cause and effect or before the cause or after the effect So the scam it supposes that you know, we're not just looking at our cause I'm sorry a bunch of causes and an effect and then this particular cause of interest and the effect We're not lumping all the causes together. We're going to disaggregate. We're going to do a time slicing And then we're going to Assume just for the notational thing that the variables are time ordered, which is why since these are causal equations we have the triangular form because You know nothing later causes anything earlier Each equation gives the direct causes relative to the time slicing Where's the potential outcome equation Just lumps together a bunch of sufficient causes. They don't go through. I don't don't have a they're not necessarily direct causes And then our outcome the one we're interested in I'm going to call it E Could be anywhere could be say x5 and our cause of interest Could also be anywhere could be x2 Now the point is just that you know that the economists do that potential outcome equation stuff and The scam is just a generalization of that but it's richer than the single potential outcome equation because it represents causes of causes in an intermediate steps and then further effects and that Being having a richer representation can actually be very informative. So just for example Think about further effects so here's a Mention in process tracing the existence of official minutes of a meeting as we know if authentic provide strong proof That a meeting took place. So what we're doing there is we're looking at effects of say the effect We were interested in whether the effect was did a meeting take place We're actually not looking at intermediate steps did the cause produce the meeting, but did it okay? So so now here are my categories of evidence for C caused E and Situation in individual eye and situation this very specific targeted situation You'll notice that I've just Repeated it and it could repeat and repeat because if you're looking at If you're looking for instance whether the Moderators operated, how would you know whether the moderators? Sorry the mediators did their job Well, you could again, that's a causal connection Did this mean you know did the cause cause the next step and the next step cause the next step and the next step Cause the next step so each of those is a causal connection that itself can have Direct and indirect evidence so this chart can just keep going on Okay now what I I've been very quick 42 slides and I'm going to finish them in half an hour Justifying these as evidence so the point of the scam is That each of these types of evidence Helps establish something about a feature in the scam relevant to the existence of a causal pathway from C to E So this is where the we're getting the real derivation business and I can do that for Every one of those evidence types, but I'm just going to give you a kind of example, and I'm sorry I don't have a real example. I was looking for one and they were also controversial I thought you know as soon as I give this example people are going to say well, you know You haven't got quite the kind of the example quite right But say look at cause characteristics and effect characteristics and the congruence between them We we see this in the size relations, I mean so we're actually claiming in the scam You know you write down your causal model for the case and you're making You're making hypothesis about Not only that x1 causes x2 if if you're right if that a if that a I mean I have left the cause I've left the constants out I mean if it's actually there actually we're claiming that x1 caused x2 and That they have a certain size relation I mean you could do these a bit more qualitatively, but the point is that the scam actually posits What the kind of relation is supposed to be between them? and if you notice that you've got the right size effect relationship what you're doing is you're fixing that equality sign there or Time right are they they cut the right times well if you've done your time slicing you can see whether or not relative to your hypothesized model the Effect is occurring too soon too late, so some of my friends who do who are worried about randomized controlled trials and Said talking worry about them not being appropriate for talking therapies And are sort of worried about CBT claim that one of the problems they think with the evidence for the effectiveness of CBT's is that sometimes an awful lot of the cases the The improvement has come either too soon or too late Given the CBT theory itself Now whether that's true or not, but it's the kind of thing that you see you know would be at work here Or we would say Malcolm had CBT and well he got a whole lot better after we went to his first session. Well, that's that's Really not evidence that the CBT was responsible for Malcolm's improvement. Okay, so time the timing Comes from time slicing and the indexing of the X's What about the operation of mediators? Well, I think that's pretty obvious so Imagine that X1 is our target cause and then mediator we've hypothesized in our model. I mean this is We're making a little bit use of a logic model here or theory of change X1 causes X2 and X2 causes X3 and X3 causes X4 so if we have Evidence of the intermediaries actually occurred of the size they were supposed to that gives us as we expected right not surprisingly evidence that the initial cause produced its effect, but the point is that We see why it's evidence using a potential outcomes equation framework just like the RCTs don't do Elimination of alternatives well Here's all the pies that could bring about the effect and The question is do Or the other factors that could be doing it sufficient to do it. So do all of these guys add up to that so again the Elimination of alternatives, which is a standard and obvious way to evidence Any kind of a claim is fits into the skim model very nicely so Schems play a vital role in evidencing singular causal claims. Maybe you can do it some other way but I wanted to find a way that We know where I had a rigorous proof that these are actually relevant types of evidence They provide the framework For a principal derivation that RCTs first you know the schemes are what you use in the RCT derivation They provide a principal derivation that RCTs do what's claimed for them and evidencing their causal claims that say somebody in the treatment group up better somebody we know not whom and they Help us justify that features from our catalogue evidence Really do evidence singular causal connections as we actually knew all along So when we're thinking about scams, I think they have a Lot going for them. They provide a rigorous justification that our evidence types are evidence for singular causal claims And how I mean you can begin to see how they're evidence Now that's I think important because they thereby provide a way of Systematizing the evidence and you kept you've got all these things that you that in daily life say in a trial We're used to collecting together and then we have odd bits here and there and If we have our the catalogue of evidence and then we connect it with a hypothesized causal model We can see exactly where each piece of evidence fits and then we can begin to question You know How to put the evidence together for instance you can see let's see if I have this written down so it helps us assess the strength of the evidence because you can make explicit and And so you can see What's being assumed right so the first thing is you it's like writing down your logic model only you've got a very extended logic model That actually has the moderator variables and causes of causes and effects of effects You can see and make you make explicit and hence can see just what's assumed And then you can see what rule each piece of evidence plays in which piece what is it doing? What's it teaching us about the scam? Does it teaches enough that we feel confident that? We you can make the inference from start to finish and it tells you what's missing and all of that's necessary to deciding how? How strong your case is so I mean do you have a vital piece missing an intermediary missing? Well, how troubling should that be but when you've got the scam You've got you're in a very good position to to see exactly what's going on and Okay, now if you don't like them There's some easy things that we cook to to to criticize them for The one thing I didn't write down But you can't use them to estimate an effect size because after all you're just doing a single case I mean you can actually use them to estimate How much I mean if you had a really good causal model that you felt fairly confident about you you could actually use it to estimate How much the cause? Contributed to the effect in this case As long as you had relatively good evidence about the other The other pies which one often doesn't but so you Depending on how much evidence you have you can estimate The size of the effect in a single case Okay, which is the analog of getting the average treatment effect and of course it's better It's harder to get because you need you need to have good luck with the kind of evidence you can get but you've got Evidence really about you a particular individual Apart from that There's no There's no way to if you thought that p values were important or you thought that having a likelihood or confidence interval I mean a confidence interval for the single case meant how Confident can we be that the singular causal claim? We've now got a lot of evidence for but then there are gaps in our evidence How confident can we be about the singular causal claim well as far as I can make out There's no formula for estimating a probability like that. I don't mind that Because I'm not a Bayesian and I don't think there are objective probabilities for the single case So I'm not quite sure what one would be doing and estimating a probability, but it would be nice to have some way You know that you didn't have to think and you didn't have to make a bet Be nice if somebody God gave you a formula and said well if you've got these six-piece kinds of evidence It's pretty confident and if you don't but there ain't any such thing But I think that's just life in the social sciences Okay the other major the really major criticism I find of evidencing the single case in The people who don't like doing it is that it's too model heavy. I mean after all I said you've got to produce this model and I'm told that we do RCTs because we don't trust Models right there are too many assumptions that go into them that it cannot be independently verified So we're told they're too model heavy Whereas I'm constantly being told I had a hard time finding if anybody can send me quotes I'm told this all the time in discussion that and with RCTs you do inference by design It's the study design. You don't it's like part of the credibility revolution We don't have to make incredible or unbacked up unwarranted assumptions You it's just the study design that entitles you to the inference But here's one place I found it remark of it a research design is a characterization a design right of the logic that connects the data to the causal inferences it Is essentially an argument as to why someone ought to believe the results in the case of randomized controlled trial There's little room for doubt about the causal effects of the treatment So there's hardly any argument necessary that you know the result is supposed to be carried by design alone We don't need to make the additional assumptions and defend them. Well, no, we don't it's just not true That with an RCT you do inference by design we all know Well, we all know that there's a lot of statistical fiddling that has to happen With problems of drop-out and non-compliance and so forth But the thing I find there's no system in but that I have fine. There's a lot of systematic work on I Want to point out that the systematic work all has to make a substantive assumptions I mean you can't adjust for Non-compliance and drop-out and so forth without making assumptions about the character of the population you're studying But worse than that I feel is things life exists between random assignment and the final reporting of Recording and reporting and analyzing the data There's a ton of time in between and things happen to the treatment group and the control group and to the People administering them and to the statisticians the people who do the measurements, etc Tons of stuff happens post random assignment almost always when I've been reading I've been reading a lot on you know unbiased estimates and do we really what good is an unbiased estimate would we rather a precise estimate than an unbiased estimate but standardly almost everyone says you know the the random assignment Guarantees that you get an unbiased estimate But it doesn't of course it has to be that nothing happens. No confounding happens post randomization and Lots and lots of things happen post randomization we all know Already about quadruple blinding in order to stop some of the causal factors that we know could create a An imbalance in expectation You know we want to blind the the people receiving the treatment you want to blind the people delivering it You want to blind the people making the measurements of the outcomes? You want to blind the statisticians? We know we want to blind the statisticians and we think about the worm wars where the epidemiologists Reanalyze the data from the original Kenyan study and claim they got quite different results than when the economists did it and then the economists and epidemiologists Was it two summers ago had rather violent clashes over in which was the right analysis of the data? Okay, so But apart from that, I mean those are things that are known. How do we know them? Where do they fit? I go to but there are lots of other things that can happen You will know real-life cases that happen in villages or in schools Where things happen post randomization that the treatment group and the control group have some Systematic difference between them that ought not to have happened. I was thinking I go to an eye clinic and I sit in the eye clinic For hours because it's a serious clinic. I have to see a real consultant and it's very busy I sit in a waiting room and I sit next to a machine that dispenses for a two-pound coin Sugary snacks crisps Etc. Now I imagine I'm in the treatment group Because I have to see the real consultant So I'm the one who's sitting there next to the machine for three hours I think the control group right whisks gets whisked in to see another pretend consultant And never sits by them. It's a good thing. It's not a diabetes study. That's my feeling Okay, so things happen post randomization and it really annoys me that The people whom I know who defend randomized control trials and how really much more super they are than a lot of the other methods you use they don't Approach this explicitly. So my feeling is the things happen post randomization. You may be lucky and they didn't But you can handle that casually in piecemeal Which doesn't seem to me goes along with the calls for rigor that we hear You know that rcts are supposed to be so good because they're rigorous So you can either handle that casually in piecemeal or you can build a scam and do it properly You know, you can't actually Patrol for External causes that affect the effect it differentially between the treatment and the control group without having some hypotheses about what they are And what causes them and how you'd know whether they were there or not and that's really what you do in building a scam You know as you build an extended model of what besides the treatment could be That's happening kind of outside that you didn't think of could be affecting the effect So I don't actually think you can justify your rct results without building a scam anyway So conclusions which are very short We can have principled evidence for singular causation and it's and it's Evidence of just the kind that we normally produce in process tracing and case studies And in trials We can have lots of evidence for causation in the single case whether it's know whether you're lucky and you can get enough to feel The matters been settled. That's a That's a question. You mean you can't always get enough evidence But absence of evidence doesn't mean you can conclude that it the Causal hypothesis is false, of course, but you just can't to our conclusion all but you can have lots of evidence of different kinds for singular causation and you can do that without Any thoughts of establishing a counterfactual So you're definitely not stuck with where's walley study You can look at each walley individually Thank you