 Hey everyone it's Veronica Howard. Remember visual analysis is one of the most important skills in our field. So when I'm saying that I really encourage you to get an opportunity to walk through a lot of these to find as many chances you can to actually practice this this skill. I really mean that this this makes or breaks your ability to understand causality. So make sure you're having you're taking an opportunity to walk through some of these. Now what I want to do for you is to go through the process of visual analysis step by step step by step. Unfortunately since this is visual analysis what I'm going to try to aim for here is is parsimony. I'm going to try to move through this quickly. If you are a person who would like some assistance with the visual portion, if you need an adaptive version of this presentation please let me know. But for the most part I'm going to move through quickly the actual data themselves and I'm going to show you for a person with no visual impairment how to quickly move through this. Remember if you need some accommodation no worry just let me know. So here's a very simple graph. In visual analysis I'm answering four different questions. Are the data divided? Are they stable? Am I convinced by the data and can I determine if I have a causal relationship? So starting here with divided I'm going to begin here. What I'm looking for is the level of overlap between the conditions. When visually analyzing look at only the last three data points per condition. So I don't really care about these for the purposes of visual analysis and I don't care about these. I'm only going to focus on these points and these points for the time being. Okay so just really quickly. Divided means is there any overlap? I'm going to compare the baseline condition to the treatment condition and then if there was a reversal condition I would compare the treatment condition to the reversal but there is no reversal condition. So for now I just look at okay do I see any overlap here and we see these points three two three do not overlap they do not overlap with these three points which are like six eight and nine. So because there's no overlap here I would say that yes the data are divided there's no overlap. All right let me clean up a little bit. Next question is stability. What I'm looking at with stability is just a question of are the behavior is the data is kind of staying low or low data staying low or high data staying high. I want to make sure that the baseline is not getting better it's not improving and I want to make sure that the treatment is maintaining. Okay remember I look at only the last three points of each condition so I don't really care about these I don't really care about these. What I'm going to do is draw a line from the first point through the third point to make a kind of arrow from the first point through the third point again to show me this is my trend of my data and are these low data staying low yes are these high data staying high yes so I can say the data are stable. Now let me just clarify here when I say high data what I mean is my treatment condition here these data are high relative to baseline if I had a condition where my baseline had values of like 60 and 70 and 80 this value of 10 and 8 and so on and so forth those would actually be low but these data are high relative to baseline. Okay so we talked about divided do we make sure that there's no overlap we talked about stable the data aren't getting better on their own and the treatment effects are staying robust and they're staying high that one those steps are done when we're moving down here to convincing all I'm looking at is where the data divided and where they stable these if these are both yes then my data is convincing if my data are not stable but they're divided or if they're not divided but they are stable then my data would not be convincing in this case because my answers to divided and stable are both yes then I can say my my data are also convincing. Now finally do I have causality and with causality I need a particular kind of design what I need to have are convincing data and I need to have a strong design in this particular case what I have is a comparison design and unfortunately this design is weak so in this case I cannot say that my intervention was the cause of the behavior change the reason that I can't is that I don't know what else could have happened at that point anything could have happened at that point to result in the the increase in the number of cases that we see here so because I only have a comparison design without a strong design like a reversal or a multiple baseline design or a changing criterion design if I don't have a strong design I cannot say that this was the cause of the behavior change no matter how good because look these data are amazing but no matter how amazing the data I cannot determine causality because I don't have a strong design okay let's do another one so what we have here this is a reversal design so that's good because we know it's a strong design and I'm going to look at the y-axis this is our dependent variable number of workouts across time weeks and what I'm going to look at here let's start with divided I'm looking for overlap but I'm checking the last three three points of each condition this means I don't care about this one I don't care about that first point I don't care about this first point I don't care about that first point now what I mean is I do care I mean I love my data I love all my data but for the purposes of visual analysis in the Miller method he says focus on the last three because the last three show you what the behavior was like after treatment had a chance to take effect so focus on the last three and look for overlap look to see whether there's any overlap between data in adjacent conditions do any of these data overlap with these the answer is no there's no overlap there which is good that means those are those are divided okay so I'm going to do the same thing but this time I'm only going to compare this condition the treatment condition doctor reminder I'm only going to compare that to the no reminder condition I'm going to check for overlap here is there any overlap between those data and these data down here no so this is good it means my data are divided so let's talk about stability let me clean up my graph here for a second okay let's talk about stability stability remember means that low data are staying low high data are staying high low data are staying low so I draw my line from the first through the third these are very stable I draw my line from the first through the third we see a kind of downward trend here draw my line from the first through the third nope that's not the way that the arrow goes okay so what we see when we compare baseline to treatment treatment to reversal we see in the baseline the data do not look like they're getting better on their own which is great these low data are staying low I just want to make sure that we don't see any kind of like creep like that that would be terrible so this data great that stable these data here in the treatment condition are these high data staying high right so where are they going it looks like they're just getting better so yes that means that these data are stable what about in this next baseline condition are these low data are they staying low yes so we have low data staying low in baseline we have high data staying high in treatment we have low data staying low in treatment oh no let me go back low data staying low in this reversal condition which means our data are stable if I need to when I'm determining whether I have convincing data I just check to see whether my data are divided and stable and so in this case we had both divided and stable data which means that they are convincing and for causality this is where I need to know that I have a reversal design because if I have strong data right if I or excuse me I have a strong design and I have good data that are convincing then I can say yes in this case I have convincing data and I have a strong design which means yes we have a functional relation here between a doctor sending a reminder and the number of workouts that our client actually engaged in let's do another one so let's talk about we don't know what our actual interventions are here but we're very fortunate because we see we have another reversal design and we have the dependent measure number of work hours we have the time number of weeks so we're looking at number of work hours across time so let's talk about divided remember we're focusing on the last three points of each condition so I don't really care too much about these don't really care too much about these don't really care too much about these I only want to look at the last three points divided we check and see is there any overlap between this baseline condition and the treatment condition and it does look like there's overlap which is bad because if there's overlap what that means is we didn't actually produce much of a meaningful effect so in this particular case we see this overlap there's not divided and it kind of doesn't even matter but just for the sake of argument let me clean up that box let's do divided here there's overlap between these conditions even if it's just a single point there's overlap so these data are not divided and that's problematic let me clean up here just a little bit remember we we do want to keep going because I want to make sure that you're doing all of them stability refers to the extent to which our treatment remains permanent our baseline is not improving on its own so draw the first through the third we draw our stability line this way this is our trend we go from the first through the third that way draw from the first through the third now this one's kind of difficult because which one's low which one is high that's its own issue I think in this case we see that these data are relatively lower and this baseline does not appear to be improving on its own these data are relatively higher which is good because high data here are staying high and these data are relatively lower just like the baseline and these low data are staying low so I can say that the data are stable but because the data are not divided and they're stable they're not convincing if even one of these if one of them is wrong like divided is not our data are not divided we have to conclude that our data are not convincing which then gives you kind of a leg up here because remember causality means convincing data plus strong design we had a no in convincing which then also gives us a no for causality because no effect we didn't have a large effect size our data were not divided which means they're not convincing which means I cannot determine that it was my treatment and only my treatment that caused that change let's get into something a little bit more advanced here so here's another reversal design what we're doing here is we've got a kind of switch here where our baseline is higher than our treatment and I'm going to do the same process remember I only care about the last three points in each condition so I'm going to ignore anything else don't worry about these only look at these only look at the last three points so division we're checking that box we're going to see is there any overlap between baseline and treatment between treatment and reversal and we see there's no overlap here there's no overlap here which means our data are divided let me clean up let's talk about stability shall we so we we ask do we have high data staying high and so we draw the line from the first through the third we see these high data are staying high our low data staying low we draw the line from the first through the third and I'm doing this freehand so you'll have to forgive me because the lines are not perfectly straight in this case our data are stable our data are stable now let's look at this reversal condition remember we're going to ignore these first two we draw the trend line from the first through the third we draw this line this way now here's the problem relative to the treatment condition these data are higher data right so when you compare these data to the ones next door these are higher data the problem is that they trend like they're going back towards the treatment condition which means that we see an improvement on its own it means that there's a potential confound going on here so we might have something going on here we say the data are not stable because higher data are not staying high we don't have stable data this makes it actually a little bit easier for us because remember for convincing data they have to be both divided and stable and in this case they are not stable which means they're not convincing and even though we have a reversal design here we have weak data we have not convincing data which means it's not the cause of the change so let's move on to multiple baseline designs these are kind of challenging for folks because with the reversal designs we were comparing one condition to the next condition to the next condition but with a multiple baseline design you actually do these steps twice so i'm going to ask for the top graph are the data divided and i'm checking for overlap here and i'm free handing it but it looks like there's no overlap in the last three points of this condition in the last three overlap or last three points of this condition so these data are divided right we say yes for the first one and the next one we check for overlap and this one's pretty clear there's no overlap here so these are also divided when we talk about convincing we want to make sure that our baseline the behavior is not getting better on its own that the treatment effects remain in place this one's challenging so draw from the first through the third of the last three points to show you your trend line first through the third of the last three points to show you your trend line first through the third show you your trend line first through the third to show you your trend line we have data that are relatively higher than the adjacent condition and we have data that are relatively lower so we want these high data to stay high and they are. So that's good. And we want these low data to stay low so they're not creeping back up towards baseline. So these data are stable, which is great. Let's look down here. Again, we have relatively higher data. We want it to stay high. We don't want to see the behavior getting lower or better on its own. But let's look at this condition. We have lower data. And we want to make sure these data are staying low, that they're not creeping back up towards the baseline level. And unfortunately, these are not stable. So data are not stable here. Even though we have data that are divided, because this is divided from this and this is divided from this, our data are not stable, which is problematic. So we have unstable data, which means we have data that are not convincing. Now, can we then determine causality? The multiple baseline design, this is a multiple baseline design. Let me clean this up so I can show you. This is a multiple baseline design. And I can tell you that because let's check the number of points here we've got in baseline. Five points here. And we've got one, two, three, four, five, six, seven, eight points here. For a multiple baseline design, we need to have three or more points in one baseline so that we can show that the behavior changes when and only when treatment goes into effect. Behavior reduces and then behavior remains unchanged here. So you have to have the start of treatment separated by three or more points, according to Miller. And this is a multiple baseline design, which is a strong design. And we love that. But we have unconvincing data. So this is not the cause of the behavior change. The problem is here in this condition, because it kind of looks like there's a possibility it'll creep back up. Let's do one last observation. Let's do one last graph. Okay. Behavior, behavior, time, baseline treatment. See, we're getting very nondescript here. This is just a practice graph. So let's check this out. In the baseline condition, we're going to do divided first. So let's look here to see whether we've got overlap. I don't care about any points except the last three points. Then I'm going to check to see is there any overlap between these. Remember, I'm looking at only these last three. I'm only looking at these last three. We see there's no overlap here, which is good. That means it's divided. I don't care about any points except these last three. I only care about these last three. Is there any overlap between these points and these points? And there's no overlap, which is good. These are both good. It means the data are divided. Let's look at stability. Stable means that the baseline data isn't getting better on its own. It means that treatment is staying robust and sound. Okay, so these data are lower and these data are lower than the treatment condition, which is higher. That's higher. These data are higher than the baseline condition. So what I want to see is do low data stay low in each of the baseline conditions to high data stay high in each of those. Remember, I look at only the last three points. I draw my trend line from the first through the third from the first through the third from the first through the third. So trend line, trend line, trend line, trend line. And let's go, I'm going to go from the bottom. These lower data look like they're going to stay low. They look like they're going this way, which is good. It means they're not trending towards data in the adjacent condition. These data look like they are totally stable, perfectly horizontal. So high data are staying high. They're not creeping down. In the top graph in the treatment condition, again, these high data are staying high, which is great. They don't look like they're creeping back down towards that lower data condition in the baseline. But here's the problem. These low data are not staying low. It looks like, let me clean this up a little bit. It kind of looks like from our trend line that if we had done nothing, it's perfectly reasonable to guess that the behavior would have just done this naturally. I can take out this condition phase, this condition change line there, and you wouldn't even be able to tell that it wasn't all part of the same data set. So these data are not stable because the baseline is trending up towards treatment. Not stable means they're not convincing. And even though this is a strong design, because we have one, two, three, four, five, five more points in this baseline. So this is a multiple baseline design. Even though we have this beautiful multiple baseline design, we cannot determine its causality because our data are not convincing. Again, this is a really challenging skill. So if you need any additional help with this, make sure to check us out, visit us during office hours, check in with your peer learning assistant. Otherwise, if you have any questions, please let me know.