 Hi there, my name is Wolfgang Fichtbauer from Maastricht University and I will be talking to you today about location scale models for meta-analysis and how you can fit these types of models with a metaphor package. So to get started, let's first talk about a standard meta-analysis and one of its main goals, namely to estimate the size of the average effect. And for this, we typically use a random effects model, which says that for every study, we have an observed effect for each of our case studies. And what we want to estimate here is mu, the average effect, but there are two sources of variance. First of all, there's epsilon sub i. This represents sampling variance. And these variances, these v sub i's, these are known. This represents variance in the estimates around their true effects, while an observed effect is not going to be equal to its true effect because of the sampling variance. And these are heteroscedastic by construction. That is, we have bigger studies, which tend to have smaller sampling variances. And then we have smaller studies with bigger sampling variances. And then we have another variance component here, tau squared. This denotes heterogeneity. This represents the variance in the underlying true effects. And this is assumed to be homoscedastic. That is, it's a constant, it doesn't vary. You can easily fit such a model with a metaphor package, which of course you have to first install if you don't already have it. And let's look at an example. And this is one of my favorite examples, the meta-analysis on the effectiveness of the BCG vaccine. And for each of the studies included in this meta-analysis, we have the number of TB positive and negative cases for the vaccinated and the non-vaccinated groups. And we will use this information to compute log-transformed risk ratios and the corresponding sampling variances. So let's first load the metaphor package, and then we can take a look at the BCG data set, which includes 13 trials. And here we have the number of TB positive and negative cases in the treated or vaccinated group, and the number of positive and negative cases in the control group. And there are a couple other variables included in this data set, one of which, the alloc variable. We will take a look at this in a little bit. Now we can compute the log-risk ratios with the ESCalc function. We have to specify the type of measure, RR stands for log-risk ratios. And we have to give the function the information so it can compute these log-risk ratios. And once you run this, the data set includes two additional variables, Yi, those are the log-risk ratios. And the negative value here represents a lower risk of infection in the vaccinated group. And Vi, those are the sampling variances, and which we can see are sometimes large or sometimes smaller, depending on the size of the studies and the amount of information that is provided by them. Finally, we can fit the random effects model with the rma function. We just give it the effect sizes and the corresponding variances. And here are the results. So we get our estimate of tau squared. We get our estimate of mu, the average effect, and a corresponding test, whether this is significantly different from zero and a corresponding confidence interval. On this slide, you can see a fairly typical force plot showing the results of the individual studies. And at the bottom, the results from the random effects model, namely the estimated average effect and the confidence interval. And since this excludes zero, we know that this is a statistically significant effect and showing that the vaccine is indeed effective on average for reducing the risk against tuberculosis. But there is a fair amount of heterogeneity in these results. We can see this here at the top. This is the estimated distribution of the true effects based on mu hat, the average effect, and tau square representing heterogeneity. So there might be circumstances where the vaccine is even more effective or where it is possibly not even effective at all. To explore such heterogeneity, we can use meta regression, where we use one or multiple moderators or study characteristics to examine if they're related to the size of the effects. To give a simple example, say the studies fall into two groups, the randomized versus non-randomized studies. And we can represent this as a dummy variable, which we code zero for the first group, the non-randomized studies, and one for the second group, the randomized studies. The alloc variable in the dataset actually provides information about the method that was used for assigning participants to either the treatment or the vaccine or the control condition, although it makes a distinction between three different forms of allocation. We will simplify a bit and create this dummy variable by coding this variable as one if the method of allocation was random and zero otherwise. Now we can include this dummy variable in our meta regression model. And we are especially interested in the results down here. The intercept represents the estimated average effect for studies not using random allocation. And the coefficient for random represents the difference in the effect for studies that do use random allocation compared to those who do not. And we can also test if that difference is statistically significant, which is not the case here. Let's ignore this though for the moment and look at this forest plot, which shows the studies ordered by whether they did not or whether they used random allocation. And up top here we now see two distributions, the estimated distribution of true effects for studies that do use random allocation versus those that do not. What we see here is that these two distributions have different means, but they have the same tau square. The amount of heterogeneity within the two subgroups is assumed to be the same. So again, tau square is assumed to be homoscedastic. But this assumption may simply not be true, which brings us to the location scale model. The first part of the model looks pretty much like a standard meta regression model, but the difference is now that tau square has a subscript i, so tau square is allowed to differ across studies. And in fact, what we have now is a model for how tau square or log transform tau square differs across studies, namely as a function of one or multiple study characteristics. So here now we can make a distinction between predictors for the size of the effect. These are so-called location variables and predictors for the amount of heterogeneity. These are so-called scale variables and they may or may not be the same. And of course, again, you can have multiple such location and or scale variables. And I recently extended the metaphor package to also fit those types of models. To illustrate this, let's extend the previous meta regression model by including random, also as a scale variable. So like before, we get information about the estimated average effect for the non-randomized studies and how different the effect is for the randomized studies. But in addition to this, we now get information about the amount of heterogeneity for the non-randomized studies and how different the amount of heterogeneity is for the randomized studies. As noted earlier, heterogeneity is expressed on the log scale for this model. So to get the estimated tau square for the non-randomized studies, we just take the intercept and exponentiate it. So we get around 0.2. And to get the estimated tau square for the randomized studies, we take the intercept plus the coefficient that represents the difference between the randomized and the non-randomized studies. And we exponentiate this and we get about 0.4. So we estimate that the amount of heterogeneity is about twice as large for the randomized studies compared to the non-randomized studies. You can also see this now in the forest plot, where the distribution of true effects for the randomized studies has a different mean compared to the non-randomized studies, but also has a different amount of variance. So this model allows the amount of heterogeneity to depend on one or multiple predictors or scale variables. And so tau square now is allowed to be heteroscedastic. Now it turns out that the model above yields identical results to fitting separate random effect models within the two subgroups, which is what I'm doing down here. I fit a standard random effects model to the subset of studies where random is 0 and where random is 1, then I collect some of the information from these two models and a table. And so we get the estimated mu for the two subgroups and we get the two tau squares and these are identical to the two tau squares that we got from our location scale model. But the location scale model is much more flexible, for example you can include none, one or multiple location and scale variables in the same model. And these variables can also be different. The variables can be categorical, so representing subgroups as we have seen before, or there can be also quantitative variables. And you can also now test if the amount of heterogeneity is related to a scale variable or differs across subgroups, and for this we have several different types of tests, so called wall type tests, likelihood ratio tests and permutation tests. For example, in the model above, alpha 1 represented the difference in heterogeneity between the randomized and the non-randomized studies. So testing whether this is equal to 0 is the same as testing whether tau square for the non-randomized studies is equal to tau square for the randomized studies. On this slide you can see how these different types of tests can be conducted. If you fit a location scale model and you look at the scale part of the output, here is our estimate of alpha 1, then you immediately get the p-value from the wall type test, testing whether this is significantly different from 0. If you want to conduct a likelihood ratio test, you fit another model where you drop the scale variable, so here I just have an intercept, so this is a reduced model, and then I compare these two different models, I get the likelihood ratio test and the corresponding p-value. And finally, if you want to do a permutation test, now that reshuffles the data repeatedly in a certain way, so there is a certain element of randomness which I can make reproducible by setting the seed of the random number generator here, then I get the p-value here in the output from the permutation test. Now in all three cases here, all these tests suggest that there is no significant difference or sufficient evidence for a significant difference in the amount of heterogeneity between the randomized and the non-randomized studies. As a more elaborate example, let's consider the data from the meta-analysis by Banger Drowns and Karlicks, who examined the effectiveness of a particular intervention for improving educational achievement. In this meta-analysis, the effect sizes of standardized mean differences, and the studies differed in their sizes, so we have small studies and quite large studies, and also in the subject matter that was examined, so 28 studies looked at mathematics performance, 9 studies looked at a science subject and 11 studies at a social science subject. We will now use both of these variables as location and scale variables in our model, and so once we fit this model, we get these results, and so here the intercept corresponds to mathematics for the average effect size and amount of heterogeneity. The coefficients here represent the difference in effect size for science and social science compared to mathematics or for science and social science compared to mathematics with respect to the amount of heterogeneity, and the coefficient for sample size represents the slope of the relationship in terms of effect size and amount of heterogeneity. And here we do find some significant relationships, in particular this one and this one. The results therefore suggest that larger studies tended to yield smaller effects, and that studies that examined the effectiveness of the intervention in science subjects tended to yield more heterogeneous effects, but there was no evidence to suggest that the average effect itself differed for science subjects. So this example shows that there can be different types of relationships in terms of the location or the size of the effects and the amount of heterogeneity. These types of models therefore open up the possibility to examine entirely new research questions. For example, comparing different types of interventions, not only in terms of the average effect size that they yield, but also in terms of the consistency of the effects. But it should be pointed out that due to the increased complexity of these models, you tend to require a larger K or number of studies to obtain meaningful answers here. And given that there are different types of tests for testing scale coefficients, these wall type tests like the ratio test and permutation test, the question arises, which type of test should you pay most attention to? This is something that we are currently examining in a simulation study. And finally, I am looking at the possibility to fit such types of models in the context of the rma.mv function from the metaphor package, which is for multilevel and multivariate models, although that type of extension is not entirely trivial. This brings me to the end of this talk. On this slide you have the references. And if you have any questions, comments or suggestions on this slide, you can see how to get in touch with me. Thank you for your attention.