 Yeah, hello. My name is Daniel Heck from the University of Marburg, and today I will present the R-Packaged Meter BMA, which can be used to perform Bayesian model averaging for meter analysis. And the example here concerns the effectiveness of descriptive social norms. And more specifically the question how we can make hotel guests reuse their towels. And the control group, the hotel guests were simply told, please reuse your towels because it's good for the environment. And the treatment group, they were told that actually a majority of guests reuse their towels. And the effect size here in seven studies was quantified by log odds ratios on the X-axis and positive values mean that the treatment was effective, so that actually the descriptive social norms increased the reuse of towels. And the question we have now is how can we accurate the results here from the seven studies? And in meter analysis there are two standard models which are commonly used. The fixed effects model assumes that each study has the same constant true effect size mu and that this effect size is identical in all studies. The random effects model in contrast assumes that each study has a different effect theta i which varies according to a normal distribution across studies. So first the studies are sampled essentially from the normal distribution as shown on the right hand side and the parameters of this distribution are mu, the average effect size and tau, the standard deviation of effect sizes. And this tau is also known as the heterogeneity and its amount of differences of effect sizes across studies. Now the question of interest are two-fold. First we are interested in the question whether there is an effect, so whether the overall effect mu differs from zero. And the second question concerns heterogeneity, specifically is the heterogeneity, so is tau larger than zero? And if you combine these two aspects into models you get four different models which are denoted here by an H4 hypothesis, H0 is the null hypothesis and H1 the alternative and the F is for fixed and R for random and then you get the four models where you assume either the effect is zero or not and second the heterogeneity is zero or not. And the question is obviously which model should we pick. If we select one of the four models we ignore uncertainty which of the four models is the correct one and so it might be better to actually consider the uncertainty which is the best model. And exactly this is done by Bayesian model averaging, so first you have to define prior model probabilities which is a question how plausible is each of the four models before you see the data. And here you see an illustration where there are uniform priors used, so on the top there are these four boxes for the four hypothesis and each has a probability of 25%. What follows from this is on the bottom on the left and right hand side the plausibility of our questions. First is there an effect, if you combine the null models and the alternative models you see it's 50-50. We don't know yet. This is why we do the meta-analysis. On the right hand side you see the heterogeneity and there you also combine models but on the left hand side the fixed ones and the right hand side the random ones. And here it's also 50-50. Now if you collect data the probabilities are updated. So we now have posterior model probabilities and these are determined essentially by the data we collect. These inform the estimation by the prior model probabilities and as we will see later by the prior distributions on the parameters mu and tau. Now what happens is here that the models under the alternative become more plausible. They have larger posterior probabilities as indicated by probabilities of 40% and 35%. And if you now ask is there an effect then their combined probability is 75% compared to only 25% for the null models. This indicates some evidence that there might be an effect while you consider uncertainty weather effects vary across studies. The same can be done on the right hand side for heterogeneity. And this is what the inclusion-based factor quantifies. How much evidence is there for a non-zero effect? So whether mu is different from zero and this is done by comparing two models the alternative models against two null models and thereby we account for the uncertainty should we assume fixed or random effects. And the same you can do for the heterogeneity just that you bundle together different models. So here you compare the random effects models against the two fixed effects models. However this only works in a Bayesian framework if you assume prior distributions and specifically this means you need to make assumptions what size of effect do you expect and how much heterogeneity do you expect. And here these prior distributions are illustrated for the fixed effects models. So you see on the right hand side two plots where there's just a point mass at zero. So essentially the fixed effects models assume there is no heterogeneity at zero. On the left hand side it differs between the null and the alternative model whether you assume an effect the null model assumes no effect and the alternative assumes that the effect is likely between minus two and two with higher probabilities assigned to smaller effect sizes. And now I switch to the random effects model and you see on the right hand side now a prior distribution on the heterogeneity tau on the standard deviation of effect sizes and this year's default prior used and based on a literature review. Now what are the basic functions of the package you can specify priors you can fit models you can average across the models and you can complete these inclusion base factors I mentioned before. These are essentially the components I mentioned already and all of these four steps are combined in a single function called meter BMA and here you see how this is called. Essentially you have four lines where you specify the input. This is a standard input for meter analysis as you would also do in other packages and then you have to specify a prior for the overall effect size here and here you see it's a normal distribution with mean zero and standard deviation 0.3 truncated at zero and on the bottom you see the prior for the heterogeneity. The output is rather large this is a separate tutorial video that explains all of this but essentially you get base factors, you get model averaged results, posterior probabilities and estimates for the parameters. The plotting functions are very helpful because they show what happens here. Essentially you have a posterior for the fixed effects model which is violet, a posterior for the random effects model which is dotted and red and the model averaged one is in between. It's a blue one which is kind of a combination of these two and how far they are combined depends on the posterior probabilities of the models. Here you see also a forward plot where you have the descriptive effects as circles and as triangles the shrinked estimates for the studies and you see that there's quite some shrinkage going on so the model estimates that the effects are rather similar even though descriptively the effects vary much more. On the bottom you also see the three estimates fixed effects random effects and the averaged one which is a combination of both so what are the benefits of this approach? I think it's very nice because it's principled the assumptions are very transparent and we consider the uncertainty regarding auxiliary assumptions and you can do it sequentially so you can continuously update your beliefs as more studies are published this is shown here so on the x-axis you accumulate studies you can update your meter analysis and this is what usually happens is that more and more studies come in. However it could be criticized that we consider the fixed effects model at all. However this is a plausible model and for instance if you perform direct replications there's some evidence that this model is supported and also if you only have few studies you might not have sufficient evidence for heterogeneity. It might be better to assume that the effect is identical to get a higher predictive accuracy which means on the bias variance trade-off that you have less variance in predictions and this is beneficial compared to fitting a complex random effects model. And of course there are priors the question is how to pick these you could ask experts with prior elicitation you could do literature reviews and of course you should perform sensitivity analysis to make sure that the results are robust for different priors and of course it's possible to pre-register the priors. So thanks for your attention thanks to my collaborators at the University of Amsterdam and if you're interested in these topics you can look up four main papers at my web page one the first concerns the meta package meta-bma package itself the second is a primer on the methodology there's a tutorial on JASP you can use these methods in a point-and-click interface and also we applied these methods to a set of pre-registered pre-registered replications. So thanks a lot and see you around.