 Hi everyone, in this video tutorial, I will present a practical example of a three-level meta-analysis using the R software. Specifically in this example, we look at a three-level meta-analysis focused on the association between paternal anxiety and child emotional problems, which is also the broad focus of my PhD project. The dataset that I'm going to use in this tutorial is available as an attachment so that you can have a closer look at it in your own time. And this is just a screenshot of the CSV file. I have already described what each of the columns contains. Anyway, just looking at it, we can see that the data has a nested structure with the individual effect sizes representing level two nested within samples, which represent level three. This is a helpful schematic representation of the structure of the data from the image which I have borrowed from one of the recommended resources. The three-level organization of the data becomes clear. So here we have the results of individual participants that are pooled in primary studies. Here in level two, we can see that different primary studies use the same sample. And here we have level three, where the aggregated cluster effects are pooled in a three-level meta-analysis. Okay, so here we are in R studio. The first step is to install, if we haven't already, and call the packages that we need to run our analysis. Also, the R code is provided as an attachment so that you can try to conduct analysis that are part of these tutorials, but also adapt the code for your own studies and your own data. I have also included this bit of code because it is recommended to keep the R environment clean and tidy. Once we have called the packages, we want to import and read our data that we have named the anxiety dataset. Once imported in R, it is helpful to use the head function to see the general structure of our data and also to keep it as a point of reference while conducting our analysis. Okay, so how do we fit our random effects three-level model in R? We can use the RMA-MV function in the Metacol package, and here I have listed the most important arguments for this function, and I've also specified what they refer to in our data file. So we definitely need the calculated effect sizes, and when using Pearson's R correlation coefficients, like in our case, it is recommended to transform them to Fisher's Z for the analysis, and then when interpreting the results, we can transform them back to Pearson's R. So in our case here, we refer to the Fisher's Z column in our data file. We then need the variance, the calculated effect sizes, the study labels, and also the name of the dataset. Then we need the test that we want to apply for our regression coefficients, and the method used to estimate the model parameters. Just enough to say that this is a matter of choice, and I do recommend to check the options that are available and to choose what is best for you and your own data. Random is actually the most important argument for our function, because it is where we specify that we want nested random effects model. In our example, just as a reminder, individual effect sizes represent level 2, and they are nested within samples which represent level 3. So how do we specify this in our function? The general structure of the formula is this one, and this translates in our example to this formula. So the cluster is represented by the sample IDs, and the effects within the cluster is represented by the effect sizes of the individual study's IDs. So the complete function in our R code looks like this. So this is filled with all the actual columns in our data file. And here we have the output in R. So what is important to note here is this part, where the random effects variances are calculated for each level of our model. So the sigma 2.1 shows the level 3 between cluster variance, and the sigma 2.2 shows the variance within clusters, so the level 2. We can also see, as I specified here, this part, so the number of groups on each level. Level 3 has 27 groups, which is the 27 included samples, and these 27 samples contain 116 effect sizes. Then importantly, we have the model result. So here we see that the estimate of our pooled effect is Z0.15, and we also see the 95% confidence interval here. To facilitate the interpretation of the results, we can transform the effect back to a normal Pearson's heart correlation coefficient, and to do this we can use the Ask package. And this is the bit of code that we need, so we convert Z to R, and here are the results. So it is also important to quantify the heterogeneity in our meta-analysis, so both for level 2 and level 3, because we are running a three-level meta-analysis. So this is the code that we need to do that, and for this we are using the I-squared statistics, that is just one of the options that are available to us. And here is the summary of the I-squared statistic. We can see the total heterogeneity for our meta-analysis and also how it is distributed in level 2 and in level 3. It is also interesting to see this visual representation of the heterogeneity. So here we can use the plot I-squared code, and here we see the variance that is not part of the sampling error. This is the total, and this is how it is split in the two levels, while here is the sampling error variance. It is also very useful and very common to obtain a visual summary of the meta-analysis results, and this can be done using forest plots. This one, so the general forest function, would just give a list of all the effect sizes in our meta-analysis, and it would not be representative of the nested structure of our data. Anyway, I do recommend checking and learning more about forest plots directly in R, because there are useful examples. So to obtain a forest plot that reflects the nested structure of our data, we can create a forest plot that shows only aggregated effects for each sample, and this is just an approximate representation of the three-level meta-analysis. This is the bit of code that we need to do that, and I won't explain this in details, but in any case we are basically running a new meta-analysis of aggregated effects. So this is what the code looks like, and of course I specified what I want in my forest plot, but as I said before, it is useful to see all the potential arguments that are in this function and the options that are available. And this is our forest plot, so we can see that we have the samples, here we have the number of estimates for each sample, here we have the effect sizes and their confidence interval, and here we also have the weight of each of them. Just to remind that these are Fisher's Z, and also the final, the pooled estimate is Fisher's Z, and we have the 95% confidence intervals. Okay, so we run our three-level meta-analysis in R, we have obtained our pooled effect size, our variance components, we have quantified our heterogeneity in level 3 and level 2. How do we present the findings? It is common and also very useful to have a summary table. So in this table we can see that we have the number of effect sizes, the number of samples, the pooled R, so the effect size, the 95% confidence interval, the P and the I squared statistic for level 3 and level 2. We can also summarize our results in written format, and this is an example of how we can report our results in an academic paper, for example. Finally, I recommend these two resources to learn more about meta-analysis in general, meta-analysis in R, three-level meta-analysis. These are really helpful resources. Okay, so we have reached the end of the tutorial. Thank you for listening, and I hope you find these resources a helpful introduction to three-level meta-analysis in R.