 Creating interactive shiny apps for meta-analysis with Meta UI. There is no straight path from the data that you collected for your meta-analysis to the results. It's rather a labyrinth through many different choices. Should you exclude certain outliers, how to deal with different effect sizes, and which type of model should you choose? These choices affect the results in critical ways that meta-analysis have been often criticized for choosing one path over another. In practice, researchers usually have one planned or even pre-registered path that they take, and then they had a few more analysis such as robustness checks or some analysis for exploratory purposes. But still, according to other researchers or reviewers, there is often a path missing. So, how can we do justice to the large number of equally justified but different paths? What we have observed so far are three approaches. First, in what we call the shotgun approach, all types of analysis that make more or less sense are shot at the data. This typically yields a very large table in the manuscript, as you can see on the right. Second, some researchers have run a multiverse analysis. That is, they first conduct many different meta-analysis, and then they run a meta-meta-analysis on the results. The third approach, which is what we have been focusing on, is what we call the open box. Thereby, you provide others with your dataset and tools to run all sorts of different analysis. What if we exclude effect sizes plus minus two standard deviations? What if we only take studies that feature a cover story? How will the results change if I filter out non-pre-registered studies? Can you tell me what effect size you get with the selection model? Sure, the open box is a published shiny app that allows everybody to access the dataset via a website and explore and download the original or filtered dataset. The open box approach is simple as anybody can go on the website instead analyzing. It is also flexible. A researcher once asked us why we chose to report a p-curve but not a z-curve. At that time, we had not really been aware of z-curve, which is why it wasn't there yet. So noting that this would provide us with additional information, we simply added a z-curve analysis to our shiny app. So if you can do it in R, which is the case for pretty much everything, I guess, then you can also feature it in a shiny app. The downside, however, is that creating shiny apps is a bit difficult. And to overcome this problem, we created an app package that creates the shiny app for you. Make sure to check out the tutorial at EsmarConf on how to use our package. But in a nutshell, these are the steps that you need to go through to create a website where you and others can interactively analyze a meta-analytical dataset. To let you know how easy this is, I would like to give you a brief look at the code. Again, we're still at an early stage, so more functions may be added soon and things might look a bit different later on. And everything is also discussed in depth in the tutorial. So your dataset goes up here. Then we need to know what the effect size and standard error variables and so on are called in your dataset. And you can determine filters or moderators that you want to apply. Here we have pre-reg, which includes information about whether or not a particular finding was pre-registered. We also have demographics such as the mean age. And the app will later see the range of these variables and let you pick a value along it. In practice, you might have downloaded data from a meta-analysis that is a few years old. And you added a couple of new studies that have been published since. And now you want to know whether including the new studies affects the results or maybe whether the new studies differ from the old studies. So you could just create a variable indicating whether or not a finding is new and put it right here. And the app will analyze whether this moderates the effect size or not. So the second function needs the formatted dataset. You can also customize labels and you can even decide whether you want to launch the app or whether you want us to save the entire shiny code on your computer. So that you can customize it, for example. So here's an example of the shiny app that includes anchoring and anchoring dataset. And before we get results, we have to filter or apply our filters. And you can see the mean age, which is the smallest mean age in all of the datasets included here is 18.8. And now we may want to select only items where the scale type was open or visual. And so we have applied, we have selected the filters and now we have to click on analyze data. And here you go. We currently have effect size estimates from several different models below here and a small sample description. And now let's continue conducting some moderator analysis. And again, as you can see, the app recognized that this is a metric variable and creates the plot and the table accordingly. But now we might want to know whether the scale type has an effect on the effect sizes. This is a categorical variable. And here you are just significant with everything that you would need to know for reporting this. You can also download the dataset and it will save the filters that you applied while it's still the entire dataset. And then you can update it later on to continue where you left off. So this was just a small excerpt from the analysis that you can run with the MetaUI package. Currently you also get a sample breakdown with respect to the moderator variables, a forced plot of all included effect size, a final plot with Agis test, p-curve, z-curve and information about the effect size distribution. So thank you for joining us for this presentation and we are looking forward to your thoughts on this.