 Welcome to this tutorial. In the next 15 minutes, I will introduce you to the meta UI package. This new app package has been created to support you in creating shiny web apps that allow readers and other consumers of your meta analysis to explore the datasets for themselves, to make it transparent how different analytical choices influence results, but also to allow people to subset the data to only look at specific outcomes, study types, regions, whatever it may be that are relevant to their interests, and also to update or edit the dataset. For example, see if the inclusion of more recent studies or the inclusion of other languages make a difference. I created this package together with Lukas Rosler and Brad Bushman. Lukas Rosler has also recorded a talk that details the use case more broadly, so you might actually want to start there. In this tutorial, I will provide a demonstration of what the resulting shiny app looks like and then show you how to get there quite quickly using the package. I picked this meta analysis on Notching as an example because it's come out relatively recently and triggered lots of discussions around application bias and around other choices that might have influenced their results. The data is available on the OSF, and from that data, using the meta UI package, I spent about 10 minutes to build this web app that's now available on shinyapps.io. When you open it up, it tells you to click on Analyze Data, possibly after filtering. I'll just analyze the data as a whole, the full data set. And once that is done, I can explore it in different ways. So on this first page, we can see the results of different meta analytical models starting with normal random effects, monthly level model, and then moving on to various approaches to controlling for application bias and to otherwise potentially making the results more robust. So here we can already see quite a range of estimates. I can look at the sample to understand more specifically what time period, topic area, and so on was covered, and also search easily through the entire data set. I can test for moderation. For example, here, see that the effect declined over time. There's information on application bias with some very basic analysis, a thumbnail plot, a P-curve, a Z-curve, and I can explore outliers. And now obviously in the combination with the sample description, I can now actually get some more information on Gaia at all here. See why they have such a large impact size. And see that, for example, this happened in a conventional lab study looking at food. If you know something about this area, you know there have been some scandals around nudging and food research, not this particular paper, but it might still encourage me to investigate further. Now, what I can also do here is obviously filter the data and say, well, I might believe that from about 2014 we maybe had more robust research practices and actually I'm only interested in field research, so removing the lab research. I can again analyze data for the subset and see how that changes the effect sizes. In this case, it leads to lower estimates throughout, but still probably strong evidence in favor of the hypothesis that nudging does something. I can download the data set based on the filters I set, edit it, and then re-upload it with changed data. So I could add new studies. I could take out a certain set of studies to see how that affects the results. So it's the kind of app that you can build with MetaUI. The package creates all the code for you, but the code is fully transparent, so if you know how to use Shiny or, in fact, if you know how to use the search and replace function, you can customize many of the aspects of the app rather easily. But how do we get to this stage? Well, the first step is to install the package into your local R installation. The instructions are on the web page, but once the package is there, we recommend starting from a template. So if I go to the new file R Markdown template, I will find a MetaUI template to create this Shiny app. If you display this in the visual editor, it becomes a bit more readable and now can take you through the process of creating this kind of app step by step. The first step is just to run the setup, load the required packages. Then for MetaUI to work, we need a range of information on each of the studies included. It's listed here. So what I would always do first is load in the data. I downloaded it and can select it here just by running this chunk. And then I would use the glimpse function to actually see what's currently in the data set. Now I can see some of the things already exist. I have a study ID. I also have a reference. I'll use the reference as the label just because it's a bit clearer. I have an effect size measure, co and st. I have a variance measure, a variance d. I do not have a standard error. I do have a sample size. I don't have a p-value. And then I do have details on a couple of moderators or filters that I want to display in the app. Now the next field here already gives some examples for what you might need to do in the follow up. I don't need to do anything about the study label, but I exactly do need to calculate a standard error from the variance as the square root of the variance. I also have some categorical moderators that I will turn into factors. So I have the type of the experiment that I will turn into a factor. I will not relabel them for now. The labels are good enough. But I will also take the domain and turn the domain into a factor. The remaining missing bit of information is the p-value. Fortunately, that can be calculated based on the d-score and the sample size. So I've created a little helper function in the background based on formulae that are quite straightforward to find online. I get a p-value. When I specify all of this, I can run the cell to adjust the data. Once I have all the required fields in the data, I can use the prepareData function from the meta UI package to move one step closer to the app. The main purpose of this function is just to label which field in your source data contains which bit of information. So firstly, the effect size is contained in the coinsD field. The study label is contained in the reference field. Standard error, we just calculated. It's in SE, p-value is in p-value. The sample size is in n comparison. The variance is already there. And now I need to specify the moderators and filters that will be displayed. They are here treated as the same set. So I can use them to filter the data set and I can test whether they moderate the effect size. I already said that I want to use the domain and I want to use the type of experiment. I will also want to use the year. Now, this is a named vector because quite often we might want to use different labels here just to make it more and more legible, to make it very clear what these filters actually are. The only final thing I need to indicate is the type of effect size and that needs to follow the metaphor and appropriations. But in this case, it's very clear as SMD. It's a standard as mean difference. If I run this and now have the data set ready to go for MetaUI, the only other thing I will want to provide to get a first version of the app is just the title for the data set. And then I can use the generate shiny function to get a first version of the app. Now, this is already exactly what we saw online. I can calculate the estimates, look at all the other tabs, but it's still running on my own computer, so I can't share it. And it's not saved. So to get it to this stage, I would need to go through all of the data preparation steps again. So obviously what I want to do, I will stop this app again and I'll generate it again with a difference particularly with the request to save it to a folder. So to create a static customized app. To do that again, give it a data set name. I now also specify a better label for the effect size, so the type is still SMD. But in the app, I'd rather want to show Cohen's D because it's clearer. I will also want to add a citation before I publish this. And I will want to add some contact details. So for example, just my email address, if I now run this, it will prompt me for a folder and then it will save the new app in that new folder. It consists of quite a lot of different files. And therefore, if you might want to adjust in your own use cases. So in models.r, there's a list of the meta analytic models and the specific code used to run them. So you could add models there. You could very easily take out models. If you think Trim and Fill is so outdated it's not worth showing that literally takes a single character, a single comment, hashtag sign. Then you can modify the UI to possibly hide some elements. If you think we don't need a p-curve and a z-curve, you would hide one of those. In the server, you can edit what is actually shown. For example, in terms of chart themes, chart labels, so on. And then there's a labels and options file where you can also edit some of the text that is displayed and change some of the parameters that are at the moment still hard-coded under the hodge. So for example, some options used for aggregating dependent effect sizes. But I don't need to do any of this. This is now ready to go. If I restart my session, I start in a fresh R session, I can run this app just by opening the new app.r and running it. Now, I have this app here. It's running again. I can just put this package into an email onto GitHub wherever you want to put it. And other people can use it to run the app. Obviously, to share it with the world more broadly, shinyups.io is a really useful platform. So the next step in the template provides instructions for how to deploy this to shinyups.io. If you've never used it before, you need to set up an account and connect the account to RStudio. There's a link here. Once you have done that, you literally just need to run this line, this line of code. Need to first stop the app. Then I can run this line of code, select a folder, and then it is getting uploaded. Now, I told you you can build an app in 10 minutes. This probably took about 10 minutes now. You can't get it onto shinyups. Just in 10 minutes, this will now take a little while for it to actually be uploaded. Once that is done, it opens up in a browser. You get the URL, and you can share the URL with anyone who wants to have a look at your data set. So this much is a demonstration of the MetaUI package would be very happy if you give it a go and let us know how you find it. We're always happy to add further models, to add further customization options, particularly if you submit them on GitHub as pull requests. And if you publish an app based on this, please let us know that we can create a gallery of use cases that others can be inspired by. So enjoy experimenting with this. We look forward to hearing from you.