 I'm David Hobby from Camarades Berlin based at the Quest Center of the Berlin Institute of Health. I'm going to discuss a web app that we have built for teaching meta-analysis and art. Camarades Berlin is a group located at the Cherite Health Network in Berlin for the promotion of the benefits of systematic review and meta-analysis of animal studies. Camarades stands for collaborative approach to meta-analysis and review of animal data from experimental studies and is part of an international network of pre-clinical researchers. We do our own research and we also provide methodological support for researchers to perform robust, high-quality reviews. One of the ways we do this is by providing free introductory workshops on systematic reviews and meta-analysis, in which we have a session demonstrating the practical examples of meta-analysis in art using the metaphor and meta-packages. During the pandemic, the tech hurdles, which have always been present when teaching this material, have been exacerbated by the remote learning framework. Most of the participants in our workshops come from disciplines such as biomedical and medical research or epidemiology and are generally not familiar with programming languages or statistical programming software. In previous iterations of the course, we would provide an R script and some CSV data files, lay out the technical requirements in advance with the expectation that all required software be installed and functional on session day. However, some subset of the participants would often arrive without having read the requirements or installing R or the required packages or without a date or conflicting packages. This often turned the first hour or hours of the course into a troubleshooting session where we'd have to go around and figure out what was going on with each individual person who's having issues. In light of this issue, we had the idea to build a self-contained web app based on the Learn R package, which walks the user through the steps to recreate the analyses of a published meta-analysis of animal data in the biomedical field involving a comparison of controlled intervention studies affecting infarct volume. This publication used all the medical and analytical methods which we would like to teach, namely random and fixed-effect models, meta-regressions and visualizations of heterogeneity with study design characteristics, forest and bubble plots. The two R meta-analysis packages which we used, meta and metaphor, contain the necessary functions for performing these analyses. In order to present these analyses in a useful way, we also used two additional packages. These packages were Shiny and Learn R. Shiny is a package for the creation of interactive web apps in R, which when combined with Learn R allows R markdown documents to be converted into interactive tutorials with live code exercises. Here, for example, is one code chunk taken from section one of our app where the user is asked to examine a data set that was just loaded. When you click Run Code, the code is executed and the output is displayed in the browser. After each major section, we also have questions with quizzes to check for understanding with immediate feedback. So if you get the wrong answer, you can try again until you get it right. In section five, participants are asked to create their own code based on what they have learned from the preceding sections in order to answer a set of questions. If they get stuck, the hint button will point them in the right direction. Other features that can be implemented in Learn R and Shiny are embedded videos and interactive Shiny elements. We actually didn't use either of these in the current iteration of the app, but it could be possible in the future. Using a teaching app allows us to sidestep any software-associated problems that individual students may have so we can focus on the actual material and on the principles of meta-analysis. The app runs in a browser window, which means that from the student's perspective, it is an interactive website. The course is presented as a step-by-step tutorial with interactive code exercises which walk the student from loading the data through to performing the final analyses. At each step, there are short quizzes to check comprehension. Progress is saved as the user progresses through the app so that they can leave and come back whenever they want. All code is run on our own server. We make sure that all packages are up to date to ensure full functionality. This does, however, present its own issues on our side. The biggest issue we've had is the performance. Our courses have between 10 and 35 participants, which means an instance of R running on our server for each participant. And even with lower numbers of participants, we have had slowdowns and crashes when everyone is running analyses at once. Secondly, some participants reported that the user interface is still quite unintuitive and could be streamlined. One of the sections in particular is very long and is difficult to tell how far you are through it. We could probably break this up to make it more intuitive. We welcome other feedback about the app in general and the user experience in order to improve it. The biggest benefit and the primary goal of using this approach is that we have had zero technical issues on the participant side since we have been using the app. The lowered barrier of entry for those students without prior experience or exposure to programming languages and we have received generally positive feedback from our students about it with the majority reporting that it was easy to use for the purposes of our tutorial. We provide access to the tutorial 24-7 on our website, which means participants can access the tutorial at any time and review the material. It is accessible to anyone else who may have interest. It was developed collaboratively via GitHub using free and open source software. And lastly, it has been shared with other camarades locations. Our colleagues at Edinburgh are also using the app in their tutorials and they are able to run it on their own server after downloading it from GitHub. So the app is not yet in its final form. The next steps are firstly to optimize server usage to improve performance when many users are running it. We have already optimized caching and we are investigating methods of parallelization. This will reduce and hopefully prevent the slowdowns and crashes that we have experienced so far. Secondly, we are going to clean up the user interface to make it easier and more intuitive to walk through. The one very long section that I mentioned earlier could easily be broken up into at least two other sections as a starting point. We don't want to remove any material. We believe it is important to present the full analysis from the published paper, but we could definitely arrange it in a more digestible way. Lastly, we would like to refine the questions and provide more in-depth feedback as to why answers are correct or incorrect. Thank you very much for your attention. I would like to thank everyone on the Camarades Berlin team and would like to thank our funders at the Cherrytay 3R department as well as the developers of META, META4 Shiny and LearnR packages making this project possible. Again, we welcome any feedback regarding the app or the experience and I'm looking forward to the discussion and questions coming through on Twitter.