 So good evening and thank you for having me. My name is Clarice Neville and I work at the University of Leicester as part of the Complex Review Support Unit, the CRSU. Today I'll be talking about a pilot for an interactive living network meta-analysis web application that we developed last year. So 2020 and the delights of COVID-19. Trials for COVID-19 started being published in April 2020. There are now 2,533 clinical trials registered worldwide. Within these trials, various treatments are being tested simultaneously. These include pharmacological therapies such as hydroxychloroquine and Remdesivir and biological therapies to name just a few. As a consequence, an increasing number of protocols for synthesizing COVID-19 evidence have been registered worldwide. Currently there are 489 systematic review protocols on Prospero, aiming to synthesize evidence on COVID-19 treatments on patients on bearing severity. As a group, we saw that there was potentially huge amounts of duplication or unnecessary work being created. We noted that a lot of the systematic reviews being conducted were using the same pool of data, but the difference between the protocols lied in drawing varied subgroups to soups, if it research questions. This motivated us to try and develop a central platform that could hold and synthesize COVID-19 study data, but allow users to adjust the analysis to suit their own research questions. So very briefly, network meta-analysis NMA synthesizes study results involving more than two interventions and creates comparisons between all interventions involved. The CRSU already host a web app to let users carry out NMA without the need of specialist software, namely Meta Insight. It was built using R and the Shiny package. This is a screenshot of the app. You can navigate through the app using the tabs at the top, including a tab to load in your own data. On the left is a panel to choose further options. You can also use this to do a sensitivity analysis by deselecting certain studies and comparing the results to the original analysis as seen here between the two network diagrams. There is a data summary tab, which gives data characteristics, study results and network plots. There is a tab to run the analysis under frequentist methods, and this uses the net meta package in R. And then there's also a tab to do so in Bayesian methods instead. This uses the GEMTC package in R where models are estimated using JAGS. So using Meta Insight as our base, we created a spin-off version named Meta Insight COVID-19. In a nutshell, we developed an interactive web-based platform to allow exploration, various analyses and interrogation of up-to-date data from existing living systematic reviews or treatment for COVID-19. As we wanted to focus on developing the user interface and functionality, we didn't actually carry out a systematic review ourselves. Instead, we extracted systematic review data from the COVID-NMA initiative up to the 19th of October, which was the end date of our pilot project. We stored the data on a Google Sheet, which was linked to the app via the Google Sheets 4 package. So each week the data would be extracted and the Google Sheet updated. And having the app linked to the Google Sheet ensured that the analysis results were always automatically updated. By building the app on top of the existing Meta Insight app, all of the functionality that I described earlier is in the COVID app. The tab to load in your own data was removed and a summary front page was built instead. So now I'm going to try and give a quick demonstration of the app. So here we have it. At the top you can see when the data was last updated. We decided to look at three outcomes in particular, mortality, viral negative conversion and serious adverse events. Here you can view the data and you can view all the different pages of the data. You can see the network plot here and you can choose the different style of network plot if you wish. And here we have a summary of the results as a forest plot. So you can change which studies you want, you can choose them by like clicking them. Or you can use the filter function here, so I want to do standard care and that one. And that will then show you which studies used it. I won't show you the results because you need to make sure that everything is still connected in the network and that all the elements of the studies are there. And then if you go up here you would then go to all the other functionalities of Meta Insight that are in the basic version. So I just wanted to quickly go over two examples of challenges that we came across during development and how we tackled them. So firstly quite early on we found that the size of the network became quite large. This had an impact on that visually network diagrams were becoming cluttered with text overlaying each other, and forest plots were actually running off the plot space and moving on to a second plot. And secondly, we tackled this case by case by altering the user interface parts of shiny to cope with the size. And secondly, there are a few instances where we found that it would have been beneficial if we could include node merging and splitting. So nodes represent the treatments within your network. So node merging would involve say grouping all the intravenous treatments in your network into one treatment group, and then they would remain as one group throughout the analysis and node splitting would then do the reverse. So instances when we had to decide in our group, whether to have nodes together or separate, and this decision remained fixed in the app. Whereas if we had a node merging splitting feature this would allow the user to decide rather than us. So after the process of developing meta inside COVID-19, can we develop a living meta inside app. From what we've learned from this pilot app, I think there are two options moving forward. One, similar to the COVID-19 version, we just create bespoke user interfaces for presenting living systematic reviews of individual disease areas. The users then explore and interrogate the data and analyze as they see fit. The second option is to try and create a generic user interface similar to meta insight where users can upload and update their own living database and then use the functionalities of the app in a similar way. Whichever way it goes, there are lots of other things to consider if we did develop a living meta insight from user functionality to intrinsic challenges within the field of living systematic reviews. But overall, the pilot project was successful and that we learned a lot about what would be needed to develop a user interface for living enemies and we're keen to develop this further and would love to hear any feedback if you do have it. So thank you for your time and I look forward to any questions or discussions.