 Hello, everyone. In this talk, I will present NMA Studio, which is a new online and interactive web application to produce and visualize results from network method analysis. This is all joined work with Anna Heimani, also from the University of Paris. So the main reason behind NMA Studio is that we wanted to provide a tool that could not only enhance but also facilitate the interpretation of the main findings from a network method analysis. We know that network method analysis by integrating direct evidence with what is called indirect evidence can simultaneously compare many treatments. And for this, typically there is a very large number of results and outputs being produced. So visualization can be challenging, especially when we have large networks with many treatments. Also more recently, it seems that we are moving towards the context of leaving evidence so we data collected weekly or monthly. So we need new software to keep up with this fast production of evidence. However, existing software in the field so far are not fully interactive in the same way we hope NMA Studio can be. So a bit more specifically, NMA Studio is a Python application that connects to R to produce the NMA results using the library net meta, then those results are imported back in Python where the app is built. In NMA Studio users upload their data and interact directly with a customizable natural plot by clicking on different nodes, so the treatments, or different edges, so the comparisons. And based on their selection different outputs will be displayed. So NMA Studio will follow all of the key steps which are typically performed in any published NMA, starting from the very first assumption of and fundamental assumption of transitivity. And then, once this is met, typically a large part is dedicated to the reporting of the summary of the treatment effects in the form of force plots or league tables. Then, importantly, we need to assess what the consistency holds in the network so if there is a statistical agreement between direct and indirect evidence, and also to assess for the presence of a small study effects which is a phenomenon related to the problem of publication bias. Finally, a ranking of treatments is usually provided. So NMA Studio will assist the user in each of these steps and we will see this how with the demonstration. So our data come from a recently updated systematic review about chronic plates rises, which comprehends 158 RCTs and compares 20 different drugs. There are two primary outcomes that will be analyzed, one for efficacy and one for safety. In this case, both outcomes are binary and risk ratios will be used. So we can just have a look at the app. Okay, so this is the main homepage of NMA Studio and first thing you will need to upload your own data from here. Selecting your data file, for example, and data format or type of outcome. Here I've already uploaded the data and this is permanently loaded actually this example into the app but once the data process is completed, your network plot will appear here on the left. This is a completely interactive object so you can move it around, zoom in or out or also drag every item. Also, there are additional settings here starting from the layout so you can change layout from a list, you can change edge size or node size, for example, and also a few options about coloring the nodes for example by risk of bias that you will have in your data or by class of treatment if you have that and also just choosing your desired color and same for the edges. Then you can download the plot from here or look at an expanded version from here. Still in terms of network visualization, we added an option to look at the evolution of the network over time with the slider here on the right. Also, if we click on the very first data available for this data in 1963, we can have a look at the first trial and then we move on over time and we can see how evidence is added. And also you will see that the data table is filtered in real time accordingly. So in terms of data filtering, you can filter your data set just by clicking on a few comparisons or edges in the network or a few nodes and the data table will be filtered accordingly again. You can also have a look at the expanded table and export the data in CSV. Okay, so now we are ready to start all of these key steps of the animated I was referring to before starting from transitivity. So to check transitivity we need to check whether the distributions of the potential effect modifiers that you have in your data set are similarly distributed across comparisons. You can choose your effect modifier here for example, we have a look at age which is typically provided and the corresponding plot will appear here. Also, these late access labels and this is true for all the subsequent plots is that are editable. So for example here you can just choose your label. And also if you want to highlight a few comparisons of interest, you just have to click on the corresponding edge in the plot. Moving on, we have the summary of the effects so starting from forest plots here we have three options and NMA for a spot the typical pair was for a spot and the bidimensional forest plot in case of course two outcomes are given. So starting from the first one. Here of course we will need to choose a reference treatment and we do so just by clicking on a node on the network. We can have a look at the plot. Also, you will have to choose the correct direction of the outcome to interpret the plot correctly, and we can also have a look at the plot for the second outcome in this case, the network plot will be automatically updated for outcome two. Then for the pair wise for first plot of course in this case we will have to choose a comparison, and you can resize a little bit. The objects and also again you can have a look at the plot here for outcome one or two and you can save it all the plots will. You can save it from from here. Moving on again, we have the bidimensional forest plot so you will have the forest plots for outcome one on the x axis and for outcome two on the y axis. Again we just choose a reference treatment, the forest plot will appear here, and in this case you will have to pick the correct region of the plot to interpret the results correctly based on the direction of the outcome. Also, you can click on the legend to remove sequentially a few treatments if that's needed. Then again we have the part dedicated to the league table so league tables report all of the possible treatment effects between any possible pair of treatments in the network so they tend of course to be very large. And here you can see that you can scroll the table. We also allow for two different options of coloring the table. First and default option by risk of bias in the direct comparisons you will have the average risk of bias, but also coloring by report from cinema assessment so cinema is a software that reports. And overall rating in the confidence of the evidence, which is graded as very low confidence, low confidence, moderate or high in each comparison. So what you will have to do is to upload the results from the from that you get from cinema for both outcomes of course, and then have a look at the corresponding coloring here on the lower part of the triangle we have results from outcome one and on the upper for outcome to also you can have a look at the expanded table here and you can export it maintaining coloring and formatting. But what is important here is that as we said the table sent to be very large so you might want to have a look just as a subset of notes which are more of interest. So you can do so just by clicking sequentially on a few notes, and you see that the filter date table will appear here. Then moving on, we have other checks. So starting from consistency. In this case in enemy studio we allow for two different options, a global test for inconsistency which is the design by treatment interaction model, and a local test for inconsistency, which is the notes splitting approach. You will see that you have all of your results here in the tables for the notes splitting for all of the different direct comparisons you can scroll down and you can see that suspicious values are flagged in red or yellow. Also here you can filter the table just speaking a few comparisons of interest. Then we have to assess for the presence of the small study effects and we do so using the typical final plots. In this case, comparison adjusted fun plots. Here also we have to pick a reference treatment so again we just do so clicking on metric, and the corresponding plot will appear here for outcome one or outcome two. And finally, we said we have the ranking of the treatments in anime studio we allow for two different plots heat map reporting peace scores for both outcome or for one outcome only, and they scatter plot of the peace course. Peace scores are the frequentist analog of the more common sucre values. And what is important here is that you will have to choose the correct direction of the outcome. So for example outcome two for us is harmful. So we change this here and we will have our heat map here with treatment sorted from best to worst. Then we have our scatter plots reporting peace scores for outcome one on the x axis and for outcome two on the y axis. Again, you choose the correct direction, and you will have of course on the upper right part of the plot treatments that appear to be the best in both outcomes. But what is important here is that you can use the network plot to assist in the interpretation of these results. For example, if we just look at some more details. We can see that the best treatment is actually a treatment which is okay at low risk of bias, but also not very well connected. There is only one trial assessing this treatment. So it is always important to use the network to assist in the interpretation of the findings. So this is pretty much it for functionalities of the app we also have a documentation page and the news page. And so, just to conclude, we have seen how anime studio is a full interactive and flexible application, and that it can simplify the full anime process while also assisting in the interpretation of findings. However, as all softwares, it comes with many benefits, but also with many risks. So we always really highly recommend to use anime studio following advice from inexperience statistician. There are many features that we will, we would like to add to anime studio. And just to name a few here, we would like to add more options to customize our network plot and also more robust system of alerts or warnings for example printing the errors from the our console directly. For sure, we will add an option for performing Bayesian enemies. And also we are looking into ways to provide each user with a permanent link to their project. And I mean she is also a campaign by Python package which is currently under development, but that will be available soon. Of course, this is not an exhaustive list so we can add more and if you have any suggestion or ideas that will be more than welcome. So you can get in touch with me at this email address here if you want to discuss more. Thank you very much everyone.