 Hello, everyone. It's my pleasure to introduce Marley Gatti and Jake Aiden. Marley is a senior data scientist at Biogen and Jake is a senior principal biostatistician. They're going to talk to you about a shiny-based ANOVA and power analysis tool. Please welcome them. This is joint work between myself and Marley, a senior data scientist at Biogen. The status quo at Biogen currently is that the statisticians often receive many requests from the research scientists to perform ANOVA or sample size calculations of three clinical studies. Unfortunately, the scientists often use pre-existing software packages such as Prism, which do not perform ANOVA in a rigorous manner and does not perform sample size or power calculations. In addition, these software packages do not include QQplot, Residual Tynastic Plot or other optional diagnostics, and they don't suggest an appropriate model for the scientists to use. Sometimes the scientists may perform these ANOVA analysis incorrectly, which may lead to incorrect interpretations for their internal biogen experiments. They may use incorrect models or analysis submitted to journals, and they also have to be careful about ANOVA assumptions. If the ANOVA assumptions are violated, this may impact the p-values or significance of findings. Now, for the sample size calculations for pre-clinical studies, sometimes the scientists may just pick a number for the sample size without doing a rigorous calculation. If they pick a number that's too small and the study is underpowered, if they pick a number too large, that's a waste of time and resources. Now, ANOVA app offers a lot of benefits. So, the app we're going to be using to train the scientists to check the assumptions of ANOVA and how to make their analysis more rigorous. So, we're hoping to create an app that is a little bit more rigorous than pre-existing tools such as Graphcat Prism, and we're striving to increase the reproducibility of pre-clinical studies. We're also hoping to increase the quality of analysis that are bound in for pre-clinical studies and streamlining of paper acceptance. Some other benefits of ANOVA app is that it relates to automation and standardization. So, by streamlining our calculation for ANOVA-empowered calculations, we are now obtaining results in less than 30 minutes rather than waiting weeks for the statistician to do ANOVA calculations for the scientists. So, this leads to both increase efficiency for both of the scientists and statisticians. We're also hoping to empower scientists to perform ANOVA-empowered calculations themselves after we properly train them on how to use the app. Another advantage of our ANOVA app is we're presenting one consensus analysis protocol to the scientists for specific scenarios such as comparing means across independent groups. And lastly, the ANOVA app is standardizing both the data input and data output. Now, we have to be mindful of the scope and limits of our app. So, within scope of the app is ANOVA and ANOVA-related sample-heist calculations for pre-clinical studies, and this app is only for exploratory analysis, not for clinical trial analysis. In terms of models, we are allowing one-way and two-way ANOVA models as well as interaction effects or no interaction effects. The app also supports well just ANOVA and non-permeasure ANOVA. And the app is only handling unpaired design and no repeated measures. Out of scope of the app is sample-heist calculations that don't relate to ANOVA, such as non-tuner survival analysis of sample-heist calculations. Clinical trial analysis is also out of scope, and the app does not support repeated measures at ANOVA. I want to discuss some of the features of our app. So, the ANOVA section of our app allows the user to import in vivo data and visualize data. The user can then assess the normality assumption, equal variance assumption of ANOVA, and then the user can decide whether or not a log transformation is necessary. The app also checks for outliers, and the user can remove outliers if they desire. Then the user can set up their comparison of interest, and then the app will provide a results table with adjusted P-values. For the power and sample size part of our app, the app does perform both parametric and non- parametric power calculations. The parametric power is based on a formula approach based on the details, and then parametric power analysis is based on simulation. One main feature of our app is the app automatically suggests a model for the user to use. So, after the user selects their interpretation of each of the diagnostic plots, the app will automatically suggest a model. So, for example, if the user selects that the diagnostic plot is showing a normal distribution and an equal variance, then the app automatically suggests an ANOVA model. If the diagnostic plot suggests a normal data and an equal variance, the app will suggest a well-tos ANOVA. I also want to give an overview of the power analysis tab of our ANOVA app. So, within the power analysis tab, the user can choose their primary and secondary objectives for the next experiment, and then the app will calculate the sample size per group required to observe at least one of their primary hypotheses in the next experiment at the desired effect size for 80% power and a 5% of type 1 family-wise error rate. The app for robustness will plot the sample size requirements per group or a range of effect size scenarios, and the app will also plot the statistical power for a range of sample size per group scenarios. And then finally, the user can output a report of their ANOVA power analysis findings. Some of the parameters the user can select during the power analysis tab is the desired effect size. The app will automatically calculate the observed effect sizes based on the input of pilot data. The user can specify their family-wise error rates and their expected mortality rate in a very pre-penical study. The app will automatically calculate the variances based on the pilot data. And the app will perform a different calculation based on the user's review of the equal variance diagnostic plots of the app supports both equal and unequal variance power calculations. In addition, as I mentioned, the app supports both parametric and non-parametric calculations for power. And now I want to hand over to the presentation to my colleague, Marley, to perform a demo of our ANOVA app. Thank you. For our demo, we're going to use a data set of infar volumes of mice after cerebral artery occlusion. And this data set has three main groups, vehicle, compound X and compound Y, and the vehicle is going to be the control group. So let's take a look at the application. So when you load the application, you have three tabs enabled. The first one is the introduction tab. And in this tab, you can see information about the application, like the type of data sets, as well as the type of analysis included. This tab is just informative. The setup tab contains information about the type of ANOVA. You can upload a data set here. And you have a violent plot, a dot plot, and a review of the data set selected. By default, the application has three these two data sets, iris and empty parts. So you can play around with ANOVA application. For purposes, we're going to use the infar data set. So we're going to perform in one way ANOVA, and our grouping variable is going to be group, 10 points going to be the volume, and the unit identity part is going to be 18. Now, by looking at the data, we're going to be able to create the next tab that the variance is equal. And again, this dropdown is just optional. It's just a way for you to tell the application that you expect that to be the variance. Now, for the covariance tab, we can see that the variance is actually a new code. And so once you click on the covariance, a new tab will load. And in this tab, you'll be able to select if the data is normal on the Rina scale, normal after log transformation, or not normal. Now, since the data is small, I'm going to reduce the number of things so that three percent change is what I create. So this does look like a bell curve shape. And so if we go to the QQ plus, we can actually see that the first QQ plus has all the points on the diagonal line. So we can safely say that is normal on the Rina scale. Now, a new tab opens, which is the liar tab. And this tab allows you to remove all liars from the data set as well as identify them. Now, reviewing this chart, we can see that there are no points greater than three, nor less than minus three. So there are no outliers. So they're going to select no liar. In the final check tab, the application is going to recommend you a test based on the previous selected choices. So in this case, he suggests a Welsh test. So this bar is only implemented for ANOVA. We can take a look at the ANOVA data here. So we're going to skip to the multiple comparison blocks since that is relevant to what we're doing. And so let's select the pairs for multiple comparison, have the X and the Y compounds. Now, by looking at the comparison results, we can see that the X vehicle is not statistically significant, since the adjusted P value is greater than 0.5. And the Y vehicle is statistically significant since the adjusted P value is less than 0.5. And by reviewing the tables below, the chart below, you can see how similar our vehicle on X groups and how the similar is that from the Y grouping variable. Now, the application throughout has a lot of choices, like increasing the dot size of the multiple comparison plots, the height of the plot, the title. So the application has embedded a lot of extensions that you can take advantage of in case that you have a grouping variable, for example. Okay, let's go to the sample size calculation tab. In the sample size calculation tab, let us choose very close to control group, the primary hypothesis as the Y compound vehicle, and as our secondary hypothesis, the X compound vehicle. Now we'll assume a certain percentage of mortality rate and a 50% of desired effect size. Now below, you can see that based on the previous elections, this table of effect size on the X axis versus sample size per group on the Y axis displays actually that the observed effect for the primary hypothesis is negative. So actually, we expect that the duration of effect is decreased in response. So if we go over the effect size of minus 50, we'll see that the sample sign is 25, which means that we need 25 animals per group to detect at least one of the primary hypothesis in the next experiment. We can also take a look at the power versus sample size. And here power means the probability of detecting an effect size in the next experiment. And when we hover over the default 80% statistical power, we see that we need 24, 25 animals per group to detect at least one of our primary hypotheses in the next experiment. Now application after you have reviewed all the tabs, you can generate reports. And for the sake of time, I've pre-generated for you. And you can take a look at the different sections. So it actually records each step of the application, the analysis set up, the initial diagnosis, the final diagnosis, their respective plots, the comparison table, and all, and the two plots on the sample size calculation for a one-way and all. We would like to end the presentation thanking the rest of the team. I went in one Taylor Reynolds, Michael Pearson, and Bob Inko. And thank you. So let's see, we have a question that's been upgraded. Does the app generate root reducible code for a specific analysis that's been run? It's like that one. So that's a very good question. And so far the application does not. But once application is open source and default, we are open for anyone making requests and working on them. Have you had a chance to look at any other ANOVA R packages? For instance, there's one called DOEX, D-O-E-X. Any thoughts on those? No, I'm not familiar with that one. Yeah, neither. We'll be interesting to look into. It's just too many R packages. Well, I don't see any more questions from the audience, but thank you. And you know, I really think that you've got a winner there with that and I hope that you get lots of users. Thank you. Thank you very much for joining us. Great. Thanks.