 Hi everyone, my name is Erin Buchanan and I'm here to teach you today about the R package MOTE. MOTE stands for Magnitude of the Effect or Measure of the Effect. We actually picked one a long time ago and we forgot, so whatever it says on GitHub. But either way, MOTE is a package that lets you calculate effect sizes and their confidence intervals and especially uses confidence intervals that are non-centralized. So you can use this package on many types of effect sizes that you would find in reported research like D, R squared, eta, G E S, and omega squared. So let's check out how to do this. I built a little template markdown. If you have not installed MOTE, it is available on CRAN, so you just simply use installed packages. You can also install it directly from GitHub. I won't run this because I have already installed it, but you should at first install that package. I do assume you have some knowledge, working knowledge of R when you are watching this tutorial. To work today, I'm going to load MOTE, dplyr, because it's my favorite, and easy, so I can show you an example of an ANOVA. So let's run all those bad boys. Cool. So the first thing I really want to do is say that we spend a lot of time on the help pages, so I hope that you look at them. And so to look at those, you can just do question mark or and type MOTE as a package, or come over to the packages window, find wherever MOTE is at, and click on it. Click on the name and not just the description, but then we have a whole bunch of examples. So here are all the things that MOTE includes so far, but it is an actively developed package when I get a break, but we have in general, the rule is that the name of the function starts with the effect size you're calculating. So everything that starts with the D is for Cohen's D or some form of T as an effect size. Everything that starts with Delta, his glasses, Delta, right? Epsilon, eta, GES for the generalized effect size, right? Odds ratios omega, R, and chi-square, so Kramer's V. The other things to say data next to them are example data points from each of these particular pages. So if I click on eta here, it tells you about the function, the usage for the function, so how, what all the arguments are, a description of what you should put in each argument for these functions. The details where we tell you what the formula is for the function, right? So this is how eta squared is calculated, given the inputs that you put in. You can click on this link to go to the example page that has even more stuff on it. It still works, fingers crossed. This will render in latex format after it thinks for a hard while and you can read it a little better. It has an example from each page. Now, the website has more examples and these might be really good for your students who are trying to translate output from different statistical programs, right? So here's an example from JAS, an example from SPSS, an example from SAS, and how that function runs in R. In a different video, I'll teach you how to use the online shiny app version of this. So it explains all of the outputs that you'll get once you run this effect size code and then examples. So these examples are all from like intro stats books, but it shows you how to calculate something. So all of these examples will work directly from the code. So you can just copy them or actually I think you can just click run examples and it will show you the output where it won't get hit enter and you'll get output. So we put a lot of time and effort into that documentation. I would love for you to look at it. Now, what all do we have? Well, we have the main effect sizes that you will find in the literature, other effect sizes forthcoming, but mostly the common pairwise test the glasses delta G and then the common linear model tests, right? So eta, omega, generalized effect size, the one for chi-square, Kramer's V, and some odds ratios. So we have a good number of those different effect sizes and they are based on the type of research design that you have. And then I will say that if you're writing your markdowns in R, maybe with the fabulous papaya, you can also pull out the effect sizes in latex format and tell it to print directly. It mostly works in PDF-nitted versions, but it will format it in APA style for you. So let's look at a quick example, two quick examples. So to do this, I'm going to use the Palmer Penguins data from the Palmer Penguins package in R. It's a really great package if you are tired of iris or empty cars. And I'm just omitting the missing data from the Penguins data set. So it's got a list of penguins, their species, their buildings, their body mass, and gender. And so I'm going to use TAPI. You can also do this with dplyr and do calculate by groups. But one thing that we've done so far with Moet is simply entering the numbers directly, but you don't want to type them because typing them directly means you might make a typo or this won't be dynamic with changes in your code. And so what you can do is save your mean, your standard deviation, and your sample size. So let's look at those. The body mass difference, it does seem that females are lighter than males in penguins. The standard deviation for that body mass and grams and the sample sizes for each of those. Okay, so how many of each do I have? Now that is two distinct groups of data. So this is between subjects effect size. And since there's only two groups, I can calculate d. I can also calculate r, but for a lot of folks d makes more sense as the magnitude of the difference between them, as opposed to the correlation between them. So we'll do d dot i and d for independent samples dot t. And again, if you're not sure what you're supposed to enter, use a question mark. So i and d meaning independent samples, t test as the test. If I did i and d dot t dot t, this was before I've figured out that Python does lots of dots, so please forgive me. i and d dot t dot t means I want to calculate d for independent samples, t test on the t value. So you can convert directly from t. However, it's usually better to convert directly from the means, so that's what we'll do. And these two, they're equivalent in repeated measures tests. They may not be equivalent. So we'll say mean one equals mean one. Mean two equals, sorry, too much scrolling. Mean two, standard deviation one, standard deviation two. I should label this better. In one and into, you can see all of those listed if I back up in order that you're supposed to write them in. And this way I'm just saying for the first mean and standard deviation, compared to the second mean and standard deviation, my alpha, my significance level is 0.05, that means I'll get a 95% confidence interval. So let's see what happens. Well, I saved this on purpose so I could show you that it prints a ton of output. One way to look at it is just print it out. The other way to look at it is use the environment and look at everything. So we'll give you the effect size with its label, so d, low and high, the 95% non-centralized confidence interval for that effect size. And then for this statistic, we actually will give you the mean, standard deviation, which you entered directly, but also the standard error, the confidence interval for that mean, right, based on its standard error and sample size, mean two, standard deviation, standard error, and that's confidence interval, s pooled and standard error pooled, which are part of the calculation, a for t and one and two, the degrees of freedom, t and p. So you actually get like the entire t-test, including the really fancy way to put it out into your document. So if I wanted to print this in late-time format, I could say effect dollar statistic, which will print the t-test statistic, okay, the dollar signs will give it the right appropriate formatting look in late-time. And so I could say, wow, that's a really big effect size. And I could also do effect dot, I forgot, estimate, okay, and that'll give you the estimate for Cohen's D. Notice here that it's got this s statistic, okay, that's Cohen's D for independent samples, okay, that's from the Lakin's paper on the best way to label all these different versions of the what's called Cohen's D. And so this will print for you in nice pretty APA format in latex style if you're using that. Otherwise you can grab each piece one at a time, effect dollar D, dollar mean, etc. Now that's how we might look at t. Let's look at a effect size for ADA. Okay, so I'm gonna use easy ANOVA for this example because it's one of my favorite ANOVA functions because I can make the output match SPSS kind of easily to show people that they do provide the same answer. I do have to have an ID number for easy ANOVA to work, and I'm just going to run a very basic one way between subjects ANOVA, comparing the different types of species of penguins on their bill lengths. So is there a difference between the different beak lengths on penguins? I didn't know this until I ran this. Now looking at the easy output, right, it gives me Levine's test, which is the second one I think, prints kind of funny in our markdown, but the first one is the ANOVA. So that we see that, yes, we do have a significant difference in the buildings of different penguins, but the nice thing about easy is it will give you generalized ADA squared, and in between subjects test, that's the same thing as ADA squared, so we can make sure that we're actually doing it right, and then it'll give you Levine's test. So let's see. I'm going to calculate ADA F, so ADA directly from F. Now I could calculate ADA fully from the sum of squares or partial ADA squared from the sum of squares, but we'll just do the simple one, which is directly from F. You put in your degrees of freedom model. Now I'm using the terminology from the anti-field book of degrees of freedom model. It's the same thing as degrees of freedom numerator, degrees of freedom error here. It's the same thing as DFD, or degrees of freedom denominator. We put in F, which is this number, and our alpha, which we're going to stick with point of five. And what do we get? Same kind of output ADA, so 70.70. It's a really big effect size. I don't make them this large on purpose, right? But if we compare that, we get the same number as our generalized effect size. And then going back to it, we get a low and a high statistic, model error F, and it calculates P again for us. So I can again print out the estimate and the statistic so that I have the complete package of all the numbers I should report in my APA output. So it's a very quick and easy version of how to use moat, right? Like I said, we have so much documentation. Please love to look at it. Tell us what's wrong with it and give us some feedback. But the documentation should help you if you're not sure where the number is coming from. It's got examples for three or four different types of output. So I'm glad that you're here today. Enjoy moat. Hi guys, my name is Erin Buchanan and I'm here now to show you how to use the online shiny version of moat. You want the R package? That's a different video. So one thing first is that we have this really great website that shows you everything about the package and has a really great documentation. So you can click on tests and look at all the different tests that we have sorted by the type of test that each effect size goes with. The references for how we pulled a lot of this. Big props to Ken Kelly's M-Best package to help us out. And just an introduction of like what the heck is a confidence interval on an effect size anyway. So all this goes with our shiny app, which hopefully will load. All right, so when you're working the shiny app, this is really great like in the classroom or for students who need to calculate on their own and you don't want to teach them coding but you do want to teach them about effect sizes. So what you do is you pick the type of statistical test normally that you're pairing this effect size with. As we organized it around that because it's generally what students are learning, right, is the type of test. So let's do an easy one and do an independent t-test. I'm going to calculate that from means. Then over here you enter all the information that you might find. What's really nice and thank goodness to some graduate students of mine at the time is that for each test we have examples. So where's our independent t? Between subjects with, here we go, okay for your pulled standard deviation, the denominator, that's independent t, right. As we have a description of the formulas here in latex format, so it tells you the formula for the test. It tells you the formula for t and how those two things are different. So you could use this for teaching. Relevant r information gives you an example and the data is available on GitHub. Then for three of the biggest popular programs we have JASP, which Jmovies Output is very similar, with SPSS and SAS all shown here on this data pictures of what that output looks like so that you could show students, you know, this is the t value you'd need. Here's the mean that you'd need, grab it from here, take this from there, okay. And then it also shows you how to calculate this in moat. So how you'd enter information into the package or the shiny app and then what effect size that you might get and how you would interpret that output all the way down through the interpretation. I also have a whole series of videos on YouTube beyond this one that go over these exact things as well. So this is like a summary of the summaries, almost, but let me show you the package in action. So we're just going to make up some numbers here. So let's compare two to four, something nice and easy, right, with a standard deviation of two in each one. So that would be two, four minus two, right, divided by two, so our effect size should be approximately one. Give or take, you know, sample size. So let's see, we've got a hundred and a hundred just to make these numbers nice and round and easy. And then we'll hit calculate. Now one thing that's confused people in the past is there's a number of us supposed to go here and so sometimes people hit calculate and it doesn't do anything. You do actually have to type the number in the box. So we just put in the default of what you should be putting there, right. If you're a student you may remember that alpha is that p less than 0.05 thing. Hit calculate. It'll tell you the definition of that particular effect size. It'll give you the effect size and its confidence interval. So our effect size is one. It will interpret the effect size, so if it doesn't include zero it will spit out your effect size does not include zero. So most people might think that that's not different from zero, something very simple like that. It actually will print the group summary for you, so it'll calculate standard error and the 95% confidence interval for each group and it'll actually print the t-test as well. If you're trying to transition students from SPSS to R you could also show them how it works in R. And so this is the help guide pulled directly from R. And then the help page is a video that shows people how to pull information. And so between the online website and the data embedded directly in the app you should be able to calculate just about everything on these two. And these are the students that helped with this. They're wonderful. They're my favorite heroes because this took a lot of stuff to set up and they did a really great job making sure all of the help pages look as good as they do. So I will not take any credit for that but we'll take credit for putting together a very extensive list of possible effect sizes for you guys to use. And hopefully this will also help with teaching because we put a lot of time and thought into having the information and the interpretations listed on the pages themselves. So let me know if you have any questions but this is how you use Moat the Shiny app and where to find all the information in case you get stuck.