 going to be giving an introduction to calculating confidence intervals around effect sizes. More and more journals now are starting to require that authors include confidence intervals along with effect sizes and test statistics in their papers. And so learning to calculate these confidence intervals is becoming a necessity for many researchers. So today I'm just going to give a short overview on a couple of tools that you can use to calculate these confidence intervals. So some of you may have seen the following formula for calculating 95% confidence intervals around effect sizes. So the classic formula is the effect size plus or minus 1.9 times the standard error. Now you'll notice that this will result in perfectly symmetric confidence intervals because both the upper and the lower bound are 1.9 times the standard error away from the effect size. So this is fine for some types of effect sizes but the problem is that many test statistics like the t, the f and the chi-squared test statistics aren't actually symmetric when the null hypothesis is false. So if you get a p-value of less than 0.05 that distribution is not going to be symmetric anymore. And so the confidence intervals also won't be symmetric. To give you an idea of what this looks like this is an example of four different t distributions. So if you look at the purple and the orange one that's when there is no difference. And you'll see that those look pretty symmetric. Those look like the regular t distribution that we probably all remember way back when from when we took interest statistics. However if we look at this blue and this blue and this green where there is statistically significant difference you notice that the distributions have started to become skewed. And the more of an effect there is the more skewed these distributions will become. And so because of these non-central distributions that's why that equation from before won't work. Because the equation assumes symmetric confidence intervals and when the distribution is not symmetric the confidence interval should also not be symmetric. In order to calculate confidence intervals for these types of distributions we actually need to calculate something called a non-centrality parameter. And use this non-centrality to determine the confidence intervals. So calculating non-centrality is actually rather intensive. It's an iterative process you would really never want to do this by hand. And so programs have actually been developed to do this for us. Today I'm specifically going to be showing how to calculate confidence intervals using an R package called the M best package. Now different statistical software packages have different ways and different functions for calculating confidence intervals. I'm going to be showing how to do it in R today for two specific reasons. One is that the M best function is probably the most extensive function for calculating confidence intervals. For example there are add-ons for SPSS that will calculate a very small subset of confidence intervals but the M best package is far more extensive. Now you might say well I'm not typically an R user. The M best package is standalone. You don't really have to learn how to use the entirety of R or be a good R user to use the M best package. It's a pretty simple package in terms of the coding required and so I actually think that even if you don't use R for your normal everyday statistics that the M best package is still a good way to go for calculating confidence intervals around effect sizes especially if the package the statistical package you currently use doesn't have a lot of good resources for calculating confidence intervals in the statistical program itself. So in order to calculate these confidence intervals we're going to need a few things. The first is we're going to need to know our test statistic for the function we want to calculate. We're also going to need to know our sample size so how many samples were there in the study and our desired level of confidence. A lot of times 95 percent confidence intervals are used as the default and the reason that this is used as the default is that if you use a 95 percent confidence interval that's basically another way of doing a p is less than 0.05 null hypothesis test. If the 95 percent confidence interval does not include zero then you know that p will be less than 0.05 and so you can get the same information from the p value and from the confidence interval if you want. So now I'm actually going to show you how to code a few examples in R. So the first thing I want to do is install the package if I don't have it installed already. So I'll type install.packages and then put in quotes and best and then I have to load the library. So I already have this package installed so I'm just going to run the library command. So for this example I'm going to assume that I did a t test it had 38 degrees of freedom and it was equal to 2.31. So the way to actually code this in the r function for t tests independent samples homogeneity variance assumed I'm going to use the function ci.smd so the confidence interval around a standardized mean difference and there are two ways that I can type this in. The first is if I know my t statistic. So in that case it will ask for non-centrality parameter which is just the t statistic. I also need to know the number of samples in each group. So for a t test that's independent samples if I have 38 degrees of freedom I had 40 samples total so I'm going to assume that these samples were equally distributed between my two groups so I'm going to say n.1 equals 20 and n.2 equals 20 as well and then I need to put in my confidence level. As I mentioned in many fields the default confidence level is a 95 confidence level so that's what I'm going to put in here and then if I run this you can see from this output I'm going to get a couple of things. The first is I'm going to get my lower confidence limit so the lower bound is 0.08 and then I have my upper confidence limit which is 1.37. What's nice is that also actually gives out the standardized mean difference so the coincidence d which is 0.73 so I don't need to know my effect size beforehand the package will give it to me as well as my confidence intervals. However you might come across a case where you know the coincidence d but you don't know the t statistic so I can also type this function out with a slightly different input so rather than inputting my non-centrality parameter I can actually input my standardized mean difference so my coincidence d so I'm going to do the exact same thing as before just input my coincidence d which I know from the previous command is 0.73 and then I'm going to do the exact same code for the rest of it and then run this and so what you'll see which is what we would expect is even though I have a slightly different input because it's the same function and the input is has the same meaning that my lower and my upper confidence bounds are exactly the same as they were before. So one thing I should note I mentioned that I'm using 0.95 because it's the default and before I mentioned that 0.95 is often used as the default so that you can basically do a p is less than 0.05 null hypothesis test by looking at the confidence interval so there are times when a 95% confidence interval makes sense if in your particular field there's a different default of course use that but even within fields where the 95% confidence interval is the default there are times when it makes more sense to use other things like a 90% confidence interval and these can be things like when you're running a one-tailed test or when you're running a test that has a one-tailed distribution like an f-test a 90% confidence interval actually makes more sense and will be analogous to a p is less than 0.05 null hypothesis test or for example if you wanted your confidence intervals to correct for multiple testing you could use different confidence intervals and in a different video I'll go over the specifics of when you might want to do that and what confidence intervals you'll be using I just wanted to mention in this video that don't always just default to 0.95 because it is the standard in many fields it makes sense a lot of times but there are exceptions to the rule so I would say the one drawback of the mbes function is that the documentation for the function can sometimes be very confusing so this is the actual crayon documentation pdf for this function and if you'll notice there are many different functions within it so if I actually go to this ci.smd so from the function description I can tell that this is used for a cohen's d so it's probably going to require a t-test now the problem is that this actually leaves some information out for example it's a little hard to tell that this is specifically for a t-test where I can assume homogeneity of variance some functions are also not this specific so if I go to another function ci.sm all it says is this function this is the function to obtain exact confidence intervals for a standardized mean you might be able to figure out that this is the function for a one sample t-test but it's not abundantly clear just from the documentation that's provided for the package and so it can become a little tricky if you know the test you ran to figure out based just on the documentation exactly what function within the mbes program you would actually want to run so one thing we did was we're creating a spreadsheet that is publicly available to try and make it easier for people to figure out which function they want to run based on these statistical tests that they ran so this is the spreadsheet we're putting together you can see that this is still very much a work in progress but what you'll notice is that there will be different test statistic families and then different types of those tests so within the t-test family right you have independent sample homogeneity of variance independent sample non-homogeneity of variance one sample paired and one-tailed tests and then for each of those there's the specific function within the mbes library that you would use to calculate confidence intervals for those tests we're also going to start adding information about other statistical programs such as spss, sass and stata if it's possible to calculate those confidence intervals in that program so you can use this spreadsheet as a reference to make sure you're using the right function to calculate your confidence intervals so hopefully this has been useful for you if you have any questions about anything that we went over in the video feel free to email us at stats-consulting at sos.io and we'd be happy to answer any of your questions you can also email us at contact at sos.io with questions comments or feedback if you have any thanks