 Kun luokasimme sähköisellä analysiit, voimme aina alkaa poittia, tai joudun koko ajan suurin, koko ajan suurin tai koko ajan. Sitten pitäisi tulla, miten järjestämme sähköisellä ja se on kenttävä kenttävä kenttävä. Se kenttävä kenttävä kenttävä ja käytämme sähköisellä ja sähköisellä kenttävä ja kenttävä kenttävä joka on kenttävä. Nyt esimerkiksi, Stannadero on hirveä, tai kalkkulajan Stannadero on jotain, joka on tarpeeksi, että me ei olemme voineet tehdä, tai asumme, joilla me tiedämme, että ne eivät ole todellisia perustamme dataa ja analysoidaan. Bootstrappingi on tullut uudestaan kalkkulajan Stannaderoa, tai esim. kuinka paljon, että statistiikki tarvitaan yksi samppalista. Bootstrappingi on täysin komputationaalinen approaches to the problem of calculating a Stannadero. How bootstrapping works is that we have our original sample. So we have a sample of 10 observations here from a normally distributed population with mean of zero and standard deviation of one. So that's our original sample here. The mean is 0.13 from that sample. And if we take multiple samples from the same population, here is the sampling distribution of the sample mean if the sample size is 10 from this population. Most of the time we get values close to the zero, which is the population value of mean. And then we have some, sometimes we get estimates that are far from the actual population value. The idea of bootstrapping is that if we don't know how we estimate this the width of this sampling distribution or the shape using statistical theory or a closed form equation, then we can do that empirically. So instead of calculating it using an equation, we take repeated samples from our original sample. So that's our original sample. It forms the population for the bootstrap. Then we take a repeated sample. So we take first 0.31, it is here. Then we put it back. So we allow every observation to be included in the sample multiple times. Then we take randomly another one 0.83, it's here. We put it back. Then we take yet another number, yet another number. We take the minus 0.84, the second time and so on. So we take these samples from our original data. And every observation can be included in the sample multiple times. So each of these randomly chosen numbers doesn't depend on any other previous choices. So we get using this bootstrap sample, we get 0.34 as sample mean. We calculate many, many times, typically we do 100, 500 or 1000 times or even 10,000 times depending on the complexity of the calculates. 1000 repetitions is quite normal nowadays. So we can see that from sample to sample this sample mean varies and this variance of sample mean, calculator, the distribution of this sample mean from the bootstrap samples, calculated from 1000 bootstrap replications here is about the same shape as that if we would take the samples from the actual populace. So these two distributions are quite similar and we can use that information, the knowledge that these two distributions are similar. They approach each other when the sample size increases. We can use that knowledge to say that this distribution here is a good representation of that distribution and if we want to estimate the standard deviation of this distribution which is what standard error quantifies or estimates then we can just use the standard deviation of that distribution. Here we can see that the mean of this distribution is slightly off. That's called the bootstrap bias. So this mean here is roughly at the mean here. So it's not that the populace mean instead of it's closer to the mean of this particular sample. Then we also the width of this distribution is in this case slightly smaller. So the dispersion here is slightly smaller than the dispersion here and that is also something that we sometimes need to take into consideration. The key thing in bootstrapping is that once sample size increases then this mean and this standard deviation will be closer to that mean and that standard deviation. Let's take a look at a demonstration of how bootstrapping works. This is a video from a statistics department from University of Auckland and they demonstrate that you have your original sample here. So we have two variables. We have the x variable and the y variable and then we have a regression coefficient. So we calculate the regression coefficient here and we are interested in how much this regression coefficient the slope would vary if we were to take this sample over and over from the same population. So that's what the standard error quantifies. For some reason we don't want to use the normal formula that our statistical software uses to calculate the standard error. We want to do it by bootstrapping. So we take some samples from our original data. So we take samples from the original data like so. You can see here that each observation can be included multiple times. Sometimes an observation is not included in the sample. Then we get the regression coefficient that is slightly different from the original one. We do another bootstrap sample. We get another regression coefficient again slightly different from the original one. We take yet another bootstrap sample. We get slightly different one and we go on a hundred times, a thousand times and ultimately we get an estimate of how much this regression coefficient would really vary if we were to take multiple different samples. So that's when you get a thousand samples or a hundred samples. Then you can see that the variance of the regression coefficient is that much. A bit in the bootstrap samples and if samples are large enough, this variation of the bootstrap samples is a good approximation of how much the regression coefficient would vary if we were to repeat the same independent samples from the same population and calculate the regression analysis again and again from those independent samples. Bootstrapping can be used to calculate the standard error, in which case we just take a standard deviation of these regression slopes and then that is our standard error estimate. We can also use bootstrapping to calculate confidence intervals. So the idea of a confidence interval is that instead of estimating a standard error and a p-value, we estimate a point estimate, so for example a value of a correlation, one single value and then we estimate an interval, let's say 95% interval, which has an upper limit and lower limit. And then if we repeat the calculation many, many times from independent samples, then the population value will be within the interval, if it's a valid interval, 95% of the times. So this is an example of a correlation and we can see that the correlation estimates when there is zero correlation in the population, we have a small sample size, they vary within 0.-0.2 and plus 0.2 and most of the time when we draw the confidence interval, which is the line here, the line includes the population value. This is 2.5% of the replications here and it doesn't include the population value. So the population value here falls above the upper limit. Here we have extremely large correlations and the population value for about 2.5% of the replications falls below the lower limit. In 95% of the cases here, the population value is within the interval. So that's the idea of confidence intervals. Here we can see that when the population value is large, then the width of the confidence interval depends on the correlation estimate. So when the correlation estimate is very high, then the confidence interval is narrow, when the correlation estimate is very low, then it's a little wider here, than the confidence interval. So the confidence interval depends on the value of the statistic and also it depends on the estimated standard of the statistic. Now there are a couple of ways that bootstrapping can be used for calculating a confidence interval. In normally when we do confidence intervals, we use the normal approximation. So the idea is that we assume that the estimate is normally distributed over repeated samples. Then we calculate the confidence interval. It is our estimate plus or minus 1.96, which covers 95% of the normal distribution multiplied by the standard error. So that gives us the plus or minus. So if we have an estimate of correlation that is here, then we multiply the standard error by 1.96. Minus, estimate minus that is the lower limit estimate plus 1.9 times the standard error is here. So that gives us the upper and lower limit. In this example 1% and 13% when the actual estimate is about 5%. So we calculate how we use bootstrapping for this calculation is that the standard error is simply the standard deviation of the bootstrap estimate. So if we take a correlation, we bootstrap it, then we calculate how much the correlation varies between the bootstrap samples using standard deviation metric. And then we use that plug that into that formula gives us the confidence interval. So that works when we can assume that the estimate is normally distributed. What if we can't assume that the estimates are normal? That is the case when we can use empirical confidence intervals based on bootstrapping. So the idea of the normal approximation interval is that the estimate is normally distributed, then we can use this equation or we can use empirical confidence intervals. The idea of an empirical confidence interval is that we do the bootstrapping and then let's say we take 1000 bootstrap replications, then we take the 25th from smallest to largest, we take the 25th value of the bootstrap replicates and that is our lower limit for the confidence interval. Then we take the 975th and that is the upper limit. So that's 2.5 percent and 97.5 percent and that's the upper limit of our confidence interval. So that's called percentile intervals. So when we have this kind of bootstrap distribution, we will take a replication here, the 25th replication, that is our lower limit and we take the 975th replication here, that is our upper limit. So that gives us the confidence interval for the mean that is estimated here. That has two problems, these approach. First, the bootstrap distribution is biased. So the mean of these bootstrap replications is about 0.15 and the actual sample value for the mean is 0. To account for that bias, we have a bias corrected confidence intervals. The idea of bias corrected confidence intervals is that instead of taking the 25th and 975th bootstrap replicate as the end points, we first estimate how much the bootstrap bias is and then based on that estimate we take, for example, the 40th and 980th replication. So instead of taking the fixed 25th and fixed 975th, we adjust which replicats we take as the end points. There's also the problem that the variance, the standard deviation here, is not always the same as the standard deviation of here. So in the correlation example, you saw that the confidence interval decreased as the actual correlation estimate went up. So the idea is that the width of the interval depends on the value of the estimate. To take that into account, we have a bias corrected and accelerated confidence intervals, which apply the same idea as the bias corrected ones, but instead of just taking the bias into account, they take the estimated differences in variance of these two distributions into account when we choose the end points for the confidence intervals. Now the question is, this looks really good, so we can estimate the variance of any statistic empirically, and we don't have to know the math. And yeah, that's basically, it's true with some qualifications. The qualifications are that bootstrapping requires large sample size. There is a good article or a book chapter by Koopman and co-authors in a book edited by van der Berg about statistical myths and urban legends, and they point out that there are three different claims made in the literature. There's the claim that bootstrapping works well in small samples, and there is a fact that bootstrapping assumes that sample is representative of the population. So if our sample is very different from the population, then the bootstrap samples that we take from our original sample cannot approximate how the samples would actually behave from the real population. Then our sampling error which means how different the sample is from the population is troublesome in small samples. So in small samples the sample may not be very accurate representation of the population. So if small samples are not representative of population and if we require that sample is must be representative of the population, then bootstrapping cannot work in small samples. So bootstrapping generally requires a large sample size, then there are also some boundary conditions under which bootstrapping doesn't work even if you have a large sample. So there are that kind of scenarios, but for most practical applications only the sample size is the thing that you need to be concerned about. The problem is that it is very hard to say when your sample size is large enough.