 In a typical research paper that uses multiple regression analysis, we do many different regression models. And the reason for that is that we want to do model comparisons. Now we will take a look at why we compare models and how we do that. In Hekman's paper, which is our example for this video, we will be focusing on their first study. So they say that they used hierarchical moderated regression analysis. So what does that mean? The hierarchical here is the key term. It simply means that you are estimating multiple models. Start with a simple one, then add more variables, compare, add more variables and compare. The moderated part here means that they have interaction terms in the model. They could just as well have said that they use regression analysis, because we use regression analysis nearly always in a hierarchical way. And it's obvious based on the regression results that they contain interaction terms. So this is a bit unnecessary, complicated way of saying we did regression, we estimated multiple models. Now let's look at the actual models and the modeling results and the logic for multiple model comparisons. They say that they entered in the first model, they have the contour variables only here. And in the second model they included some of the interesting variables. So we'll be focusing on the first two models, model one and model two. Model one is the control variables only, model two is controls and some interesting variables. The logic in model comparison when we do that kind of comparison is to ask the question, do the interesting variables and the controls together explain the dependent variable more than the controls only. If the control variables and the interesting variables together don't explain the data more than the controls only, then we conclude that the interesting variables are not very useful in explaining the dependent variable. And we can conclude that they don't really have an effect. How we do that model comparison is that we compare the R-square statistic. So here they have the adjusted R-square and the actual R-square. The model comparison, if we just want to assess the magnitude, how much better model two is, in small samples the more appropriate statistic is the adjusted R-square. However, the adjusted R-square statistic doesn't really have a well-known test. So instead of looking at the adjusted R-square, we test the R-square difference. They present the R-square difference here. So this is the difference between the first model R-square and the second model R-square and they have some stars here. So the important question is, does the second model explain the data better than the first model? The delta R, the adjusted R-square difference is 4. The actual R-square difference is 7 or 0.07, 7%. So the interesting variables explain the data a bit more than the control variables. Now we will be focusing on these test statistics here. So where do these stars come from? These stars come from an F-test that tests the null hypothesis that all the recursion coefficients for every variable added to this model are 0. We'll look at the logic of the test now. So the idea of the F-test between the first two models is that it is a nested model comparison test. So one model is nested in another. That means that one model is a special case of another. So in this case, model 2 is the unrestricted model or unconstrained model, model 1 is the restricted model or constrained model. So why can we say that model 1 is a special case of more general model, model 2? The reason is that model 1, which leaves out these variables, is the same model as model 2 except that the effects of these variables are constrained to be 0. So by leaving out variables we constrain the recursion coefficient of that variable to be 0 and that's the reason why we say that this model is a constrained version of that model. Here the effects of the last three variables are freely estimated. Here they are constrained to be 0. So how do we test these differences? Whether the difference in R-square is more than what we can expect by chance only. Remember that every time we add something to the model, the R-square can only go up. It can stay the same or go up. Typically it goes up. So is that increase in R-square statistically significant? To answer that question we do the t-test and let's do the t-test by hand now. We need to first have the decrys of freedom for the first two models to do the F-test. The decrys of freedom for recursion model is n, the sample size minus k, the number of estimated parameters, or recursion coefficients, or number of variables in the model minus 1 for the intercept. So we estimate, we have a sample that provides us 113 units of information. We estimate for the first model effects of 15 variables, we estimate the intercept. So we have 97 decrys of freedom remaining for the restricted model. In the unrestricted model we estimate three more things. So it's 113 minus 18 minus 1 is 94 decrys of freedom for that model. So these decrys of freedom calculations are pretty simple. It's just basic subtraction. Now we need to have a test statistic as well and the F-statistic can be defined based on the R-square values. So it's the R-square difference divided by the decrys of freedom difference divided by that thing there. So that's the F-statistic, your econometric textbook will explain where that comes from. But importantly we are here interested in how much the R-square increases per decrys of freedom consumed when we estimate the model. Quite often we compare increased explanation against increased complexity. That's a fairly general comparison which we use in multiple different tests. So we do that, we plug in the numbers, we get the result of 3.22. We compare that against the proper F-distribution, we get the p-value of 0.026 which has one significant star. So they present two stars, the reason I have no idea. But I've done this example in multiple classes over multiple years and I don't know why this is different. It could be that there's a typo in the paper or it could be, but that's probably the case. Because this kind of difference getting that because of rounding error in the R-square is quite unlikely. So that's the idea of F-test. You take a constraint model and you take an unconstraint model. You calculate the difference per, there are decrys of freedom difference. You scale it with this thing and then you will get a test statistic that you compare against F-distribution. In more complicated models for which we don't know how they behave in small samples, we use the chi-square distribution instead of the F-distribution. But the principle is the same. In practice your software will do the calculation for you, but it is useful to understand that these calculations are not complicated. And I have a little bit of understanding of the logic behind the calculus.