 Se on hyvin common to compare models after regression estimation. For example, we can use the F test for comparing or testing to nested models, or we could compare adjusted R-squares for two non-nested models. Similar comparisons can be done after maximum like estimation of simultaneous equation models. Let's take a look at different comparisons that are possible. Comparisons can be understood along three different dimensions. We could compare two different models. We could compare models using different samples, or we could have two different sets of variables. This latter comparison really makes sense. And even if you wanted to make that kind of comparison, it reduces the model comparison. So if you want to compare two different sets of variables, then you form a superset and you estimate one model to both supersets to the superset, concentrating some variables to have no effects or allowing them to be freely correlated with everything in the model, including error terms, in which case they wouldn't cause any misfit depending on what kind of research question you would be answering with the comparisons of two different sets of variables. But I don't remember seeing any studies ever that would actually have been published using this kind of comparisons. Then the comparison is basically reduced to four different combinations. Out of these, simply this same sample, two different models is a comparison. The reason is that if we have the same model and same sample, well, that's not a comparison. And if we have two different samples, then that is actually a multi-group structuralism modeling, which is estimated in one go and it's not a post estimation comparison. So comparing models, post estimation, we have the same sample or the same groups if we do a multi-group SCM, but two different models. Before we go into detail on these comparison techniques, we need to understand that for model comparisons, the models must be over identified. If you have a just identified model, it will fit perfectly to the data and comparisons involving just identified models don't make sense. If you are comparing two just identified models, then both will fit the data perfectly and there is really nothing to compare. If the comparison model is over identified and the benchmark model is just identified, then that basically reduces just to a test of the over identified model and it is not a model comparison. So if for model comparison purposes, both models should be over identified. If you have a nested model, then we do a likelihood ratio test or chi-square difference test. And here we have two example models. So we have this model here. This is the unconstrained four degrees of freedom. So it is a full media model where we allow endogeneity. So these errors are correlated. And then we have a full media model that doesn't allow endogeneity. So we assume that M and Y don't have any common causes that are excluded from the model. So the first model has five degrees of freedom and the second model has four degrees of freedom. And how the comparison works is that we calculate the difference in chi-square and difference in degrees of freedom. So these models have a chi-square of 0.03 and this one has 0.59. And then decrease of freedom difference is 1, chi-square difference is 9.2 and getting that kind of difference by chance only would be very unlikely according to the chi-square 1 distribution. So what's being tested here is that the constraint model is not substantially worse than the unconstrained model. So the null hypothesis is that these models fit equally well and the only reason why there is a difference in the chi-square statistical model fit is due to the sampling error. So in this case the test would say that this is the right proper model and this is an incorrect model that we should not use. There is one important thing that clients book note. So we have the issue of model fit. If your less constrained model is something that doesn't fit the data at all then your more constrained model will miss fit even worse. But really if you have two models that neither of them fit the data then what's the point in the comparison? So generally when you are before you compare models use it first establish that at least one of those models fit the data reasonably well and if it doesn't then you need to understand why. So another kind of model comparison is the non-nested model comparisons. So we can compare what client calls non-hierarchical models. And the most commonly used statistic for this comparison is the AIC statistic. The idea of AIC statistic is that we compare the likelihoods which are also used to calculate the chi-square statistic but then we also penalize for model complexity. So every time we add something to the model the minus 2 log likelihood will get smaller. So in other words the likelihoods are actually very small numbers. So they are well below, they are very close to zero. And when we take a log of the likelihood then we get a very small, very large negative number. And when the fit improves then the log likelihood gets closer and closer to zero. So the minus 2 log likelihood will get always larger when we add parameters. We always get smaller when we add parameters to the model so we penalize for model complexity. Because comparing a simple model against a complex model wouldn't be a fair comparison. With AIC we know that less is more so smaller AIC is a better model. So we calculate AICs for two non-nested models and then we choose the one with the smallest AIC as the better model. This has a couple of disadvantages. The most important disadvantage is that this assessment does not tell us or cannot tell us if the models are equally good. So it always, this comparison always gives us one model that we should prefer. Also the difference in AIC doesn't really have a meaning. So we can't really say how much different better the other model is compared to the comparison model. And this is not the proper statistical test. It doesn't have a p-value. It is just an informal comparison for model selection purposes. But for non-nested models this is a very commonly used technique and it's useful if you understand its limitations. Whereas in nested model comparisons the likelihood ratio test is basically the final thing we have known for it for long time and that's statistically appealing. There is lots of ongoing research on non-hierarchical models. So if you want to understand what's the, the state of the art you need to do some reading because by the time you watch this video it's probably different from the time of recording. But generally the AIC is the most commonly applied way of comparing non-nested models. So summary of model comparisons. You need to understand about model comparisons. The first is that caveats for model testing apply. So you need to understand what are the constraints in both models. So what is being actually tested in those models that you are comparing. So what is the source of misfit in those models. And the second thing that you need to understand that model tests and model comparisons neither can tell us pretty much anything about the direction of influence and you need to consider that in your research design. That is pretty much beyond what covariance model fitting can tell us. Then there are two comparison strategies. We have the likelihood ratio test for nested models and we have AIC comparisons and others for non-nested models and this is an active research topic. So if you want to know what the state of the art is you need to do some work on Google Scholar to find out. Finally, comparing models that fit badly does not make much sense. You need to first understand why one of the models or both of the models don't fit the data because comparing two in your correct models doesn't really give us any additional information about the phenomenon.