 There are two bad practices in confronted factor analysis diagnostics that I want to address. Unfortunately, these two bad practices are really common. The first bad practice is dropping indicators based on reliability estimates. So here we have a confronted factor analysis model with a single factor of four indicators x1 through x4 and the results show that x4 has a lower loading than the other indicators and the standardest loading of x4 is also below the 0.7 cut-off that some people use as an acceptable reliability. Quite often, when faced with this kind of result, researchers drop the indicator because it is less reliable than other indicators. So why is this bad practice? This is bad practice for a couple of reasons. It is bad practice because scale modifications should be justified by the items, not by the results. So you should actually look at the item x4 and then tell why you think that it is less reliable instead of just saying that it is estimated to be less reliable. Also estimating a reliability to be lower can be an unreliable estimate itself if the model is mis-specified. So it is possible that there is some dimensionality in the scale that you don't model and for that reason one of the indicators is estimated to be less reliable than others. Also the model needs to be diagnosed. It has a non-significant chi-square which means that there is unmodeled dimensionality. A poor reliability or low reliability of an indicator itself does not cause any problems because it just increases the error variance of the indicator and that is appropriately modeled in the model. So the fact that we have a significant chi-square means that there is dimensionality and dimensionality needs to be explored and addressed instead of just dropping an indicator based on reliability estimate. And items with reliability, if the model is correctly specified, they wouldn't lead to bias in any case. So if you have a computer factory analysis model and then you move to structurally custom model it is actually better to keep those unreliable items in the model than to drop them as long as the items are valid which means that the model fits well and if it doesn't at least you know what is the source of mis-spec. So what is the model here that generated the data like? So in this case we actually had a secondary factor F2 that loaded on X1, X2 and X3 and that explained why X4 was loading less because the main factor actually captured some of the secondary factor which was not included in the model and if we take a look at this data generating model more closely we can see that X4 is actually more reliable. If it is loading of 1.2 compared to loading of 1 it is more reliable than X1 or X2. So when we are dropping X4 yes we will get a well-fitting model because it is just identified but we are also dropping the best indicator that we have and we will not notice that there is actually a secondary dimension to the scale that we will probably want to model with the by factor model instead. So this is the first bad practice. You should never drop indicators based on the reliability in a confirmed factor analysis model. If you do scale scores after factor analysis then things are a bit different. In that case if you have extremely small reliability and if you know that that reliability is not just because of mis-specification then in some scenarios you would be better of dropping the item but in most cases you should keep all the items in the model unless you can show that there are cross-loadings or other dimensionality problems. Reliability problem is generally not a good reason for dropping an indicator. So this is the first problem. The second problem is following modification indices blindly. So we have another data set, we have the same model. We have a highly significant chi-square and we start addressing this highly significant chi-square by adding things to the model. And what we do first is that we take the modification indices, we show that xc, that x1, x4 correlation, if we add that to the model then our modification, our chi-square will increase a lot. So that is bad practice for a couple of reasons. So quite often we would add an error correlation between the variables indicated and we would be looking at what is the expected parameter change and try to think if that makes sense and well, that's bad practice. It's bad practice because scale modifications will be justified again by the items, not by the results. And we would need to understand why is there a correlation by looking at items and then that is the justification, not the fact that we have a modification index. The model needs to be diagnosed. The modification indices does not really, it's not really an appropriate way of doing diagnostics. You need to understand, if you're looking at one value, you need to take a look at the residuals and understand the big picture. So modification indices are useful, but they are not very useful unless you also combine that to, for example, expert factor analysis for discovering dimensionality or residual analysis to understand patterns in the variables. The modification index always focuses on a single parameter, but often adding factors with multiple parameters would be more appropriate. Particularly in this case, using a by factor model would be better. It is almost always generally better to use a by factor model than freeing a correlation because by factor forces you to interpret the model and also if the right choice to do would be to actually add a factor that loads on multiple items, then the modification indices would indicate that the by factor actually needs more indicators than just two, which you would get when you do a correlated error. So what is the data generation process like? And here we have a model with one main factor F1, all indicators load on F1 equally. And then we have F2 and F3, two minor factors F2 and F3. F2 affects the first three indicators, F3 affects the last three indicators and why X1 and X4 are correlated less than the other indicators is that X2 and X3 are affected by both minor factors, but X1 and X2 are affected only by a factor that does not affect the other one. So misspecification here is two minor factors, not a single correlated error. There's always a reason for a correlated error and the reason relates to dimensionality and instead of following the modification indices blindly, you should be looking at the diagnostics more holistically, running different factor analysis, looking at patterns in residuals and then looking at the survey items or whatever data you have are actually about and whether that tells you about something about dimensionality. Instead of just looking at what is my highest modification index and then adding parameters based on modification indices to get a correct world feeding model.