@camerinobarron It is on my wiki: statwiki. kolobkreations. com Also, there is a better way to do this kind of analysis now. Check out my new video "multigroup moderation in amos made easy".
Your Stat Tools package is fantastic. Thank you very much. Do you happen to have a version which would allow for testing multigroup moderation between more than two groups? E.g. four groups?
Testing differences between more than two groups is always a case of testing the pairwise differences (i.e., between group 1 and 2, then 1 and 3, then 2 and 3). This excel tool will do it; you just have to do it two at a time.
Thanks for this, very interesting. I did wonder why you didn't just do a multi-group analysis though. Wouldn't that be easier? You could then also ask for Critical Ratios of Differences - That way you can check which paths differ across the groups without having to resort to laboriously checking each one. Or am I missing something?!
Sorry for the second question. We are running into a problem with "just identified models" with these models. Unfortunately for females ALL pathways are sig, whereas with males, several pathways are not sig so I cannot reasonably trim any pathway. So we cannot test model differences because with a just-identified model you don't get a Chi-square value. Anyone out in the you tube universe have a suggestion for us??
@sgcraig84 There are a couple work-arounds. The best method is probably to add a control variable and hope it is not significant for females... Or, add another independent variable that you expect will be significant for males, but not for females. Or, simply trim the least significant path for females (assuming it is not the only path tying the exogenous variable to the model). Feel free to email me directly if these options don't sound appealing or if they don't work. email: jeg82@case.edu
@Gaskination Much appreciated your sharing. A Quick Question: To achieve a unconstrained model, we need to trim (delete) the path that is insignificant. In your example, the paths deleted are insignificant for both groups (male & female). What if the specific path is significant for female and not significant for male? Then should we delete that path to reach unconstrained model?
I believed it is to delete the path if the path is significant on either 1 group.
In my case, there are only 3 paths (in the whole model, and All three are hypothesized to be moderated by the moderating variables). 1 path is insignificant on both groups, so we trim the path. The other 2 paths are insignificant only on group A, but significant on group B.
Is this means that I don't have to continue to proceed to the constrained model? or there is other ways?
@XenovaKyo Great question. The answer is that you only remove the paths that are not significant for BOTH groups. If it is significant on one, but not the other, then you leave it in. Also, the easier way to do this is shown in my other youtube video: "Multigroup Moderation in Amos - Made Easy" I highly recommend doing it the way shown in this other video. Much easier, much faster, less room for error, and better results, plus you can do it for more than two groups.
@Gaskination Thank you for your quick response :) Yes. I have watch the easier way recommended by you. Indeed it is much error-proof and faster way of doing the same process. Just because of academic requirement, we are suggested to show the step-by-step process of constrain the model and of cause to better understand the works behind it. But I found the stat tools created by you is a really valuable tool for many situations, I have suggested it to many of my colleagues and friends here.
@Gaskination Thank you for your quick response :) Yes. I have watch the easier way recommended by you. Indeed it is much error-proof and faster way of doing the same process. Just because of academic requirement, we are suggested to show the step-by-step process of constrain the model and of cause to better understand the works behind it. But I found the stat tools created by you is a really valuable tool for many situations, I have suggested it to many of my colleagues and friends here.
@sgcraig84 You can get it from my wiki. I can't post the link here in the comments section (like, youtube won't post the comment if there is a url in it...), so I created a note in the video that has the url. The note pops up around 3:15.
@Gaskination Thanks so much. I was trying to do mine at the same time and must have missed the URL :).
Another quick question, would this be appropriate if I am looking for say gender differences with a moderator? ex. a moderator is sig for females but not males? I am having issues finding a 3 way interaction due to sample size
Another thought...You could just run a fully unconstrained model and check the box for Critical Ratios for Differences. This will give you a table of z-scores and you can compare whether or not parameter values across groups vary for each parameter. If the parameter is >1.96 you will not have invariance for that relationship.
This is good, but are'n't most multigroup models shifting to comparisons between the CFIs or RMSEA rather than the Chi-Square tests alone to truly determine multi-group invariance? I am thinking here of large sample sizes in which chi-square tests really are poor indicators of model fit. Any thoughts?
@jrm236 There are different methods to test this, and none of them are great. Additionally, if you are comparing a group of 100 to a group of 1000, the chi-square difference test may pose problems, and instead you could look at the change in CFI (typically looking for a difference of more than a tenth).
@Gaskination Thank you for your quick response. I just feel like I am hitting a wall with my own research right now (2 groups, n = 720, n = 900) and cannot find a solid answer regarding overall fit differences especially since I have about 300 missing cases total (sample attrition). Can one compare the CFI's just looking at how they differ? I mean, as you know, you cannot simply compare the chi-squares of each group by just noting their diff. you have to calculate w/ chi-square diff. test.
@jrm236 It might help to know I am strictly concerned with structural invariance and not measurement invariance. My current solution for model fit has been that since the original model with both groups fit well the next most important step is not statistically significant differences in model fit between groups, but instead statistically significant differences in parameters (regression weights) between groups using the z-test or critical ratios comparisons.
@jrm236 Can't you just do a path by path chi-square difference test as demonstrated in this video? Or do you mean that you have 300 attrition from the 700 and 900? in which case you have smaller n and serious attrition issues.
@Gaskination Yes it appears that may be best. I was just concerned considering the fact that my original model fit shows to be poor (chi-square p<.003) but my CFI and RMSEA are both good. So I was concerned if discussing chi-squared differences are irrelevant since the original theoretical model showed it was not good (most likely because of the large sample size), which is why the CFI and RMSEA where then used. Attrition not an issue (roughly same amnt lost from each).
@jrm236 the p value is highly sensitive to large sample size and complex models. I would not rely on it. Just go with the chi-square difference test. And for fit, just rely on the CFI and RMSEA, and the Chi-square/DF.
@Gaskination excellent advice, I appreciate all of your help. I can tell you are dedicated to your craft! Crossing my fingers for at least an R&R on the final copy.
@Gaskination Attrition occurred because although my structural model is cross-sectional the original study was longitudinal and I used Time 2 measures for my primary latent construct to avoid issues with tautology (a common occurrence with the particular theory I am testing). Thus, attrition of 300 for a sample of 1672 (spread out about the same across each group) does not appear to be a problem (except for the fact that I cannot get Modification Indicies because of the need to use ML!).
wow, excellent,
can you send me the excel file that you used?
MSA76Y 3 weeks ago
@MSA76Y
It's on my wiki: statwiki. kolobkreation. com on the home page. Also, use the updated video "moderation made easy"
Gaskination 3 weeks ago
How can I find the spreadsheet from Excel you're using?
camerinobarron 4 weeks ago
@camerinobarron It is on my wiki: statwiki. kolobkreations. com Also, there is a better way to do this kind of analysis now. Check out my new video "multigroup moderation in amos made easy".
Gaskination 4 weeks ago
Excellent. Many thanks.
catsfancyful 2 months ago
Your Stat Tools package is fantastic. Thank you very much. Do you happen to have a version which would allow for testing multigroup moderation between more than two groups? E.g. four groups?
catsfancyful 2 months ago
@catsfancyful
Testing differences between more than two groups is always a case of testing the pairwise differences (i.e., between group 1 and 2, then 1 and 3, then 2 and 3). This excel tool will do it; you just have to do it two at a time.
Gaskination 2 months ago
Thanks for this, very interesting. I did wonder why you didn't just do a multi-group analysis though. Wouldn't that be easier? You could then also ask for Critical Ratios of Differences - That way you can check which paths differ across the groups without having to resort to laboriously checking each one. Or am I missing something?!
zapatistathistle 6 months ago
Sorry for the second question. We are running into a problem with "just identified models" with these models. Unfortunately for females ALL pathways are sig, whereas with males, several pathways are not sig so I cannot reasonably trim any pathway. So we cannot test model differences because with a just-identified model you don't get a Chi-square value. Anyone out in the you tube universe have a suggestion for us??
sgcraig84 7 months ago
@sgcraig84 There are a couple work-arounds. The best method is probably to add a control variable and hope it is not significant for females... Or, add another independent variable that you expect will be significant for males, but not for females. Or, simply trim the least significant path for females (assuming it is not the only path tying the exogenous variable to the model). Feel free to email me directly if these options don't sound appealing or if they don't work. email: jeg82@case.edu
Gaskination 7 months ago
@Gaskination Much appreciated your sharing. A Quick Question: To achieve a unconstrained model, we need to trim (delete) the path that is insignificant. In your example, the paths deleted are insignificant for both groups (male & female). What if the specific path is significant for female and not significant for male? Then should we delete that path to reach unconstrained model?
Thanks in advanced.
XenovaKyo 5 months ago
@Gaskination
I believed it is to delete the path if the path is significant on either 1 group.
In my case, there are only 3 paths (in the whole model, and All three are hypothesized to be moderated by the moderating variables). 1 path is insignificant on both groups, so we trim the path. The other 2 paths are insignificant only on group A, but significant on group B.
Is this means that I don't have to continue to proceed to the constrained model? or there is other ways?
XenovaKyo 5 months ago
@XenovaKyo Great question. The answer is that you only remove the paths that are not significant for BOTH groups. If it is significant on one, but not the other, then you leave it in. Also, the easier way to do this is shown in my other youtube video: "Multigroup Moderation in Amos - Made Easy" I highly recommend doing it the way shown in this other video. Much easier, much faster, less room for error, and better results, plus you can do it for more than two groups.
Gaskination 5 months ago
@Gaskination Thank you for your quick response :) Yes. I have watch the easier way recommended by you. Indeed it is much error-proof and faster way of doing the same process. Just because of academic requirement, we are suggested to show the step-by-step process of constrain the model and of cause to better understand the works behind it. But I found the stat tools created by you is a really valuable tool for many situations, I have suggested it to many of my colleagues and friends here.
XenovaKyo 5 months ago
@Gaskination Thank you for your quick response :) Yes. I have watch the easier way recommended by you. Indeed it is much error-proof and faster way of doing the same process. Just because of academic requirement, we are suggested to show the step-by-step process of constrain the model and of cause to better understand the works behind it. But I found the stat tools created by you is a really valuable tool for many situations, I have suggested it to many of my colleagues and friends here.
XenovaKyo 5 months ago
Where did you get the chi-square calculator for excel?
sgcraig84 7 months ago
@sgcraig84 You can get it from my wiki. I can't post the link here in the comments section (like, youtube won't post the comment if there is a url in it...), so I created a note in the video that has the url. The note pops up around 3:15.
Gaskination 7 months ago
@Gaskination Thanks so much. I was trying to do mine at the same time and must have missed the URL :).
Another quick question, would this be appropriate if I am looking for say gender differences with a moderator? ex. a moderator is sig for females but not males? I am having issues finding a 3 way interaction due to sample size
sgcraig84 7 months ago
@sgcraig84 Yes this is a good way to try gender differences. You will still have sample size issues though.
Gaskination 7 months ago
Will that really work? I'll have to try that. That would be stupendous!
Gaskination 8 months ago
Another thought...You could just run a fully unconstrained model and check the box for Critical Ratios for Differences. This will give you a table of z-scores and you can compare whether or not parameter values across groups vary for each parameter. If the parameter is >1.96 you will not have invariance for that relationship.
jrm236 8 months ago
This is good, but are'n't most multigroup models shifting to comparisons between the CFIs or RMSEA rather than the Chi-Square tests alone to truly determine multi-group invariance? I am thinking here of large sample sizes in which chi-square tests really are poor indicators of model fit. Any thoughts?
jrm236 8 months ago
@jrm236 There are different methods to test this, and none of them are great. Additionally, if you are comparing a group of 100 to a group of 1000, the chi-square difference test may pose problems, and instead you could look at the change in CFI (typically looking for a difference of more than a tenth).
Gaskination 8 months ago
@Gaskination Thank you for your quick response. I just feel like I am hitting a wall with my own research right now (2 groups, n = 720, n = 900) and cannot find a solid answer regarding overall fit differences especially since I have about 300 missing cases total (sample attrition). Can one compare the CFI's just looking at how they differ? I mean, as you know, you cannot simply compare the chi-squares of each group by just noting their diff. you have to calculate w/ chi-square diff. test.
jrm236 8 months ago
@jrm236 It might help to know I am strictly concerned with structural invariance and not measurement invariance. My current solution for model fit has been that since the original model with both groups fit well the next most important step is not statistically significant differences in model fit between groups, but instead statistically significant differences in parameters (regression weights) between groups using the z-test or critical ratios comparisons.
jrm236 8 months ago
@jrm236 Can't you just do a path by path chi-square difference test as demonstrated in this video? Or do you mean that you have 300 attrition from the 700 and 900? in which case you have smaller n and serious attrition issues.
Gaskination 8 months ago
@Gaskination Yes it appears that may be best. I was just concerned considering the fact that my original model fit shows to be poor (chi-square p<.003) but my CFI and RMSEA are both good. So I was concerned if discussing chi-squared differences are irrelevant since the original theoretical model showed it was not good (most likely because of the large sample size), which is why the CFI and RMSEA where then used. Attrition not an issue (roughly same amnt lost from each).
jrm236 8 months ago
@jrm236 the p value is highly sensitive to large sample size and complex models. I would not rely on it. Just go with the chi-square difference test. And for fit, just rely on the CFI and RMSEA, and the Chi-square/DF.
Gaskination 8 months ago
@Gaskination excellent advice, I appreciate all of your help. I can tell you are dedicated to your craft! Crossing my fingers for at least an R&R on the final copy.
jrm236 8 months ago
@Gaskination Attrition occurred because although my structural model is cross-sectional the original study was longitudinal and I used Time 2 measures for my primary latent construct to avoid issues with tautology (a common occurrence with the particular theory I am testing). Thus, attrition of 300 for a sample of 1672 (spread out about the same across each group) does not appear to be a problem (except for the fact that I cannot get Modification Indicies because of the need to use ML!).
jrm236 8 months ago
Thank you so much, you saved me!
borisherbas 8 months ago
This is an excellent tool. Thanks a lot. Good luck with your study.
jefferychang0703 9 months ago
Where can I find the excel table like yours? Can you share? Tks
jefferychang0703 9 months ago
@jefferychang0703 You can find it on my statwiki: statwiki. kolobkreations. com
Gaskination 9 months ago
Helpful video, but it is NON-significant, not insignificant. Insignificant is not a statistical term and means something different.
allje599 9 months ago
@allje599 Thanks for catching that. Slip of the tongue...
Gaskination 9 months ago