Factors Affecting Measures of Fit David A. Kenny

Background Introduction Measures of Fit

Factors that Affect Fit Indices Number of Variables Model Complexity Sample Size Non-normality Click to go to a specific topic.

Number of Variables (v)   Anecdotal evidence that models with many variables have poor fit. Kenny and McCoach (2003) show that the RMSEA gets lower as more variables are added to the model and that the TLI and CFI are relatively stable, but tend to decline slightly. We still do not understand why it is that models with more variables tend to have poor fit.

Model Complexity (df) One way to think about model complexity is the number of parameters estimated or conversely the model df. So for a given dataset, a model with more parameters and so smaller df is more complex than a model with fewer parameters and a larger df.   The question asked for a given fit index, especially if the question is model comparison, is how much c2 needs to change per df for that fit index not to change.

How much c2 needs to change per df for the fit index not to change (theoretical values and for two studies) Measure Theoretical Value  Study 1b Study 2c Bentler & Bonett 0  0  0 CFI 1  1  1 AIC 2  2  2 Tucker-Lewisa c2/df 3.56  2.62 RMSEAa c2/df 3.56  2.62 SABIC ln[(N +2)/24]  2.54  2.79 BIC ln(N) 5.71  5.96 ac2 value used from the final model. bN = 301, v = 9, c2(24) = 88.31 (Holzinger-Swineford) cN = 388, v = 8, c2(19) = 49.87 (Moreland & Beech) √

Implications of the Previous Slide The different fit indices change by a constant amount, regardless of the df change.   Larger values reward parsimony and smaller values reward complexity.  The BIC rewards parsimony most, and the CFI (after the Bentler and Bonett) the least.

√[(ln[(N + 2)/24] – 1)/ (N - 1)] SABIC Penalty If the RMSEA equals √[(ln[(N + 2)/24] – 1)/ (N - 1)] then the penalty for the SABIC equals the same value as the RMSEA or the TLI. Also the penalty for the SABIC is greater than the AIC if N > 175.

Sample Size (N) Bentler-Bonett fails to adjust for sample size: Models with larger sample sizes have smaller values.  The TLI and CFI do not vary much with sample size.  The RMSEA and the SRMR are larger with smaller sample sizes. Differences in the AIC get larger as N increases and so is affected by N.

Non-normality Non-normal data (especially high kurtosis) inflate c2. Absolute measures of fit, RMSEA and SRMR, are likely inflated.  Presumably, incremental and perhaps comparative measures of fit are less affected by non-normality.

Click here to go to references.