1. SOC 681 James G. Anderson, PhD
Testing Model Fit
SOC 681
James G. Anderson, PhD

2. Limitations of Fit Indices
Values of fit indices indicate only the average or overall fit of a model. It is thus possible that some parts of the model may poorly fit the data.
Becauwe a single index reflects only a particul;ar aspect oof model fit, a favorable value of that indesx does not by itself indicate good fit. That is why model fit is assessed based on the values of multiple indices.

3. Limitations of Fit Statistics
Fit indices do not indicate whether the results are theoretically meaningful.
Values of fit indices that suggest adequate fit do not indicate the predictive power of the model is also high.
The sampling distribution of many fit indices are unknown.

4. Assessment of Model Fit
Examine the parameter estimates
Examine the standard errors and significance of the parameter estimates.
Examine the squared multiple correlation coefficients for the equations
Examine the fit statistics
Examine the standardized residuals
Examine the modification indices

5. Measures of Fit Measures of fit are provided for three models:
Default Model – this is the model that you specified
Saturated Model – This is the most general model possible. No constraints are placed on the population moments It is guaranteed to fit any set of data perfectly.
Independence Model – The observed variables are assumed to be uncorrelated with one another.

9. Overall measures of Fit
NPAR is the number of parameters being estimated (q)
CMIN is the minimum value of the discrepancy function between the sample covariance matrix and the estimated covariance matrix.
DF is the number of degrees of freedom and equals the p-q
p=the number of sample moments
q= the number of parameters estimated

10. Overall measures of Fit
CMIN is distributed as chi square with df=p-q
P is the probability of getting as large a discrepancy with the present sample
CMIN/DF is the ratio of the minimum discrepancy to degrees of freedom. Values should be close to 1.0 for correct models.

11. Chi Square: 2 Best for models with N=75 to N=100
For N>100, chi square is almost always significant since the magnitude is affected by the sample size
Chi square is also affected by the size of correlations in the model: the larger the correlations, the poorer the fit

12. Chi Square to df Ratio: 2/df
There are no consistent standards for what is considered an acceptable model
Some authors suggest a ratio of 2 to 1
In general, a lower chi square to df ratio indicates a better fitting model

13. Transforming Chi Square to Z
Z = (2*2) - (2*df-1)

14. CMIN Model NPAR CMIN DF P CMIN/DF Default model 22 10.335 14 .737 .738
Saturated model 36
.000
Independence model 8 28
8.706

15. RMR, GFI
RMR is the Root Mean Square Residual. It is the square root of the average amount that the sample variances and covariances differ from their estimates. Smaller values are better
GFI is the Goodness of Fit Index. GFI is between 0 and 1 where 1 indicates a perfect fit. Acceptable values are above 0.90.

16. GFI and AGFI (LISREL measures)
The AGFI takes into consideration the df available to test the model.
Values close to .90 reflect a good fit.
These indices are affected by sample size and can be large for poorly specified models.
These are usually not the best measures to use.

17. RMR, GFI
AGFI is the Adjusted Goodness of Fit Index. It takes into account the degrees of freedom available for testing the model. Acceptable values are above 0.90.
PGFI is the Parsimony Goodness of Fit Index. It takes into account the degrees of freedom available for testing the model. Acceptable values are above 0.90.

18. RMR, GFI Model RMR GFI AGFI PGFI Default model .003 .975 .935 .379
Saturated model
.000
1.000
Independence model
.023
.570
.447
.443

19. Comparisons to a Baseline Model
NFI is the Normed Fit Index. It compares the improvement in the minimum discrepancy for the specified (default) model to the discrepancy for the Independence model. A value of the NFI below 0.90 indicates that the model can be improved.

20. Bentler-Bonett Index or Normed Fit Index (NFI)
Define null model in which all correlations are zero:
2 (Null Model) - 2 (Proposed Model) 2 (Null Model)
Value between .90 and .95 is acceptable; above .95 is good
A disadvantage of this index is that the more parameters, the larger the index.

21. Comparisons to a Baseline Model
RFI is the Relative Fit Index This index takes the degrees of freedom for the two models into account.
IFI is the incremental fit index. Values close to 1.0 indicate a good fit.
TLI is the Tucker-Lewis Coefficient and also is known as the Bentler-Bonett non-normed fit index (NNFI). Values close to 1.0 indicate a good fit.
CFI is the Comparative Fit Index and also the Relative Noncentrality Iindex (RNI). Values close to 1.0 indicate a good fit.

22. Tucker Lewis Index or Non-normed Fit Index (NNFI)
Value: 2/df(Null Model) - 2/df(Proposed Model) 2/df(Null Model)
If the index is greater than one, it is set to1.
Values close to .90 reflects a good model fit.
For a given model, a lower chi-square to df ratio (as long as it is not less than one) implies a better fit.

23. Comparative Fit Index (CFI)
If D= 2 - df, then:
D(Null Model) - D(Proposed Model)
D(Null Model)
If index > 1, it is set to 1; if index <0, it is set to 0
A lower value for D implies a better fit
If the CFI < 1, then it is always greater than the TLI
The CFI pays a penalty of one for every parameter estimated

24. Baseline Comparisons Model NFI Delta1 RFI rho1 IFI Delta2 TLI rho2 CFI
Default model
.958
.915
1.016
1.034
1.000
Saturated model
Independence model
.000

25. Parsimony Adjusted Measures
PRATIO is the Parsimony Ratio. It is the number of constraints in the model being evaluated as a fraction of the number of constraints in the independence model.
PNFI is the result of applying the PRATIO to the NFI.
PCFI is the result of applying parsimony adjustments to the CFI.

26. Parsimony-Adjusted Measures
Model
PRATIO
PNFI
PCFI
Default model
.500
.479
Saturated model
.000
Independence model
1.000

27. Measures Based on the Population Discrepancy
NCP is an estimate of the noncentrality parameter obtained by fitting a model to the population moments rather than to the sample moments.
The 90% confidence interval is also computed for the NCP.

28. NCP Model NCP LO 90 HI 90 Default model .000 7.102 Saturated model
Independence model

29. The Minimum Sample Discrepancy Function
FMIN is the minimum value of the discrepancy .

30. FMIN Model FMIN F0 LO 90 HI 90 Default model .107 .000 .073
Saturated model
Independence model
2.513
2.224
1.748
2.778

31. Root Mean Square Error of Approximation (RMSEA)
Value: [(F0pop-F0est)/n]/df
F0 is the minimum vale of the discrepancy function.
If 2 < df for the model, RMSEA is set to 0
Good models have values of < .05; values of > .10 indicate a poor fit.
It is a parsimony-adjusted measure.
Amos provides upper and lower limits of a 90% confidence interval for the RMSEA

32. PCLOSE
PCLOSE is the probability for testing the null hypothesis that the population RMSEA is no greater than 0.05.

33. RMSEA Model RMSEA LO 90 HI 90 PCLOSE Default model .000 .072 .877
Independence model
.282
.250
.315

34. Information Theoretic Measures
These indices are composite measures of badness of fit and complexity.
Simple models that fit well receive low scores. Complicated poorly fitting models get high scores.
These indices are used for model comparison not to evaluate a single model.

35. AIC Model AIC BCC BIC CAIC Default model 54.335 58.835 111.204 133.204
Saturated model
72.000
79.364
Independence model

36. Akaike Information Criterion (AIC)
Value: 2 + k(k-1) - 2(df)
where k= number of variables in the model
A better fit is indicated when AIC is smaller
Not standardized and not interpreted for a given model.
For two models estimated from the same data, the model with the smaller AIC is preferred.

37. Information Theoretic Measures
BCC is the Browne-Cudeck Criterion
BIC is Bayes Information Criterion.
CAIC is the consistent AIC
ECVI except for a constant scale factor it is the same as AIC.
MECVI except for a scale factor is the same as the BCC.

38. ECVI Model ECVI LO 90 HI 90 MECVI Default model .560 .598 .671 .607
Saturated model
.742
.818
Independence model
2.678
2.202
3.231
2.695

39. Nonhierarchical Models

42. Hierarchical Models

43. Difference in Chi Square
Value: X2diff = X2 model 1 -X2 model 2
DFdiff = DF model 1 –DFmodel 2

46. Miscellaneous Measures
HOELTER is the largest sample size for which one would accept the hypothesis that a model is correct.

47. Hoelter Index Value: (N-1)*2(crit) + 1 2
Where 2 (crit) is the critical value for the chi-square statistic
The index should only be calculated if chi square is statistically significant.

48. Hoelter Index (2)
If the critical value is unknown, can approximate: [ ( (2df-1) ]2 + 1
2 2/ (N-1)
For both formulas, one rounds down to the nearest integer
The index states the sample size at which the chi square would not be significant

49. Hoelter Index (3)
In other words, how small one’s sample size would have to be for chi square to no longer be significant
Hoelter Recommends values of at least 200
Values < 75 indicate poor fit

50. HOELTER Model HOELTER .05 HOELTER .01 Default model 223 274
Independence model 17 20

52. Which Fit Indices to Report?/
Chi Square and df
RMSEA
CFI
AGFI
Hierarchical Models: Difference in Chi Square
Nonhierarchical Models: AIC