1. Detecting Differential Item Functioning using Mplus
Richard N. Jones, Sc.D.
Hebrew Rehabilitation Center for Aged
Research and Training Institute
Boston, MA
University of Washington
Psychometrics Workshop
September 20-25, 2004
Please send questions and corrections to

2. Overview of this talk Other Resources Statistical model
Implementation in Mplus
Example
Discussion
Limitations
Advantages
Mplus (

3. Other Resources (Don’t take my word for it)
Glockner-Rist, A., & Hoijtink, H. (2003). The best of both worlds: Factor analysis of dichotomous data using item response theory and structural equation modeling. Structural Equation Modeling, 10(4),
Mislevy, R. J. (1986). Recent developments in the factor analysis of categorical variables. Journal of Educational Statistics, 11(1), 3-31.
Macintosh, R., & Hashim, S. (2003). Variance Estimation for Converting MIMIC Model Parameters to IRT Parameters in DIF Analysis. Applied Psychological Measurement, 27(5),
Muthén, B., & Lehman, J. (1985). Multiple group IRT modeling: Applications to item bias analysis. Journal of Educational Statistics, 10(2),
Muthén, B. (1988). Some uses of structural equation modeling in validity studies: Extending IRT to external variables. In H. Wainer & H. Braun (Eds.), Test validity (pp ). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Muthén, B. O. ( ). Mplus Technical Appendices. Los Angeles, CA: Muthén & Muthén.
Muthén, B., & Asparouhov, T. (2002). Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus (Mplus Web Note No. 4). Los Angeles: University of California and Muthén & Muthén.(
Mplus Short Courses (November 2004, Alexandria VA) see
Lord, F., & Novick, M. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
Takane, Y., & De Leeuw, J. (1987). On the relationship between item response theory and factor analysis of descretized variables. Psychometrika, 52(3),
Also see
Jones, R. N. (2003). Racial bias in the assessment of cognitive functioning of older adults. Aging & Mental Health, 7(2),
Jones, R. N., & Gallo, J. J. (2002). Education and sex differences in the Mini Mental State Examination: Effects of differential item functioning. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 57B(6), P

4. Implementing IRT Models in Mplus
Mplus has capabilities of popular SEM software packages (LISREL, EQS, AMOS)
Mplus has unique features, e.g.
Latent class and mixture modeling
handling categorical dependent variables
Regression based treatment of exogenous variables (covariate)
These special features make possible the implementation of IRT models
Normal ogive
Logistic (Version 3)
“2-P” models (difficulty, discrimination)
Uniform DIF detection
Non-Uniform DIF detection a special challenge

5. Structural Equation Modeling
Includes path analysis, confirmatory factor analysis
Unobserved (latent) variables account for covariation among observed variables
Mplus uses y* variables when y are categorical

6. Latent Response Variable
Latent Response Variable (y*)
aka Unobserved Variable Approach
Assume observed ordinal y has corresponding latent (unobserved) form (y*)
e.g., propensity to endorse sadness
When y is binary, y=1 if y* exceeds a threshold value t
Analysis concerns these latent response variables

7. Correlation among y’s, y*’s
Correlation among binary y’s
Phi coefficients
Range determined by base rate
Lead to spurious “difficulty factors” when used in factor analysis
Correlation among y*’s
Tetrachoric coefficients if y’s are binary
Polychoric correlation coefficients if y’s have more than 2 categories
Same scale as Pearson correlation coefficients for continuous observed variables

8. Mplus Categorical Data Modeling

9. Latent Variable Modeling with Categorical Dependent Variables

10. Latent Variable Modeling with Categorical Dependent Variables

11. Summary

12. Correspondence of IRT and MIMIC terms

13. Understanding the Mplus and LISCOMP Models

14. Example Perturbed data
Epidemiologic Catchment Area (ECA) study participants ( )
Original sample: N=1,358 Aged 70+
Perturbed sample: N=1,358
Total N = 2,716
Five binary indicators of perfect performance on sections of MMSE

15. Steps in Analysis Exploratory data analysis (missingness, guessing)
Exploratory factor analysis (EFA)
Confirmatory factor analysis (CFA)
CFA with covariates (MIMIC)
Identify likely model mis-specification corresponding to direct effects from among matrix of derivatives
Stepwise model building relaxing measurement non-invariance assumptions (use WLS)
Possible multiple-group models
Final model estimation (use WLSMV estimator)

16. Doing DIF Detection with Mplus
Mplus requires data as raw text file
fixed, free, delimited
stata2mplus module available
Raw text command file
Use Mplus language generator
Have the manual handy
runmplus module (contact me)

17. Mplus Command File Two-Parameter Logistic IRT Model
TITLE: Example from rich jones' thesis DATA: File = c:\work\thesis\example\eg_5y.dat; format is 6f1.0; VARIABLE: Names = time place regrecp sevens lang x; Categorical = time place regrecp sevens lang; ANALYSIS: Estimator = MLR; OUTPUT: standardized TECH1 TECH2 ; MODEL: mmse by time place regrecp sevens lang; mmse on x; time-lang on
TITLE: Example from rich jones' thesis DATA: File = c:\work\thesis\example\eg_5y.dat; format is 6f1.0; VARIABLE: Names = time place regrecp sevens lang x; Categorical = time place regrecp sevens lang; ANALYSIS: Estimator = WLS ; ! note WLSMV is default OUTPUT: standardized modindices TECH2 ; MODEL: mmse by time place regrecp sevens lang; mmse on x; time-lang on
... using runmplus from within STATA
runmplus time-lang x, categorical(time-lang) ///
estimator(wls) ///
model(mmse by time place regrecp sevens lang; ///
mmse on x; time-lang on

18. Discussion Advantages Disadvantages Anchor items Model Fit
Simultaneous modeling of differences in ability and item-level performance
Mixed metrics for dependent variables
No linking
Capable of handling multidimensional constructs
Disadvantages
Guessing not accommodated
Modeling Non-Uniform DIF a challenge (Multiple Group models required)
Anchor items
Model Fit

19. Diagram of a MIMIC Model for Detecting DIF

20. Race Bias in Mental Status Assessment
Applied Example
Race Bias in
Mental Status
Assessment
Jones, R. N. (2003). Racial bias in the assessment of cognitive functioning of older adults. Aging & Mental Health, 7(2),

21. Health and Retirement Study
Nationally representative, very large sample (N=15,257)
Over-sample of Black or African-Americans (N=2,090)
Assessment of cognition
Very adequate assessment of SES (education, income, occupation)

22. Objective
Evaluate the extent to which item level performance is due to race (White, non-Hispanic vs. Black or African-American participants)
Control for main and potentially differential effects of background variables
Sex, Age
Educational attainment
Household income, occupation groups
Health Conditions and Health Behaviors

23. AHEAD Measures of Cognitive Function (Herzog 1997)
Points
Orientation to time
(weekday, day, month, year) 4
Name President, Vice-President 2
Name two objects (cactus, scissors) 2
Count Backwards from 20 1
Serial Sevens 5
Immediate recall (10 nouns) 10
Delayed free-recall (10 nouns, 5 min delay) 10

24. Background Variables Sex Age (9 groups) Education (6 groups)
Household Income (5 groups)
‘Highest’ household occupation (8 groups)
Health Conditions
HBP DM
Heart Stroke
Arthritis Pulmonary
Cancer
Health Behaviors
current smoking
drinking [three groups]

25. MIMIC Model

26. Multiple Group MIMIC Model

27. All items show DIF by race, some by sex, age, education
Results
All items show DIF by race, some by sex, age, education
Effect of covariates (age, education, occupation, income, smoking status) significantly different across racial group
Greater variance in latent cognitive function for Black or African-American participants
No significant race difference in mean latent cognition by race after adjusting for measurement differences
Jones. Aging Ment Health, 2003; 7:

28. Differences in Underlying Ability between Whites and African Americans
60% is due to measurement differences (DIF, item bias)
12% is due to main effect of background variables
7% is due to structural differences (i.e., interactions of group and background variables)
What remains is not significantly different from no difference
Jones. Aging Ment Health, 2003; 7:

29. Differences in Underlying Ability ignoring measurement bias
Jones. Aging Ment Health, 2003; 7:

30. Differences in Underlying Ability after controlling for measurement bias
Jones. Aging Ment Health, 2003; 7:

34. Model Fit / Parsimony
Model fitting accomplished more than shifting group differences in mental status to item-level
New model provides greater fit to observed data using fit statistics that reward model parsimony

35. Implications
Important measurement differences of cognition across racial group
Analyses can adjust for this measurement bias in latent variable framework
Practical solution in raw score space is needed to address measurement issues
Cognitive Status Assessment Device needed that is
Reliable
Small floor/ceiling effect
Brief
Capable of being administered by Telephone
Little influenced by construct irrelevant factors (education, ethnicity, culture)