Explanatory Secondary Dimension Modeling of Latent Different Item Functioning Paul De Boeck, Sun-Joo Cho, and Mark Wilson

DIF Conditional on person latent trait, different manifest groups have different probabilities to answer correctly/endorse an item. (manifest DIF) DIF as the consequence of neglecting the “nuisance dimensions”. (secondary dimensions)

This study Latent DIF: DIF between latent classes, i.e., a non- DIF vs a DIF latent class group membership not observed or unobservable The observed membership not reliable or not valid Mixed dimensionality: the secondary dimension in the DIF latent class, but not in the non-DIF latent class Multidimensional model for DIF (MMD) (Roussos & Stout, 1996) One dimensionality approach: Multiple-indicator multiple- cause (MIMIC) approach

MMD: a secondary dimension in both manifest groups MIMIC: neither of the classes has a secondary dimension but a direct effect of the group variable on the primary dimension (exceptional: Glockner-Rist & Hoijtink, 2003) MIMIC Mixed dimensionality: a secondary dimension but only in the DIF latent class (but may have some problems)

Measurement Invariance Notions Scalar variance: difference in item difficulties (intercept in factor model) and implies uniform DIF Metric variance: difference in item discriminations (loadings in factor model) and implies nonuniform DIF Relate the idea of mixed dimensionality to measurement invariance: With secondary dimension, scalar invariance (equivalence of difficulties) are regain (M2a vs M1a).

Outline of the Study Aim: to show the implication and application of a mixed dimensionality model into the latent DIF (1) Models specification (2) Model Estimation and Evaluation (3) Model application (Speededness & Arithmetic operation)

Models Non-DIF One-Dimensional Mixture Models (1d-non-DIF; Model 0) α i and  i are the same in both groups;, (model identification) 2I + 3 para (I difficulties, I discriminations, a mean and variance of DIF latent class, and a mixing probability (Mixture 2PL model)

One dimensional difficulty DIF mixture model (1d-  dif; Model 1a)  i are different between groups Lack of scalar invariance, uniform DIF 2I+D+3 para (D items showing latent DIF)

One dimensional difficulty and discrimination DIF mixture model (1d-  dif; Model 1b) α i and  i are different between groups Lack of scalar and metric invariance, uniform and nonuniform DIF 2(I+D)+3 para (D items showing latent DIF)

Mixture of one-dimensional and two- dimensional model(1&2d-DIF, Model 2a) For DIF latent class, (  f1,  f2 )’ ~ BVN( ,  ). ( ) 2I+D+4 free paras One discrimination for a DIF items in dimension 2 of DIF latent class is fixed to (over constrain)

Features of Model 2a: (1) no difference in difficulty between two latent traits (2) metric invariance of primary dimension & scalar invariance Implication of Model 2a: (1) Respondents from the DIF latent class differ with respect to how much DIF they show (  pf2 ); All DIF is assumed to rely on the secondary dimension

Mixture of one-dimensional and two- dimensional model with discrimination DIF (1&2d-  dif, Model 2b) 2(I+D)+3 free parameters Scalar invariance Cov  f12 is fixed to zero (over constrained)

Two-dimensional mixture models(2d-DIF and 2d-  DIF/ Models 3a and 3b): introduce the second dimension to the non-DIF latent class Model 3a: Measurement-invariance Model 3b: scalar invariance, configural invariance

Relationships between dimensional latent DIF mmodels

Aim of Applications Preliminary question: whether any DIF occurs at the level of latent classes (M0 vs M1a & 1b) Whether a secondary dimension in the DIF latent class can explain the  DIF? (M1a vs M2a; or M1b vs M2b) Is the DIF of a kind that the discrimination on the primary dimension is affected? (M1a vs M1b or M2a vs M2b) Is the secondary dimension limited to the DIF latent class or does it also apply to the non-DIF latent class? (M2a vs M3a, or M2b vs M3b)

Model Estimation & Evaluation LatentGOLD 4.5 Syntax Cluster module (Vermunt & Magidson, 2007) Two major problems in mixture model: (1) Label switch; (2) multiple local maxima Model specification in LatentGOLD LRT (for nested models), AIC, BIC

Results of Speededness Data First 23 items were assumed no speededness while last 8 items has (item location). Speeded vs no speeded latent class

M1a and 1b fit much better than M0, suggesting that DIF exist at the level of latent class; First horizontal comparison (M2a> M1a; M2b>M1b) suggests the mixed dimensionality approach is favored. Vertical comparison (M1b>M1a;M2b>M2a) suggests discrimination DIF is related to the primary dimension. Thus, M2b is chose. Second comparison (M3b is not significantly better than M2b) suggests secondary dimension not need to extend to the non-DIF latent class.

 =0.311, speeded class is a minority class; Nonzero discriminations on the second dimension for the DIF items in the speeded class; the discrimination of items on the primary dimension differ between two latent classes. For DIF class, mean=(-0.634,-0.987)’,var-cov=(0.787, 0; 0, 1), suggesting that the speeded class has a lower ability and being speeded at the end of the test seems not to help. Validation check for the DIF items

Confirmatory and exploratory two- dimensional models without latent classes were not supported by AIC and BIC.

Results of Arithmetic Operations Data Class 1 (Poorer on the division items) vs class 2 (proficient)

DIF exist between two latent classes; Secondary dimension helps; The primary dimension discrimination DIF is not strongly supported. Hence, M2a is favored. No needed to extend the second dimension to the non-DIF latent class.

For DIF class, mean=(-1.150,-2.945)’,var- cov=(1.034, 0.353*sqrt(1.034); 0.353*sqrt(1.034), 1).  =0.292 AIC supported exploratory two-dimensional model without latent class but BIC did not.

Conclusions Mixed dimensionality approach are supported by the generally better goodness of fit. It was sufficient to include the secondary dimension in only one of the latent class. Mixed dimensionality approach has some merits (for latent DIF, explaining DIF…) Mixed dimensionality approach should primarily be used for confirmatory and explanatory purposes, but not for exploratory or detection purpose. (problematic)

Discussions & Future Studies The mixed dimensionality approach have some problems (not usual; local dependence). To define item properties which are not available as the clue of the suspected DIF before the secondary dimension approach can be implemented. Applied mixed dimensionality approach to manifest groups, to study cognitive development, mutiple strategies…

Applied the mixture approach within the cognitive development theoretical framework Whether individual differences in DIF is proper?