Effects of unequal indicator intercepts on manifest composite differences Holger Steinmetz and Peter Schmidt University of Giessen / Germany

Introduction Importance of analyses of mean differences For instance: - gender differences on wellbeing, self-esteem, abilities, behavior - differences between leaders and non-leaders on intelligence and personality traits - differences between cultural populations on psychological competencies, values, wellbeing Usual procedure: t-test or ANOVA with manifest composite scores Latent variables vs. manifest variables Manifest mean = indicator intercept + factor loading * latent mean → Will unequal intercepts lead to wrong conclusions regarding composite differences?

Intercepts and latent means X Y Y X B1B1 B0B0

xixi  i ii x1x1 x4x4 x2x2 x3x3         

xixi  i ii x1x1 x4x4 x2x2 x3x3         

x1x1 x4x4 x2x2 x3x3  xixi  i ii  M(x i )        

Intercepts and latent means x1x1 x4x4 x2x2 x3x3   xixi i ii  M(x i )        

Group differences in intercepts and factor loadings xixi   M(x i ) x1x1 x4x4 x2x2 x3x3  x1x1 x4x4 x2x2 x3x3  Group AGroup B

Group differences in intercepts and factor loadings xixi   M(x i ) x1x1 x4x4 x2x2 x3x3  x1x1 x4x4 x2x2 x3x3  Group AGroup B

Group differences in intercepts and factor loadings xixi   M(x i ) x1x1 x4x4 x2x2 x3x3  x1x1 x4x4 x2x2 x3x3  Group AGroup B

Meaning of (unequal) intercepts Associated terms used in the literature - Item bias - Differential item functioning - Measurement/factorial invariance ("strong factorial invariance", "scalar invariance") Meaning - Response style (acquiescence, leniency, severity) - Response sets (e.g., social desirability) - Connotations of items - Item specific difficulty

The study Partial invariance Research question: Is partial invariance enough for composite mean difference testing? - Pseudo-differences - Compensation effects Procedure (Mplus): - Step 1: Specification of two-group population models with latent mean and intercept differences; 1000 replications, raw data saved - Step 2: Creation of a composite score - Step 3: Analysis of composite differences - Step 4: Aggregation (-> sampling distribution)

The study Design (population model): - Two groups - One latent variable - 4 vs. 6 indicators - All intercepts equal vs. one vs. two intercepts unequal in varying directions (+.30 vs. -.30) - Latent mean difference: 0 vs Loadings kept equal with ‘s =.80; latent variance = 1 - N = 2 x 100 vs. 2 x Latent models as comparison standard for each condition Dependent variables - Average composite mean difference - Percent of significant composite differences („% sig“)

Full scalar invariance (Latent mean difference =.30) Avg. composite difference %sig N = 2 x 100 N = 2 x Indicators 6 Indicators 4 Indicators 6 Indicators

Pseudo-Differences (Latent mean difference = 0; unequal intercept(s) 4 Ind.6 Ind. N = 2 x 300N = 2 x 100N = 2 x Intercepts unequal (.30) 4 Ind.6 Ind.4 Ind. 6 Ind.4 Ind. 6 Ind. N = 2 x Intercept unequal (.30) Avg. composite difference %sig

Pseudo-Differences (Latent mean difference = 0; unequal intercept(s) 4 Ind.6 Ind. N = 2 x 300N = 2 x 100N = 2 x Intercepts unequal (.30) 4 Ind.6 Ind.4 Ind. 6 Ind.4 Ind. 6 Ind. N = 2 x Intercept unequal (.30) Avg. composite difference %sig

Compensation effects (Latent mean difference =.30; negative intercept difference) 4 Ind.6 Ind. N = 2 x 300N = 2 x 100N = 2 x Intercepts unequal (-.30) 4 Ind.6 Ind.4 Ind. 6 Ind.4 Ind. 6 Ind. N = 2 x Intercept unequal (-.30)

Compensation effects (Latent mean difference =.30; negative intercept difference) 4 Ind.6 Ind. N = 2 x 300N = 2 x 100N = 2 x Intercepts unequal (-.30) 4 Ind.6 Ind.4 Ind. 6 Ind.4 Ind. 6 Ind. N = 2 x Intercept unequal (-.30)

Summary Full latent variable models have more power than composite analyses Pseudo-differences - Even one unequal intercept increases the risk to find spurious composite differences - High sample size increases risk - Number of indicators reduces the risk – but not substantially Componensation effects - Even one unequal intercept reduces the size of the composite difference to 50% - In small samples little chance to find a significant composite difference (power = ) - Two unequal intercepts drastically reduce the composite difference: The power in the „best“ condition (2x300, 6 Ind.) is only.50

Conclusons Most comparisons of means rely on traditional composite difference analysis These methods make assumptions that are unrealistic (i.e., full invariance of intercepts) Even minor violations of these assumptions increase the risk of drawing wrong conclusions Advantages of SEM: - Assumptions can be tested - Partial invariance implies no danger - Greater power even in small samples

Thank you very much!