Effects of unequal indicator intercepts on manifest composite differences Holger Steinmetz and Peter Schmidt University of Giessen / Germany.

Slides:



Advertisements
Similar presentations
Applied Structural Equation Modeling for Dummies, by Dummies February 22, 2013 Indiana University, Bloomington Joseph J. Sudano, Jr., PhD Center for.
Advertisements

Myers’ PSYCHOLOGY (7th Ed)
Models for Measuring. What do the models have in common? They are all cases of a general model. How are people responding? What are your intentions in.
Jeff Beard Lisa Helma David Parrish Start Presentation.
ASVAB: E Pluribus Unum? Martin J. Ippel, Ph.D. CogniMetrics Inc, San Antonio,TX Steven E. Watson, Ph.D. U.S. Navy Selection & Classification (CNO 132)
Analysis of Variance (ANOVA) Statistics for the Social Sciences Psychology 340 Spring 2010.
BHS Methods in Behavioral Sciences I April 25, 2003 Chapter 6 (Ray) The Logic of Hypothesis Testing.
Chapter 8 The t Test for Independent Means Part 1: March 6, 2008.
Using Statistics in Research Psych 231: Research Methods in Psychology.
ANOVA Analysis of Variance: Why do these Sample Means differ as much as they do (Variance)? Standard Error of the Mean (“variance” of means) depends upon.
SADC Course in Statistics Comparing Means from Independent Samples (Session 12)
Inferential Stats for Two-Group Designs. Inferential Statistics Used to infer conclusions about the population based on data collected from sample Do.
PSY 1950 Fixed and Random Effects October 20, 2008.
How to evaluate the cross-cultural equivalence of single items Melanie Revilla, Willem Saris RECSM, UPF Zurich – 15/16 July.
Experimental Design & Analysis
Testing factorial invariance in multilevel data: A Monte Carlo study Eun Sook Kim Oi-man Kwok Myeongsun Yoon.
Analyzing observed composite differences across groups: Is partial measurement invariance enough? Holger Steinmetz Faculty of Economics and Business Administration.
© 2005 The McGraw-Hill Companies, Inc., All Rights Reserved. Chapter 13 Using Inferential Statistics.
Today Concepts underlying inferential statistics
LECTURE 16 STRUCTURAL EQUATION MODELING.
Chapter 9 Flashcards. measurement method that uses uniform procedures to collect, score, interpret, and report numerical results; usually has norms and.
Lorelei Howard and Nick Wright MfD 2008
Richard M. Jacobs, OSA, Ph.D.
Regression and Correlation Methods Judy Zhong Ph.D.
Selecting the Correct Statistical Test
Basic Statistics Michael Hylin. Scientific Method Start w/ a question Gather information and resources (observe) Form hypothesis Perform experiment and.
Comparing Means From Two Sets of Data
McGraw-Hill/Irwin Copyright © 2007 by The McGraw-Hill Companies, Inc. All rights reserved. Statistical Inferences Based on Two Samples Chapter 9.
1 Today Null and alternative hypotheses 1- and 2-tailed tests Regions of rejection Sampling distributions The Central Limit Theorem Standard errors z-tests.
A Latent Class Call-back Model for Survey Nonresponse Paul P. Biemer RTI International and UNC-CH Michael W. Link Centers for Disease Control and Prevention.
Slide 1 Estimating Performance Below the National Level Applying Simulation Methods to TIMSS Fourth Annual IES Research Conference Dan Sherman, Ph.D. American.
Multi-sample structural equation models with mean structures Hans Baumgartner Penn State University.
Introduction Multilevel Analysis
Statistical Power 1. First: Effect Size The size of the distance between two means in standardized units (not inferential). A measure of the impact of.
Chapter 7 Experimental Design: Independent Groups Design.
Click to edit Master title style t = df = = - ( ) df: df MS error pv =pv =
Statistics (cont.) Psych 231: Research Methods in Psychology.
Confirmatory Factor Analysis Psych 818 DeShon. Construct Validity: MTMM ● Assessed via convergent and divergent evidence ● Convergent – Measures of the.
With or without constraints? An empirical comparison of two approaches to estimate interaction effects in the theory of planned behavior Eldad Davidov,
Experimental Design and Statistics. Scientific Method
Analyzing Statistical Inferences How to Not Know Null.
KNR 445 Statistics t-tests Slide 1 Introduction to Hypothesis Testing The z-test.
Chapter 12 Introduction to Analysis of Variance PowerPoint Lecture Slides Essentials of Statistics for the Behavioral Sciences Eighth Edition by Frederick.
Analyzing Statistical Inferences July 30, Inferential Statistics? When? When you infer from a sample to a population Generalize sample results to.
The Analysis of Variance ANOVA
Chapter 8: Between Subjects Designs
Statistics for the Social Sciences Psychology 340 Spring 2009 Analysis of Variance (ANOVA)
Units 1 & 2. Prescientific Psychology  Is the mind connected to the body or distinct?  Are ideas inborn or is the mind a blank slate filled by experience?
PART 2 SPSS (the Statistical Package for the Social Sciences)
T-tests Chi-square Seminar 7. The previous week… We examined the z-test and one-sample t-test. Psychologists seldom use them, but they are useful to understand.
SUMMARY EQT 271 MADAM SITI AISYAH ZAKARIA SEMESTER /2015.
Scenario-Based Scales Measuring Cultural Orientations of Business Owners Christine Koenig, Holger Steinmetz, Michael Frese, Andreas Rauch University of.
Independent Samples ANOVA. Outline of Today’s Discussion 1.Independent Samples ANOVA: A Conceptual Introduction 2.The Equal Variance Assumption 3.Cumulative.
Aron, Aron, & Coups, Statistics for the Behavioral and Social Sciences: A Brief Course (3e), © 2005 Prentice Hall Chapter 10 Introduction to the Analysis.
Statistics (cont.) Psych 231: Research Methods in Psychology.
Chapter 22 Inferential Data Analysis: Part 2 PowerPoint presentation developed by: Jennifer L. Bellamy & Sarah E. Bledsoe.
Chapter 11: Test for Comparing Group Means: Part I.
Definition Slides Unit 2: Scientific Research Methods.
Definition Slides Unit 1.2 Research Methods Terms.
Chapter 12 Introduction to Analysis of Variance
Classical Test Theory Psych DeShon. Big Picture To make good decisions, you must know how much error is in the data upon which the decisions are.
Descriptive Statistics Report Reliability test Validity test & Summated scale Dr. Peerayuth Charoensukmongkol, ICO NIDA Research Methods in Management.
CHAPTER 15: THE NUTS AND BOLTS OF USING STATISTICS.
Two-Way Analysis of Variance Chapter 11.
By: Aniruddha Deshmukh – M. Sc. Statistics, MCM
Modify—use bio. IB book  IB Biology Topic 1: Statistical Analysis
Holger Steinmetz Dep. of Work and Organizational Psychology
The t Test for Independent Means
Chapter 10 Introduction to the Analysis of Variance
Introduction to the t Test
Presentation transcript:

Effects of unequal indicator intercepts on manifest composite differences Holger Steinmetz and Peter Schmidt University of Giessen / Germany

Introduction Importance of analyses of mean differences For instance: - gender differences on wellbeing, self-esteem, abilities, behavior - differences between leaders and non-leaders on intelligence and personality traits - differences between cultural populations on psychological competencies, values, wellbeing Usual procedure: t-test or ANOVA with manifest composite scores Latent variables vs. manifest variables Manifest mean = indicator intercept + factor loading * latent mean → Will unequal intercepts lead to wrong conclusions regarding composite differences?

Intercepts and latent means X Y Y X B1B1 B0B0

xixi  i ii x1x1 x4x4 x2x2 x3x3         

xixi  i ii x1x1 x4x4 x2x2 x3x3         

x1x1 x4x4 x2x2 x3x3  xixi  i ii  M(x i )        

Intercepts and latent means x1x1 x4x4 x2x2 x3x3   xixi i ii  M(x i )        

Group differences in intercepts and factor loadings xixi   M(x i ) x1x1 x4x4 x2x2 x3x3  x1x1 x4x4 x2x2 x3x3  Group AGroup B

Group differences in intercepts and factor loadings xixi   M(x i ) x1x1 x4x4 x2x2 x3x3  x1x1 x4x4 x2x2 x3x3  Group AGroup B

Group differences in intercepts and factor loadings xixi   M(x i ) x1x1 x4x4 x2x2 x3x3  x1x1 x4x4 x2x2 x3x3  Group AGroup B

Meaning of (unequal) intercepts Associated terms used in the literature - Item bias - Differential item functioning - Measurement/factorial invariance ("strong factorial invariance", "scalar invariance") Meaning - Response style (acquiescence, leniency, severity) - Response sets (e.g., social desirability) - Connotations of items - Item specific difficulty

The study Partial invariance Research question: Is partial invariance enough for composite mean difference testing? - Pseudo-differences - Compensation effects Procedure (Mplus): - Step 1: Specification of two-group population models with latent mean and intercept differences; 1000 replications, raw data saved - Step 2: Creation of a composite score - Step 3: Analysis of composite differences - Step 4: Aggregation (-> sampling distribution)

The study Design (population model): - Two groups - One latent variable - 4 vs. 6 indicators - All intercepts equal vs. one vs. two intercepts unequal in varying directions (+.30 vs. -.30) - Latent mean difference: 0 vs Loadings kept equal with ‘s =.80; latent variance = 1 - N = 2 x 100 vs. 2 x Latent models as comparison standard for each condition Dependent variables - Average composite mean difference - Percent of significant composite differences („% sig“)

Full scalar invariance (Latent mean difference =.30) Avg. composite difference %sig N = 2 x 100 N = 2 x Indicators 6 Indicators 4 Indicators 6 Indicators

Pseudo-Differences (Latent mean difference = 0; unequal intercept(s) 4 Ind.6 Ind. N = 2 x 300N = 2 x 100N = 2 x Intercepts unequal (.30) 4 Ind.6 Ind.4 Ind. 6 Ind.4 Ind. 6 Ind. N = 2 x Intercept unequal (.30) Avg. composite difference %sig

Pseudo-Differences (Latent mean difference = 0; unequal intercept(s) 4 Ind.6 Ind. N = 2 x 300N = 2 x 100N = 2 x Intercepts unequal (.30) 4 Ind.6 Ind.4 Ind. 6 Ind.4 Ind. 6 Ind. N = 2 x Intercept unequal (.30) Avg. composite difference %sig

Compensation effects (Latent mean difference =.30; negative intercept difference) 4 Ind.6 Ind. N = 2 x 300N = 2 x 100N = 2 x Intercepts unequal (-.30) 4 Ind.6 Ind.4 Ind. 6 Ind.4 Ind. 6 Ind. N = 2 x Intercept unequal (-.30)

Compensation effects (Latent mean difference =.30; negative intercept difference) 4 Ind.6 Ind. N = 2 x 300N = 2 x 100N = 2 x Intercepts unequal (-.30) 4 Ind.6 Ind.4 Ind. 6 Ind.4 Ind. 6 Ind. N = 2 x Intercept unequal (-.30)

Summary Full latent variable models have more power than composite analyses Pseudo-differences - Even one unequal intercept increases the risk to find spurious composite differences - High sample size increases risk - Number of indicators reduces the risk – but not substantially Componensation effects - Even one unequal intercept reduces the size of the composite difference to 50% - In small samples little chance to find a significant composite difference (power = ) - Two unequal intercepts drastically reduce the composite difference: The power in the „best“ condition (2x300, 6 Ind.) is only.50

Conclusons Most comparisons of means rely on traditional composite difference analysis These methods make assumptions that are unrealistic (i.e., full invariance of intercepts) Even minor violations of these assumptions increase the risk of drawing wrong conclusions Advantages of SEM: - Assumptions can be tested - Partial invariance implies no danger - Greater power even in small samples

Thank you very much!