SEM Model Fit: Introduction David A. Kenny January 12, 2014.

Slides:



Advertisements
Similar presentations
Tests of Hypotheses Based on a Single Sample
Advertisements

1 Model Evaluation and Selection. 2 Example Objective: Demonstrate how to evaluate a single model and how to compare alternative models.
Inferential Statistics and t - tests
Structural Equation Modeling
1 COMM 301: Empirical Research in Communication Lecture 15 – Hypothesis Testing Kwan M Lee.
Objectives 10.1 Simple linear regression
CHAPTER 25: One-Way Analysis of Variance Comparing Several Means
Hypothesis Testing Steps in Hypothesis Testing:
Decision Errors and Power
SOC 681 James G. Anderson, PhD
Growth Curve Model Using SEM
Empirical Analysis Doing and interpreting empirical work.
Structural Equation Modeling
Ch 15 - Chi-square Nonparametric Methods: Chi-Square Applications
Aaker, Kumar, Day Seventh Edition Instructor’s Presentation Slides
G Lecture 51 Estimation details Testing Fit Fit indices Arguing for models.
Copyright © 2010 Pearson Education, Inc. Chapter 24 Comparing Means.
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 8 Tests of Hypotheses Based on a Single Sample.
Stages in Structural Equation Modeling
Introduction to Multilevel Modeling Using SPSS
AM Recitation 2/10/11.
Aaker, Kumar, Day Ninth Edition Instructor’s Presentation Slides
Structural Equation Modeling 3 Psy 524 Andrew Ainsworth.
Correlation and Linear Regression
Hypothesis Testing. Distribution of Estimator To see the impact of the sample on estimates, try different samples Plot histogram of answers –Is it “normal”
Inference in practice BPS chapter 16 © 2006 W.H. Freeman and Company.
Confirmatory Factor Analysis Psych 818 DeShon. Purpose ● Takes factor analysis a few steps further. ● Impose theoretically interesting constraints on.
6.1 - One Sample One Sample  Mean μ, Variance σ 2, Proportion π Two Samples Two Samples  Means, Variances, Proportions μ 1 vs. μ 2.
Copyright © 2010, 2007, 2004 Pearson Education, Inc. Chapter 24 Comparing Means.
CJT 765: Structural Equation Modeling Class 7: fitting a model, fit indices, comparingmodels, statistical power.
Statistical Decision Making. Almost all problems in statistics can be formulated as a problem of making a decision. That is given some data observed from.
Variance Harry R. Erwin, PhD School of Computing and Technology University of Sunderland.
Random Regressors and Moment Based Estimation Prepared by Vera Tabakova, East Carolina University.
VI. Evaluate Model Fit Basic questions that modelers must address are: How well does the model fit the data? Do changes to a model, such as reparameterization,
Ordinary Least Squares Estimation: A Primer Projectseminar Migration and the Labour Market, Meeting May 24, 2012 The linear regression model 1. A brief.
CJT 765: Structural Equation Modeling Class 8: Confirmatory Factory Analysis.
Chapter 11: Inference for Distributions of Categorical Data Section 11.1 Chi-Square Goodness-of-Fit Tests.
Measurement Models: Exploratory and Confirmatory Factor Analysis James G. Anderson, Ph.D. Purdue University.
Testing Hypothesis That Data Fit a Given Probability Distribution Problem: We have a sample of size n. Determine if the data fits a probability distribution.
Assessing Hypothesis Testing Fit Indices
Measures of Fit David A. Kenny January 25, Background Introduction to Measures of Fit.
Hypothesis Testing An understanding of the method of hypothesis testing is essential for understanding how both the natural and social sciences advance.
CFA: Basics Beaujean Chapter 3. Other readings Kline 9 – a good reference, but lumps this entire section into one chapter.
Copyright © 2010, 2007, 2004 Pearson Education, Inc. Chapter 24 Comparing Means.
SEM Basics 2 Byrne Chapter 2 Kline pg 7-15, 50-51, ,
Copyright © 2010 Pearson Education, Inc. Warm Up- Good Morning! If all the values of a data set are the same, all of the following must equal zero except.
Assessing Hypothesis Testing Fit Indices Kline Chapter 8 (Stop at 210) Byrne page
CJT 765: Structural Equation Modeling Class 8: Confirmatory Factory Analysis.
Other Types of t-tests Recapitulation Recapitulation 1. Still dealing with random samples. 2. However, they are partitioned into two subsamples. 3. Interest.
Examples. Path Model 1 Simple mediation model. Much of the influence of Family Background (SES) is indirect.
ALISON BOWLING CONFIRMATORY FACTOR ANALYSIS. REVIEW OF EFA Exploratory Factor Analysis (EFA) Explores the data All measured variables are related to every.
How to Fool Yourself with SEM James G. Anderson, Ph.D Purdue University.
Comparing Means Chapter 24. Plot the Data The natural display for comparing two groups is boxplots of the data for the two groups, placed side-by-side.
Confirmatory Factor Analysis of Longitudinal Data David A. Kenny December
Comparing Counts Chapter 26. Goodness-of-Fit A test of whether the distribution of counts in one categorical variable matches the distribution predicted.
Statistical Inference for the Mean Objectives: (Chapter 8&9, DeCoursey) -To understand the terms variance and standard error of a sample mean, Null Hypothesis,
Chapter 11: Categorical Data n Chi-square goodness of fit test allows us to examine a single distribution of a categorical variable in a population. n.
Class Seven Turn In: Chapter 18: 32, 34, 36 Chapter 19: 26, 34, 44 Quiz 3 For Class Eight: Chapter 20: 18, 20, 24 Chapter 22: 34, 36 Read Chapters 23 &
The SweSAT Vocabulary (word): understanding of words and concepts. Data Sufficiency (ds): numerical reasoning ability. Reading Comprehension (read): Swedish.
Testing the Equality of Means and Variances in Over-Time Data David A. Kenny.
Multiple Regression Analysis: Inference
STAT 312 Chapter 7 - Statistical Intervals Based on a Single Sample
Measures of Fit David A. Kenny.
CJT 765: Structural Equation Modeling
Writing about Structural Equation Models
Factors Affecting Measures of Fit
Confirmatory Factor Analysis
Inferential Statistics
Factors Affecting Measures of Fit
Testing Causal Hypotheses
Presentation transcript:

SEM Model Fit: Introduction David A. Kenny January 12, 2014

2 Definition Fit refers to the ability of a model to reproduce the data (i.e., usually the variance-covariance matrix). A good-fitting model is one that is reasonably consistent with the data and so does not necessarily require any major respecification. A good-fitting measurement model is required before interpreting the causal paths of the structural model.

3 However… A good fitting model does not necessarily mean a good model. For instance, a model all of whose parameters are zero is likely a "good-fitting" model. Additionally, models with nonsensical results (e.g., paths that are clearly the wrong sign) and models with poor discriminant validity or Heywood cases can be “good-fitting” models. Parameter estimates must be carefully examined to determine if one has a reasonable model as well as the fit statistics.

4 Additionally A good fitting model can still be improved. You can make it a better fitting model. Also too you can make the model simpler and not appreciably change the fit. A good fitting model is not necessarily the best model.

5 How Big a Sample Size? Rules of Thumb –Ratio of N to the Number of Free Parameters –Absolute N Power Analysis

6 Ratio of N to Number of Free Parameters Tanaka (1987): 20 to 1, but that is unrealistically high. Bentler & Chou (1987): 5 to 1 Many published studies fail to meet this goal!

7 Absolute N 200 is seen as a goal for SEM research Lower sample sizes can be used for –Models with no latent variables –Models where all loadings are fixed (usually to one) –Models with strong correlations –Simpler models –Models for which there is an upper limit on N (e.g., countries or years as the unit)

8 Power Analysis The best way to determine if you have a large enough sample is to conduct a power analysis. Either use the Sattora and Saris (1985) method or conduct a simulation. To determine the power to detect a poor fitting model, you can use Preacher and Coffman’s web-based calculator.web-based calculator

9  2 Test For models with about 75 to 200 cases, the chi square test is a reasonable measure of fit. But for models with more cases (e.g., 400 or more), the  2 test is almost always statistically significant. This is why fit indices were invented. They provide a way to claim that one has a good model, despite the fact that the  2 test is statistically significant. Sometimes chi square is more interpretable if it is transformed into a Z value. The following approximation can be used: Z = √(2χ 2 ) - √(2df - 1)

10 More on  2 An old measure of fit is the chi square to df ratio or  2 /df. A problem with this fit index is that there is no universally agreed upon standard as to what is a good- and a bad-fitting model. Note, however, that two very popular fit indices, TLI and RMSEA, are largely based on this old- fashioned ratio. The chi square test is too liberal (i.e., too many Type 1) errors when variables have non-normal distributions, especially distributions with kurtosis. Moreover, with small sample sizes, there are too many Type I errors. Note the  2 test is asymptotic test, and so it works best with large sample sizes.

11 Typology of Fit Indices This typology is very different from the mainstream definitions. Please note differences. Typology –Incremental –Absolute –Comparative

12 Incremental Fit Indices An incremental (sometimes called relative) fit index is analogous to R 2, and so a value of zero indicates having the worst possible model and a value of one indicates having the best possible. So my model is placed on a continuum. In terms of a formula, it is Worst Possible Model – My Model Worst Possible Model – Best Possible Model

13 The Best Model The standard definition of best possible model is one in which equaled it degrees of freedom (the expected value of chi-square given the null hypothesis of perfect fit). An older definition (Bentler-Bonnet) is assume the best model has a chi square of zero, but that definition ignores sampling error.

14 The Worst Model The worst possible model is called the null or independence model and the usual convention is to allow all the variables in the model to have variation but no correlation. The usual null model is to allow the means to equal their actual value. However, for growth curve models, the null model should set the means as equal, i.e., no growth.

15 df of the Null Model The degrees of freedom of the null model are k(k – 1)/2 where k is the number of variables in the model. If the null model sets the means equal, as in a growth-curve model, its df are (k + 2)(k – 1)/2.

16 Alternative Null Models Alternative null models might be considered (but almost never done). One alternative null model is that all latent variable correlations are zero and another is that all exogenous variables are correlated but the endogenous variables are uncorrelated with each other and the exogenous variables. O’Boyle and Williams (2011) suggest two different null models for the measurement and structural models.

17 Absolute Fit Indices An absolute measure of fit presumes that the best fitting model has a fit of zero. The measure of fit then determines how far the model is from perfect fit. These measures of fit are typically “badness” measure of fit in that a bigger number implies worse fit. Common absolute fit indices are SRMR and RMSEA.

18 Comparative Fit Indices A comparative measure of fit is only interpretable when comparing two different models. This term is unique to this PowerPoint in that these measures are more commonly called absolute fit indices. However, it is helpful to distinguish absolute indices that do not require a comparison between two models. One advantage of comparative fit indices is that often they can be computed for models that are just-identified. Examples are AIC, BIC, and SABIC.

19 Controversy about Fit Indices There is considerable controversy about fit indices. Some researchers do not believe that fit indices add anything to the analysis (e.g., Barrett, 2007) and only the chi square should be interpreted. The worry is that fit indices allow researchers to claim that a miss-specified model is not a bad model. Others (e.g., Hayduk, Cummings, Boadu, Pazderka-Robinson, & Boulianne, 2007) argue that cutoffs for a fit index can be misleading and subject to misuse.

20 Consensus View Most analysts believe in the value of fit indices, but caution against strict reliance on cutoffs. Particularly, problematic is the “cherry picking” a fit index. That is, you compute many fit indices and you pick the one index that allows you to make the point that you want to make. If you decide not to report a popular index (e.g., the TLI or the RMSEA), you need to give a good reason why you are not.

21 Compute a Fit Index? Kenny, Kaniskan, and McCoach (2014) have argued that fit indices should not even be computed for small degrees of freedom models. Rather for these models, the researcher should locate the source of specification error by determining what parameter should be added to the model and then test that parameter.

22 Additional Powerpoints Standard measures of fit –CFI and TLI –RMSEA –SRMR Non-standard measures of fit

23 References Click hereClick here to go to references.