10 SEM is Based on the Analysis of Covariances! Why?Analysis of correlations represents loss of information. A B r = 0.86r = 0.50 illustration with regressions having same slope and intercept Analysis of covariances allows for estimation of both standardized and unstandardized parameters.

11 I. SEM Essentials: 1. SEM is a form of graphical modeling, and therefore, a system in which relationships can be represented in either graphical or equational form. x1x1 y1y1 11  11 graphical form y 1 = γ 11 x 1 + ζ 1 equational form 2. An equation is said to be structural if there exists sufficient evidence from all available sources to support the interpretation that x 1 has a causal effect on y 1.

12 y2y2 y1y1 x1x1 y3y3 ζ1ζ1 ζ2 ζ2 ζ3ζ3 Complex Hypothesis e.g. y 1 = γ 11 x 1 + ζ 1 y 2 = β 21 y 1 + γ 21 x 1 + ζ 2 y 3 = β 32 y 2 + γ 31 x 1 + ζ 3 Corresponding Equations 3. Structural equation modeling can be defined as the use of two or more structural equations to represent complex hypotheses.

14 a. manipulations of x can repeatably be demonstrated to be followed by responses in y, and/or b. we can assume that the values of x that we have can serve as indicators for the values of x that existed when effects on y were being generated, and/or c. if it can be assumed that a manipulation of x would result in a subsequent change in the values of y Relevant References: Pearl (2000) Causality. Cambridge University Press. Shipley (2000) Cause and Correlation in Biology. Cambridge 4. Some practical criteria for supporting an assumption of causal relationships in structural equations:

15 The Methodological Side of SEM

16 6. SEM is a framework for building and evaluating multivariate hypotheses about multiple processes. It is not dependent on a particular estimation method. 7. When it comes to statistical methodology, it is important to distinguish between the priorities of the methodology versus those of the scientific enterprise. Regarding the diagram below, in SEM we use statistics for the purposes of the scientific enterprise. Statistics and other Methodological Tools, Procedures, and Principles. The Scientific Enterprise

17 The Relationship of SEM to the Scientific Enterprise modified from Starfield and Bleloch (1991) Understanding of Processes univariate descriptive statistics exploration, methodology and theory development realistic predictive models simplistic models multivariate descriptive statistics detailed process models univariate data modeling Data structural equation modeling

18 8. SEM seeks to progress knowledge through cumulative learning. Current work is striving to increase the capacity for model memory and model generality. exploratory/ model-building applications structural equation modeling confirmatory/ hypothesis-testing applications one aim of SEM

An interest in systems under multivariate control motivates us to explicitly consider the relative importances of multiple processes and how they interact. We seek to consider simultaneously the main factors that determine how system responses behave. 12. SEM is one of the few applications of statistical inference where the results of estimation are frequently “you have the wrong model!”. This feedback comes from the unique feature that in SEM we compare patterns in the data to those implied by the model. This is an extremely important form of learning about systems.

Illustrations of fixed-structure protocol models: Univariate Models x1x1 x2x2 x3x3 x4x4 x5x5 y1y1 Multivariate Models x1x1 x2x2 x3x3 x4x4 x5x5 F y1y1 y2y2 y3y3 y4y4 y5y5 Do these model structures match the causal forces that influenced the data? If not, what can they tell you about the processes operating?

Structural equation modeling and its associated scientific goals represent an ambitious undertaking. We should be both humbled by the limits of our successes and inspired by the learning that takes place during the journey.

22 x1x1 y1y1 y2y2 11 22 Some Terminology exogenous variable endogenous variables  21  11  21 path coefficients direct effect of x 1 on y 2 indirect effect of x 1 on y 2 is  11 times  21

23 First Rule of Path Coefficients: the path coefficients for unanalyzed relationships (curved arrows) between exogenous variables are simply the correlations (standardized form) or covariances (unstandardized form). x1x1 x2x2 y1y1.40 x 1 x 2 y x x y

24 x1x1 y1y1 y2y2  11 =.50  21 =.60  (gamma) used to represent effect of exogenous on endogenous.  (beta) used to represent effect of endogenous on endogenous. Second Rule of Path Coefficients: when variables are connected by a single causal path, the path coefficient is simply the standardized or unstandardized regression coefficient (note that a standardized regression coefficient = a simple correlation.) x 1 y 1 y x y y

25 Third Rule of Path Coefficients: strength of a compound path is the product of the coefficients along the path. x1x1 y1y1 y2y Thus, in this example the effect of x 1 on y 2 = 0.5 x 0.6 = 0.30 Since the strength of the indirect path from x 1 to y 2 equals the correlation between x 1 and y 2, we say x 1 and y 2 are conditionally independent.

26 What does it mean when two separated variables are not conditionally independent? x 1 y 1 y x y y x1x1 y1y1 y2y2 r =.55r = x 0.60 = 0.33, which is not equal to 0.50

27 The inequality implies that the true model is x1x1 y1y1 y2y2 Fourth Rule of Path Coefficients: when variables are connected by more than one causal pathway, the path coefficients are "partial" regression coefficients. additional process Which pairs of variables are connected by two causal paths? answer: x 1 and y 2 (obvious one), but also y 1 and y 2, which are connected by the joint influence of x 1 on both of them.

28 And for another case: x1x1 x2x2 y1y1 A case of shared causal influence: the unanalyzed relation between x 1 and x 2 represents the effects of an unspecified joint causal process. Therefore, x 1 and y 1 connected by two causal paths. x 2 and y 1 likewise.

29 x1x1 y1y1 y2y How to Interpret Partial Path Coefficients: - The Concept of Statistical Control The effect of y 1 on y 2 is controlled for the joint effects of x 1. I have an article on this subject that is brief and to the point. Grace, J.B. and K.A. Bollen Interpreting the results from multiple regression and structural equation models. Bull. Ecological Soc. Amer. 86:

30 Interpretation of Partial Coefficients Analogy to an electronic equalizer from Sourceforge.net With all other variables in model held to their means, how much does a response variable change when a predictor is varied?

31 x1x1 y1y1 y2y2 Fifth Rule of Path Coefficients: paths from error variables are correlations or covariances. R 2 = R 2 = 22 1 equation for path from error variable.56 alternative is to show values for zetas, which = 1-R 2.84

32 x1x1 y1y1 y2y2 R 2 = 0.16 R 2 = 0.25 22 1 x 1 y 1 y x y y Now, imagine y 1 and y 2 are joint responses Sixth Rule of Path Coefficients: unanalyzed residual correlations between endogenous variables are partial correlations or covariances.

34 Seventh Rule of Path Coefficients: total effect one variable has on another equals the sum of its direct and indirect effects. y1y1 x2x2 x1x1 y2y2 ζ1ζ1 ζ2ζ x 1 x 2 y y y Total Effects: Eighth Rule of Path Coefficients: sum of all pathways between two variables (causal and noncausal) equals the correlation/covariance. note: correlation between x 1 and y 1 = 0.55, which equals *0.11

Path Tracing Rules No loops No going forward & then backward A maximum of one curved arrow per path 35

37 Suppression Effect - when presence of another variable causes path coefficient to strongly differ from bivariate correlation. x 1 x 2 y 1 y x x y y y1y1 x2x2 x1x1 y2y2 ζ1ζ1 ζ2ζ path coefficient for x 2 to y 1 very different from correlation, (results from overwhelming influence from x 1.)

42 Σ = { σ 11 σ 12 σ 22 σ 13 σ 23 σ 33 } Model-Implied CorrelationsObserved Correlations* { } S = * typically the unstandardized correlations, or covariances 2. Estimation (cont.) – analysis of covariance structure The most commonly used method of estimation over the past 3 decades has been through the analysis of covariance structure (think – analysis of patterns of correlations among variables). compare

43 x1 y1 y2 Hypothesized Model Σ = { σ 11 σ 12 σ 22 σ 13 σ 23 σ 33 } Implied Covariance Matrix Observed Covariance Matrix { } S = compare Model Fit Evaluations + Parameter Estimates estimation (e.g., maximum likelihood) 3. Evaluation

44 1. The Multiequational Framework (a) the observed variable model We can model the interdependences among a set of predictors and responses using an extension of the general linear model that accommodates the dependences of response variables on other response variables. y = p x 1 vector of responses α = p x 1 vector of intercepts Β = p x p coefficient matrix of y s on y s Γ = p x q coefficient matrix of y s on x s x = q x 1 vector of exogenous predictors ζ = p x 1 vector of errors for the elements of y Φ = cov ( x ) = q x q matrix of covariances among x s Ψ = cov (ζ) = q x q matrix of covariances among errors y = α + Βy + Γx + ζ

47 The LISREL Equations Jöreskög 1973 (b) the latent variable model η = α + Β η + Γξ + ζ x = Λ x ξ + δ y = Λ y η + ε where: η is a vector of latent responses, ξ is a vector of latent predictors, Β and Γ are matrices of coefficients, ζ is a vector of errors for η, and α is a vector of intercepts for η (c) the measurement model where: Λ x is a vector of loadings that link observed x variables to latent predictors, Λ y is a vector of loadings that link observed y variables to latent responses, and δ and ε are vectors are errors

Notation Covariance Matrixes of Interest: – Φ Exogenous Constructs – Ψ Structural Error Terms – Θ δ Measurement Error X – Θ ε Measurement Error Y

Trust in Individuals people are helpful (x1) people can be trusted (x2) people are Fair (x3) 1 ξ1ξ1 δ1δ1 δ2δ2 δ3δ3 Confirmatory Factor Analysis λ 11 λ 21

G Lecture 556 Form of  The form of  can be worked out with expectation operators

G Lecture 557 Fitting Functions ML minimizes ULS minimizes GLS minimizes ADF minimizes a weighted least squares criterion, but with a weight different than GLS

G Lecture 561 Estimation Example using Excel

G Lecture 562 Measures of Fit: Chi Square If model is not saturated, and If residuals of Y have multivariate normal distribution – ML*(N-1) and GLS*(N-1) have large sample chi squared distributions – Degrees of freedom given by difference in number of parameters in model compared to saturated model

63 Illustration of the use of Χ 2 X 2 = 3.64 with 1 df and 100 samples P = X 2 = 7.27 with 1 df and 200 samples P = x y1y1 y2y r xy2 expected to be 0.2 (0.40 x 0.50) X 2 = 1.82 with 1 df and 50 samples P = 0.18 correlation matrix issue: should there be a path from x to y 2 ? Essentially, our ability to detect significant differences from our base model, depends as usual on sample size.

G Lecture 564 Chi Square Test Issues Appeal of Chi Square Test – Makes model fit appear confirmatory – Can reject an ill fitting model – Can compare nested models – Can calculate power Problems with Chi Square Test – Global test will reject a good model if data are not multivariate normal – Usual issues of significance testing

65 Residuals: Most fit indices represent average of residuals between observed and predicted covariances. Therefore, individual residuals should be inspected. Correlation Matrix to be Analyzed y1 y2 x y y x Fitted Correlation Matrix y1 y2 x y y x residual = 0.15 Diagnosing Causes of Lack of Fit (misspecification) Modification Indices: Predicted effects of model modification on model chi- square.

Six Steps to Modeling Specification Implied Covariance Matrix Identification Estimation Model Fit Respecification

Specification Theorize your model – What observed variables? How many observed variables? – What latent variables? How many latent variables? – Relationship between latent variables? – Relationship between latent variables and observed variables? – Correlated errors of measurement?

Identification Are there unique values for parameters? Property of model, not data 10 = x + y x = y 2, 8 -1, 11 4, 6

Identification Underidentified Just identified Overidentified

Identification Rules for Identification – By type of model Classic econometric – e.g., recursive rule Confirmatory factor analysis – e.g., three indicator rule General Model – e.g., two-step rule

Identification Identified? Yes (just) by 3-indicator rule. Trust in Individuals people are helpful (x1) people can be trusted (x2) people are Fair (x3) 1 ξ1ξ1 δ1δ1 δ2δ2 δ3δ3 λ 11 λ 21

A Perspective on “Fit”