Data validation for use in SEM Ned Kock

Validity and reliability Whenever perception-based variables are used in inferential studies, measurement errors can bias the results. One effective technique employed to minimize the impact of such measurement errors on results is to measure each latent variable based on multiple indicators. This technique also allows for validity and reliability tests in connection with the measurement model used.

Indicators in reflective models Each set of related indicators is designed, often in the form of related question-statements, to “load on” (or correlate with) what is referred to as a latent variable. The above rule refers to reflective measurement models, and does not apply to models in which latent variables are measured in a formative way. Formative measurement is not widely discussed in SEM texts because it cannot be employed in classic factor-based SEM (e.g., LISREL); it can be employed in classic variance-based SEM (e.g., PLS) and factor-based PLS-SEM.

Reflective measurement example … of a process modeling approach. Latent variable Ease of generation Question-statements easgen1: It is easy to conceptualize a process using this approach. easgen2: It is easy to create a process model using this approach. easgen3: This approach for process modeling is easy to use. easgen4 (reversed): It is difficult to use this process modeling approach. Answers provided on a Likert-type scale ranging from 1 (Very strongly disagree) to 7 (Very strongly agree).

Validity assessment Among the most common validity tests are those in connection with the assessment of the convergent and discriminant validity of a measurement model. Convergent validity tests are aimed at verifying whether answers from different individuals to question-statements are sufficiently correlated with the respective latent variables. Discriminant validity tests are aimed at checking whether answers from different individuals to question-statements are either lightly correlated or not correlated at all with other latent variables.

Reliability assessment Reliability tests have a similar but somewhat different purpose than validity tests. They are aimed at verifying whether answers from different individuals to question-statements associated with each latent variable are sufficiently correlated. Validity and reliability tests allow for the assessment of whether the individuals responding to question-statements understood and answered the question-statements reasonably carefully; as opposed to answering them in a hurry, or in a mindless way.

A convergent validity test Loadings obtained from a confirmatory factor analysis are obtained with WarpPLS – combined or pattern loadings can be used. The loadings above are rotated, using an oblique rotation method similar to Promax. Whenever factor loadings associated with indicators for all respective latent variables are .5 or above the convergent validity of a measurement model is generally considered to be acceptable (Hair et al., 1987; Kock, 2015).

Good convergent validity Loadings obtained from a confirmatory factor analysis. Shown in shaded cells are the loadings expected to be conceptually associated with the respective latent variables (all above .5).

A discriminant validity test A measurement model containing latent variables is generally considered to have acceptable discriminant validity if the square root of the average variance extracted for each latent variable is higher than any of the bivariate correlations involving the latent variables in question (Fornell & Larcker, 1981; Kock, 2015). An even more conservative discriminant validity assessment would involve comparing the average variances extracted (as opposed to their square roots) with the bivariate correlations.

Good discriminant validity Notes on table: Correlation coefficients shown are Pearson bivariate correlations (calculated by WarpPLS): * = correlation significant at the .05 level. ** = correlation significant at the .01 level. Average variances extracted (AVEs) are shown on diagonal. Good discriminant validity because: -All average variances extracted (AVEs) are higher than the correlations shown below them or to their left. -The above is a conservative criterion; square roots of the AVEs are usually used in this type of test.

A reliability test Reliability assessment usually builds on the calculation of reliability coefficients, of which the most widely used is arguably Conbrach’s alpha. The reliability of a latent variable-based measurement model is considered to be acceptable if the Cronbach’s alpha coefficients calculated for each latent variable are .7 or above (Nunnaly, 1978; Kock, 2015). In SEM, the composite reliability coefficient can be used instead of the Cronbach’s alpha (Fornell, & Larcker, 1981; Kock, 2015), with the same .7 rule of thumb as above.

alpha = Cronbach’s alpha coefficient (calculated by WarpPLS). Good reliability Notes: alpha = Cronbach’s alpha coefficient (calculated by WarpPLS). The coefficients of reliability (alpha’s) range from .81 to .93 (all above .7), suggesting that the measurement model presents acceptable reliability.

Acknowledgements Adapted text, illustrations, and ideas from the following sources were used in the preparation of the preceding set of slides: Fornell, C., & Larcker, D.F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of marketing research, 18(1), 39-50. Hair, J.F., Anderson, R.E., & Tatham, R.L. (1987). Multivariate data analysis (2nd Edition). New York, NY: Macmillan. Kock, N. (2015). WarpPLS 5.0 User Manual. Laredo, TX: ScriptWarp Systems. Nunnaly, J. (1978). Psychometric theory. New York, NY: McGraw Hill. WarpPLS software. Final slide