1. 1 Structural validity of psychiatric scales Jouko Miettunen, PhD Department of Psychiatry University of Oulu e-mail: jouko.miettunen@oulu.fi

2. 2 Topics of this presentation n Exploratory factor analysis n Confirmatory factor analysis n Structural equation modeling n Cronbach’s Alpha n Latent class analysis n Examples u Rutter u Swan

3. 3 Exploratory factor analysis n Based on correlations between variables u Dichotomous, ordinal or continuous items n Data requirements u Tests for data u N should be about 5* n of variables u Depends on skewness of variables

4. 4 Exploratory factor analysis n Communalities u Variables effect in EFA based on loadings n Selection of N of factors u Eigenvalues > 1 (or 1.5) u Scree test u Theory? n Interpreting loadings u E.g. > 0.40

5. 5 Exploratory factor analysis n Factor analysis vs. principal component analysis u EFA maximizes variance explained by all factors in solution u PCA maximizes first variance explained by 1st factor, then by next etc. n Rotation techniques u Orthogonal or obligue

6. 6 Confirmatory factor analysis n For testing presented models n Test statistics u Chi-square test u Akaike’s Information Criteria (AIC, CAIC) u Root Mean Square Error Of Approximation ( RMSEA) u Goodness of Fit Index (GFI, AGFI) u CFI u Tucker-Lewis Index (TLI)

7. 7 Confirmatory factor analysis n ????

8. 8 Structural equation modeling n Combination of factor analysis and regression n Continuous and discrete predictors and outcomes n Relationships among measured or latent variables

9. 9 Structural equation modeling Caring orientation Expertise orientation Life orientation Catalytic- co-operational nursing Controlling nursing Confirming nursing male, p=.002 older, p<.0001 no children, p=.048 Swedish, p<.0001 older, p<.0001 no children, p=.036 Finnish, p=.020 younger, p=.0003 sairaanhoit, p=.020 no children, p<.0001 older, p=.034 Swedish, p<.0001 older, p0.002 older, p=.030 + (r=.64) + (r=.11) + (r=.27) + (r=.47) + + + + + + + (r=.22) + (r=.44) + (r=.18) + (r=.19) Orientation to nursing Orientation to learning nursing Vanhanen-Nuutinen et al. (manuscript)

10. 10 Structural equation modeling n References u Bentler & Stein. Structural equation models in medical research. Stat Methods Med Res 1: 159–181, 1992. u Bollen. Structural equations with latent variables. John Wiley & Sons, Inc, New York, 1989. u MacCallum & Austin. Applications of structural equation modeling in psychological research. Annu Rev Psychol 51: 201–226, 2000.

11. 11 Cronbach’s Alpha n Are the items measuring same phenomenon? u For the whole scale? u For subscales? n Based on variances between items n Varies between 0 and 1 n Improves with more items u Validity of mean of the scale not validity of one item

12. 12 Latent class analysis n Specific statistical method developed to group subjects according to selected characteristics u Classifies subjects to groups u Identifies characteristics that indicate groups

13. 13 Example: Anti-Social Behavior n Damaged property n Fighting n Shoplifting n Stole <$50 n Stole >$50 n Use of force n Seriously threaten n Intent to injure n Use Marijuana n Use other drug n Sold Marijuana n Sold hard drugs n ‘Con’ somebody n Stole an Automobile n Broken into a building n Held stolen goods n Gambling Operation n National Longitudinal Survey of Youth (NLSY) n Respondent ages between 16 and 23 n Background information: age, gender and ethnicity n N=7,326 17 antisocial dichotomously scored behavior items: Reference: http://www.ats.ucla.edu/stat/mplus/seminars/lca/default.htm

14. 14 Example: Anti-Social Behavior Damage Property FightingShopliftingStole <$50Gambling... Male Race Age C

15. 15 Example: Anti-Social Behavior probabilities

16. 16 Relationship between class probabilities and age by gender FemalesMales 16 17 18 19 20 21 22 23 (age)16 17 18 19 20 21 22 23

17. 17 n Summary of four classes: u Property Offense Class (9.8%) u Substance Involvement Class (18.3%) u Person Offenses Class (27.9%) u Normative Class (44.1%) n Classification Table: 1234 10.8540.0310.0700.040 20.0410.9170.0400 30.0580.0210.8200.100 40.03800.0800.880 Example: Anti-Social Behavior Rows: Average latent class probability for most likely latent class membership Columns: Latent class

18. 18 Latent class analysis n References u Muthén & Muthén. Integrating person- centered and variable-centered analyses: Growth mixture modeling with latent trajectory classes. Alcohol Clin Exp Res, 24, 882-91, 2000. u LCA in ADHD ?????????????? u http://www.ats.ucla.edu/stat/mplus/semi nars/lca/default.htm n More references and examples u Homepage of Mplus software: F www.statmodel.com

19. 19 Rutter items n Rutter 1. Child is restless, does not have patience to sit down for along period of time n Rutter 2. Stays out of school without reason n Rutter 3. Wriggles and is restless n Rutter 4. Often ruins and brakes his/her own or other's things n Rutter 5. Fights every so often or quarrels often with other children n Rutter 6. Other children don't particularly like him/her n Rutter 7. Is often worried n Rutter 8. Has tendency towards being alone, is quite seclusive n Rutter 9. Is irritable, takes offence quickly, flares up easily n Rutter 10. Seems often low-spirited, unhappy, weepy or anguished n Rutter 11. Child has twitching in his/her face or compulsive movements in his/her body n Rutter 12. Sucks often his/her thumb or fingers n Rutter 13. Bites often nails or fingers n Rutter 14. Stays out of school for unimportant reasons n Rutter 15. Is often disobedient n Rutter 16. Is not able to concentrate on anything for a longish period n Rutter 17. Is often scared of new things or situations n Rutter 18. Is meticulous pedantic n Rutter 19. Lies often n Rutter 20. Has stolen things once or more often n Rutter 21. Is passive, slack or apathetic n Rutter 22. Complains often of aches and pains n Rutter 23. Child has had tears in his/her eyes when coming to school or has refused to come into the school building n Rutter 24. Child stutters n Rutter 25. Gets annoyed or behaves aggressively when corrected n Rutter 26. Teases other children

20. 20 Example: Rutter Eigenvalues 110.98 2 3.79 3 1.73 4 1.62 5 1.12 6 0.94 Northern Finland 1986 Birth Cohort 7-year follow-up (N=8228) PROMAX ROTATED LOADINGS 1 2 3 ________ ________ ________ ITEM1 0.910 -0.184 0.076 ITEM2 0.028 0.048 0.923 ITEM3 0.858 -0.037 0.042 ITEM4 0.850 0.001 0.069 ITEM5 0.963 -0.061 -0.084 ITEM6 0.610 0.356 -0.047 ITEM7 -0.093 0.819 -0.039 ITEM8 -0.252 0.782 0.067 ITEM9 0.684 0.310 -0.200 ITEM10 -0.002 0.825 0.107 ITEM11 0.306 0.463 -0.123 ITEM12 0.217 0.215 0.202 ITEM13 0.230 0.265 0.206 ITEM14 -0.031 0.132 0.870 ITEM15 0.978 -0.104 -0.019 ITEM16 0.753 -0.015 0.140 ITEM17 -0.178 0.869 0.035 ITEM18 0.046 0.584 -0.276 ITEM19 0.718 -0.024 0.195 ITEM20 0.660 -0.067 0.170 ITEM21 0.009 0.529 0.237 ITEM22 0.090 0.323 0.249 ITEM23 -0.009 0.581 0.258 ITEM24 0.068 0.356 -0.019 ITEM25 0.752 0.236 -0.226 ITEM26 0.959 -0.122 -0.014 alpha=0.25

21. 21 Example: Swan Eigenvalues 111.38 2 1.68 3 0.62 Northern Finland 1986 Birth Cohort 15-year follow-up (N=6643) PROMAX ROTATED LOADINGS 1 2 ______ ______ ITEM1 0.000 0.875 ITEM2 0.088 0.813 ITEM3 0.364 0.438 ITEM4 0.056 0.824 ITEM5 -0.013 0.864 ITEM6 -0.044 0.830 ITEM7 0.061 0.840 ITEM8 0.210 0.566 ITEM9 0.259 0.533 ITEM10 0.729 0.133 ITEM11 0.816 0.099 ITEM12 0.873 -0.031 ITEM13 0.875 -0.003 ITEM14 0.908 0.018 ITEM15 0.794 0.051 ITEM16 0.678 0.195 ITEM17 0.841 0.049 ITEM18 0.564 0.193

22. 22 Example: Swan Confirmatory factor analysis (N=5987) CFI 0.880 TLI 0.985 RMSEA 0.140 SRMR 0.044

23. 23 Example: Swan Alpha  Whole scale (21 items) 0.45  ADHD (18 items) 0.33  Attention-deficit (first 9 items) 0.25  Hyperactivity (items 10-18) 0.26