1 Structural validity of psychiatric scales Jouko Miettunen, PhD Department of Psychiatry University of Oulu

2 Topics of this presentation n Exploratory factor analysis n Confirmatory factor analysis n Structural equation modeling n Cronbach’s Alpha n Latent class analysis n Examples u Rutter u Swan

3 Exploratory factor analysis n Based on correlations between variables u Dichotomous, ordinal or continuous items n Data requirements u Tests for data u N should be about 5* n of variables u Depends on skewness of variables

4 Exploratory factor analysis n Communalities u Variables effect in EFA based on loadings n Selection of N of factors u Eigenvalues > 1 (or 1.5) u Scree test u Theory? n Interpreting loadings u E.g. > 0.40

5 Exploratory factor analysis n Factor analysis vs. principal component analysis u EFA maximizes variance explained by all factors in solution u PCA maximizes first variance explained by 1st factor, then by next etc. n Rotation techniques u Orthogonal or obligue

6 Confirmatory factor analysis n For testing presented models n Test statistics u Chi-square test u Akaike’s Information Criteria (AIC, CAIC) u Root Mean Square Error Of Approximation ( RMSEA) u Goodness of Fit Index (GFI, AGFI) u CFI u Tucker-Lewis Index (TLI)

7 Confirmatory factor analysis n ????

8 Structural equation modeling n Combination of factor analysis and regression n Continuous and discrete predictors and outcomes n Relationships among measured or latent variables

9 Structural equation modeling Caring orientation Expertise orientation Life orientation Catalytic- co-operational nursing Controlling nursing Confirming nursing male, p=.002 older, p<.0001 no children, p=.048 Swedish, p<.0001 older, p<.0001 no children, p=.036 Finnish, p=.020 younger, p=.0003 sairaanhoit, p=.020 no children, p<.0001 older, p=.034 Swedish, p<.0001 older, p0.002 older, p= (r=.64) + (r=.11) + (r=.27) + (r=.47) (r=.22) + (r=.44) + (r=.18) + (r=.19) Orientation to nursing Orientation to learning nursing Vanhanen-Nuutinen et al. (manuscript)

10 Structural equation modeling n References u Bentler & Stein. Structural equation models in medical research. Stat Methods Med Res 1: 159–181, u Bollen. Structural equations with latent variables. John Wiley & Sons, Inc, New York, u MacCallum & Austin. Applications of structural equation modeling in psychological research. Annu Rev Psychol 51: 201–226, 2000.

11 Cronbach’s Alpha n Are the items measuring same phenomenon? u For the whole scale? u For subscales? n Based on variances between items n Varies between 0 and 1 n Improves with more items u Validity of mean of the scale not validity of one item

12 Latent class analysis n Specific statistical method developed to group subjects according to selected characteristics u Classifies subjects to groups u Identifies characteristics that indicate groups

13 Example: Anti-Social Behavior n Damaged property n Fighting n Shoplifting n Stole <$50 n Stole >$50 n Use of force n Seriously threaten n Intent to injure n Use Marijuana n Use other drug n Sold Marijuana n Sold hard drugs n ‘Con’ somebody n Stole an Automobile n Broken into a building n Held stolen goods n Gambling Operation n National Longitudinal Survey of Youth (NLSY) n Respondent ages between 16 and 23 n Background information: age, gender and ethnicity n N=7, antisocial dichotomously scored behavior items: Reference:

14 Example: Anti-Social Behavior Damage Property FightingShopliftingStole <$50Gambling... Male Race Age C

15 Example: Anti-Social Behavior probabilities

16 Relationship between class probabilities and age by gender FemalesMales (age)

17 n Summary of four classes: u Property Offense Class (9.8%) u Substance Involvement Class (18.3%) u Person Offenses Class (27.9%) u Normative Class (44.1%) n Classification Table: Example: Anti-Social Behavior Rows: Average latent class probability for most likely latent class membership Columns: Latent class

18 Latent class analysis n References u Muthén & Muthén. Integrating person- centered and variable-centered analyses: Growth mixture modeling with latent trajectory classes. Alcohol Clin Exp Res, 24, , u LCA in ADHD ?????????????? u nars/lca/default.htm n More references and examples u Homepage of Mplus software: F

19 Rutter items n Rutter 1. Child is restless, does not have patience to sit down for along period of time n Rutter 2. Stays out of school without reason n Rutter 3. Wriggles and is restless n Rutter 4. Often ruins and brakes his/her own or other's things n Rutter 5. Fights every so often or quarrels often with other children n Rutter 6. Other children don't particularly like him/her n Rutter 7. Is often worried n Rutter 8. Has tendency towards being alone, is quite seclusive n Rutter 9. Is irritable, takes offence quickly, flares up easily n Rutter 10. Seems often low-spirited, unhappy, weepy or anguished n Rutter 11. Child has twitching in his/her face or compulsive movements in his/her body n Rutter 12. Sucks often his/her thumb or fingers n Rutter 13. Bites often nails or fingers n Rutter 14. Stays out of school for unimportant reasons n Rutter 15. Is often disobedient n Rutter 16. Is not able to concentrate on anything for a longish period n Rutter 17. Is often scared of new things or situations n Rutter 18. Is meticulous pedantic n Rutter 19. Lies often n Rutter 20. Has stolen things once or more often n Rutter 21. Is passive, slack or apathetic n Rutter 22. Complains often of aches and pains n Rutter 23. Child has had tears in his/her eyes when coming to school or has refused to come into the school building n Rutter 24. Child stutters n Rutter 25. Gets annoyed or behaves aggressively when corrected n Rutter 26. Teases other children

20 Example: Rutter Eigenvalues Northern Finland 1986 Birth Cohort 7-year follow-up (N=8228) PROMAX ROTATED LOADINGS ________ ________ ________ ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM alpha=0.25

21 Example: Swan Eigenvalues Northern Finland 1986 Birth Cohort 15-year follow-up (N=6643) PROMAX ROTATED LOADINGS 1 2 ______ ______ ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM ITEM

22 Example: Swan Confirmatory factor analysis (N=5987) CFI TLI RMSEA SRMR 0.044

23 Example: Swan Alpha  Whole scale (21 items) 0.45  ADHD (18 items) 0.33  Attention-deficit (first 9 items) 0.25  Hyperactivity (items 10-18) 0.26