1. Karen Bandeen-Roche October 27, 2016
Latent Class Analysis
Karen Bandeen-Roche
October 27, 2016

2. Objectives For you to leave here knowing…
When is latent class analysis (LCA) model useful?
What is the LCA model its underlying assumptions?
How are LCA parameters interpreted?
How are LCA parameters commonly estimated?
How is LCA fit adjudicated?
What are considerations for identifiability / estimability?

3. Motivating Example Frailty of Older Adults
“…the sixth age shifts into the lean and slipper’d pantaloon, with spectacles on nose and pouch on side, his youthful hose well sav’d, a world too wide, for his shrunk shank…”
-- Shakespeare, “As You Like It”

4. The Frailty Construct
Fried et al., J Gerontol 2001; Bandeen-Roche et al., J Gerontol, 2006

5. Frailty as a latent variable
“Underlying”: status or degree of syndrome
“Surrogates”: Fried et al. (2001) criteria
weight loss above threshold
low energy expenditure
low walking speed
weakness beyond threshold
exhaustion

6. Part I:
Model

7. Latent class model ε1 Y1 Frailty η Ym εm Structural … Measurement
Increasingly learning what I just showed you seems to be oversimplified in a number of ways. Work I’m reporting on today primarily addresses one of the added complexities, that is, indirect measurement (box), such that boxes in the previous slide should really have been circles. Particularly, we don’t have a clean direct measurement of “inflammation”—rather, we measure a bunch of individual cytokines (X here), each with appreciable assay error, must infer “inflammation” from those.
A second challenge for analysis exists whether or not there’s a complex measurement problem (box), and that is, how to isolate the effect of inflammation on mobility functioning from the influences of potentially confounding variables in an observational study. (1 min) Ym εm
Measurement

8. Well-used latent variable models
Latent variable scale
Observed variable scale
Continuous
Discrete
Factor analysis
LISREL
Discrete FA
IRT (item response)
Latent profile
Growth mixture
Latent class analysis, regression
General software: MPlus, Latent Gold, WinBugs (Bayesian), NLMIXED (SAS)

9. Analysis of underlying subpopulations Latent class analysisUi … P1 PJ
∏11
∏1M
∏J1
∏JM … … Y1 YM Y1 YM
Lazarsfeld & Henry, Latent Structure Analysis, 1968; Goodman, Biometrika, 1974

10. Latent Variables: What? Integrands in a hierarchical model
Observed variables (i=1,…,n): Yi=M-variate; xi=P-variate
Focus: response (Y) distribution = GYx(y/x) ; x-dependence
Model:
Yi generated from latent (underlying) Ui:
(Measurement)
Focus on distribution, regression re Ui:
(Structural)
Overall, hierarchical model:

11. Latent Variable Models Latent Class Regression (LCR) Model
Structural model:
Measurement model:
= “conditional probabilities”
> is MxJ
Compare to general form:

12. Latent Variable Models Latent Class Regression (LCR) Model
Measurement assumptions:
Conditional independence
{Yi1,…,YiM} mutually independent conditional on Ui
Reporting heterogeneity unrelated to measured, unmeasured characteristics

13. Latent Variable Models Latent Class Regression (LCR) Model
Measurement assumptions:
Conditional independence
{Yi1,…,YiM} mutually independent conditional on Ci
Reporting heterogeneity unrelated to measured, unmeasured characteristics

14. Analysis of underlying subpopulations Method: Latent class analysis
Seeks homogeneous subpopulations
Features that characterize latent groups
Prevalence in overall population
Proportion reporting each symptom
Number of them
= least to achieve homogeneity / conditional independence

15. Latent class analysis Prediction
Of interest: Pr(C=j|Y=y)
= posterior probability of class membership
Once model is fit, a straightforward calculation
Pr(C=j|Y=y) = =
= ij when evaluated at yi

16. Part II:
Fitting

17. Estimation Broad Strokes
Maximum likelihood
EM Algorithm
Simplex method (Dayton & Macready, 1988)
Possibly with weighting, robust variance correction
ML software
Specialty: Mplus, Latent Gold
Stata: gllamm
SAS: macro
R: poLCA
Bayesian: winBugs

18. Estimation Methods other than EM algorithm
Bayesian
MCMC methods (e.g. per Winbugs)
A challenge: label-switching
Reversible-jump methods
Advantages: feasibility, philosophy
Disadvantages
Prior choice (high-dimensional; avoiding illogic)
Burn-in, duration
May obscure identification problems

19. Estimation Likelihood maximization: E-M algorithm
A process of averaging over missing data – in this case, missing data is class membership.

20. Estimation Likelihood maximization: E-M algorithm
Rationale: LVs as “missing” data
Brief review
“Complete” data
Complete data log likelihood
taken as a function of ϕ
Iterate between
(K+1) E-Step: evaluate
(K+1) M-Step: maximize wrt ϕ
Convergence to a local likelihood maximum under regularity
Dempster, Laird, and Rubin, JRSSB, 1977

21. Estimation EM example: Latent Class Model

22. EM-Algorithm Latent class model
A process of averaging over missing data – in this case, missing data is class membership.
1. Choose starting set of posterior probabilities
Use them to estimate P and π (M-step)
Calculate Log Likelihood
Use estimates of P and π to calculate posterior probabilities (E-step)
Repeat 2-4 until LL stops changing.

23. Global and Local Maxima
Multiple starting values very important!

24. Example: Frailty Women’s Health & Aging Studies
Longitudinal cohort studies to investigate
Causes / course of physical and cognitive disability
Physiological determinants of frailty
Up to 7 rounds spanning 15 years
Companion studies in community, Baltimore, MD
≥ moderately disabled women 65+ years: n=1002
≤ mildly disabled women years: n=436
This project: n=786 age years at baseline
Probability-weighted analyses
Guralnik et al., NIA, 1995; Fried et al., J Gerontol, 2001

25. Example: Latent Frailty Classes Women’s Health and Aging Study
Conditional Probabilities (π)
Criterion
2-Class Model
3-Class Model
CL. 1
“NON-FRAIL”
CL. 2
“FRAIL”
CL. 1 “ROBUST”
“INTERMED.”
CL. 3
Weight Loss
.073
.26
.072
.11
.54
Weakness
.088
.51
.029
.77
Slowness
.15
.70
.004
.45
.85
Low Physical Activity
.078
.000
.28
Exhaustion
.061
.34
.027
.16
.56
Class Prevalence (P) (%)
73.3
26.7
39.2
53.6
7.2
Bandeen-Roche et al., J Gerontol, 2006

26. Example: Latent Frailty Classes Women’s Health and Aging Study
Conditional Probabilities (π)
Criterion
2-Class Model
3-Class Model
CL. 1
“NON-FRAIL”
CL. 2
“FRAIL”
CL. 1 “ROBUST”
“INTERMED.”
CL. 3
Weight Loss
.073
.26
.072
.11
.54
Weakness
.088
.51
.029
.77
Slowness
.15
.70
.004
.45
.85
Low Physical Activity
.078
.000
.28
Exhaustion
.061
.34
.027
.16
.56
Class Prevalence (P) (%)
73.3
26.7
39.2
53.6
7.2
We estimate that 26% in the “frail”
Subpopulation exhibit weight loss”
Bandeen-Roche et al., J Gerontol, 2006

27. Part III:
Evaluating Fit

28. Choosing the Number of Classes
a priori theory
Chi-Square goodness of fit
Entropy
Information Statistics
AIC, BIC, others
Lo-Mendell-Rubin (LMR)
Not recommended (designed for normal Y)
Bootstrapped Likelihood Ratio Test

29. Entropy Measures classification error 0 – terrible 1 – perfect Ci=j
Dias & Vermunt (2006)

30. Information Statistics
s = # of parameters
N= sample size
smaller values are better
AIC: -2LL+2s
BIC: -2LL + s*log(N)
BIC is typically recommended
- Theory: consistent for selection in model family
- Nylund et al, Struct Eq Modeling, 2007

31. Likelihood Ratio Tests
LCA models with different # of classes NOT nested appropriately for direct LRT.
Rather: LRT to compare a given model to the “saturated” model
LCA df (binary case): J J*M
Saturated df: 2M -1
Goodness of fit df: 2M – J(M+1)
P parameters
(sum to 1)
π parameters
(M items*J classes)

32. Bootstrapped Likelihood Ratio Test
In the absence of knowledge about theoretical distribution of difference in –2LL, can construct empirical distribution from data.
per Nylund (2006) simulation studies, performs “best”

33. Example: Frailty Construct Validation Women’s Health & Aging Studies
Internal convergent validity
Criteria manifestation is syndromic
“a group of signs and symptoms that occur together and characterize a particular abnormality”
- Merriam-Webster Medical Dictionary

34. Validation: Frailty as a syndrome Method: Latent class analysis
If criteria characterize syndrome:
At least two groups (otherwise, no co-occurrence)
No subgrouping of symptoms (otherwise, more than one abnormality characterized)

35. Conditional Probabilities of Meeting Criteria in Latent Frailty Classes WHAS
Criterion
2-Class Model
3-Class Model
CL. 1
“NON-FRAIL”
CL. 2
“FRAIL”
CL. 1 “ROBUST”
“INTERMED.”
CL. 3
Weight Loss
.073
.26
.072
.11
.54
Weakness
.088
.51
.029
.77
Slowness
.15
.70
.004
.45
.85
Low Physical Activity
.078
.000
.28
Exhaustion
.061
.34
.027
.16
.56
Class Prevalence (%)
73.3
26.7
39.2
53.6
7.2
Bandeen-Roche et al., J Gerontol, 2006

36. Results: Frailty Syndrome Validation
Data: Women’s Health and Aging Study
Single-population model fit: inadequate
Two-population model fit: good
Pearson χ2 p-value=.22; minimized AIC, BIC
Frailty criteria prevalence stepwise across classes—no subclustering
Syndromic manifestation well indicated

37. Example Residual checking
Frailty construct
Write the hypothesis in above

38. Identifiability / Estimability
Part IV:
Identifiability / Estimability

39. Identifiability
Rough idea for “non”-identifiability: More unknowns than there are (independent) equations to solve for them
Definition: Consider a family of distributions
The parameter is (globally)
identifiable iff

40. Identifiability Related concepts
Local identifiability
Basic idea: ϕ identified within a neighborhood
Definition: F is locally identifiable at if there exists a neighborhood τ about for all τ Φ.
Estimability, empirical identifiability: The information matrix for ϕ given y1,…,yn is non-singular.

41. Identifiability Latent class (binary Y)
Latent class analysis (measurement only)
Parameter dimension: 2M -1
Unconstrained J-class model: J-1 + J*M
Need 2M ≥ J(M+1) (necessary, not sufficient)
Local identifiability: evaluate the Jacobian of the likelihood function (Goodman, 1974)
Estimability: Avoid fewer than 10 allocation per “cell”
n > 10*(2M) (rule of thumb)

42. Identifiability / estimability Frailty example
Latent class analysis
Need 2M ≥ J(M+1) (necessary, not sufficient)
M=5; J=3;
32 ≥ 3∙(5+1) – YES
By this criterion, could fit up to 9 classes
Local identifiability: evaluate the Jacobian of the likelihood function (Goodman, 1974)
Estimability: n > 10*(2M)
n > 10*(25) = YES

43. Objectives For you to leave here knowing…
When is latent class analysis (LCA) model useful?
What is the LCA model its underlying assumptions?
How are LCA parameters interpreted?
How are LCA parameters commonly estimated?
How is LCA fit adjudicated?
What are considerations for identifiability / estimability?