Measurement invariance of the Beck Depression Inventory in assessing and comparing treatment effects Marjolein Fokkema, MSc Niels Smits, PhD Henk Kelderman, PhD Pim Cuijpers, PhD Faculty of Psychology and Education Vrije Universiteit Amsterdam, the Netherlands

Introduction Increasing use of routine outcome monitoring in clinical practice No big or consistent differences found between therapies in depression research Real change or response shift? Measurement invariance across timepoints

Data National Institute of Mental Health Treatment of Depression Collaborative Research Program First major attempt at comparing efficacy of treatments for depression: 2 psychotherapy conditions (PSY) 2 medication conditions (MED) 239 subjects entered treatment (T0), 155 subjects completed treatment (T1).

Beck Depression Inventory (BDI) BDI (Beck et al, 1961) assesses depressive symptoms 21 items with 4 response categories, ranging from neutral to maximum severity

Example: item 8 Self Accusations (0) I don’t feel I am any worse than anybody else (1) I am critical of myself for my weakness or mistakes (2) I blame myself all the time for my faults (3) I blame myself for everything bad that happens

Example: item 15 Work Inhibition (0) I can work about as well as before (1) It takes an extra effort to get started at doing something (2) I have to push myself very hard to do anything (3) I can’t do any work at all

Beck Depression Inventory (BDI) BDI (Beck et al, 1961) assesses depressive symptoms 21 items with 4 response categories, ranging from neutral to maximum severity Total score is widely used as outcome measure

Methods SEM approach (LISREL) –Measurement invariance between T0 and T1 : Configural invariance Metric invariance Scalar invariance Invariant uniqueness

Configural invariance Modeldfχ2χ2 RMSEANNFICFIAIC 1D D D

Invariance constraints Modeldfχ2χ2 RMSEANNFICFIAIC Configural Metric Partial Metric * Scalar Partial scalar ** * Freed loadings for items 18, 19, 5 & 3 ** Freed intercepts for items 14, 7, 8, 3, 5, 21 & 9

Methods SEM approach (LISREL) –Measurement invariance between T0 and T1 IRT approach (MULTILOG) –Comparisons between groups (PSY and MED) and time points (T0 and T1)

IRT analysis (MULTILOG) Samejima’s (1969) Graded Response Model

Item Characteristic Curves Item 4: Lack of satisfaction

PSY T0 MED T0MED T1 PSY T1

Item Characteristic Curves Item 8: Self Accusations

PSY T0 MED T0MED T1 PSY T1

Item Characteristic Curves Item 16: Sleep Disturbance

PSY T0 MED T0MED T1 PSY T1

Item Characteristic Curves Item 19: Weight Loss

PSY T0 MED T0MED T1 PSY T1

IRT analysis (MULTILOG) Samejima’s (1969) Graded Response Model Freely estimated model, compared with model with a-parameters constrained to be equal –across time points –across groups

Invariance between T0 and T1 -2 times log likelihood PSYMED Free Constr. a’s Difference* p-value< * ~ χ 2 (21)

Invariance between PSY and MED -2 times log likelihood T0T1 Free Constr. a’s Difference* p-value * ~ χ 2 (21)

Discussion Response shift occured –Therapies are very effective at improving ICC’s! –Response shift seems different in different groups Future research: –Proper control group –Larger sample –Different measures for depression

Questions, remarks, suggestions? Thank you for your attention!