Class 13 and 14 Jacobson et al (1996) APA (2006)Evidence Based Practice in Psychology 1

2 Experimental Between Group Designs 1. Post-Test Only Control 2. Pre-Test -- Post-Test Control 3. Solomon Four Group (combination of 1 and 2 above) Factorial Design more than one independent variable; interactions gender x treatment Dependent Sample Design (Matching) 2

3 Experimental Between Group Designs Post-Test Only Control Pre-Test -- Post-Test Control Solomon Four Group Factorial Design (Treatment X Therapist) Dependent Sample Design (Matching) # of previous episodes and severity depression, presence of dysthimia, gender, marital stauts 3

Types Outcome Studies Kazdin (chap 18) 1. Treatment Package Strategy 2. Dismantling Strategy 3. Constructive Strategy 4. Parametric Strategy (structural components) 5. Common factors Control Group 6. Comparative Outcome Strategy 7. Client and Therapist Variation Strategy Moderation Designs

Types Outcome Studies Kazdin (chap 18) 1. Treatment Package Strategy 2. Dismantling Strategy What are the active ingredients ? 1. Constructive Strategy 2. Parametric Strategy (structural components) 3. Common factors Control Group 4. Comparative Outcome Strategy 5. Client and Therapist Variation Strategy Moderation Designs

Research Focus How treatment effects change: identify change mechanisms VS. How well treatment works 6

Change Mechanisms Activation Hypotheses ( BA) (activation) Change behaviors- become active and access sources of reinforcement- occurs early in therapy Coping Skills Hypotheses (AT) Learn effective coping strategies (coping + activation) Beck Hypotheses (CT) Change cognitive structures or core schemas (cognitive schemas + activation + coping) 7

Chapter 78 Treatment Groups BA Condition: Behavioral Activation Hypotheses: Behavioral Activation AT Condition: Coping Skills Hypotheses Behavioral Activation Coping Skills - Automatic Thoughts CT Condition: Cognitive Therapy Hypotheses Behavioral Activation Automatic Thoughts Core Schemas 8

Chapter 79 Mechanisms of Change Construct Measure Behavioral Activation Dysfunctional Thinking Cognitive Schemas Pleasant Events Schedule Automatic Thoughts Q. Expanded Attributional Styles How did the authors examine if the mechanisms of change worked in therapy as predicted by theory? 9

How Treatment Effects Change To what extent treatment groups differed in post-test measures of each change mechanism Measure Ttreat. G Behavioral activation – Pleasant Event SCd BA Coping skills-- AT (Dist Thinking) AT Core schemas- Exp Attrb Style CT To what extent was there change from pre-to post- treatment in each change mechanism 10

How Well Treatments Work Which of the three treatment conditions yielded better outcomes regarding Depression at termination and at 6 month follow up? HRSD BDI Recovered Improved Rates 11

Research Questions Analyses Treatment Outcome group differences Depression Type of Analyses CVs IVS DVs Mechanism of Change: Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) Type of Analyses CVs IVS DVs Mechanism of Change: Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS) Type and # of Analyses 12

Research Questions Analyses Treatment Outcome group differences in Depression 3 x 4 MANCOVA CVs Pre-test BDI HRSD IVs Treatments: CT, AT, BA Therapists (n=4) Treatment X Therapist DVs Post-Test BDI HRSD Mechanism of Change: Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) Mechanism of Change: Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS) 13

How Well Treatments Work Which of the three treatment conditions yielded better Depression outcomes at termination and at 6 month follow up? 3 X 4 MANCOVA IVS 3 Treatments (CT, AT, BA) 4 Therapists Treatment X Therapist (Interaction) DVs - Depression HRSD – Clinician BDI - Self- Report 14

Which group to use in outcome analyses GroupSessions Intent-to treatTotal sample n = 149 Dropouts (not included in MANCOVA analyses; p. 299) 1<Sessions <12 n = 12 CompletersAt least 12/20 sessions n = 137 Maximum Completers20 sessions n =

Results MANCOVAsTreatment Outcome: Post- Test Depression Main Effect Treatment ? Main Effect Therapist ? Treatment X Therapist Interaction ? Results p ANOVAs Table 3 16

Results MANCOVAs Treatment Outcome: Post- Test BID & HRSD Treatment Group N= Main Effect Treatment NS NS NS Main Effect Therapist NS NS NS Treatment X Therapist NS NS NS Results p ANOVAs Table 3 17

Table 3 -- ANCOVAS ?? 18

19 ANCOVA Post-Test Main Effects for Therapists Post-Test Measures TherapistsBDIHRSD TH-1 BDIT1HRSDT1 TH-2 BDIT2HRSDT2 TH-3 BDIT3HRSDT3 TH-4 BDIT4HRSDT4 19

20 ANCOVA - Post-Test Effects Therapist X Treatment Interaction Therapist BDI Post-test Scores by Treatment HRSD Post-test Scores by Treatment BAATCTBAATCT TH-1 BDIT1 HRSDT1 TH-2 BDIT2 HRSDT2 TH-3 BDIT3 HRSDT3 TH-4 BDIT4 HRSDT4 20

Outcome: Follow-up 6 months Overall Impact of Therapy ANCOVAS IV:Treatments DVs: Follow-Up HRSD BDI Covariate: Pretest Score Changes in Follow-up Time ANCOVAS IV: Treatments DVs: Follow-Up HRSD BDI Covariate: Post-test Score 21

Clinical Significance Improved No major depression at post-test Recovered No major depression and BDI<8 22

23

Clinical Significance: Mean Improvement/Recovered Rates across Treatments – Post -Test (p.299) Group ImprovedRecovered Intent-to treat Total sample (149) 62.3%51.5% Completers 12/20 N= % Maximum Completers N = %54.5% Dropouts N= %5.6% 24

25

Chapter 726 Mechanisms of Change Construct Measure Behavioral Activation Dysfunctional Thinking Cognitive Schemas Pleasant Events Schedule Automatic Thoughts Q. Expanded Attributional Styles How did the authors examine to what extent the mechanisms of change worked in therapy as predicted by theory? 26

How Treatment Effects Change To what extent treatment groups differed in post-test measures of each change mechanism Measure Ttreat. Grps Behavioral activation – Pleasant Event SCd BA Coping skills-- AT (Dist Thinking) AT Core schemas- Exp Attrb Style CT To what extent was there change from pre-to post- treatment in each change mechanism 27

Research Questions Analyses Treatment Outcome group differences Depression 3 x 4 MANCOVA CVs Pre-test BDI HRSD IVs Treatments: CT, AT, BA Therapists (n=4) Treatment X Therapist DVs Post-Test BDI HRSD Mechanism of Change: Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) Mechanism of Change: Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS) 28

Research Questions Analyses Treatment Outcome group differences Depression 3 x 4 MANCOVA CVs Pre-test BDI HRSD IVs Treatments: CT, AT, BA Therapists Treatment X Therapist DVs Post-Test BDI HRSD Mechanism of Change: Treatment groups differed in post- test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) 3 ANCOVAS (one CV & DV per analysis) CV Pre-test: E Attrb Q, ATQ, PES IV Treatments: (CT, AT, BA) DV Post-test: E Attrb Q, ATQ, PES Mechanism of Change: Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS) 29

Research Questions Analyses Treatment Outcome group differences in Depression 3 x 4 MANCOVA CVs Pre-test BDI HRSD IVs Treatments: CT, AT, BA Therapists Treatment X Therapist DVs Post-Test BDI HRSD Mechanism of Change: Treatment groups differed in post- test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) 3 ANCOVAS (one CV & DV per analysis) CV Pre-test: E Attrb Q, ATQ, PES IV Treatments: (CT, AT, BA) DV Post-test: E Attrb Q, ATQ, PES Mechanism of Change: Treatment groups change from pre- to post- treatment in each change mechanism (BA – AT-CS) 3 Paired T Tests per Treatment Pre-post differences in each mechanism of change for each treatment : E Attrb Q, ATQ, PES 30

Paired T Tests: Pre-test/Post-test Change Treatment Pleasant Events Schedule Automatic Thoughts Questionnaires Expanded Attributional Styles Behavioral Activation PESch**ATQEAS Automatic Thoughts PESch**ATQ**EAS Cognitive Therapy PESch**ATQ**EAS** Clients in all conditions significantly improved on the three measures (p. 301) 31

Residual change score in Mechanism of Change from Pre-to Mid-T and RCS in Depression from Mid-T to Post-T (p.301) 32

Threats to Statistical Conclusion Validity Jacobson et al. (1996) Are the observed relations among variables accurate? 1.Power 2.Unreliability of Treatment Implementation 33

Threats to Statistical Conclusion Validity Are the observed relations among variables accurate? 1.Power N=149 and 3 groups/about 50 participants per group. Large N for an Exp study (+) Outcome measures are well-known – high internal reliability (+) No information is given about alphas with study sample Power analyses not reported 2.Unreliability of Treatment Implementation Therapists were experienced in CT and trained for study (+) Training followed a manual prepared for each treatment group (+) Therapy tapes were listened on an on- going basis and therapists flagged if they deviated (+) Analyses of adherence based on 27 taped sessions showed that treatments were distinct and consistent with their respective protocols (+) 34

Threats to Statistical Conclusion Validity Are the observed relations among variables accurate? 3.Extraneous Variance in the Experimental Setting 4.Heterogeneity of Participants 35

Threats to Statistical Conclusion Validity Are the observed relations among variables accurate? 3.Extraneous Variance in the Experimental Setting Do not appear to be any – The same four therapists administered all treatments However procedures and settings are not described in detail 4.Heterogeneity of Participants Study had many exclusion criteria including co-morbidity, taking psychotropic medication, suicidal… p (+) However, don’t know #s in pool of volunteers from which the 152 accepted patients were taken from 72% were women (+) No info regarding race/ethnicity, SES (-) 36

Threats to Internal Validity Can we conclude that there is a causal relation between the IV and the DV? 1.Selection to Treat. Groups 2.History 3.Attrition 4.Repeated Testing Effects 5.Reaction to Control Group Assignment 37

Threats to Internal Validity Can we conclude that there is a causal relation between the IV and the DV? 1.Selection to Treat. Groups Used Randomization after matching for episodes dep., dysthimia, severity of depression, gender, marital status (+) 2.History Therapy appeared to occur for everyone at once – but this is not addressed 3.Attrition Small -only 15 out of % -Attrition during acute treatment was comparable across treatment conditions – p.296 right (+) 4.Repeated Testing Effects Five administration of measures – at pre-test; post –test and 6, 12, 18 month follow-up (-) 5.Reaction to Control Group Assignment No placebo or no-treatment control group (+) Every one received treatment (+). Maybe some realized not receiving whole treatment (?) 38

Threats to Construct Validity To what extent variables capture desired constructs 1.Mono-Operation Bias 2. Mono-Method Bias 3.Experimenter Expectancies 39

Threats to Construct Validity To what extent variables capture desired constructs 1.Mono-Operation Bias Used two measures to assess outcome: Depression (+) 2. Mono-Method Bias HRSD is interview based and BDI is self-report (+) 3.Experimenter Expectancies Potential risk (-) Should have worked in favor of the CT condition that included all aspects of CBT-- therapists and researchers were aligned with CBT 40

Threats to External Validity Can we generalize observed relations across persons, settings and times 1.Person-Units 2. Outcome Measures 4.Settings 41

Threats to External Validity Can we generalize observed relations across persons, settings and times 1.Person-Units Highly selected sample – p. 296 (-) Exclusionary criteria (-) Primarily female = 73% (-) No info regarding race/ethnicity, SES (-) 2. Outcome Measures They used interview based and self report measures of Depression + Examined outcome using clinical significance index + 4.Settings Empirical Question….. 42

Empirically Supported Therapies - EST 1993 APA Division 12 Identify Efficacious treatments for specific disorders FDA Criteria: Specificity Specific ingredients vs. placebo effects Randomized double-blind, placebo control design 43

44 What components of treatment are responsible for therapy effects? Specific Ingredients unique to each therapy approach Common Factors --- Placebo underlie most approaches

Scientific advances that lead to the Randomized Placebo Control Group Design Germany: 1850 Wundt’s experimental method in psychology – 1 st psychology lab Britain:1800 Galton & Pearson – use of normal curve in assessment of distribution mental abilities France: Physicians start comparing treatments across groups of patients: treatment groups 45

Scientific advances that lead to the Randomized Placebo Control Group Design 1930s Fisher- ANOVA statistics - randomization Placebo treatment - physiochemical ingredients vs. patients’ expectations, hopes, beliefs Mesmerized…. Placebo 46

Wampold & Bhati’s Review: Therapy Outcome Research  Well conducted placebos of therapy = outcomes of bonafide treatment  Not much evidence that treatments work through mechanisms specified  Mixed findings regarding adherence to manuals and outcome  Regarding outcome: therapists’ effects > than type of treatments’ effects, but,  therapists and patients’ subjective exp are ignored 47

Well-established treatments Division 12 Task Force Criteria I. At least two good between-group design experiments must demonstrate efficacy in one or more of the following ways: A. Superiority to pill or psychotherapy placebo, or to other treatment (wait list control is not enough) B. Equivalence to already established treatment with adequate sample sizes 48

OR Well-established treatments (2) II. A large series of single-case design experiments must demonstrate efficacy with A. Use of good experimental design and B. Comparison of intervention to another treatment 49

And, Well-established treatments (3) III. Experiments must be conducted with treatment manuals or equivalent clear description of treatment IV. Characteristics of samples must be specified V. Effects must be demonstrated by at least two different investigators or teams 50

Probably efficacious treatments I. Two experiments must show that the treatment is superior to waiting-list control group OR II. One or more experiments must meet well-established criteria IA or IB, III, and IV above, but V is not met (2 investigator teams) OR III. A small series of single-case design experiments must meet well-established- treatment criteria 51

Treatment: Efficacious or Probably Efficacious ? Family Intervention (Smith et al.) Anorexia in adolescent and young adult women Superior cognitive behavioral intervention (well established treatment) in 2 randomized clinical studies Both studies conducted by Smith et al. 52

53 Evidence Based Psych. Practice Best available research Results from randomized Controlled Clinical Trials and other types of empirical studies Clinical Expertise Client Characteristics

Concerns EVT movement: Brief, straightforward manualized treatments: Cognitive Behavioral Emphasis on specific effects/ignore common factors Lack of applicability to wide range of clients; comorbidity, males, race, ethnicity, social class Mandates to use EVT- restriction to choice of treatment 54

APA Task Force Acceptable Research Designs Clinical observation Qualitative studies Systematic case studies Single-case experimental designs Public health and ethnographic research Process–outcome studies Studies in naturalistic settings Randomized Controlled Clinical Trials Meta-analysis 55

APA Task Force Research Designs Clinical observation Qualitative studies Systematic case studies Single-case experimental designs Public health and ethnographic research Process–outcome studies Studies in naturalistic settings Randomized Controlled Clinical Trials Meta-analysis 56

Evidenced Based PracticeEmpirically Supported Tmts. Effectiveness/Clinical utility:Efficacy; Main Focus: Main Focus: Knowledge from clinical expertise Knowledge from clinical expertise Clients’ individual differences Clients’ individual differences: Therapists’ individual differences: Therapists’ individual differences; 57

Evidenced Based PracticeEmpirically Supported Tmts. Effectiveness: does it work; generalizability; feasibility Efficacy; does treatment cause outcome Main Focus: Client Main Focus: Intervention Knowledge from clinical expertise is directly applied Knowledge fromclinical expertise Indirect- to generate hypotheses Clients’ individual differences Central Clients’ individual differences: Nuisance Therapists’ individual differences: Central Therapists’ individual differences; Nuisance 58

EBP or EST?? Know the person who has the disorder Know the disorder the person has 59

EBP or EST?? Know the person who has the disorder Know the disorder the person has 60

61 Components Shared by Approaches to Psychotherapy (Wampold, 2000) 1. Emotionally charged, confiding relationship 2. In a healing setting 3. Rationale, conceptual scheme, or myth to explains patient’s symptoms 4. There is a ritual or procedure that engages client and therapist based on the rationale ( Frank & Frank, 1991 ).

62 Common Elements in Therapeutic Rituals and Procedures (1/2) 5. The relationship helps combat clients’ feelings of alienation 6. The process of therapy provides hope for improvement 7. Therapist provides new learning experiences

63 Common Elements in Therapeutic Rituals and Procedures (2/2) 8. Client’s emotions are aroused as a result of therapy and the clients expects to improve 9. Therapist enhances client’s sense of mastery and self-efficacy 10. Therapist provides opportunities for practice

64 Wampold concluded that: Specific theoretical ingredients are not differentially related to outcome Specific ingredients are necessary to construct a coherent treatment In which therapists have faith, and that provides a convincing rational for clients Therefore, knowledge of theoretical approaches is necessary to build coherent interventions and treatments