1. Class 13 and 14 Jacobson et al (1996) APA (2006)Evidence Based Practice in Psychology 1

2. 2 Experimental Between Group Designs 1. Post-Test Only Control 2. Pre-Test -- Post-Test Control 3. Solomon Four Group (combination of 1 and 2 above) Factorial Design more than one independent variable; interactions gender x treatment Dependent Sample Design (Matching) 2

3. 3 Experimental Between Group Designs Post-Test Only Control Pre-Test -- Post-Test Control Solomon Four Group Factorial Design (Treatment X Therapist) Dependent Sample Design (Matching) # of previous episodes and severity depression, presence of dysthimia, gender, marital stauts 3

4. Types Outcome Studies Kazdin (chap 18) 1. Treatment Package Strategy 2. Dismantling Strategy 3. Constructive Strategy 4. Parametric Strategy (structural components) 5. Common factors Control Group 6. Comparative Outcome Strategy 7. Client and Therapist Variation Strategy Moderation Designs

5. Types Outcome Studies Kazdin (chap 18) 1. Treatment Package Strategy 2. Dismantling Strategy What are the active ingredients ? 1. Constructive Strategy 2. Parametric Strategy (structural components) 3. Common factors Control Group 4. Comparative Outcome Strategy 5. Client and Therapist Variation Strategy Moderation Designs

6. Research Focus How treatment effects change: identify change mechanisms VS. How well treatment works 6

7. Change Mechanisms Activation Hypotheses ( BA) (activation) Change behaviors- become active and access sources of reinforcement- occurs early in therapy Coping Skills Hypotheses (AT) Learn effective cognitive coping strategies (coping + activation) Beck Hypotheses (CT) Change cognitive structures or core schemas (coping + activation + cognitive schemas) 7

8. Chapter 78 Treatment Groups BA Condition: Behavioral Activation Hypotheses: Behavioral Activation AT Condition: Coping Skills Hypotheses Behavioral Activation Coping Skills - Automatic Thoughts CT Condition: Cognitive Therapy Hypotheses Behavioral Activation Automatic Thoughts Core Schemas 8

9. Chapter 79 Mechanisms of Change Construct Measure Behavioral Activation Dysfunctional Thinking Cognitive Schemas Pleasant Events Schedule Automatic Thoughts Q. Expanded Attributional Styles How did the authors examine if the mechanisms of change worked in therapy as predicted by theory? 9

10. How Treatment Effects Change To what extent treatment groups differed in post-test measures of each change mechanism Measure Ttreat. G Behavioral activation – Pleasant Event SCd BA Coping skills-- AT (Dist Thinking) AT Core schemas- Exp Attrb Style CT To what extent was there change from pre-to post- treatment in each change mechanism 10

11. How Well Treatments Work Which of the three treatment conditions yielded better outcomes regarding Depression at termination and at 6 month follow up? HRSD BDI Recovered Improved Rates 11

12. Research Questions Analyses Treatment Outcome group differences Depression Type of Analyses CVs IVS DVs Mechanism of Change: Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) Type of Analyses CVs IVS DVs Mechanism of Change: Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS) Type and # of Analyses 12

13. Research Questions Analyses Treatment Outcome group differences in Depression 3 x 4 MANCOVA CVs Pre-test BDI HRSD IVs Treatments: CT, AT, BA Therapists (n=4) Treatment X Therapist DVs Post-Test BDI HRSD Mechanism of Change: Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) Mechanism of Change: Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS) 13

14. How Well Treatments Work Which of the three treatment conditions yielded better Depression outcomes at termination and at 6 month follow up? 3 X 4 MANCOVA IVS 3 Treatments (CT, AT, BA) 4 Therapists Treatment X Therapist (Interaction) DVs - Depression HRSD – Clinician BDI - Self- Report 14

15. Which group to use in outcome analyses GroupSessions Intent-to treatTotal sample n = 149 Dropouts (not included in MANCOVA analyses; p. 299) 1<Sessions <12 n = 12 CompletersAt least 12/20 sessions n = 137 Maximum Completers20 sessions n = 129 15

16. Results MANCOVAsTreatment Outcome: Post- Test Depression Main Effect Treatment ? Main Effect Therapist ? Treatment X Therapist Interaction ? Results p. 300 -- ANOVAs Table 3 16

17. Results MANCOVAs Treatment Outcome: Post- Test BID & HRSD Treatment Group N= 149 129 137 Main Effect Treatment NS NS NS Main Effect Therapist NS NS NS Treatment X Therapist NS NS NS Results p. 300-- ANOVAs Table 3 17

18. Table 3 -- ANCOVAS ?? 18

19. 19

20. 20 ANCOVA Post-Test Main Effects for Therapists Post-Test Measures TherapistsBDIHRSD TH-1 BDIT1HRSDT1 TH-2 BDIT2HRSDT2 TH-3 BDIT3HRSDT3 TH-4 BDIT4HRSDT4 20

21. 21 ANCOVA - Post-Test Effects Therapist X Treatment Interaction Therapist BDI Post-test Scores by Treatment HRSD Post-test Scores by Treatment BAATCTBAATCT TH-1 BDIT1 HRSDT1 TH-2 BDIT2 HRSDT2 TH-3 BDIT3 HRSDT3 TH-4 BDIT4 HRSDT4 21

22. Outcome: Follow-up 6 months Overall Impact of Therapy ANCOVAS IV:Treatments DVs: Follow-Up HRSD BDI Covariate: Pretest Score Changes in Follow-up Time ANCOVAS IV: Treatments DVs: Follow-Up HRSD BDI Covariate: ??? 22

23. Outcome: Follow-up 6 months Overall Impact of Therapy ANCOVAS IV:Treatments DVs: Follow-Up HRSD BDI Covariate: Pretest Score Changes in Follow-up Time ANCOVAS IV: Treatments DVs: Follow-Up HRSD BDI Covariate: ??? 23

24. Outcome: Follow-up 6 months Overall Impact of Therapy ANCOVAS IV:Treatments DVs: Follow-Up HRSD BDI Covariate: Pretest residualized change score Changes in Follow-up Time ANCOVAS IV: Treatments DVs: Follow-Up HRSD BDI Covariate: Pre-test score 24

25. Clinical Significance: Termination Improved No major depression at post-test: LIFE Interview: psychiatric rating Recovered No major depression and BDI<8 25

26. 26

27. Clinical Significance: Mean Improvement/Recovered Rates across Treatments – Post -Test (p.299 text) Group ImprovedRecovered Intent-to treat Total sample (149) 62.3%51.5% Completers 12/20 N=37 58.3% Maximum Completers N = 129 66.0%54.5% Dropouts N=12 16.7%5.6% 27

28. 28

29. Conclusion 29

30. Conclusion 12 to 20 therapy sessions of either Behavioral Activation (BA), Automatic Thoughts (AT) and BA or Schema Restructuring and BA and AT results in similar outcome in treating depression for adult clients who meet exclusionary criteria and were treated by highly trained and monitored therapists. Regarding the mechanisms of change in CBT?? 30

31. Chapter 731 Mechanisms of Change Construct Measure Behavioral Activation Dysfunctional Thinking Cognitive Schemas Pleasant Events Schedule Automatic Thoughts Q. Expanded Attributional Styles How did the authors examine to what extent the mechanisms of change worked in therapy as predicted by theory? 31

32. How Treatment Effects Change To what extent treatment groups differed in post-test measures of each change mechanism Measure Ttreat. Grps Behavioral activation – Pleasant Event SCd BA Coping skills-- AT (Dist Thinking) AT Core schemas- Exp Attrb Style CT To what extent was there change from pre-to post- treatment in each change mechanism 32

33. Research Questions Analyses Treatment Outcome group differences Depression 3 x 4 MANCOVA CVs Pre-test BDI HRSD IVs Treatments: CT, AT, BA Therapists (n=4) Treatment X Therapist DVs Post-Test BDI HRSD Mechanism of Change: Treatment groups differed in post-test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) Mechanism of Change: Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS) 33

34. Research Questions Analyses Treatment Outcome group differences Depression 3 x 4 MANCOVA CVs Pre-test BDI HRSD IVs Treatments: CT, AT, BA Therapists Treatment X Therapist DVs Post-Test BDI HRSD Mechanism of Change: Treatment groups differed in post- test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) 3 ANCOVAS (one CV & DV per analysis) CV Pre-test: E Attrb Q, ATQ, PES IV Treatments: (CT, AT, BA) DV Post-test: E Attrb Q, ATQ, PES Mechanism of Change: Treatment groups change from pre-to post- treatment in each change mechanism (BA – AT-CS) 34

35. Research Questions Analyses Treatment Outcome group differences in Depression 3 x 4 MANCOVA CVs Pre-test BDI HRSD IVs Treatments: CT, AT, BA Therapists Treatment X Therapist DVs Post-Test BDI HRSD Mechanism of Change: Treatment groups differed in post- test measures of each change mechanism (Behavioral Activation, Automatic Thoughts & Core Schemas) 3 ANCOVAS (one CV & DV per analysis) CV Pre-test: E Attrb Q, ATQ, PES IV Treatments: (CT, AT, BA) DV Post-test: E Attrb Q, ATQ, PES Mechanism of Change: Treatment groups change from pre- to post- treatment in each change mechanism (BA – AT-CS) 3 Paired T Tests per Treatment Pre-post differences in each mechanism of change for each treatment : E Attrb Q, ATQ, PES 35

36. Paired T Tests: Pre-test/Post-test Change Treatment Pleasant Events Schedule Automatic Thoughts Questionnaires Expanded Attributional Styles Behavioral Activation PESch**ATQEAS Automatic Thoughts PESch**ATQ**EAS Cognitive Therapy PESch**ATQ**EAS** Clients in all conditions significantly improved on the three measures (p. 301) 36

37. Residual change score (RCS) in Mechanism of Change from Pre-to Mid-T and RCS in Depression from Mid-T to Post-T (p.301) 37

38. Threats to Statistical Conclusion Validity Jacobson et al. (1996) Are the observed relations among variables accurate? 1.Power 2.Unreliability of Treatment Implementation 38

39. Threats to Statistical Conclusion Validity Are the observed relations among variables accurate? 1.Power N=149 and 3 groups/about 50 participants per group. Large N for an Exp study (+) Outcome measures are well-known – high internal reliability (+) No information is given about alphas with study sample Power analyses not reported 2.Unreliability of Treatment Implementation Therapists were experienced in CT and trained for study (+) Training followed a manual prepared for each treatment group (+) Therapy tapes were listened on an on- going basis and therapists flagged if they deviated (+) Analyses of adherence based on 27 taped sessions showed that treatments were distinct and consistent with their respective protocols (+) 39

40. Threats to Statistical Conclusion Validity Are the observed relations among variables accurate? 3.Extraneous Variance in the Experimental Setting 4.Heterogeneity of Participants 40

41. Threats to Statistical Conclusion Validity Are the observed relations among variables accurate? 3.Extraneous Variance in the Experimental Setting Do not appear to be any – The same four therapists administered all treatments However procedures and settings are not described in detail 4.Heterogeneity of Participants Study had many exclusion criteria including co-morbidity, taking psychotropic medication, suicidal… p. 296. (+) However, don’t know #s in pool of volunteers from which the 152 accepted patients were taken from 72% were women (+) No info regarding race/ethnicity, SES (-) 41

42. Threats to Internal Validity Can we conclude that there is a causal relation between the IV and the DV? 1.Selection to Treat. Groups 2.History 3.Attrition 4.Repeated Testing Effects 5.Reaction to Control Group Assignment 42

43. Threats to Internal Validity Can we conclude that there is a causal relation between the IV and the DV? 1.Selection to Treat. Groups Used Randomization after matching for episodes dep., dysthimia, severity of depression, gender, marital status (+) 2.History Therapy appeared to occur for everyone at once – but this is not addressed 3.Attrition Small -only 15 out of 152 - 8% -Attrition during acute treatment was comparable across treatment conditions – p.296 right (+) 4.Repeated Testing Effects Five administration of measures – at pre-test; post –test and 6, 12, 18 month follow-up (-) 5.Reaction to Control Group Assignment No placebo or no-treatment control group (+) Every one received treatment (+). Maybe some realized not receiving whole treatment (?) 43

44. Threats to Construct Validity To what extent variables capture desired constructs 1.Mono-Operation Bias 2. Mono-Method Bias 3.Experimenter Expectancies 44

45. Threats to Construct Validity To what extent variables capture desired constructs 1.Mono-Operation Bias Used two measures to assess outcome: Depression (+) 2. Mono-Method Bias HRSD is interview based and BDI is self-report (+) 3.Experimenter Expectancies Potential risk (-) Should have worked in favor of the CT condition that included all aspects of CBT-- therapists and researchers were aligned with CBT 45

46. Threats to External Validity Can we generalize observed relations across persons, settings and times 1.Person-Units 2. Outcome Measures 4.Settings 46

47. Threats to External Validity Can we generalize observed relations across persons, settings and times 1.Person-Units Highly selected sample – p. 296 (-) Exclusionary criteria (-) Primarily female = 73% (-) No info regarding race/ethnicity, SES (-) 2. Outcome Measures They used interview based and self report measures of Depression + Examined outcome using clinical significance index + 4.Settings Empirical Question….. 47

48. Empirically Supported Therapies - EST 1993 APA Division 12 Identify Efficacious treatments for specific disorders FDA Criteria: Specificity Specific ingredients vs. placebo effects Randomized double-blind, placebo control design 48

49. 49 What components of treatment are responsible for therapy effects? Specific Ingredients unique to each therapy approach Common Factors --- Placebo underlie most approaches

50. Scientific advances that lead to the Randomized Placebo Control Group Design Germany: 1850 Wundt’s experimental method in psychology – 1 st psychology lab Britain:1800 Galton & Pearson – use of normal curve in assessment of distribution mental abilities France: Physicians start comparing treatments across groups of patients: treatment groups 50

51. Scientific advances that lead to the Randomized Placebo Control Group Design 1930s Fisher- ANOVA statistics - randomization 1930- 1950 Placebo treatment - physiochemical ingredients vs. patients’ expectations, hopes, beliefs Mesmerized…. Placebo 51

52. Wampold & Bhati’s Review: Therapy Outcome Research  Well conducted placebos of therapy = outcomes of bonafide treatment  Not much evidence that treatments work through mechanisms specified  Mixed findings regarding adherence to manuals and outcome  Regarding outcome: therapists’ effects > than type of treatments’ effects, however  therapists and patients’ subjective experiences are ignored 52

53. Well-established treatments Division 12 Task Force Criteria I. At least two good between-group design experiments must demonstrate efficacy in one or more of the following ways: A. Superiority to pill or psychotherapy placebo, or to other treatment (wait list control is not enough) B. Equivalence to already established treatment with adequate sample sizes 53

54. OR Well-established treatments (2) II. A large series of single-case design experiments must demonstrate efficacy with A. Use of good experimental design and B. Comparison of intervention to another treatment 54

55. And, Well-established treatments (3) III. Experiments must be conducted with treatment manuals or equivalent clear description of treatment IV. Characteristics of samples must be specified V. Effects must be demonstrated by at least two different investigators or teams 55

56. Probably efficacious treatments I. Two experiments must show that the treatment is superior to waiting-list control group OR II. One or more experiments must meet well-established criteria IA or IB, III, and IV above, but V is not met (2 investigator teams) OR III. A small series of single-case design experiments must meet well-established- treatment criteria 56

57. Treatment: Efficacious or Probably Efficacious ? Family Intervention (Smith et al.) Anorexia in adolescent and young adult women Superior cognitive behavioral intervention (well established treatment) in 2 randomized clinical studies Both studies conducted by Smith et al. 57

58. 58 Evidence Based Psych. Practice Best available research Results from randomized Controlled Clinical Trials and other types of empirical studies Clinical Expertise Client Characteristics

59. Concerns EVT movement: Brief, straightforward manualized treatments: Cognitive Behavioral Emphasis on specific effects/ignore common factors Lack of applicability to wide range of clients; comorbidity, males, race, ethnicity, social class Mandates to use EVT- restriction to choice of treatment 59

60. APA Task Force Acceptable Research Designs Clinical observation Qualitative studies Systematic case studies Single-case experimental designs Public health and ethnographic research Process–outcome studies Studies in naturalistic settings Randomized Controlled Clinical Trials Meta-analysis 60

61. APA Task Force Research Designs included in Div. 12 criteria for EVT Clinical observation Qualitative studies Systematic case studies Single-case experimental designs Public health and ethnographic research Process–outcome studies Studies in naturalistic settings Randomized Controlled Clinical Trials Meta-analysis 61

62. Evidenced Based PracticeEmpirically Supported Tmts. Effectiveness/Clinical utility:Efficacy: Main Focus: Main Focus: Knowledge from clinical expertise Knowledge from clinical expertise Clients’ individual differences Clients’ individual differences: Therapists’ individual differences: Therapists’ individual differences; 62

63. Evidenced Based PracticeEmpirically Supported Tmts. Effectiveness: does it work; generalizability; feasibility Efficacy: does treatment cause outcome Main Focus: Client Main Focus: Intervention Knowledge from clinical expertise is directly applied Knowledge fromclinical expertise Indirect- to generate hypotheses Clients’ individual differences Central Clients’ individual differences: Nuisance Therapists’ individual differences: Central Therapists’ individual differences; Nuisance 63

64. EBP or EST?? Know the person who has the disorder Know the disorder the person has 64

65. EBP or EST?? Know the person who has the disorder Know the disorder the person has 65

66. 66 Components Shared by Approaches to Psychotherapy (Wampold, 2000) 1. Emotionally charged, confiding relationship 2. In a healing setting 3. Rationale, conceptual scheme, or myth to explains patient’s symptoms 4. There is a ritual or procedure that engages client and therapist based on the rationale ( Frank & Frank, 1991 ).

67. 67 Common Elements in Therapeutic Rituals and Procedures (1/2) 5. The relationship helps combat clients’ feelings of alienation 6. The process of therapy provides hope for improvement 7. Therapist provides new learning experiences

68. 68 Common Elements in Therapeutic Rituals and Procedures (2/2) 8. Client’s emotions are aroused as a result of therapy and the clients expects to improve 9. Therapist enhances client’s sense of mastery and self-efficacy 10. Therapist provides opportunities for practice

69. 69 Wampold concluded that: Specific theoretical ingredients are not differentially related to outcome Specific ingredients are necessary to construct a coherent treatment In which therapists have faith, and that provides a convincing rational for clients Therefore, knowledge of theoretical approaches is necessary to build coherent interventions and treatments